Bash script to mirror directory and database of website between servers

Preface

When you have a site that’s an on-going project, and you’re using a development, staging, and production server configuration, time and time again you’ll have a need to mirror the  production site content with your other environments.

Previously, this either meant to manually (FTP) or semi-manually (rsync) update the files, export the database, and in the case of WordPress MU, update certain records in the database. While the entire act isn’t hard, it has a zero-fun factor.

I put some time aside to write a small bash script that could automate this for multiple sites with just two commands. The script could definitely be improved, but it’s functional and gets the job done nicely, but I’m always open to suggestions and improvements.

I’ll go through the entire script, and if you find it useful, you can adapt it to meet your needs. Any variables or data you may need to change, will be highlighted in red below the code box.

You can find the entire code in a Gist. Note that there are two versions. One for server to server, the other for server to localhost. We’ll discuss the server edition below.

Table of Contents

  1. Program Skeleton
  2. Server Environment
  3. Production/Local Server Variables
  4. Program Arguments
  5. Display Functions (page 2)
  6. MySQL Functions
  7. Main Functions (page 3)
    1. Download Files
    2. Export Database
    3. Upload Files
    4. Import Database
  8. Run Functions (page 4)
    1. Run Synchronize
    2. Run Update
    3. Run
  9. Run Program (page 5)

1. Program Skeleton

We start with the bash shebang, followed by some program meta data, which will print out on the screen when the program runs. You’ll find a sample output of this at the end of post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/bin/bash
 
### Program meta data
 
# Program name
NAME='site'
 
# Program version
VERSION='1.0.0'
 
# Today's date
DATE=$(date +'%x %r %Z')
 
# Program header title
TITLE="Site v$VERSION on $HOSTNAME"
 
# Program header information
STATUS="$DATE by $USER"
 
# Program usage help
USAGE="Usage: $NAME [sync|update] [sitename]"
#!/bin/bash

### Program meta data

# Program name
NAME='site'

# Program version
VERSION='1.0.0'

# Today's date
DATE=$(date +'%x %r %Z')

# Program header title
TITLE="Site v$VERSION on $HOSTNAME"

# Program header information
STATUS="$DATE by $USER"

# Program usage help
USAGE="Usage: $NAME [sync|update] [sitename]"

Things to edit:

  • Line 21: sitename

2. Server Environment

Next, we determine on which server the program is being executed on. This allows us to use the very same program on all servers, meaning we only have to maintain one instance of it.

1
2
3
4
5
6
7
8
9
10
11
### Server environment
 
ENV=''
case $HOSTNAME in
    DEV01.DOMAIN.COM)
        ENV='DEV'
        ;;
    STG01.DOMAIN.COM)
        ENV='STG'
        ;;
esac
### Server environment

ENV=''
case $HOSTNAME in
	DEV01.DOMAIN.COM)
		ENV='DEV'
		;;
	STG01.DOMAIN.COM)
		ENV='STG'
		;;
esac

Things to edit:

  • Line 5: DEV01.DOMAIN.COM
  • Line 8: STG01.DOMAIN.COM

If you type in hostname in Terminal on each of your servers, you’d replace DEV01.DOMAIN.COM and STG01.DOMAIN.COM with those values. You can add as many server environments as you have. What this will do is set a variable called ENV that we can later check to determine which set of credentials and domain to use.

3. Production/Local Server Variables

The program needs to know where the production web and database server is, and with which username the program should attempt to connect to each. You’ll be prompted for those passwords when the time comes.

The program also needs to know where your sites live on the production server, where they live on your local server, and where you want to temporarily store the files that will be downloaded from the production server.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
### Production database server
 
# Hostname or IP address
MYSQL_PRD_HOSTNAME='hostname'
 
# Username
MYSQL_PRD_USERNAME='username'
 
### Production web server
 
# Hostname or IP address
SSH_PRD_HOSTNAME='hostname'
 
# Username
SSH_PRD_USERNAME='username'
 
### Local server paths
 
# Absolute path to local sites directory
# Ex: /var/www/domains
LOCAL_SITES_PATH='/var/www/domains'
 
# Absolute path to production sites directory
# Ex: /var/www/domains
PRD_SITES_PATH='/var/www/domains'
 
# Absolute path to local temporary sites directory
# Ex: /opt/sites
TEMP_SITES_PATH='/opt/sites'
### Production database server

# Hostname or IP address
MYSQL_PRD_HOSTNAME='hostname'

# Username
MYSQL_PRD_USERNAME='username'

### Production web server

# Hostname or IP address
SSH_PRD_HOSTNAME='hostname'

# Username
SSH_PRD_USERNAME='username'

### Local server paths

# Absolute path to local sites directory
# Ex: /var/www/domains
LOCAL_SITES_PATH='/var/www/domains'

# Absolute path to production sites directory
# Ex: /var/www/domains
PRD_SITES_PATH='/var/www/domains'

# Absolute path to local temporary sites directory
# Ex: /opt/sites
TEMP_SITES_PATH='/opt/sites'

Things to edit:

  • Line 4: hostname
  • Line 7: username
  • Line 12: hostname
  • Line 15: username
  • Line 21: /var/www/domains
  • Line 25: /var/www/domains
  • Line 29: /opt/sites

As a side note, I created a new database user that has read-only access to the production site, to ensure that nothing would be manipulated by accident.

4. Program Arguments

The program will take two arguments, which we will save as arg1 and arg2. This allows us to reference those arguments in our functions later.

1
2
3
4
### Save program arguments
 
arg1=$1
arg2=$2
### Save program arguments

arg1=$1
arg2=$2

The first argument will tell the program what to do, and the second which site to do it to.

2 thoughts on “Bash script to mirror directory and database of website between servers

  1. mahender

    Hi,

    We have site for library.We will keep on updating the site.We are planing for mirror site for it.
    I have understood with the help of rsync we can update the /var/www folder of mirror site.
    How mysql database of mirror site will be in sync with production server.Could you please tell me how it can be achieved.

    Thanks in advance,
    Raja

    Reply
    1. Ryan Sechrest Post author

      The key commands are rsync (to synchronize files), mysqldump to export a database to a file, and mysql used to import a file into another database. With those three commands, all showcased in the above code, you can mirror files and databases between servers, or more specifically, keep one server (primary) synchronized with another (secondary).

      If you look at my development kit page, toward the very bottom are links to the individual documentation of those commands.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *