Bash script to mirror directory and database of website between servers

7. Main Functions

Now we get into the meat and potatoes of this program. This section contains four functions:

  1. download_files – Download the files from production to a temporary directory on the local server
  2. export_database – Export the database from production to a temporary directory on the local server
  3. upload_files – Copy the files from the temporary directory on the local server to where they need to go on the local server
  4. import_database – Import the database into the local website that we want to update

As a side note, the reason we’re using a temporary directory in the first place is two-fold. One, it allows us to reset our local site easily at any point, and two, it allows you to pull those changes into another instance of that site. The beauty is that it can do all that without having to reconnect to production, i.e., less strain on production server.

7.1 Download Files

First, the program will check if the temporary directory you specified in step 3 exists. If not, it will create it.

Second, it will start synchronizing the src location (production) with the dst location (server you are on). It will delete anything that wasn’t present in the source, and skip any directories specified in ex (exclude).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# rsync
# -a, --archive         archive mode; same as -rlptgoD
#   -r, --recursive     recurse into directories
#   -l, --links         copy symlinks as symlinks
#   -p, --perms         preserve permissions
#   -t, --times         preserve times
#   -g, --group         preserve group
#   -o, --owner         preserve owner (super-user only)
#   -D                  same as --devices --specials
#       --devices       preserve device files (super-user only)
#       --specials      preserve special files
# -v, --verbose         increase verbosity
# --exclude=PATTERN     exclude files matching PATTERN
# --delete              delete files that don’t exist on sender
# --delete-excluded     also delete excluded files on receiver
 
# Copy files from production web server to temporary directory
# $1    path to files on production server
# $2    path to files on local server
# $3    rsync exclude option(s)
function download_files {
    src=$1
    dst=$2
    ex=$3
 
    if [ ! -d "$dst" ]; then
        echo "> Creating temporary directory..."
        echo "  $dst"
        mkdir -p "$dst"
    fi
 
    if [ -d "$dst" ]; then
        echo "> Synchronizing production files with temporary directory..."
        echo "  Source      : $src"
        echo "  Destination : $dst"
        rsync --archive --verbose $ex --delete --delete-excluded "$src" "$dst"
    else
        echo "> Temporary directory does not exist"
        echo "  $dst"
    fi
}
# rsync
# -a, --archive			archive mode; same as -rlptgoD
# 	-r, --recursive		recurse into directories
# 	-l, --links			copy symlinks as symlinks
# 	-p, --perms			preserve permissions
# 	-t, --times			preserve times
# 	-g, --group			preserve group
# 	-o, --owner			preserve owner (super-user only)
# 	-D					same as --devices --specials
#		--devices		preserve device files (super-user only)
#		--specials		preserve special files
# -v, --verbose			increase verbosity
# --exclude=PATTERN		exclude files matching PATTERN
# --delete 				delete files that don’t exist on sender
# --delete-excluded		also delete excluded files on receiver

# Copy files from production web server to temporary directory
# $1 	path to files on production server
# $2 	path to files on local server
# $3	rsync exclude option(s)
function download_files {
	src=$1
	dst=$2
	ex=$3

	if [ ! -d "$dst" ]; then
		echo "> Creating temporary directory..."
		echo "  $dst"
		mkdir -p "$dst"
	fi

	if [ -d "$dst" ]; then
		echo "> Synchronizing production files with temporary directory..."
		echo "  Source      : $src"
		echo "  Destination : $dst"
		rsync --archive --verbose $ex --delete --delete-excluded "$src" "$dst"
	else
		echo "> Temporary directory does not exist"
		echo "  $dst"
	fi
}

7.2 Export Database

First, the program will ensure the temporary folder exists (as before), and create it if it doesn’t.

Second, it will connect to the production database and export everything, but excluding any tables specified in ignore.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# Export database from production database server to temporary directory
# $1    database hostname
# $2    database username
# $3    database name
# $4    path to save mysql export
# $5    mysqldump ignore-table option(s)
function export_database {
    host=$1
    user=$2
    db=$3
    dst=$4
    ignore=$5
 
    if [ ! -d "$dst" ]; then
        echo "> Creating temporary directory..."
        echo "  $dst"
        mkdir -p "$dst"
    fi
 
    if [ -d "$dst" ]; then
        echo "> Exporting production database to temporary directory..."
        echo "  Database    : $db"
        echo "  Destination : $dst"
        echo "  Filename    : $db.sql"
        mysqldump --verbose --host=$host --user=$user --password $db $ignore > "$dst/$db.sql"
    else
        echo "> Temporary directory does not exist"
        echo "  $dst"
    fi
}
# Export database from production database server to temporary directory
# $1 	database hostname
# $2 	database username
# $3	database name
# $4 	path to save mysql export
# $5 	mysqldump ignore-table option(s)
function export_database {
	host=$1
	user=$2
	db=$3
	dst=$4
	ignore=$5

	if [ ! -d "$dst" ]; then
		echo "> Creating temporary directory..."
		echo "  $dst"
		mkdir -p "$dst"
	fi

	if [ -d "$dst" ]; then
		echo "> Exporting production database to temporary directory..."
		echo "  Database    : $db"
		echo "  Destination : $dst"
		echo "  Filename    : $db.sql"
		mysqldump --verbose --host=$host --user=$user --password $db $ignore > "$dst/$db.sql"
	else
		echo "> Temporary directory does not exist"
		echo "  $dst"
	fi
}

7.3 Upload Files

The program will mirror the temporary directory with the equivalent directory of the local site.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# Copy files from temporary directory to local site
# $1    path to temporary files on local server
# $2    path to real files on local server
function upload_files {
    src=$1
    dst=$2
 
    if [[ (-d "$src") && (-d "$dst") ]]; then
        echo "> Synchronizing temporary files with local site directory..."
        echo "  Source      : $src"
        echo "  Destination : $dst"
        rsync --archive --delete --delete-excluded "$src" "$dst"
    else
        echo "> Temporary or local site directory does not exist"
        echo "  Source      : $src"
        echo "  Destination : $dst"
    fi
}
# Copy files from temporary directory to local site
# $1 	path to temporary files on local server
# $2	path to real files on local server
function upload_files {
	src=$1
	dst=$2

	if [[ (-d "$src") && (-d "$dst") ]]; then
		echo "> Synchronizing temporary files with local site directory..."
		echo "  Source      : $src"
		echo "  Destination : $dst"
		rsync --archive --delete --delete-excluded "$src" "$dst"
	else
		echo "> Temporary or local site directory does not exist"
		echo "  Source      : $src"
		echo "  Destination : $dst"
	fi
}

7.4 Import Database

The program will replace your local database with the exported database from production. This shouldn’t be an issue, but if you have some test data in there, remember it’ll all be overwritten.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Import production database from temporary directory to local database
# $1    database hostname
# $2    database username
# $3    database password
# $4    database name
# $5    path to mysql export
function import_database {
    host=$1
    user=$2
    pass=$3
    db=$4
    src=$5
 
    if [ -f "$src/$db.sql" ]; then
        echo "> Importing temporary database export into local database..."
        echo "  Filename : $db.sql"
        echo "  Source   : $src"
        echo "  Database : $db"
        mysql --host=$host --user=$user --password=$pass $db < "$src/$db.sql"
    else
        echo "> Temporary database export does not exist"
        echo "  Filename : $db.sql"
        echo "  Source   : $src"
    fi
}
# Import production database from temporary directory to local database
# $1 	database hostname
# $2 	database username
# $3	database password
# $4 	database name
# $5 	path to mysql export
function import_database {
	host=$1
	user=$2
	pass=$3
	db=$4
	src=$5

	if [ -f "$src/$db.sql" ]; then
		echo "> Importing temporary database export into local database..."
		echo "  Filename : $db.sql"
		echo "  Source   : $src"
		echo "  Database : $db"
		mysql --host=$host --user=$user --password=$pass $db < "$src/$db.sql"
	else
		echo "> Temporary database export does not exist"
		echo "  Filename : $db.sql"
		echo "  Source   : $src"
	fi
}

2 thoughts on “Bash script to mirror directory and database of website between servers

  1. mahender

    Hi,

    We have site for library.We will keep on updating the site.We are planing for mirror site for it.
    I have understood with the help of rsync we can update the /var/www folder of mirror site.
    How mysql database of mirror site will be in sync with production server.Could you please tell me how it can be achieved.

    Thanks in advance,
    Raja

    Reply
    1. Ryan Sechrest Post author

      The key commands are rsync (to synchronize files), mysqldump to export a database to a file, and mysql used to import a file into another database. With those three commands, all showcased in the above code, you can mirror files and databases between servers, or more specifically, keep one server (primary) synchronized with another (secondary).

      If you look at my development kit page, toward the very bottom are links to the individual documentation of those commands.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *