Backing Up Your Linux

Run a Linux-based web server? If so, your backup possibilities have increased dramatically. Depending on whether you are copying to, or grabbing from the web server, there are two applications I love to use: rsync, as you have likely already guessed, and also LFTP, a command-line-based FTP application that is feature-rich and a pleasure to use.

Let’s delve into our LFTP usage first, since our rsync techniques will take a little more time. First, if you are unsure whether or not this application is installed, simply type lftp at the command-line. If you are using a distro with a repository, chances are that it will be found there. If not, feel free to compile by source.

Suppose you are the head of a popular open-source project and would like to upload nightly builds of your source code. The easiest way would be first creating an LFTP script, followed by a shell script and adding it to your crontab. Let’s tackle the first challenge first.

If you have a directory full of nightly builds (which could be built with our scripts on the next page), you can use LFTP to automate the process of uploading them. First you would want to create a simple script that LFTP understands, which are thankfully, similar to shell scripts. I -highly- recommend saving this under a root account because your password will be clear as the day (unless you happen to live in a fog-belt, whereas that would be a bad example).

lftp open -u username,password -p 21 domain.com ;
lftp mput -O public_html/nightly /home/techgage/projects/appname/nightly/*.tar.gz

Running this script with lftp -f lftp_script would proceed to upload all *.tar.gz files found in that directory. If the tarball exists on the server already, it will not be overwritten. So that way, it will only upload whatever is new, which is likely what you want. On the server-side, you could create a script which would symlink the new file to say, latest.tar.gz, since LFTP and FTP in general doesn’t have that kind of functionality.

You want more than just that one filetype to be uploaded? How about the entire folder? Let’s move onto the mirror command with our sample script:

open -u username,password -p 21 domain.com ;
mirror -R /home/techgage/projects/appname public_html/appname

This script will upload all the files found under your /appname folder, including subfolders. That way, if you have a mounted folder on your local machine that has all of your nightly builds, hash files, readmes or anything else, then the same will be found on the server. To reverse the process (mirror a remote folder to a local one), remove the -R toggle and switch the directories around:

open -u username,password -p 21 domain.com ;
mirror public_html/appname /home/techgage/projects/appname

LFTP is a superb program for simple file exchanging, but for more in-depth and important tasks, you should trust rsync.

Using Rsync Without Passwords

If you run a website or have files on a server that need to be updated often, then rsync can become your best friend, very quickly. If you rsync your site on a regular basis, it means you -always- have a recent copy stored on your local machines hard-drive, NAS or external storage. So, if something breaks, you will be quickly able to restore the file thats gone awry.

Using rsync manually is simple… just type in the command and then the password, then wait. However, if you want to make nightly backups or sync twice a day, it can become an arduous chore. That’s why you will be interested in setting up rsync keys and automating the entire process.

To sync everything without incident, it’s recommended you do this as root. However, if you are positive that all of the files you require are from a user account only (eg. /home/username), you should be fine. In my case, I want to sync more files than what’s found in /home, so I first enter su mode and then the SSH directory: cd /root/.ssh. Once in there, you will need to create an identification key. When prompted for a passphrase, hit enter twice without entering any characters.

localhost .ssh # ssh-keygen -t dsa -b 1024 -f rsync-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in rsync-key.
Your public key has been saved in rsync-key.pub.
The key fingerprint is:
0e:2d:a9:fc:34:66:86:3c:42:43:b4:d2:8e:3c:50:52 root@localhost

Once the process is finished, it will create rsync-key and rsync-key.pub. You will need to duplicate this rsync-key.pub to your servers /root/.ssh folder, to keep things simple. You can do this a variety of ways: Copy over via SSH, Copy/Paste the entire contents of the file, use SCP… it’s up to you. Me, I just used gFTP to log onto the server as root and copied it that way.

At this point, you are almost finished. Making sure that the file is on your server, SSH in and enter the same folder as on your local machine: cd /root/.ssh. In order to gain access to the server without a password, you will need to copy your rsync credentials to an authorized_keys file. Regardless of whether it is there already, add the contents of your rsync-key.pub and chmod it before you finish up.

[root@fakeserver .ssh]# cat rsync-key.pub >> authorized_keys
[root@fakeserver .ssh]# chmod 644 authorized_keys

That step was the last one, so you can now log out and back in to your local machine. From here, log back into the server you just logged out of, hopefully password free:

localhost techgage # ssh -i /root/.ssh/rsync-key domain.com
Last login: Mon Aug 20 21:06:09 2007 from xxx000000000000.cpe.cable.domain.com
[root@fakeserver ~]#

If that last step proved successful, you are ready to rsync without the hassle of a password, meaning you can set up a new entry in your crontab to automate your backups! To properly rsync with your server, use these steps:

rsync -av -e “ssh -i rsync-key” domain.com:/home/username/projects /home/techgage/projects

Now that you are all set, let’s jump right into our script-writing and the best part, crontabs.