A Brief Guide to Backing up Your Site on Joyent
January 02, 2008

I've revamped a lot of my backup strategies lately, and I feel like I need to share my latest update - backing up my joyent hosted blog to my local development server. Performing a nightly backup of the full contents of my blog assures me that, should Joyent ever perish, my beautiful writings and incredible content will not be lost.

Avast! The first order of business is getting a rake task setup to actually backup the database. This is as simple as using Craig Ambrose's Rake Backup Task with a quick little modification. Since I use postgresql, the original mysql_dump command wasn't quite up to snuff. I also added in a quick regex expression to make sure that the script was only deleting folders which matched the backup scheme as a precaution.

lib/tasks/backup.rake [updated 2008-01-02]

require 'find'
namespace :db do
  desc "Backup the database to a file. Options: DIR=base_dir RAILS_ENV=development MAX=10"
  task :backup => [:environment] do
    datestamp = Time.now.strftime("%Y-%m-%d")
    base_path = ENV["DIR"] || "db"
    backup_base = File.join(base_path, 'backup')
    backup_folder = File.join(backup_base, datestamp)
    backup_file = File.join(backup_folder, "#{RAILS_ENV}_dump.sql.gz")
    File.makedirs(backup_folder)
    db_config = ActiveRecord::Base.configurations[RAILS_ENV]
#    sh "mysqldump -u #{db_config['username']} -p#{db_config['password']} -Q --add-drop-table -O add-locks=FALSE -O lock-tables=FALSE #{db_config['database']} | gzip -c
    sh "pg_dump -U #{db_config['username']} #{db_config['database']} | gzip -c > #{backup_file}"
    puts "-- Created backup: #{backup_file}"

    deleted_backups = 0
    max_backups = ENV["MAX"] ? ENV["MAX"].to_i : 10
    unwanted_backups = Dir.new(backup_base).entries.sort[2..-1].reverse[max_backups..-1] || []
    unwanted_backups.each do |unwanted_backup|
      next unless unwanted_backup.match(/[0-9]{4}\-[0-9]{2}\-[0-9]{2}/)
      FileUtils.rm_rf(File.join(backup_base, unwanted_backup))
      puts "-- Deleted #{unwanted_backup}"
      deleted_backups+=1
    end

    puts "-- Deleted #{deleted_backups} backups."
  end

end

Go ahead and run "RAILS_ENV=production rake db:backup" (while in the rails app's root) to test the backup scheme and you should see a new folder created underneath "db/" called "backup". That should contain a folder dated after today's date. That contains your most up to date database backup. Joy!

A Joyent Note; The pg_dump requires the localhost specification otherwise the connection will be dropped.

Hold on, you say! I was prompted for a password, how can I automate that? It's actually a pretty simple matter of populating your "~/.pgpass" file with your login credentials. Here's an example;

~/.pgpass

localhost:*:YOURDATABASENAME:YOURUSERNAME:YOURPASSWORD

This file will need to be chmod'ed to 0600 in order to ensure that nobody else can read it, of course. Afterwards you should be able to run the prior rake command (RAILS_ENV=production rake db:backup) without being prompted for a password. Now you're one step closer!

Go ahead and add the Rake task to your crontab as thus;

crontab -e

23   23  *   *   *     cd WHEREVERYOURRAILSAPPIS && PATH=/usr/local/bin:/usr/bin:/bin RAILS_ENV=production rake db:backup

Cron on Joyent's servers seems to lack any pathing - so the specific PATH variable is necessary. Joyent's version of Cron also seems to lack the ability to specify environment variables directly in the file, so you have to specify a sane PATH for EACH command used. A PITA if you ask me. Whicher way you look at it, your cron task should be setup to automatically generate a backup of your database every night at 23:23.

Now that you've got your site automatically backing up on Joyent's servers you're in good shape. However, as any good BOFH should know - unless the data's on your servers it's not in your control. We need to get the data syncing up to our local development box in order to have truly powerful backups.

Go ahead and get passwordless SSH running between your development box and Joyent's server and then just set yourself up a simple nightly rsync task!

crontab -e (on your local dev box)

  2    1  *   *   *     rsync -avz USERNAME_ON_JOYENT@SERVER_ON_JOYENT.joyent.us:PATH_TO_YOUR_APP_ON_JOYENT YOUR_LOCAL_DESTINATION

That's it. You'll have a tidy nightly backup of what's up on your Joyent account. Since you're already keeping good local backups, you don't have to worry about making it any more complicated than that. Now, enjoy your new backed up data!

Update 2008-01-02

I found a rather nasty bug in the backup script that caused the removal rules to fail and simply remove all backups. The crucial item here was changing "max_backups = ENV["MAX"] ? ENV["MAX"].to_i : 10" to use ternary instead of simply being an or statement - because ENV["MAX"].to_i was returning a true value even when ENV["MAX"] wasn't specified - causing the script to believe that max_backups was 0. Damn. Fixed now.