Summer Nights

A blog on embedded and real-time systems as well as on general computing issues

backur: A Configurable Python Utility for Optimized Incremental Backups

I have been on the lookout for an efficient small utility to incrementally backup my Linux server to an external drive. I have looked at various options like Back in Time and Backup PC. They are however too advanced and too automated for my purpose. My requirements are the following:

  1. Incremental backups
  2. Console based
  3. Preservation of file times, permissions, user and group
  4. Easy configuration
  5. Logging support
  6. Mirror mode

I have been using rsync for some time in bash scripts. As Python is becoming more and more my language of scripting, so I wanted to write it in Python. With that background information, here comes “backur”. backur is very easy to configure and use. backur.conf, as given below, is self-explanatory. It also provides customizable logging including log-file rotation. Th backup and source paths are very easy to enter, and allow spaces, thanks to shlex split method. You can run backur either manually, or through as a cron job, as it suits you.

#! /usr/bin/python

# Title:    backur.py
# Author:   Asif Iqbal (aiqbalrana@gmail.com)
# Date:     11OCT2012
# Info:     "backur" can be used to automatically backup given folder locations
#           to another backup location. backur is easily configurable through a
#           configuration file. backur uses rsync to achieve highly optimized
#           data transfers by doing only the incremental updates.    

import ConfigParser
import re, os
import glob
import logging
import logging.handlers
import subprocess
import datetime
import shlex

def splitNStripArgs(inputStr, charToStrip):
    data = inputStr.split(',')
    retData = []
    for item in data:
        retData.append((item.strip()).strip(charToStrip))
    return retData

# Read configuration data
confpath = os.path.dirname(__file__)
config = ConfigParser.ConfigParser()
config.read(confpath + "/backur.conf")

rsyncflags = splitNStripArgs((config.get("Config", "RSYNCFlags")),'"')
rsyncexe   = config.get("Config", "RSYNC")
folderlist = splitNStripArgs((config.get("Folders", "FolderList")),'"')
backuploc  = config.get("Config","BackupLocation")
backupsize = int(config.get("Config","MaxLogFileSize"))
numbackups = int(config.get("Config","NumBackups"))
logfile    = config.get("Config","LogFile")

# Setup logging
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)
handler = logging.handlers.RotatingFileHandler(logfile, maxBytes=backupsize, backupCount=numbackups)
my_logger.addHandler(handler)

# Do the job
my_logger.info("#########################################################################################")
my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("Starting backup")
my_logger.info("Time: " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M"))
my_logger.info("-----------------------------------------------------------------------------------------")

# Run individual backups
for jobfolder in folderlist:
    command = rsyncexe + " " + ' '.join(rsyncflags) + " -s \"" + jobfolder + "\" \"" + backuploc + "\""
    my_logger.info("Executing: " + command)
    out = subprocess.check_output(shlex.split(command))
    my_logger.info(out)

my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("Backup Complete")
my_logger.info("Time: " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M"))
my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("#########################################################################################")

backur can be configure using backur.conf which look like the following:

[Config]
BackupLocation: /media/usb/backup/
LogFile: /var/log/backur.log
MaxLogFileSize: 2097152
NumBackups: 5
RSYNCFlags: "-a",
	        "-v",
	        "--delete"
RSYNC: /usr/bin/rsync

[Folders]
FolderList: "/mnt/Drv1/pics",
	        "/mnt/Drv2/my data",
	        "/mnt/Drv3/docs"

backur can also be downloaded in a zip file. I have written and tested backur only on Linux. However, it should also run on Windows provided that Python is installed on the system. Do let me know about your experiences or issues if you use backur.

Leave a Reply

Your email address will not be published. Required fields are marked *