Data Backups with Bacula: Client Data Backup and Restoration
When configuring the Bacula client, we don’t need the full-blown Bacula installation, but just the bacula-client package that we can simply install by using the apt-get command. The client configuration consists of editing the /etc/bacula/bacula-fd.conf configuration file and ensuring that the name and the password in the Director definition block are the same as defined in the bacula-dir.conf on the Bacula server.
# apt-get install bacula-client
When choosing to backup a new client, we should add a new client definition in the bacula-dir.conf. The Client definitions should specify the name, the address and port number of a file daemon. Special care should be taken that proper password from the bacula-fd.conf configuration from from the client endpoint is copied to the Bacula director configuration file.
Name = client-fd
Address = client
FDPort = 9102
Catalog = MyCatalog
Password = "ffAC33z3JNGCYfzj/EPW7Aisuw0JWi7SmjtnGP+"
File Retention = 30 days
Job Retention = 6 months
AutoPrune = yes
Next, we can connect to the Bacula server by using bconsole, while a new backup can be issued by using the run command in bconsole as shown below.
A job name must be specified.
The defined Job resources are:
Select Job resource (1-4): 2
Run Backup job
FileSet: Full Set
Pool: File (From Job resource)
Storage: File (From Job resource)
When: 2014-08-09 13:38:27
OK to run? (yes/mod/no): yes
Job queued. JobId=8
You have messages.
The run command printed some messages, which we can see with the messages command presented below – the run command was not run successfully, because there was some problem with permissions.
09-Aug 13:35 baculaserver-dir JobId 7: No prior Full backup Job record found.
09-Aug 13:35 baculaserver-dir JobId 7: No prior or suitable Full backup found in catalog. Doing FULL backup.
09-Aug 13:35 baculaserver-dir JobId 7: Start Backup JobId 7, Job=BackupClient.2014-09-09_13.35.16_04
09-Aug 13:35 baculaserver-sd JobId 7: Error: dev.c:120 Unable to stat device /backup/: ERR=Permission denied
The permissions of the /backup/ directory is root:root as presented below. This is clearly the problem, since Bacula storage runs under bacula:bacula.
# ls -ld /backup/
drwxr-xr-x 3 root root 4096 Aug 5 20:46 /backup/
To enable the bacula-sd daemon to write the files to the disk, we have to change the directory permissions:
# chown bacula:bacula /backup -R
Once we rerun the backup, we can again take a look at the messages created by Bacula in order to see whether everything worked. Below we can see that Bacula informed us that we have to create a new volume with the label command. A volume is a single file/tape where Bacula will write backup data, while a pool groups multiple volumes together to overcome the limitation of being restricted to the length of a single volume. Bacula stores backup data in a volume and keeps the information about which file is stored in each volume in a catalog. We can create volumes manually by using the label command or we can use the ‘Label Format‘ directive inside Pool definitions, which will automatically create a volume with a specified name.
09-Aug 13:59 backupserver-sd JobId 10: Job BackupClient.2014-08-09_13.57.49_07 is waiting. Cannot find any appendable volumes.
Please use the "label" command to create a new Volume for:
Storage: "FileStorage" (/backup/)
Media type: File
Next, we can again rerun the backup job, but this time let’s first enable the autodisplay, which will automatically display all new messages without us requiring to run messages command manually. Below we can see the output of a successful backup where a full backup was done since one wasn’t found before. The backup job was done on a BackupClient with an ID 1 and a new volume “Clients-0001” was created: this means that a file with the same name is also created in the /backup/ directory, which will hold the contents of that volume. There is also some statistical data presenting the fileset, pool, catalog, storage, priority and schedule of the backup. The important line is “Termination: Backup OK”, which states that the backup job was successfully run.
09-Aug 18:04 baculaserver-dir JobId 1: No prior Full backup Job record found.
09-Aug 18:04 baculaserver-dir JobId 1: No prior or suitable Full backup found in catalog. Doing FULL backup.
09-Aug 18:04 baculaserver-dir JobId 1: Start Backup JobId 1, Job=BackupClient .2014-08-09_18.04.12_04
09-Aug 18:04 baculaserver-dir JobId 1: Created new Volume "Clients-0001" in catalog.
09-Aug 18:04 baculaserver-dir JobId 1: Using Device "FileStorage"
09-Aug 18:04 baculaserver-sd JobId 1: Labeled new Volume "Clients-0001" on device "FileStorage" (/backup/).
09-Aug 18:04 baculaserver-sd JobId 1: Wrote label to prelabeled Volume "Clients-0001" on device "FileStorage" (/backup/)
09-Aug 18:04 baculaserver-sd JobId 1: Job write elapsed time = 00:00:01, Transfer rate = 133.0 M Bytes/second
09-Aug 18:04 baculaserver-dir JobId 1: Bacula baculaserver-dir 5.2.6 (21Feb12):
Build OS: x86_64-pc-linux-gnu debian 7.0
Backup Level: Full (upgraded from Incremental)
Client: "computer-fd" 5.2.6 (21Feb12) x86_64-pc-linux-gnu,debian,7.0
FileSet: "Full Set" 2014-08-09 18:04:12
Pool: "File" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Job resource)
Scheduled time: 09-Aug-2014 18:04:09
Start time: 09-Aug-2014 18:04:14
End time: 09-Aug-2014 18:04:15
Elapsed time: 1 sec
FD Files Written: 2
SD Files Written: 2
FD Bytes Written: 133,089,280 (133.0 MB)
SD Bytes Written: 133,089,490 (133.0 MB)
Rate: 133089.3 KB/s
Software Compression: None
Volume name(s): Clients-0001
Volume Session Id: 1
Volume Session Time: 1407513760
Last Volume Bytes: 133,188,801 (133.1 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
09-Aug 18:04 baculaserver-dir JobId 1: Begin pruning Jobs older than 6 months .
09-Aug 18:04 baculaserver-dir JobId 1: No Jobs found to prune.
09-Aug 18:04 baculaserver-dir JobId 1: Begin pruning Files.
09-Aug 18:04 baculaserver-dir JobId 1: No Files found to prune.
09-Aug 18:04 baculaserver-dir JobId 1: End auto prune.
Once the backup has been completed, we can execute the “status storage” command in bconsole to find out what is stored on the storage.
No Jobs running.
Jobs waiting to reserve a drive:
JobId Level Files Bytes Status Finished Name
1 Full 2 133.1 M OK 10-Aug-14 13:03 BackupClient
Restoring the data is equally important as backing up the data: the backup of data is done for the sole reason to be restored one day in case of an emergency. If we don’t have a clear process how to restore the data when needed, then we might as well not have a backup solution in place. Whenever introducing a backup solution to the network, we also have to provide detailed instructions for restoring the data.
To restore the data in Bacula, we can enter bconsole and issue the restore command, which will give us a list of options to choose from as presented below. Most often we’ll use option 5 to restore the most recent backup for a client, but we might restore just a specific file from a backup or select a backup from a specific time, etc.
First you select one or more JobIds that contain files
to be restored. You will be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.
To select the JobIds, you have the following choices:
1: List last 20 Jobs run
2: List Jobs where a given File is saved
3: Enter list of comma separated JobIds to select
4: Enter SQL list command
5: Select the most recent backup for a client
6: Select backup for a client before a specified time
7: Enter a list of files to restore
8: Enter a list of files to restore before a specified time
9: Find the JobIds of the most recent backup for a client
10: Find the JobIds for a backup for a client before a specified time
11: Enter a list of directories to restore for found JobIds
12: Select full restore to a specified Job date
Select item: (1-13):
After we choose option 5 to select the most recent backup for a client, Bacula will ask us to choose a client which we would like to work on. Then it will build a directory tree for the JobIds that were used to backup the data. In the file selection mode, we have to select the files we would like to restore by using the add/remove commands. The commands below present a way to restore the files from the Bacula volume. Upon entering the file selection mode, we executed the pwd command to print the name of the current working directory. Then we issued the ls command to print the files in the current directory, upon which we changed the working directory to /backup. After that we selected the file mysqldump-31_07_2014.sql for restoration by using the add command and finished the restoration.
cwd is: /
$ cd backup
$ add mysqldump-31_07_2014.sql
1 file marked.
The done command will print the information about the restore job: the name of the fileset, the backup/restore client, the priority and other options will be given. Also the location of the restored files will be saved to a directory specified with the Where option.
Bootstrap records written to /var/lib/bacula/backup-dir.restore.1.bsr
The job will require the following
Volume(s) Storage(s) SD Device(s)
Clients-0001 File FileStorage
Volumes marked with "*" are online.
1 file selected to be restored.
Run Restore job
FileSet: “Full Set”
Backup Client: computer-fd
Restore Client: computer-fd
When: 2014-08-10 13:29:50
Plugin Options: *None*
OK to run? (yes/mod/no): yes
The Where option directly corresponds with the Job definition in bacula-dir.conf as presented below.
Name = "RestoreFiles"
Type = Restore
Storage = File
Pool = Default
Messages = Standard
Where = /restore
The successful restoration will result in selected files being written into the /restore/ directory, which we can further use to restore the backup. In the example above, we’ve seen how a .sql database was restored, which still needs to be applied to the database by using the appropriate command. Often we’re restoring important files, like files uploaded to an internal document system, a web application platform, etc. In such cases, the files are already there and don’t need to be imported into an application to make them accessible from it, but merely moved to the right directory as configured.
Often there are many applications running in internal network which require some kind of database, including Bacula itself. Bacula keeps catalog metadata in the Sqlite database by default, but it’s often the case that Mysql/Postgresql/Oracle database is used. With Sqlite databases, we only have to backup a single file and thus no further actions are needed, but it’s usually the case that databases are contained on a remote central location to which applications connect. This makes them easy to maintain and simple to backup. In this tutorial we’ll concentrate on the Mysql database and configure Bacula to automatically backup all databases contained on a separate database server. Usually, a crontab entry is created to backup the database via mysqldump tool at specified intervals, like once a week.
I’ve been using the following script, which I’ve placed under /etc/cron.weekly/ to dump the webapp database to a file on a weekly basis.
/usr/bin/mysqldump -u root -ppassword webapp 2>/var/log/mysqldump.log > /backup/webapp-$(date +"%d_%m_%Y").sql
Despite the crontab being a good solution, it still requires a log-in to the remote machine, creation of the backup script, and configuring the crontab in order to dump the database to a file. Above we only dumped one database to a file, but we can copy-paste the mysqldump command to dump as many databases as we want. To let Bacula run the backup, we have to add the ClientRunBeforeJob into the Job – in bacula-dir.conf configuration file. An example of this Job can be seen below, where Bacula will run /root/mysqlbackup script prior to backing up the data.
Name = "MysqlClient"
JobDefs = "DefaultJob"
Client = mysqlclient-fd
By letting Bacula run the backup script, we’re effectively not relying on crontab to run the jobs for us. We’re also running the backup job all at the same time: we don’t have to think about running the crontab job before Bacula or calculating anything. Let’s imagine that we setup crontab to run every day at a specific time and Bacula is configured to run every week; what would happen to 7 versions of backup files (one for each day in the week)? It depends on the backup script – if the backup script overwrites the previous version of the backup file, we would only backup one of the files in a whole week. This can happen when changing the Bacula scheduling mechanism and forgetting about crontabs in Linux distributions. The best way to avoid the confusion is to let Bacula run the script by itself prior to backing up the data. By using such methodology, we’re eliminating the risk of mistakes and keeping Bacula in control of every aspect of the backup process, which indicates good design.
It’s also imperative to understand that ClientRunBeforeJob runs the script and waits for it to finish before backing up the data. Therefore, we must differentiate between a synchronous and asynchronous script. When an asynchronous script is run, the script will be executed in the background and control will be immediately returned to us. Contrary to the asynchronous script, a synchronous script runs in the foreground, thus returning control to the user once the script has finished processing. This is the script we must use, since ClientRunBeforeJob will wait for the script to give control back to Bacula before proceeding. If asynchronous script is used, the control will be given back to Bacula immediately and Bacula will backup an old copy of the backed up files, which is undesirable. To circumvent such behavior, we should use a synchronous script, which waits for the script to finish before giving control back to Bacula.
 Solid-state drive, https://en.wikipedia.org/wiki/Solid-state_drive.
 Tape drive https://en.wikipedia.org/wiki/Tape_drive.
 List of backup software https://en.wikipedia.org/wiki/List_of_backup_software.
 Bacula-Web, http://www.bacula-web.org/.
 The Bootstrap File, http://www.bacula.org/5.2.x-manuals/en/main/main/Bootstrap_File.html.
 ESXi 5.1: Using Raw Device Mappings (RDM) on an HP Microserver,
 Bacula Installation and Conﬁguration Guide, https://access.redhat.com/site/sites/default/files/attachments/install_1.pdf.
 Overview on modifying the Synology Server, bootstrap, ipkg etc, .
 Data Encryption, http://www.bacula.org/5.2.x-manuals/en/main/main/Data_Encryption.html.
 Messages Resource, http://www.bacula.org/5.2.x-manuals/en/main/main/Messages_Resource.html.