51733e5623
Add facility to borg-backup role to run a command and save the output of it to a separate archive file during the backup process. This is mostly useful for database backups. Compressed on-disk logs are terrible for differential backups because revisions have essentially no common data. By saving the uncompressed stream directly from mysqldump, we allow borg the chance to de-duplicate, saving considerable space on the backup servers. This is implemented for our ansible-managed servers currently doing dumps. We also add it to the testinfra. This also separates the archive names for the filesystem and stream backup with unique prefixes so they can be pruned separately. Otherwise we end up keeping only one of the stream or filesystem backups which isn't the intention. However, due to issues with --append-only mode we are not issuing prune commands at this time. Note the updated dump commands are updated slightly, particularly with "--skip-extended-insert" which was suggested by mordred and significantly improves incremental diff-ability by being slightly more verbose but keeping much more of the output stable across dumps. Change-Id: I500062c1c52c74a567621df9aaa716de804ffae7
1.5 KiB
Configure a host to be backed up
This role setups a host to use borgp for backup to any
hosts in the borg-backup-server group.
A separate ssh key will be generated for root to connect to the backup server(s) and the host key for the backup servers will be accepted to the host.
The borg tool is installed and a cron job is setup to
run the backup periodically.
Note the borg-backup-server role must run after this to
create the user correctly on the backup server. This role sets a tuple
borg_user with the username and public key; the
borg-backup-server role uses this variable for each host in
the borg-backup group to initalise users.
Hosts can place into /etc/borg-streams which should be a
script that outputs to stdout data to be fed into a backup archive on
each run. This will be saved to an archive with the name of the file.
This is useful for raw database dumps which allow borg to
deduplicate as much as possible.
Role Variables
The username to connect to the backup server. If this is left undefined, it will be automatically set to
borg-$(hostname)