Add documentation on how to backup and restore the Undercloud

Change-Id: Ib9f7b7ed133755a6f666e44227ae2e4dda5ee566

doc/source/post_deployment/backup_restore_undercloud.rst

@@ -0,0 +1,74 @@
+Backing up and Restoring your Undercloud
+========================================
+
+Backing up your Undercloud
+--------------------------
+
+In order to backup your undercloud you need to make sure the following items are backed up
+
+* All MariaDB databases on the undercloud node
+* MariaDB configuration file on undercloud (so we can restore databases accurately)
+* All glance image data in /var/lib/glance/images
+* All swift data in /srv/node
+* All data in stack users home directory
+
+The following commands can be used to perform a backup of all data from the undercloud node::
+
+  mysqldump --opt --all-databases > /root/undercloud-all-databases.sql
+  tar -czf undercloud-backup-`date +%F`.tar.gz undercloud-all-databases.sql /etc/my.cnf.d/server.cnf /var/lib/glance/images /srv/node /home/stack /etc/pki /opt/stack
+
+Restoring a backup of your Undercloud on a Fresh Machine
+--------------------------------------------------------
+The following restore process assumes you are recovering from a failed undercloud node where you have to reinstall it from scratch.
+It assumes that the hardware layout is the same, and the hostname and undercloud settings of the machine will be the same as well.
+Once the machine is installed and is in a clean state, re-enable all the subscriptions/repositories needed to install and run TripleO.
+
+Note that unless specified, all commands are run as root.
+
+Then install mariadb server with::
+
+  yum install -y mariadb-server
+
+Now restore the MariaDB configuration file and database backup, then start the MariaDB server and load the backup in::
+
+  tar -xzC / -f undercloud-backup-$DATE.tar.gz etc/my.cnf.d/server.cnf /root/undercloud-all-databases.sql
+  # Edit /etc/my.cnf.d/server.cnf and comment out 'bind-address'
+  systemctl start mariadb
+  cat /root/undercloud-all-databases.sql | mysql
+  # Now we need to clean out some old permissions to be recreated
+  for i in ceilometer glance heat ironic keystone neutron nova tuskar;do mysql -e "drop user $i";done
+  mysql -e 'flush privileges'
+
+Now create the stack user and restore the stack users home directory::
+
+  sudo useradd stack
+  sudo passwd stack  # specify a password
+
+  echo "stack ALL=(root) NOPASSWD:ALL" | sudo tee -a /etc/sudoers.d/stack
+  sudo chmod 0440 /etc/sudoers.d/stack
+
+Next restore the stack users home directory::
+
+  tar -xzC / -f undercloud-backup-$DATE.tar.gz home/stack
+
+We have to now install the swift and glance base packages, and then restore their data::
+
+  yum install -y openstack-glance openstack-swift
+  tar -xzC / -f undercloud-backup-$DATE.tar.gz srv/node var/lib/glance/images
+  # Confirm data is owned by correct user
+  chown -R swift: /srv/node
+  chown -R glance: /var/lib/glance/images
+
+Finally we rerun the undercloud installation from the stack user, making sure to run it in the stack user home dir::
+
+  su - stack
+  sudo yum install -y python-rdomanager-oscplugin
+  # Double check hostname is correctly set in /etc/hosts
+  openstack install undercloud
+
+Reconnect the restored undercloud to the overcloud
+--------------------------------------------------
+Having completed the steps above, the undercloud can be expected to automatically
+restore its connection to the overcloud. The nodes will continue to poll
+Orchestration (heat) for pending tasks, using a simple HTTP request issued every
+few seconds.

doc/source/post_deployment/post_deployment.rst

@@ -13,3 +13,4 @@ In this chapter you will find advanced management of various |project| areas.
    Replacing a Failed Controller Node <replace_controller>
    Import/Export of VM Snapshots <vm_snapshot>
    Updating Packages on Overcloud Nodes <package_update>
+   Backing up and Restoring the Undercloud <backup_restore_undercloud>