276 lines
14 KiB
XML
276 lines
14 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE section [
|
|
<!ENTITY % openstack SYSTEM "../../common/entities/openstack.ent">
|
|
%openstack;
|
|
]>
|
|
<section xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
version="5.0"
|
|
xml:id="backup-block-storage-disks">
|
|
<title>Back up Block Storage service disks</title>
|
|
<para>While you can use the LVM snapshot to create snapshots, you
|
|
can also use it to back up your volumes. By using LVM
|
|
snapshot, you reduce the size of the backup; only existing
|
|
data is backed up instead of the entire volume.</para>
|
|
<para>To back up a volume, you must create a snapshot of it. An
|
|
LVM snapshot is the exact copy of a logical volume, which
|
|
contains data in a frozen state. This prevents data
|
|
corruption, because data cannot be manipulated during the
|
|
volume creation process. Remember that the volumes created
|
|
through a <command>nova volume-create</command> command exist
|
|
in an LVM logical volume.</para>
|
|
<para>You must also make sure that the operating system is not
|
|
using the volume, and that all data has been flushed on the
|
|
guest file systems. This usually means that those file systems
|
|
have to be unmounted during the snapshot creation. They can be
|
|
mounted again as soon as the logical volume snapshot has been
|
|
created.</para>
|
|
<para>Before you create the snapshot, you must have enough space
|
|
to save it. As a precaution, you should have at least twice as
|
|
much space as the potential snapshot size. If insufficient
|
|
space is available, the snapshot might become
|
|
corrupted.</para>
|
|
<para>For this example, assume that a 100 GB volume named
|
|
<literal>volume-00000001</literal> was created for an
|
|
instance while only 4 GB are used. This example uses these
|
|
commands to back up only those 4 GB:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><command>lvm2</command> command. Directly
|
|
manipulates the volumes.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><command>kpartx</command> command. Discovers the
|
|
partition table created inside the instance.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><command>tar</command> command. Creates a
|
|
minimum-sized backup.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><command>sha1sum</command> command. Calculates the
|
|
backup checksum to check its consistency.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>You can apply this process to volumes of any size.</para>
|
|
<procedure>
|
|
<title>To back up Block Storage service disks</title>
|
|
<step>
|
|
<title>Create a snapshot of a used volume</title>
|
|
<substeps>
|
|
<step>
|
|
<para>Use this command to list all volumes:</para>
|
|
<screen><prompt>#</prompt> <userinput>lvdisplay</userinput></screen>
|
|
</step>
|
|
<step>
|
|
<para>Create the snapshot; you can do this while
|
|
the volume is attached to an instance:</para>
|
|
<screen><prompt>#</prompt> <userinput>lvcreate --size 10G --snapshot --name volume-00000001-snapshot /dev/cinder-volumes/volume-00000001</userinput></screen>
|
|
<para>Use the <parameter>--snapshot</parameter>
|
|
configuration option to tell LVM that you want
|
|
a snapshot of an already existing volume. The
|
|
command includes the size of the space
|
|
reserved for the snapshot volume, the name of
|
|
the snapshot, and the path of an already
|
|
existing volume. Generally, this path is
|
|
<filename>/dev/cinder-volumes/<replaceable>VOLUME_NAME</replaceable></filename>.</para>
|
|
<para>The size does not have to be the same as the
|
|
volume of the snapshot. The
|
|
<parameter>--size</parameter> parameter
|
|
defines the space that LVM reserves for the
|
|
snapshot volume. As a precaution, the size
|
|
should be the same as that of the original
|
|
volume, even if the whole space is not
|
|
currently used by the snapshot.</para>
|
|
</step>
|
|
<step>
|
|
<para>Run the <command>lvdisplay</command> command
|
|
again to verify the snapshot:</para>
|
|
<programlisting>--- Logical volume ---
|
|
LV Name /dev/cinder-volumes/volume-00000001
|
|
VG Name cinder-volumes
|
|
LV UUID gI8hta-p21U-IW2q-hRN1-nTzN-UC2G-dKbdKr
|
|
LV Write Access read/write
|
|
LV snapshot status source of
|
|
/dev/cinder-volumes/volume-00000026-snap [active]
|
|
LV Status available
|
|
# open 1
|
|
LV Size 15,00 GiB
|
|
Current LE 3840
|
|
Segments 1
|
|
Allocation inherit
|
|
Read ahead sectors auto
|
|
- currently set to 256
|
|
Block device 251:13
|
|
|
|
--- Logical volume ---
|
|
LV Name /dev/cinder-volumes/volume-00000001-snap
|
|
VG Name cinder-volumes
|
|
LV UUID HlW3Ep-g5I8-KGQb-IRvi-IRYU-lIKe-wE9zYr
|
|
LV Write Access read/write
|
|
LV snapshot status active destination for /dev/cinder-volumes/volume-00000026
|
|
LV Status available
|
|
# open 0
|
|
LV Size 15,00 GiB
|
|
Current LE 3840
|
|
COW-table size 10,00 GiB
|
|
COW-table LE 2560
|
|
Allocated to snapshot 0,00%
|
|
Snapshot chunk size 4,00 KiB
|
|
Segments 1
|
|
Allocation inherit
|
|
Read ahead sectors auto
|
|
- currently set to 256
|
|
Block device 251:14</programlisting>
|
|
</step>
|
|
</substeps>
|
|
</step>
|
|
<step>
|
|
<title>Partition table discovery</title>
|
|
<substeps>
|
|
<step>
|
|
<para>To exploit the snapshot with the
|
|
<command>tar</command> command, mount your
|
|
partition on the Block Storage service
|
|
server.</para>
|
|
<para>The <command>kpartx</command> utility
|
|
discovers and maps table partitions. You
|
|
can use it to view partitions that are created
|
|
inside the instance. Without using the
|
|
partitions created inside instances, you
|
|
cannot see its content and create efficient
|
|
backups.</para>
|
|
<screen><prompt>#</prompt> <userinput>kpartx -av /dev/cinder-volumes/volume-00000001-snapshot</userinput></screen>
|
|
<note os="debian">
|
|
<para>On a Debian-based distribution, you can
|
|
use the <command>apt-get install kpartx</command>
|
|
command to install
|
|
<command>kpartx</command>.</para>
|
|
</note>
|
|
<para>If the tools successfully find and map the
|
|
partition table, no errors are
|
|
returned.</para>
|
|
</step>
|
|
<step>
|
|
<para>To check the partition table map, run this
|
|
command:</para>
|
|
<screen><prompt>$</prompt> <userinput>ls /dev/mapper/nova*</userinput></screen>
|
|
<para>You can see the
|
|
<literal>cinder--volumes-volume--00000001--snapshot1</literal>
|
|
partition.</para>
|
|
<para>If you created more than one partition on
|
|
that volume, you see several partitions; for
|
|
example:
|
|
<literal>cinder--volumes-volume--00000001--snapshot2</literal>,
|
|
<literal>cinder--volumes-volume--00000001--snapshot3</literal>,
|
|
and so on.</para>
|
|
</step>
|
|
<step>
|
|
<para>Mount your partition:</para>
|
|
<screen><prompt>#</prompt> <userinput>mount /dev/mapper/cinder--volumes-volume--volume--00000001--snapshot1 /mnt</userinput></screen>
|
|
<para>If the partition mounts successfully, no
|
|
errors are returned.</para>
|
|
<para>You can directly access the data inside the
|
|
instance. If a message prompts you for a
|
|
partition or you cannot mount it, determine
|
|
whether enough space was allocated for the
|
|
snapshot or the <command>kpartx</command>
|
|
command failed to discover the partition
|
|
table.</para>
|
|
<para>Allocate more space to the snapshot and try
|
|
the process again.</para>
|
|
</step>
|
|
</substeps>
|
|
</step>
|
|
<step>
|
|
<title>Use the <command>tar</command> command to create
|
|
archives</title>
|
|
<para>Create a backup of the volume:</para>
|
|
<screen><prompt>$</prompt> <userinput>tar --exclude="lost+found" --exclude="some/data/to/exclude" -czf volume-00000001.tar.gz -C /mnt/ /backup/destination</userinput></screen>
|
|
<para>This command creates a <filename>tar.gz</filename>
|
|
file that contains the data, <emphasis role="italic"
|
|
>and data only</emphasis>. This ensures that you
|
|
do not waste space by backing up empty sectors.</para>
|
|
</step>
|
|
<step>
|
|
<title>Checksum calculation I</title>
|
|
<para>You should always have the checksum for your backup
|
|
files. When you transfer the same file over the
|
|
network, you can run a checksum calculation to ensure
|
|
that your file was not corrupted during its transfer.
|
|
The checksum is a unique ID for a file. If the
|
|
checksums are different, the file is corrupted.</para>
|
|
<para>Run this command to run a checksum for your file and
|
|
save the result to a file:</para>
|
|
<screen><prompt>$</prompt> <userinput>sha1sum volume-00000001.tar.gz > volume-00000001.checksum</userinput></screen>
|
|
<note>
|
|
<para>Use the <command>sha1sum</command> command
|
|
carefully because the time it takes to complete
|
|
the calculation is directly proportional to the
|
|
size of the file.</para>
|
|
<para>For files larger than around 4 to 6 GB, and
|
|
depending on your CPU, the process might take a
|
|
long time.</para>
|
|
</note>
|
|
</step>
|
|
<step>
|
|
<title>After work cleaning</title>
|
|
<para>Now that you have an efficient and consistent
|
|
backup, use this command to clean up the file
|
|
system:</para>
|
|
<substeps>
|
|
<step>
|
|
<para>Unmount the volume:</para>
|
|
<screen><userinput>umount /mnt</userinput></screen>
|
|
</step>
|
|
<step>
|
|
<para>Delete the partition table:</para>
|
|
<screen><userinput>kpartx -dv /dev/cinder-volumes/volume-00000001-snapshot</userinput></screen>
|
|
</step>
|
|
<step>
|
|
<para>Remove the snapshot:</para>
|
|
<screen><userinput>lvremove -f /dev/cinder-volumes/volume-00000001-snapshot</userinput></screen>
|
|
</step>
|
|
</substeps>
|
|
<para>Repeat these steps for all your volumes.</para>
|
|
</step>
|
|
<step>
|
|
<title>Automate your backups</title>
|
|
<para>Because more and more volumes might be allocated to
|
|
your Block Storage service, you might want to automate
|
|
your backups. The <link
|
|
xlink:href="https://github.com/Razique/BashStuff/blob/master/SYSTEMS/OpenStack/SCR_5005_V01_NUAC-OPENSTACK-EBS-volumes-backup.sh"
|
|
>SCR_5005_V01_NUAC-OPENSTACK-EBS-volumes-backup.sh</link>
|
|
script assists you with this task. The script performs
|
|
the operations from the previous example, but also
|
|
provides a mail report and runs the backup based on
|
|
the <option>backups_retention_days</option>
|
|
setting.</para>
|
|
<para>Launch this script from the server that runs the
|
|
Block Storage service.</para>
|
|
<para>This example shows a mail report:</para>
|
|
<programlisting>Backup Start Time - 07/10 at 01:00:01
|
|
Current retention - 7 days
|
|
|
|
The backup volume is mounted. Proceed...
|
|
Removing old backups... : /BACKUPS/EBS-VOL/volume-00000019/volume-00000019_28_09_2011.tar.gz
|
|
/BACKUPS/EBS-VOL/volume-00000019 - 0 h 1 m and 21 seconds. Size - 3,5G
|
|
|
|
The backup volume is mounted. Proceed...
|
|
Removing old backups... : /BACKUPS/EBS-VOL/volume-0000001a/volume-0000001a_28_09_2011.tar.gz
|
|
/BACKUPS/EBS-VOL/volume-0000001a - 0 h 4 m and 15 seconds. Size - 6,9G
|
|
---------------------------------------
|
|
Total backups size - 267G - Used space : 35%
|
|
Total execution time - 1 h 75 m and 35 seconds</programlisting>
|
|
<para>The script also enables you to SSH to your instances
|
|
and run a <command>mysqldump</command> command into
|
|
them. To make this work, enable the connection to the
|
|
Compute project keys. If you do not want to run the
|
|
<command>mysqldump</command> command, you can add
|
|
<literal>enable_mysql_dump=0</literal> to the
|
|
script to turn off this functionality.</para>
|
|
</step>
|
|
</procedure>
|
|
</section>
|