5fca287eb5
There are several instances of "Block Storage Service" in the upstream documentation. As was recently clarified in the docs mailing list, these should be "Block Storage service". This patch applies the fix. Change-Id: I591c100ab031b75f1c0d27d45f53e32401a26e30 Partial-Bug: #1217503
275 lines
14 KiB
XML
275 lines
14 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE section [
|
|
<!-- Some useful entities borrowed from HTML -->
|
|
<!ENTITY nbsp " ">
|
|
]>
|
|
<section xml:id="backup-block-storage-disks"
|
|
xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0">
|
|
<title>Back up Block Storage service disks</title>
|
|
<para>While you can use the LVM snapshot to create snapshots, you
|
|
can also use it to back up your volumes. By using LVM
|
|
snapshot, you reduce the size of the backup; only existing
|
|
data is backed up instead of the entire volume.</para>
|
|
<para>To back up a volume, you must create a snapshot of it. An
|
|
LVM snapshot is the exact copy of a logical volume, which
|
|
contains data in a frozen state. This prevents data
|
|
corruption, because data cannot be manipulated during the
|
|
volume creation process. Remember that the volumes created
|
|
through a <command>nova volume-create</command> command exist
|
|
in an LVM logical volume.</para>
|
|
<para>You must also make sure that the operating system is not using the
|
|
volume, and that all data has been flushed on the guest filesystems.
|
|
This usually means that those filesystems have to be unmounted during
|
|
the snapshot creation. They can be mounted again as soon as the logical
|
|
volume snapshot has been created.</para>
|
|
<para>Before you create the snapshot, you must have enough space
|
|
to save it. As a precaution, you should have at least twice as
|
|
much space as the potential snapshot size. If insufficient
|
|
space is available, the snapshot might become
|
|
corrupted.</para>
|
|
<para>For this example, assume that a 100 GB volume named
|
|
<literal>volume-00000001</literal> was created for an
|
|
instance while only 4 GB are used. This example uses these
|
|
commands to back up only those 4 GB:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><command>lvm2</command> command. Directly
|
|
manipulates the volumes.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><command>kpartx</command> command. Discovers the
|
|
partition table created inside the instance.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><command>tar</command> command. Creates a
|
|
minimum-sized backup.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><command>sha1sum</command> command. Calculates the
|
|
backup checksum to check its consistency.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>You can apply this process to volumes of any size.</para>
|
|
<procedure>
|
|
<title>To back up Block Storage service disks</title>
|
|
<step>
|
|
<title>Create a snapshot of a used volume</title>
|
|
<substeps>
|
|
<step>
|
|
<para>Use this command to list all volumes:</para>
|
|
<screen><prompt>#</prompt> <userinput>lvdisplay</userinput></screen>
|
|
</step>
|
|
<step>
|
|
<para>Create the snapshot; you can do this while
|
|
the volume is attached to an instance:</para>
|
|
<screen><prompt>#</prompt> <userinput>lvcreate --size 10G --snapshot --name volume-00000001-snapshot /dev/cinder-volumes/volume-00000001</userinput></screen>
|
|
<para>Use the <option>--snapshot</option>
|
|
configuration option to tell LVM that you want
|
|
a snapshot of an already existing volume. The
|
|
command includes the size of the space
|
|
reserved for the snapshot volume, the name of
|
|
the snapshot, and the path of an already
|
|
existing volume. Generally, this path is
|
|
<filename>/dev/cinder-volumes/<replaceable>$volume_name</replaceable></filename>.</para>
|
|
<para>The size does not have to be the same as the
|
|
volume of the snapshot. The
|
|
<parameter>size</parameter> parameter
|
|
defines the space that LVM reserves for the
|
|
snapshot volume. As a precaution, the size
|
|
should be the same as that of the original
|
|
volume, even if the whole space is not
|
|
currently used by the snapshot.</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Run the <command>lvdisplay</command> command
|
|
again to verify the snapshot:</para>
|
|
<programlisting>--- Logical volume ---
|
|
LV Name /dev/cinder-volumes/volume-00000001
|
|
VG Name cinder-volumes
|
|
LV UUID gI8hta-p21U-IW2q-hRN1-nTzN-UC2G-dKbdKr
|
|
LV Write Access read/write
|
|
LV snapshot status source of
|
|
/dev/cinder-volumes/volume-00000026-snap [active]
|
|
LV Status available
|
|
# open 1
|
|
LV Size 15,00 GiB
|
|
Current LE 3840
|
|
Segments 1
|
|
Allocation inherit
|
|
Read ahead sectors auto
|
|
- currently set to 256
|
|
Block device 251:13
|
|
|
|
--- Logical volume ---
|
|
LV Name /dev/cinder-volumes/volume-00000001-snap
|
|
VG Name cinder-volumes
|
|
LV UUID HlW3Ep-g5I8-KGQb-IRvi-IRYU-lIKe-wE9zYr
|
|
LV Write Access read/write
|
|
LV snapshot status active destination for /dev/cinder-volumes/volume-00000026
|
|
LV Status available
|
|
# open 0
|
|
LV Size 15,00 GiB
|
|
Current LE 3840
|
|
COW-table size 10,00 GiB
|
|
COW-table LE 2560
|
|
Allocated to snapshot 0,00%
|
|
Snapshot chunk size 4,00 KiB
|
|
Segments 1
|
|
Allocation inherit
|
|
Read ahead sectors auto
|
|
- currently set to 256
|
|
Block device 251:14</programlisting>
|
|
</step>
|
|
</substeps>
|
|
</step>
|
|
<step>
|
|
<title>Partition table discovery</title>
|
|
<substeps>
|
|
<step>
|
|
<para>To exploit the snapshot with the
|
|
<command>tar</command> command, mount your
|
|
partition on the Block Storage service
|
|
server.</para>
|
|
<para>The <command>kpartx</command> utility
|
|
discovers and maps table partitions. You
|
|
can use it to view partitions that are created
|
|
inside the instance. Without using the
|
|
partitions created inside instances, you
|
|
cannot see its content and create efficient
|
|
backups.</para>
|
|
<screen><prompt>#</prompt> <userinput>kpartx -av /dev/cinder-volumes/volume-00000001-snapshot</userinput></screen>
|
|
<note os="debian">
|
|
<para>On a Debian-based distribution, you can
|
|
use the <command>apt-get install kpartx</command>
|
|
command to install
|
|
<command>kpartx</command>.</para>
|
|
</note>
|
|
<para>If the tools successfully find and map the
|
|
partition table, no errors are
|
|
returned.</para>
|
|
</step>
|
|
<step>
|
|
<para>To check the partition table map, run this
|
|
command:</para>
|
|
<screen><prompt>$</prompt> <userinput>ls /dev/mapper/nova*</userinput></screen>
|
|
<para>You can see the
|
|
<literal>cinder--volumes-volume--00000001--snapshot1</literal>
|
|
partition.</para>
|
|
<para>If you created more than one partition on
|
|
that volume, you see several partitions; for
|
|
example:
|
|
<literal>cinder--volumes-volume--00000001--snapshot2</literal>,
|
|
<literal>cinder--volumes-volume--00000001--snapshot3</literal>,
|
|
and so on.</para>
|
|
</step>
|
|
<step>
|
|
<para>Mount your partition:</para>
|
|
<screen><prompt>#</prompt> <userinput>mount /dev/mapper/cinder--volumes-volume--volume--00000001--snapshot1 /mnt</userinput></screen>
|
|
<para>If the partition mounts successfully, no
|
|
errors are returned.</para>
|
|
<para>You can directly access the data inside the
|
|
instance. If a message prompts you for a
|
|
partition or you cannot mount it, determine
|
|
whether enough space was allocated for the
|
|
snapshot or the <command>kpartx</command>
|
|
command failed to discover the partition
|
|
table.</para>
|
|
<para>Allocate more space to the snapshot and try
|
|
the process again.</para>
|
|
</step>
|
|
</substeps>
|
|
</step>
|
|
<step>
|
|
<title>Use the <command>tar</command> command to create
|
|
archives</title>
|
|
<para>Create a backup of the volume:</para>
|
|
<screen><prompt>$</prompt> <userinput>tar --exclude="lost+found" --exclude="some/data/to/exclude" -czf volume-00000001.tar.gz -C /mnt/ /backup/destination</userinput></screen>
|
|
<para>This command creates a <filename>tar.gz</filename>
|
|
file that contains the data, <emphasis role="italic"
|
|
>and data only</emphasis>. This ensures that you
|
|
do not waste space by backing up empty sectors.</para>
|
|
</step>
|
|
<step>
|
|
<title>Checksum calculation I</title>
|
|
<para>You should always have the checksum for your backup
|
|
files. When you transfer the same file over the
|
|
network, you can run a checksum calculation to ensure
|
|
that your file was not corrupted during its transfer.
|
|
The checksum is a unique ID for a file. If the
|
|
checksums are different, the file is corrupted.</para>
|
|
<para>Run this command to run a checksum for your file and
|
|
save the result to a file:</para>
|
|
<screen><prompt>$</prompt> <userinput>sha1sum volume-00000001.tar.gz > volume-00000001.checksum</userinput></screen>
|
|
<note>
|
|
<para>Use the <command>sha1sum</command> command
|
|
carefully because the time it takes to complete
|
|
the calculation is directly proportional to the
|
|
size of the file.</para>
|
|
<para>For files larger than around 4 to 6 GB, and
|
|
depending on your CPU, the process might take a
|
|
long time.</para>
|
|
</note>
|
|
</step>
|
|
<step>
|
|
<title>After work cleaning</title>
|
|
<para>Now that you have an efficient and consistent
|
|
backup, use this command to clean up the file
|
|
system:</para>
|
|
<substeps>
|
|
<step>
|
|
<para>Unmount the volume:</para>
|
|
<screen><userinput>unmount /mnt</userinput></screen>
|
|
</step>
|
|
<step>
|
|
<para>Delete the partition table:</para>
|
|
<screen><userinput>kpartx -dv /dev/cinder-volumes/volume-00000001-snapshot</userinput></screen>
|
|
</step>
|
|
<step>
|
|
<para>Remove the snapshot:</para>
|
|
<screen><userinput>lvremove -f /dev/cinder-volumes/volume-00000001-snapshot</userinput></screen>
|
|
</step>
|
|
</substeps>
|
|
<para>Repeat these steps for all your volumes.</para>
|
|
</step>
|
|
<step>
|
|
<title>Automate your backups</title>
|
|
<para>Because more and more volumes might be allocated to
|
|
your Block Storage service, you might want to automate
|
|
your backups. The <link
|
|
xlink:href="https://github.com/Razique/BashStuff/blob/master/SYSTEMS/OpenStack/SCR_5005_V01_NUAC-OPENSTACK-EBS-volumes-backup.sh"
|
|
>SCR_5005_V01_NUAC-OPENSTACK-EBS-volumes-backup.sh</link>
|
|
script assists you with this task. The script performs
|
|
the operations from the previous example, but also
|
|
provides a mail report and runs the backup based on
|
|
the <option>backups_retention_days</option>
|
|
setting.</para>
|
|
<para>Launch this script from the server that runs the
|
|
Block Storage service.</para>
|
|
<para>This example shows a mail report:</para>
|
|
<programlisting>Backup Start Time - 07/10 at 01:00:01
|
|
Current retention - 7 days
|
|
|
|
The backup volume is mounted. Proceed...
|
|
Removing old backups... : /BACKUPS/EBS-VOL/volume-00000019/volume-00000019_28_09_2011.tar.gz
|
|
/BACKUPS/EBS-VOL/volume-00000019 - 0 h 1 m and 21 seconds. Size - 3,5G
|
|
|
|
The backup volume is mounted. Proceed...
|
|
Removing old backups... : /BACKUPS/EBS-VOL/volume-0000001a/volume-0000001a_28_09_2011.tar.gz
|
|
/BACKUPS/EBS-VOL/volume-0000001a - 0 h 4 m and 15 seconds. Size - 6,9G
|
|
---------------------------------------
|
|
Total backups size - 267G - Used space : 35%
|
|
Total execution time - 1 h 75 m and 35 seconds</programlisting>
|
|
<para>The script also enables you to SSH to your instances
|
|
and run a <command>mysqldump</command> command into
|
|
them. To make this work, enable the connection to the
|
|
Compute project keys. If you do not want to run the
|
|
<command>mysqldump</command> command, you can add
|
|
<literal>enable_mysql_dump=0</literal> to the
|
|
script to turn off this functionality.</para>
|
|
</step>
|
|
</procedure>
|
|
</section>
|