Introduces a volume driver which makes use of SMB shares for
hosting volumes as disk images, having a similar workflow with the
other NFS like drivers.
The driver uses Samba in order to access the SMB shares. It takes
SMB shares information such as share path, credentials and other
mount flags from a config file of which location can be configured
in the cinder config file.
It includes all the features required by the Juno release.
Driver cert results:
http://paste.openstack.org/show/100915/
The online snapshot related tests have been skipped as this
feature is not supported yet.
Implements: blueprint smbfs-volume-driver
DocImpact
Co-Authored-By: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
Change-Id: I3b4bebf73122d5b784228af18f9b1eddfc02b643
Besides the aesthetical aspect of replacing ad-hoc code
with an utility function, it's more correct
to perform image conversions in CONF.image_conversion_dir,
which is taken care by the image_utils.temporary_file()
context guard.
Change-Id: Ied5607156f0ab1f1f8cfc889d5b57f3486d6b407
Move RemoteFSDriver into its own file.
Refactor pieces needed for qcow2 snapshots into a class.
Also add self.SHARE_FORMAT_REGEX.
Implements bp: remotefs-snaps
Change-Id: I8f9eaed125f4b916b835a63b04239058e34492a9
To ensure that logs are properly translated and logged to
the Cinder log files (using the Cinder message catalogs)
we need to explicitly import _() in any python files that
use the _() function.
Closes-bug: 1306275
Related-Blueprint: i18n-enablement
Change-Id: I3cf1f1d63d5f8bd8ffd007104691f914e3fd8538
If a glusterfs volume is in-use, nova is called to delete a
volume snapshot. It is possible for a timeout to occur, since
cinder is polling for nova task progress. In such case, a
GlusterfsException is thrown, but nova continues to process the
snapshot delete and commits the changes. Cinder is not aware of
this, and the actual snapshot entry in cinder is prevented from
being deleted, since no backing file exists for the out-of-sync
snapshot. This patch allows a snapshot with no backing file to
be deleted from cinder and updates the volume info file.
Change-Id: I83c8a7242199edbfd1897297589ac7deb42c5865
Closes-Bug: #1311182
There is Cinder's version units module, that looks good. Considering
consistency with other projects like Nova, Glance, Cinder would be
better to use oslo-incubator's version.
Change-Id: I07e93e9d8a985df0f96c3e80de9c3f23bf6a0c1e
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
This patch adds a new hacking check - N319 - that ensures all
debug log messages don't have translations.
Change-Id: Id9c2715f25c8f2ea52235aba4bd1583655391584
Implements: blueprint debug-translation-removal
Closes-Bug: #1318713
When restarting cinder service (cinder-volume specifically),
ensure that the gluster mounts are unmounted and remounted so
that any new mount options added to the shares config file
is taken effect, post service restart.
Change-Id: I9fe3442cd4899770b4c9ea4a73a7fe62967f2c64
Gluster's driver uses warning to log mount exceptions. This patch
replaces that warning with an error message.
Change-Id: I54a10ec8abf460ca9ff7bb009b8d702cfad0fed0
Closes-Bug: #1317573
If a snapshot is taken of a volume that is attached to an active
instance, the volume file used by the instance will be switched to
the new snapshot file that is created. When you delete the
snapshot, the base volume file will be merged with the snapshot
file and the base volume is deleted. Upon a deleting the active
volume, the active snapshot file is not deleted because it does not
have the expected name that cinder is looking for, i.e.
volume-<uuid>. Instead, the snapshot file has the name
volume-<uuid>.<snapshot-uuid>. This patch looks at the volume info
file to find any active snapshot file and properly delete it when
the volume is deleted.
Change-Id: Ib0af4401d839ec3bd1eb3a81e1671811e0d4a288
Closes-Bug: #1300303
glusterfs creates a file named volume-<uuid>.info to hold volume
data, mapping cinder snapshot UUIDs and filenames. When the
volume and its associated snapshots are both deleted, this file
remains. This patch deletes the volume-<uuid>.info (if one exists)
when the volume is deleted.
Change-Id: Iefc8bd4768b30a75c648c4689d935daeab3afe98
Closes-Bug: #1299182
This commit removes the use of str() from LOG.* messages
and exceptions.
The reason we need to remove str() is because str() will fail if
it gets a Unicode string that has something that doesn't translate
to ASCII in it. If such a situation is encountered you will lose
the message string in question. In most cases, the use of str() is
unnecessary for LOG.* and exception messages. Using %s is smart
enough to figure out what to do with what it is passed. It first
tries to str() it, if this fails it falls back to using unicode. Either
way, the result will then be something that gettextutils can
handle and translate.
Change-Id: I6eb81043edd9fa5e035d81ee81e8439340546d24
Closes-bug: 1274245
Related-bp: i18n-messages
The GlusterFS driver sets specific permissions on the
volume file when volumes are created. Setting them on the
snapshot files at snapshot creation time is also required,
otherwise volume attaches will fail for volumes with snapshots
when using QEMU's libgfapi functionality.
This does not weaken the current security model in any
significant way, as it only lines up the permissions used
for snapshot files with those already used for volume files.
This will eventually be addressed by functionality in libvirt:
https://bugzilla.redhat.com/show_bug.cgi?id=1052014
Closes-Bug: #1286376
Change-Id: I93944724c557c907eb3821a061e7434f09ff20bc
Increase the timeout for Nova snapshot delete operations from ten
minutes to two hours. This helps prevent Cinder from terminating
operations prematurely that are still being processed by Nova.
It is not uncommon for snapshot delete jobs to run for longer than
ten minutes depending on the size of the snapshot and speed of the
storage backend.
This will be followed up with a more robust mechanism to keep track
of snapshot job progress as a later effort.
Related-Bug: 1273894
Change-Id: I1ad52568aed1ce1bf593e71704e481b6fe5f44fb
Allow create_backup for GlusterFS volumes, but
only when snapshots do not exist. (For now.)
Restore is a no-op for the driver, so allow it.
Related-Bug: 1247743
Closes-Bug: 1275977
Change-Id: I50b8f6cac684c967c7374bb43247a396ce936157
The action cinder backup-create fails when trying to backup a newly
created volume when using cinder.volume.drivers.nfs.NfsDriver. This
patch removes not implemented stubs for backup_volume and
restore_volume. The inherited methods in cinder/volume/driver.py
succeed in creating a backup and in restoring it. After manual
testing of this change backups/restores succeed without the not
implemented stubs. This change also adds the not implemented
stubs to the glusterfs driver until a fix is submitted.
Closes-Bug: #1247743
Change-Id: I5478d00442ce855c91a7f04c5ba2b40733d44902
Fixes an InvalidParameterValue error by passing the required
information to Brick for mounting.
See also: 1cd3626 Brick connector revised fix for NFS
Bump driver version 1.1.1.
Closes-Bug: 1238085
Change-Id: Ie45889460dd4775a556d592161c98c24a56989c7
The create_cloned_volume path could deadlock due to
create_cloned_volume and create/delete_snapshot using the same
lock for synchronization.
Refactor the calls to create/delete snapshot to call the inner
method which does not use a lock.
Introduced by "06999f6 GlusterFS: Synchronize additional op..."
Related-Bug: 1267983
Closes-Bug: 1272092
Change-Id: I84ca34b201c10644faa047f1c9274c14bcdd0359
create_cloned_volume() and copy_volume_to_image() should also be
locked against other driver operations to prevent collisions
from concurrent updates of snapshot metadata or volume file data.
Closes-Bug: #1267983
Change-Id: Idd6158bfbe4dc049c33fa58b40d3d570fc244d01
When deleting the most recent snapshot, the 'file_to_merge' field
which translates into the base= field for libvirt's blockRebase
call in Nova must be set depending on whether other snapshots exist.
If there are no other snapshots, base = None, which results in
libvirt clearing the qcow2 backing file pointer for the active
disk image.
If there are other snapshots, pass the parent of the file being
deleted as the new base file. The snapshot info pointer for the
prior base file must also be updated in this case.
Closes-Bug: #1262880
Change-Id: If7bc8259b031d0406346caafb8f688e65a38dba6
The glusterfs_disk_util option lets one choose whether to
use df or du to calculate space usage. This serves no real
useful purpose. Should remove this option to simplify things.
Since this does not meaningfully impact behavior, removal of
this option shouldn't require any special deprecation efforts.
(This same change was made for the NFS driver in 5bf7b9be.)
DocImpact: Remove config option
Change-Id: I302692454b43de15688d65873a72a3dada0c67cb
The snapshot_delete operation will fail if the snapshot info file
doesn't contain a record for the snapshot, or does not exist.
This happens in cases such as when snapshot_create fails to commit
anything to disk.
The driver should allow the manager to delete the snapshot
in this case, as there is no action required for the driver
to delete anything.
Closes-Bug: #1252864
Change-Id: I8686a1be09dbb7984072538bff6c026bb84eeb52
This is currently done when creating a new volume from scratch,
but not when cloning a new volume.
Closes-Bug: #1254768
Change-Id: I8506dc0824bacdb6cc42b4cadd41c2cad3a74b5b
Ensure the Cinder user can write to the GlusterFS share. This
is required for snapshot functionality, and means the admin
does not have to set this permission manually.
Closes-Bug: #1236966
Change-Id: I4a9ea40df9681ca6931ad6b390aa21b09d6cfec9
Operations that modify qcow2 chains or the GlusterFS driver's
snapshot info file should use locking so that more than one thread
cannot run these operations at the same time.
Without this it is possible for multiple threads to concurrently
modify snapshot information resulting in an incomplete snapshot
chain.
Closes-Bug: #1251425
Change-Id: I93c6ffecd9aec38560cb19085243390b9120b363
When cloning a volume, the volume[name] field was populated
with incorrect data, resulting in an unexpected filename containing
the volume data. This results in failures of later operations on
that cloned volume.
Change-Id: I067ed44cebdc8e91e9ded326953fd0c99d003f05
Closes-Bug: #1244238
Move the remotefs CONF options back into their
corresponding volume drivers.
Partial-Bug: #1230066
Change-Id: Ie37a803dc0f895ffd9dc2c7daf8255e6096ccee2
The GlusterFS driver's copy_volume_from_snapshot was previously
not parsing the snapshot information to determine which backing
file to copy from, instead copying from the active file at all
times. This would result in the wrong data being supplied when
a volume is cloned from a snapshot.
Add a test for the copy_volume_from_snapshot method.
Closes-Bug: #1222907
Change-Id: Ib829ca1a6812b61845f3b2eb9c5507779fa6ec15
Code added for GlusterFS snapshot support in Havana included
new code to parse qemu-img output. This removes this code
and uses image_utils qemu_img_info instead.
Add test to image_utils for parsing a different style of
qemu-img info output.
Improve GlusterFS driver test coverage with tests for cloning,
create from snapshot, and initialize_connection.
Closes-Bug: #1224030
Change-Id: I8f1811b400c06edb3cd7416c52aa297921841e54
Initial code for GlusterFS snapshot support in Havana
used qemu-img arguments which are too new to run in many
environments. Run qemu-img using only arguments which
will exist on older platforms.
Partial-Bug: #1224030
Change-Id: I5155dac492da67a951ede978c2c46a54c239eb04
Coordinate with Nova to create and delete snaphots for
GlusterFS volumes that are attached to VMs.
Cinder is responsible for creating a QCOW2 file which Nova
will activate in the VM's snapshot chain when a snapshot is
created.
When a snapshot is deleted, Cinder will request for Nova to
perform a block commit/rebase operation to logically delete
the snapshot from the QCOW2 chain.
Implements blueprint qemu-assisted-snapshots
Change-Id: I4a7f0c1bc08d88b0f75d119168dd2077487a62a0
Refactor NFS driver to use the brick library to perform mounts
rather than duplicating the mount code. Some of the NFS unit
test code was rewritten to test the brick implementation because
the old mount code was moved.
The Gluster driver was slightly affected in the refactoring, but
does not yet use brick to perform attaches.
bug 1214748
Change-Id: Ib29f83fabe407f9591191e9fd19e6306ca9960cf
Offline snapshots for the GlusterFS driver, based
on QCOW2 snapshotting.
This introduces the option to have volumes stored as
qcow2 files rather than raw files. (glusterfs_qcow2_volumes)
Bump driver version to 1.1.0.
This patch requires Nova to support loading qcow2 disks:
https://review.openstack.org/#/c/38321/
Online snapshot support (via Nova) is added in a
subsequent patch submitted in this series.
This patch also enables volume cloning for GlusterFS volumes.
Implements blueprint qemu-assisted-snapshots
Change-Id: I783a802574b835b495e343f555f7ecf9b46981dd
This patch updates each driver to provide
the same mechanism for reporting the version
of the driver. It also includes fixing the
reported driver version at get_volume_stats()
time to be the version set in the driver.
When the manager starts the driver it now
logs the driver's name and version in the log file.
fixes Bug #1212878
Change-Id: Icefd065f885ad417735490e6ac90e2b21557d2dc
Using of locals() for formatting string is a nasty thing because:
1) It is not so clear as using explicit dicts
2) It could produce hidden errors during refactoring
3) Changing name of variable causes change in message
4) Creating a lot of unused variables
Fix bug 1171936
Change-Id: I806c530851527db9da251352be45b97c183241a8
Per (proposed) H203 check, convert all uses of
the deprecated except x,y: construct to except
x as y:, which works with any Python version >= 2.6
Change-Id: I5528dc556f3ef8d356e01d59df04ba57f66c95b7
The config documentation relies on options being registered
on a modules import. Our need to move the drivers to using
self.configuration for multi-backend support means that options
wouldn't be loaded until object initialization which breaks
documentation.
This patch puts a dummy CONF init/load back in the drivers. While putting
this change together I came across a number of drivers still using FLAGS,
and even worse a number of drivers using a mixture of FLAGS and CONF and
self.configuraiton. So most of those are cleaned up here as well.
Note there are two drivers that were not updated at all here:
1. windows.py
2. zadara.py
The zadara folks have indicated that they're in the process of updating and
releasing a new version of their driver so I left that as is.
The windows driver needs a bit of work to switch over.
Fixes bug: 1179159
Change-Id: I90165299bf080da17741d027e36e361540da0ff8
Eric Harney