While creating an image, if we want to extend the volume
(to accomodate the image), we need to first detach the volume,
perform the extend and attach it again. This is inefficient
for backends that support extending attached volumes since we
are performing 3 cinder operations (the attachment call
includes 3 API calls which sum to a total of 5 API calls for 3
operations) instead of directly extending it which requires only
1 API call to cinder.
The support for extending attached volumes was added in cinder
microversion 3.42.
This patch adds a new boolean config option
``cinder_do_extend_attached`` which allows operators to specify
if the cinder backend they are using supports extending
attached (in-use) volumes. By default this will be ``false``.
Based on the parameter, we will perform the extend operation,
online (if cinder_do_extend_attached is true) and offline otherwise.
Spec: https://review.opendev.org/c/openstack/glance-specs/+/868901
Depends-On: https://review.opendev.org/c/openstack/glance/+/869021
Depends-On: https://review.opendev.org/c/openstack/cinder/+/869051
Change-Id: I5e70824e9abc5277ea25ba95704b358fe3686037
This is an effort to decouple some of the cinder backend specific
code (like nfs, scaleio) from the generic logic.
The purpose is to make the code modular and any change
for a particular cinder backend should not affect the code path
of other backends thereby reducing regression.
This is also required for another use case of supporting extend
of attached volumes added in [1].
Following are the major changes done in this patch:
1) Move cinder store to a new directory 'cinder' and rename
'cinder.py' to 'store.py' (similar to swift)
2) Create new files for nfs and scaleio backends for moving code
specific to these backends into their own separate file.
This also fixes one bug when using sparse files in nfs and
we wait for file size to be equal to volume size (initially done
for scaleio/powerflex backend) but this will never happen for
nfs sparse files. See bug: 2000584
3) Move cinder tests to 'tests/unit/cinder' directory and add tests
for base, nfs and scaleio files.
4) Modify/fix existing tests
Closes-Bug: #2000584
[1] https://review.opendev.org/c/openstack/glance_store/+/868742
Depends-On: https://review.opendev.org/c/openstack/glance/+/869021
Change-Id: I26c272b6c503e98fbbafca411d3eec47283bd6fc
tox.ini started failing with Tox4 which had some
incompatible changes.
Fixing tox.ini for tox4 changes.
Depends-On: https://review.opendev.org/c/openstack/glance/+/868530
Change-Id: I96feba843ca5ebecdc7313961207369da30e72f5
Add an option to let the operator choose a proper region_name.
Some operator are not using S3 from amazon but from other provider.
In such case, the get_s3_location cannot give good region_name.
Adding an option from config seems a better option that relying on
hardcoded stuff in the code.
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I71352e75f6415494709d8fb62c8b13f3ea7c6016
Change I59b1579dc9877668b82d4195431c1 added tests that mock an
input buffer, but the mocked return_value for __len__ exceeds the
allowable maximum value for an index in the CPython implementation
on 32-bit systems. Fix this by using half the value and allowing
the buffer to be called twice during the tests.
Change-Id: Iaba20c01fb4d994cd738a570197f8f6f04f22606
Closes-bug: #1991406
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for antelope. Also,
updating the template name to generic one.
See also the PTI in governance [1].
[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html
Change-Id: I8a2644e11afccdb9e937ec09b6766e7c7f7ae574
Add file to the reno documentation build to show release notes for
stable/zed.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.
Sem-Ver: feature
Change-Id: I9745880b98acf910c4f8cf74372a7fdc9ebac582
The *_domain_id parmaeters should not have any default. Otherwise
keystoneauth ignores the *_domain_name parameters and it requires
only *_domain_id parameters are used.
Closes-Bug: #1620999
Change-Id: I1f8c9184761313f9fc5fda2f257e52233e0196d1
when fetching images with http driver, the redirect URL can have
mandatory query in it, which must be kept intact to successfully
fetch the image.
Change-Id: I2a9d4d026b935ea6c5e5a3a46c86f70ce1e39ae7
Closes-Bug: #1633860
When debugging issues related to glance cinder store, there are
several calls to cinder and it becomes hard to determine which step
we are currently executing without going through cinder logs.
This patch adds some useful debug logs for the new attachment's code
to understand which stage of attachment we are on.
Change-Id: I491b7292a511c47c1d6148dab69ae04269e50c85
... because that old version is no longer supported.
Also the check_python_version method is removed by this change, because
the required python version is now enforced by setup.cfg.
Change-Id: Ie9c91adcb81a1a5dbca2587b6592a58d4cc91e4a
As a new feature relevant for the Cinder store, os-brick now supports
setting the location of file locks in a different location from the
locks of the service.
The functionality is intended for HCI deployments and hosts that are
running Cinder and Glance using Cinder backend. In those scenarios the
service can use a service specific location for its file locks while
only sharing the location of os-brick with the other services.
To leverage this functionality the new os-brick code is needed and
method ``os_brick.setup`` needs to be called once the service
configuration options have been loaded.
The default value of the os-brick ``lock_path`` is the one set in
``oslo_concurrency``.
This patch adds support for this new feature in a backward compatible
way so code works even if using an os-brick version that doesn't have
this feature. That is the case for many CI jobs that install os-brick
from PyPi instead of the code present in master.
Change-Id: Ib11d63e3c388e12f145f40247d17030a566b8c5e
Python 2 has been deprecated for almost two years, and has not been
guaranteed to work with glance_store for a while. This patch removes all
traces of six, unicode strings and Python 2 tweaks.
Co-Authored-By: Cyril Roelandt <cyril@redhat.com>
Change-Id: Ifa78924d7ecf4f2d9a54c677888ab2926530c487
In Zed cycle, we have dropped the python 3.6/3.7[1] testing
and its support. Moving the py36 job to py3 based so that they run
on latest supported python 3. Updating the python classifier also to reflect the same.
[1] https://governance.openstack.org/tc/reference/runtimes/zed.html
Change-Id: Ida582daf7db6fb3cedb3c7e37fc512d094b37c6e
There is a syntax error in the exception logging when attaching a
volume fails in attachment state manager. This patch corrects it
and adds a test to guard against similar changes in future.
Closes-Bug: #1970698
Change-Id: I43c407046a49bb37631113e2ea65d05450f9365d
When we try to simultaneously attach same volume multiple times
(multiattach), there are multiple attachments created, suppose
attachA and attachB. If attachA marks the volume "reserved" then
attachB can't proceed until the volume is in "in-use" or "available"
state. We retry until we reach any of these states for which we use
the retrying library.
The retrying library defaults to 1 second retry[1] (5 times) which causes
the original failure as the volume takes time to transition between
"reserved" -> "in-use" state. This patch corrects it by adding an
exponential retry time and max exponential retry i.e.
1: 2 ** 1 = 2 seconds
2: 2 ** 2 = 4 seconds
3: 2 ** 3 = 8 seconds
4: 2 ** 4 = 16 seconds (but max wait time is 10 seconds)
5: 2 ** 5 = 32 seconds (but max wait time is 10 seconds)
[1] https://github.com/rholder/retrying/blob/master/retrying.py#L84
Closes-Bug: #1969373
Change-Id: I0094b044085d7f92b07ea86236de3b6efd7d67ea
This patch adds test coverage of code paths like extend volume,
exception blocks, when image_size is zero, volume delete call etc
Change-Id: I59b1579dc9877668b82d4195431c14cc41cfe892
This patch adds coverage for the various exceptions handled and
raised in get, get_size and delete methods.
It also corrects the behavior of _test_cinder_get_size method where
client.volumes returned dictionary instead of a MagicMock and it
worked due to the existing method "get" in dictionary which has
same name as "get" method in cinderclient. The dictionary object
is replaced with the appropriate MagicMock object in this patch.
Change-Id: If63a6c810b5aab992e54857bc81f5052c2c593c4
This patch adds coverage for some parts in get_cinderclient
method like ``cinder_endpoint_template`` option and keystone
exception case when catalog info for cinder could not be found.
Also it adds tests for complete coverage of _check_context
method.
Change-Id: I18b1e5e8fd818824a2dda2ad14d9456190fe9ff4
The cinder volume create call was moved to the cinder_utils
module to keep the driver code cleaner with commit[1].
A decorator was also added to handle NotFound exception which
made the existing try/except block redundant.
Also the current try/except block has misleading messages:
1) "Invalid volume type configured"
2) "Failed to create image-volume due to invalid
`cinder_volume_type` configured."
A volume create could fail because of many reasons and "invalid
volume type" is just one of them. These messages will confuse
the operators even if the configured volume type is correct.
This patch removes them since the NotFound exception is already
handled at[2] and the messages are misleading.
[1] 1178f113c4
[2] c0c4969a4a/glance_store/common/cinder_utils.py (L37)
Change-Id: I39c9caec85f574fc717e60f11e5681860a20ed30
This patch adds code coverage (UTs) for methods in StoreLocation class:
1) process_specs
2) get_uri
3) parse_uri
It also adds a missing method (_set_url_prefix) coverage in
Multistore cinder tests (test_multistore_cinder).
Change-Id: I8ced5af11669fb131c665ce53be57143f2c7b518
Add test coverage for two config options used to create a keystone
session object for cinderclient namely ``cinder_api_insecure`` and
``cinder_ca_certificates_file``.
Change-Id: Idb6db9f87ef7290441dfe60ca2eb9a59c9ee7bd3
six has been used to maintain compatibility between python2 and
python3 code. Since current openstack development branches doesn't
support python2, we don't need the six library and can use python3
code directly.
This patch removes six usage from cinder store tests.
Change-Id: I4deb193b44394ae9f99f57e31fb81fe04394e247
This patch aims at a refactoring effort that would remove
duplicate tests (current and future) by moving them into a
common base class which is called via both single and multi store
test modules with their specific configurations.
This has a lot of benefits:
1) Removes duplicate code
2) Makes addition of new tests easier and cleaner
3) Ensuring a new method/code path added is tested in both
single and multi store configurations
4) Fixing issues detected while refactoring methods
(Eg: tests for add method in test_multistore_cinder were not
passing the hashing_algo parameter which is currently handled
by the backward compat code (back_compat_add decorator) but
those tests will break when we remove backward compatibility)
Change-Id: I12569af5623f1cd7803c00a6c3b9eb211f15b6fd
This patch aims at a refactoring effort that would remove
duplicate tests (current and future) by moving them into a
common base class which is called via both single and multi store
test modules with their specific configurations.
This has a lot of benefits:
1) Removes duplicate code
2) Makes addition of new tests easier and cleaner
3) Ensuring a new method/code path added is tested in both
single and multi store configurations
4) Fixing issues detected while refactoring methods
(Eg: fake_chown accepted an optional backend parameter which
does not match the signature of original method temporary_chown)
Change-Id: Iaacb3e117cac2a661750bdb21bd0a7496078ea26
FakeObject is used to set dynamic attributes to a fake class which
works in most cases but fails when an attribute is accessed which
is not defined/passed explicitly.
This causes failure of tests whenever a new attribute is accessed
and it's hard to maintain the set of attributes for every object
used.
MagicMock provides the same functionality with additional features
and handles the case effectively when an attribute is accessed
which is not explicitly defined in the tests.
This change will help safeguard against cases seen earlier in[1].
This patch replaces FakeObject with MagicMock in test_multistore_cinder.py
[1] https://review.opendev.org/c/openstack/glance/+/805974
Change-Id: I193bb03d174e6c010132702dedef1dea05f40e41
FakeObject is used to set dynamic attributes to a fake class which
works in most cases but fails when an attribute is accessed which
is not defined/passed explicitly.
This causes failure of tests whenever a new attribute is accessed
and it's hard to maintain the set of attributes for every object
used.
MagicMock provides the same functionality with additional features
and handles the case effectively when an attribute is accessed
which is not explicitly defined in the tests.
This change will help safeguard against cases seen earlier in[1].
This patch replaces FakeObject with MagicMock in test_cinder_store.py
[1] https://review.opendev.org/c/openstack/glance/+/805974
Change-Id: Ifd0905b139cc90c2bd7444bc9e7638f3af6879a7
Add file to the reno documentation build to show release notes for
stable/yoga.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/yoga.
Sem-Ver: feature
Change-Id: I99772281621ebce5a57ddd37cee879d7cc387c47
Zuul
OpenStack Release Bot