Add APIs to support manage and unmanage share snapshots.
Also add support in the Generic driver.
This only supports for DHSS=False driver mode.
Add provider_location column to the share_snapshots table
to save data used to identify the snapshot on the backend.
Also need to bump microversion.
APIImpact
DocImpact
Change-Id: I87a066173c85d969607d132accd9f0e9bd49c235
Implements: blueprint manage-unmanage-snapshot
The post_test_hook.sh script was hardcoding the Tempest concurrency
setting to 8 for DHSS=False test runs. This patch has it honor the
MANILA_TEMPEST_CONCURRENCY environment variable, if it exists.
Change-Id: Ia2f0495560517484ba4d9290ced2c4dc3de6fb34
Closes-bug: 1546150
- Add update_access() method to driver interface
- Move all code related to access operations to ShareInstanceAccess
class
- Statuses from individual access rules are now mapped to
share_instance's access_rules_status
- Add 'access_rules_status' field to share instance, which indicates
current status of applying access rules
APIImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Tiago Pasqualini da Silva <tiago.pasqualini@gmail.com>
Implements: bp new-share-access-driver-interface
Change-Id: Iff1ec2e3176a46e9f6bd383b38ffc5d838aa8bb8
Manila CI jobs create lots of shares and for Generic driver it means
lots of Cinder volumes. After merge of [1] to Cinder project, we started
having NoValidHostFound errors in Cinder logs, it changed default
oversubscription ratio to 1.0 from 20.0. But Manila CI requires it
be bigger than 1.0, so set it explicitly to Cinder's default value 20.0
that is used by all other driver but not LVM.
[1] I5a214462b573d00d26086512821b4f8cc1db4fc3
Change-Id: If0a6e535b92b1471572e1de98e3b44ab6a2c65c9
Closes-Bug: #1540947
After merge of change [1] to Devstack project Manila Tempest plugin not
compatible anymore with latest Devstack.
So, bump Tempest version to latest and fix compatibility with it.
[1] If66096cde31fedda407e1e6ac6899b196569a454
Change-Id: Idf9a9eabf9c98d090e47207cf89d9e5a5fc0ea97
Closes-Bug: #1539007
Reuse code of old LVM share driver.
LVM Driver is 1st party Manila driver with NFS and CIFS support
and no share server support.
LVM Driver doesn't rely on Neutron, Nova, or Cinder. As such, it
is ideal for testing purposes and, after a period of maturation,
it should be useful in production environments.
Move generic driver's helpers to manila/share/drivers/helpers.py
and reuse them in LVM driver.
Implement 'ro' access, user access, extend_share function for
LVM driver.
Implements bp lvm-driver
Change-Id: Ia46c51ed400dbb0f1d87a758fb165ca711ed3d7c
Use arguments of Devstack hook scripts to configure Manila and Tempest in
CI jobs instead of dependency on names of CI jobs.
Change-Id: I96d8bca54320f4476789c00e1061d4e96b697d05
Depends-On: I521f0419e65e27f22acbc03aaa1f0f997492e8f8
After merge of change [1] to devstack project, preconfigured creds for
Tempest cannot be used anymore. But Manila requires such.
For the moment, latest Tempest interfaces cannot allow us to use
preconfigured creds because of bug #1524717.
So, until this bug fixed, we should workaround Devstack changes and
set legacy opts explicitly.
Also, redefine new Nutron option 'dnsmasq_local_resolv' to 'False' value,
that is 'True' by default and was added in change [2].
Default value causes Nova operations initiated by Manila be very slow.
That leads to breakage of CI.
[1] I65b56ff681d6c27094380693c953fbc3664eceb0
[2] I17a884f467d307432a06f67a9dd93ed2fa6081a3
Change-Id: I516a6c9ab4396b14e3984d5f810210e4fcf7ec85
Related-Bug: #1531049
Closes-Bug: #1530847
Manila CI jobs fail very often exceeding timeout.
So, do several changes to speed up Tempest jobs:
1) Separate NFS and CIFS tests between jobs.
Run NFS jobs on MySQL ones and CIFS on PostgreSQL.
2) Refactor couple of tests removing creation of redundant shares
testing the same things.
3) Increase amount of threads, to minimize probability of
running heavy test suites in one thread.
Change-Id: I6b60db039a96b47386f8ff538d1cb74559691ccc
With the merge of 'Factor up config dependent credential classes' in tempest,
(Change-Id: I351f86f0570070105bbf24f2d904e917feea6e57)
helper methods related to credentials have been moved to
common.credentials_factory module.
manila_tempest_tests need to change accordingly.
Change-Id: I9e5c72279638e07f509a4bfe0174b5ad5ee70b45
Closes-Bug: #1519612
Just rely on the return code from "ping" instead of
parsing it's output when checking the service VM during
the CI run.
Change-Id: I20f144687662a0dda5fdfc0293ab578009658c1b
Manila CI hooks for DHSS=False job with the Generic driver wait for
service instance availability and right after it becomes available
returns control to caller. And caller can run tests with creation of
shares immediately. But, in this case, we can face race conditions when
manila-share service requires some time to inform manila-scheduler.
So, add 'sleep 10' operation to be executed after reaching 'available'
state by service instance.It should be enough to overcome races.
Change-Id: Ie4ae231701edaab319c9012e476d1f63e3806499
Commit [1] moved common logic for CI hooks to separate file, but it
depends on some devstack functions, that should be "sourced" in this
common file. So, this common file works in manila hooks, because they
import devstack functions separately, but it will not work if some other
scripts use it and do not import devstack functions.
[1] I94514405628ab8337d1f4153e431f064614e3596
Change-Id: I919fea046e556d601a7680ea9bac16c1a9b7343c
Closes-Bug: #1519028
Manila contains CI hooks located in "contrib/ci" dir.
Split common logic to separate file to be able to just "source" it and
then use env vars and bash functions in different CI jobs and projects.
Current change allows to do two things:
- Reuse env var "MANILA_TEMPEST_COMMIT" in third-party CIs
- Wait for drivers init in other projects like OpenStack Rally. Where,
for the moment, one of jobs fails because of absense of "waiting".
Usage:
$ source %manila%/contrib/ci/common.sh
$ echo $MANILA_TEMPEST_COMMIT
$ manila_wait_for_drivers_init "/etc/manila/manila.conf"
Change-Id: I94514405628ab8337d1f4153e431f064614e3596
This example consists of three parts:
- Server side hook module
'contrib.share_driver_hooks.zaqar_notification' that can be enabled in Manila
- Client side consumer module
'contrib.share_driver_hooks.zaqar_notification_example_consumer' that can be
used in any user machine.
- Common module 'contrib.share_driver_hooks.zaqarclientwrapper' that is used
by server and client side modules for initialization of Zaqar client.
Details of its usage are described in file
'contrib/share_driver_hooks/README.rst'
Change-Id: I5e802ee2e2a4dd36db92865b0ba82e73c1fa86d4
Two reasons to do it:
1) Broken compatibility of old Tempest we use with latest dependencies.
2) Manila Tempest plugin is incompatible with latest Tempest.
Change-Id: Iae5a656b72e774d9d62b52d7ffb63edecbb23ca6
Closes-Bug: #1513105
Our Tempest job "*no-share-servers*" fails after merge of change [1]
The root cause for it is in changed behaviour:
After merge of change [1] share services with driver that use Nova
started verify availability of a Nova VM by its net interface and
do not report its capabilities until it is available.
It takes about 2,5 minutes. During this time scheduler has no available
hosts and we get "no hosts available" errors from it in case of CI that
runs Tempest tests much faster than 2,5 minutes after VM creation.
So, add waiter in post_test_hook.sh script.
[1] I325d6461c4b12f481769bb01f792ef850dba789c
Change-Id: I58d617dd76cf079e5e26bf6a4ddc6b4fba4e20e6
Closes-Bug: #1507605
wating -> waiting
acknowledgement -> acknowledgment
occured -> occurred
Should never get here. If get, then error occurred.
->
Should never get here unless an error occurred.
Change-Id: Ibe89d652e7d76b3d8e6d7549140c50c88144036e
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
This patch addresses issues commented on
change I64b0a3ee77b27278cc294f72702408a27888e0e9
after it was merged, according to bug below.
Closes-bug: #1494000
Change-Id: I9079ea16e0edc359b380705bebba2a7e98446d5c
Use novaclient for creation of keypair instead of openstack client in
devstack plugin, because the latter is not compatible with
nova microversion 2.1
Also, specify explicitly Nova API version in manila wrapper over
novaclient that is used by Generic and Windows drivers.
Also, revert back amount of tempest threads for Ci jobs to avoid job
timeout failures.
Also, fix raise of SnapshotBuildErrorException in Tempest plugin.
Tempest tries to raise exception SnapshotBuildErrorException from wrong
place. It should use "share_exceptions" instead of "exceptions".
Change-Id: Ie9ea9f6d25a13f8cd8f0ccbc73f6fa75cca35e0b
Closes-Bug: #1493008
Closes-Bug: #1493072
This commit adds tempest functional tests for Manila consistency
groups and cgsnapshot objects and actions. By default these tests are
enabled; 3rd party CI systems should disabled these tests through the
RUN_MANILA_CG_TESTS environment variable if their driver does not
support consistency groups.
Partially implements bp manila-consistency-groups
Change-Id: I3297e02ad53c328f0bfe5245fefdb6af80552b4a
Recent update to Tempest plugin was moving existing test modules to different
place. And module 'test_microversions' was missed and not moved.
Change-Id: Id19fdb64f12b59c36118c097e2ca7136646c76b3
Depends-On: Ie9ea9f6d25a13f8cd8f0ccbc73f6fa75cca35e0b
Closes-Bug: #1493009
The Generic Driver in Manila has no concept of consistency
groups. This patch proposes to implement a loose
consistency group support mechanism for the Generic Driver,
using existing logic to snapshot shares and create shares
from snapshots.
The actions concerning creating and deleting a consistency
group are handled by the API and the Generic Driver does not
need to remember any relevant information persistently.
The actions concerning consistency group snapshots are not
simultaneous, but are rather let to run in succession.
DocImpact
Partially implements bp manila-consistency-groups
Change-Id: Ie5e51205a1154786f9057691aa82bffcafe04de7
Make use of the Tempest plugin interface instead of copying Manila
files into Tempest. This will remove the burden to port Manila
tests onto Tempest master recurrently.
This ports all existing Manila Tempest test to the new structure.
It uses manila_tempest_tests as new top folder for all Tempest
tests. It follow the model of Heat (see [1]).
[1]: https://github.com/openstack/heat/tree/master/heat_integrationtests
Change-Id: Ie5ed64a6777ed1acf8dd56522c26705ae897596d
Partly-implements: bp tempest-plugin-interface
Depends-On: I26dd32b1de8cceeaa6dc674092efec683df71889
Make it possible to setup password for generic driver service image and
scenario tests. Also allow to just disable its usage not unsetting the password
It is useful for testing purpose when we have some problems with SSH keys
injection to Nova VMs.
Change-Id: I26dd32b1de8cceeaa6dc674092efec683df71889
Related-Bug: #1480870
Add admin API which allows:
- Get share instances of given share
- Get all share instances
- Get share instance details by id
- Reset share instance state
- Force-delete share instance
Add appropriate unit and tempest tests.
APIImpact
Change-Id: Ie96dfc18b491cb4d9705da4eaca5bc4ce43225ea
Partially-Implements: blueprint share-instances
Two of the tests targeting share listing contain a wrong assertion on the
number of existing shares. It is asserted that more than one shares exist,
while in case snapshot tests are disabled, only one share is created
during resource setup.
This patch fixes those tests by lowering the expected minimum
number of shares to 1.
Change-Id: I28c5d7d61425c20a657af25532e5495b2ebea360
Closes-Bug: #1489846
The tests targeting user based access rules do not wait when
deleting such access rules. As multiple tests will create access
rules for the same user, this operation will fail if the previous
rule was not deleted in time.
This patch fixes those tests by waiting for the access rule delete
operation to complete.
The wait_for_resource_deletion method is updated to work with
access rules, re-adding previously existing functionality, removed
as unused and also requested by bug/1429952.
Closes-Bug: #1489367
Change-Id: Ie48fba56668ad663be84b74cd64927e3e5d6c675
The helper method "_create_delete_ro_access_rule" used by the tests
targeting user rules will actually create ip based rules.
This can easily be fixed by using the access_type and access_to
attributes instead of the hardcoded values.
Closes-Bug: #1489359
Change-Id: Ie81537c53744f33c10a6d0e0d584a6f355e47407
In order to be independent from changes in Tempest Manila should
use the tempest_lib as much as possible. Exception classes are
already copied to tempest_lib.
Closes-Bug: #1489299
Change-Id: I043be143d7d35614af1f6d7892b0a0bc71d73580
On last midcycle meetup was decided to make snapshots optional feature.
Features:
- Add new boolean capability 'snapshot_support' to base share driver
so every existing share driver will inherit it. Make value of it be
calculated from the fact of redefinition of three main driver
methods for snapshots 'create_snapshot', 'delete_snapshot' and
'create_share_from_snapshot'.
- Set extra spec 'snapshot_support' with share type creation by default
to 'True'
- Restrict deletion of extra spec 'snapshot_support' that is expected
to exist
- Allow to redefine new extra spec 'snapshot_support'
- Restrict API 'snapshot create' for share created with
share type that has extra spec 'snapshot_support' equal to 'False'.
- Add migration where new extra spec 'snapshot_support' is added
to all share types that do not have it yet.
Partially implements bp snapshots-optional
Change-Id: I069d9e911c7d7a708fa518b38ed10572a45e5f42
Rework availability zones support which was inherited from Cinder:
- Add public API extension
- Preserve AZ if creating a share from a snapshot
- Always set AZ in Share API or Share Manager
- Update db schema and create db migration
- Update appropriate unit tests
APIImpact
Partially-Implements: blueprint availability-zones
Change-Id: Iea9fbc3fea5c0128772115c028989121f397e0c5
This patch adds tempest tests that validate Manila's versions API. Since the
versions API is the first API to be versioned in Manila, the code in this
patch makes requests to invoke the versions API with valid and invalid
version codes and evaluates responses.
Change-Id: I4046f020f0e129dab639f99390367efda9cf4551
Currently, Manila Tempest test cases support only testing of shares of
type nfs or cifs protocol. Allow testing of shares that use glusterfs
protocol or hdfs protocol.
Closes-Bug: #1485987
Change-Id: Ie7b1e9638ad2afaf8388cb4d7cda8d04e89a3c0a
Currently some tests that run successfully do not clean up shares
created, wasting space on backends that run CIs.
The root cause is that shares_client used directly for creation of shares
instead of using wrapper from base which will add them to "deletion" queue.
Change-Id: I8d80a5dd37be874c852b03e204547ed43d3843d2
Closes-bug: #1483863
On last midcycle meetup was decided to make snapshots optional feature.
So, at first step, add possibility to skip all functional tests that
use snapshots.
Partially implements bp snapshots-optional
Change-Id: Ib7dd3303e3974298439b6ac543b9df628b69e6ae