400 Commits

Author SHA1 Message Date
Xing Yang
c91f27f4e1 Manage and unmanage snapshot
Add APIs to support manage and unmanage share snapshots.
Also add support in the Generic driver.
This only supports for DHSS=False driver mode.

Add provider_location column to the share_snapshots table
to save data used to identify the snapshot on the backend.

Also need to bump microversion.

APIImpact
DocImpact
Change-Id: I87a066173c85d969607d132accd9f0e9bd49c235
Implements: blueprint manage-unmanage-snapshot
2016-02-18 04:09:05 -05:00
Andrew Kerr
a659738b11 Allow DHSS=False tests to override Tempest concurrency
The post_test_hook.sh script was hardcoding the Tempest concurrency
setting to 8 for DHSS=False test runs.  This patch has it honor the
MANILA_TEMPEST_CONCURRENCY environment variable, if it exists.

Change-Id: Ia2f0495560517484ba4d9290ced2c4dc3de6fb34
Closes-bug: 1546150
2016-02-16 18:18:52 +00:00
Igor Malinovskiy
b1b723ad0b Add update_access() method to driver interface
- Add update_access() method to driver interface
- Move all code related to access operations to ShareInstanceAccess
class
- Statuses from individual access rules are now mapped to
share_instance's access_rules_status
- Add 'access_rules_status' field to share instance, which indicates
current status of applying access rules

APIImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Tiago Pasqualini da Silva <tiago.pasqualini@gmail.com>
Implements: bp new-share-access-driver-interface

Change-Id: Iff1ec2e3176a46e9f6bd383b38ffc5d838aa8bb8
2016-02-05 10:41:51 -02:00
vponomaryov
8c3c73ada6 Fix Cinder's NoValidHostFound errors
Manila CI jobs create lots of shares and for Generic driver it means
lots of Cinder volumes. After merge of [1] to Cinder project, we started
having NoValidHostFound errors in Cinder logs, it changed default
oversubscription ratio to 1.0 from 20.0. But Manila CI requires it
be bigger than 1.0, so set it explicitly to Cinder's default value 20.0
that is used by all other driver but not LVM.

[1] I5a214462b573d00d26086512821b4f8cc1db4fc3

Change-Id: If0a6e535b92b1471572e1de98e3b44ab6a2c65c9
Closes-Bug: #1540947
2016-02-02 22:21:54 +02:00
Valeriy Ponomaryov
70871c2479 Bump tempest version
After merge of change [1] to Devstack project Manila Tempest plugin not
compatible anymore with latest Devstack.
So, bump Tempest version to latest and fix compatibility with it.

[1] If66096cde31fedda407e1e6ac6899b196569a454

Change-Id: Idf9a9eabf9c98d090e47207cf89d9e5a5fc0ea97
Closes-Bug: #1539007
2016-01-29 00:24:39 +02:00
Julia Varlamova
401c8d982e Add LVM driver
Reuse code of old LVM share driver.

LVM Driver is 1st party Manila driver with NFS and CIFS support
and no share server support.

LVM Driver doesn't rely on Neutron, Nova, or Cinder. As such, it
is ideal for testing purposes and, after a period of maturation,
it should be useful in production environments.

Move generic driver's helpers to manila/share/drivers/helpers.py
and reuse them in LVM driver.

Implement 'ro' access, user access, extend_share function for
LVM driver.

Implements bp lvm-driver

Change-Id: Ia46c51ed400dbb0f1d87a758fb165ca711ed3d7c
2016-01-20 17:15:30 -05:00
vponomaryov
09e7125583 Use new approach for setting up CI jobs
Use arguments of Devstack hook scripts to configure Manila and Tempest in
CI jobs instead of dependency on names of CI jobs.

Change-Id: I96d8bca54320f4476789c00e1061d4e96b697d05
Depends-On: I521f0419e65e27f22acbc03aaa1f0f997492e8f8
2016-01-08 18:02:59 +02:00
vponomaryov
f4a00c5965 Fix CI Tempest jobs
After merge of change [1] to devstack project, preconfigured creds for
Tempest cannot be used anymore. But Manila requires such.
For the moment, latest Tempest interfaces cannot allow us to use
preconfigured creds because of bug #1524717.
So, until this bug fixed, we should workaround Devstack changes and
set legacy opts explicitly.

Also, redefine new Nutron option 'dnsmasq_local_resolv' to 'False' value,
that is 'True' by default and was added in change [2].
Default value causes Nova operations initiated by Manila be very slow.
That leads to breakage of CI.

[1] I65b56ff681d6c27094380693c953fbc3664eceb0
[2] I17a884f467d307432a06f67a9dd93ed2fa6081a3

Change-Id: I516a6c9ab4396b14e3984d5f810210e4fcf7ec85
Related-Bug: #1531049
Closes-Bug: #1530847
2016-01-07 00:52:32 +02:00
Valeriy Ponomaryov
fde99c896f [CI] Speed up Tempest jobs
Manila CI jobs fail very often exceeding timeout.
So, do several changes to speed up Tempest jobs:

1) Separate NFS and CIFS tests between jobs.
Run NFS jobs on MySQL ones and CIFS on PostgreSQL.

2) Refactor couple of tests removing creation of redundant shares
testing the same things.

3) Increase amount of threads, to minimize probability of
running heavy test suites in one thread.

Change-Id: I6b60db039a96b47386f8ff538d1cb74559691ccc
2015-12-03 16:59:46 +02:00
Jenkins
30eaf80506 Merge "Change manila_tempest_tests to use credentials_factory" 2015-12-03 02:16:55 +00:00
Sam Wan
69a3d999c2 Change manila_tempest_tests to use credentials_factory
With the merge of 'Factor up config dependent credential classes' in tempest,
(Change-Id: I351f86f0570070105bbf24f2d904e917feea6e57)
helper methods related to credentials have been moved to
common.credentials_factory module.
manila_tempest_tests need to change accordingly.

Change-Id: I9e5c72279638e07f509a4bfe0174b5ad5ee70b45
Closes-Bug: #1519612
2015-12-02 03:56:38 -05:00
Jenkins
56ccb015c8 Merge "Simplify ping usage for service VM check in CI" 2015-12-01 09:44:16 +00:00
Thomas Bechtold
6eb3a2d05b Simplify ping usage for service VM check in CI
Just rely on the return code from "ping" instead of
parsing it's output when checking the service VM during
the CI run.

Change-Id: I20f144687662a0dda5fdfc0293ab578009658c1b
2015-11-27 09:55:38 +01:00
Valeriy Ponomaryov
14d57d158f Add sleep to CI hooks to avoid races
Manila CI hooks for DHSS=False job with the Generic driver wait for
service instance availability and right after it becomes available
returns control to caller. And caller can run tests with creation of
shares immediately. But, in this case, we can face race conditions when
manila-share service requires some time to inform manila-scheduler.
So, add 'sleep 10' operation to be executed after reaching 'available'
state by service instance.It should be enough to overcome races.

Change-Id: Ie4ae231701edaab319c9012e476d1f63e3806499
2015-11-26 17:58:14 +02:00
Valeriy Ponomaryov
037ba11389 Fix import of devstack functions for common CI script
Commit [1] moved common logic for CI hooks to separate file, but it
depends on some devstack functions, that should be "sourced" in this
common file. So, this common file works in manila hooks, because they
import devstack functions separately, but it will not work if some other
scripts use it and do not import devstack functions.

[1] I94514405628ab8337d1f4153e431f064614e3596

Change-Id: I919fea046e556d601a7680ea9bac16c1a9b7343c
Closes-Bug: #1519028
2015-11-23 18:38:55 +02:00
Jenkins
6c4322495b Merge "Split common logic of CI hooks to separate file" 2015-11-14 09:48:52 +00:00
vponomaryov
6b8319e32a Split common logic of CI hooks to separate file
Manila contains CI hooks located in "contrib/ci" dir.
Split common logic to separate file to be able to just "source" it and
then use env vars and bash functions in different CI jobs and projects.

Current change allows to do two things:
 - Reuse env var "MANILA_TEMPEST_COMMIT" in third-party CIs
 - Wait for drivers init in other projects like OpenStack Rally. Where,
   for the moment, one of jobs fails because of absense of "waiting".

Usage:
 $ source %manila%/contrib/ci/common.sh
 $ echo $MANILA_TEMPEST_COMMIT
 $ manila_wait_for_drivers_init "/etc/manila/manila.conf"

Change-Id: I94514405628ab8337d1f4153e431f064614e3596
2015-11-10 11:24:51 +00:00
Valeriy Ponomaryov
2f4795f7fe Add mount automation example based on Zaqar
This example consists of three parts:
- Server side hook module
'contrib.share_driver_hooks.zaqar_notification' that can be enabled in Manila
- Client side consumer module
'contrib.share_driver_hooks.zaqar_notification_example_consumer' that can be
used in any user machine.
- Common module 'contrib.share_driver_hooks.zaqarclientwrapper' that is used
by server and client side modules for initialization of Zaqar client.

Details of its usage are described in file
'contrib/share_driver_hooks/README.rst'

Change-Id: I5e802ee2e2a4dd36db92865b0ba82e73c1fa86d4
2015-11-06 23:41:40 +00:00
Valeriy Ponomaryov
0af26bbba2 Sync Manila Tempest plugin with latest Tempest
Two reasons to do it:
1) Broken compatibility of old Tempest we use with latest dependencies.
2) Manila Tempest plugin is incompatible with latest Tempest.

Change-Id: Iae5a656b72e774d9d62b52d7ffb63edecbb23ca6
Closes-Bug: #1513105
2015-11-06 12:15:54 +02:00
Valeriy Ponomaryov
cbefc53051 Fix no-share-servers CI job
Our Tempest job "*no-share-servers*" fails after merge of change [1]

The root cause for it is in changed behaviour:
After merge of change [1] share services with driver that use Nova
started verify availability of a Nova VM by its net interface and
do not report its capabilities until it is available.
It takes about 2,5 minutes. During this time scheduler has no available
hosts and we get "no hosts available" errors from it in case of CI that
runs Tempest tests much faster than 2,5 minutes after VM creation.

So, add waiter in post_test_hook.sh script.

[1] I325d6461c4b12f481769bb01f792ef850dba789c

Change-Id: I58d617dd76cf079e5e26bf6a4ddc6b4fba4e20e6
Closes-Bug: #1507605
2015-10-27 11:54:27 +02:00
Zhao Lei
586c666e5c Fix some spelling typo in manual and error message
wating -> waiting
acknowledgement -> acknowledgment
occured -> occurred

Should never get here. If get, then error occurred.
->
Should never get here unless an error occurred.

Change-Id: Ibe89d652e7d76b3d8e6d7549140c50c88144036e
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
2015-09-24 07:05:39 +00:00
Marc Koderer
2a79beb643 Fix typo in test_hook
Fix typo according to https://review.openstack.org/#/c/221704/

Change-Id: I5efc0c4dec4917ec3686dec0044df40d4111e54a
2015-09-12 16:53:44 +02:00
Rodrigo Barbieri
72241d1036 Fix Share Migration tempest tests
This patch addresses issues commented on
change I64b0a3ee77b27278cc294f72702408a27888e0e9
after it was merged, according to bug below.

Closes-bug: #1494000
Change-Id: I9079ea16e0edc359b380705bebba2a7e98446d5c
2015-09-11 18:15:56 -03:00
Jenkins
fa10a98649 Merge "Add functional tests for Manila consistency groups" 2015-09-07 21:10:18 +00:00
Jenkins
964761ab04 Merge "Place tempest microversions test module in proper place" 2015-09-07 20:33:06 +00:00
Valeriy Ponomaryov
f88c2aaeea Fix keypair creation
Use novaclient for creation of keypair instead of openstack client in
devstack plugin, because the latter is not compatible with
nova microversion 2.1

Also, specify explicitly Nova API version in manila wrapper over
novaclient that is used by Generic and Windows drivers.

Also, revert back amount of tempest threads for Ci jobs to avoid job
timeout failures.

Also, fix raise of SnapshotBuildErrorException in Tempest plugin.
Tempest tries to raise exception SnapshotBuildErrorException from wrong
place. It should use "share_exceptions" instead of "exceptions".

Change-Id: Ie9ea9f6d25a13f8cd8f0ccbc73f6fa75cca35e0b
Closes-Bug: #1493008
Closes-Bug: #1493072
2015-09-07 14:52:11 -04:00
Andrew Kerr
333474047c Add functional tests for Manila consistency groups
This commit adds tempest functional tests for Manila consistency
groups and cgsnapshot objects and actions. By default these tests are
enabled; 3rd party CI systems should disabled these tests through the
RUN_MANILA_CG_TESTS environment variable if their driver does not
support consistency groups.

Partially implements bp manila-consistency-groups
Change-Id: I3297e02ad53c328f0bfe5245fefdb6af80552b4a
2015-09-07 13:59:22 -04:00
Valeriy Ponomaryov
8666460e8c Place tempest microversions test module in proper place
Recent update to Tempest plugin was moving existing test modules to different
place. And module 'test_microversions' was missed and not moved.

Change-Id: Id19fdb64f12b59c36118c097e2ca7136646c76b3
Depends-On: Ie9ea9f6d25a13f8cd8f0ccbc73f6fa75cca35e0b
Closes-Bug: #1493009
2015-09-07 19:23:12 +03:00
Goutham Pacha Ravi
079a3d64f5 Consistency Group Support for the Generic Driver
The Generic Driver in Manila has no concept of consistency
groups. This patch proposes to implement a loose
consistency group support mechanism for the Generic Driver,
using existing logic to snapshot shares and create shares
from snapshots.

The actions concerning creating and deleting a consistency
group are handled by the API and the Generic Driver does not
need to remember any relevant information persistently.

The actions concerning consistency group snapshots are not
simultaneous, but are rather let to run in succession.

DocImpact
Partially implements bp manila-consistency-groups

Change-Id: Ie5e51205a1154786f9057691aa82bffcafe04de7
2015-09-07 04:48:45 +00:00
Marc Koderer
09798caae1 Use Tempest plugin interface
Make use of the Tempest plugin interface instead of copying Manila
files into Tempest. This will remove the burden to port Manila
tests onto Tempest master recurrently.

This ports all existing Manila Tempest test to the new structure.

It uses manila_tempest_tests as new top folder for all Tempest
tests. It follow the model of Heat (see [1]).

[1]: https://github.com/openstack/heat/tree/master/heat_integrationtests

Change-Id: Ie5ed64a6777ed1acf8dd56522c26705ae897596d
Partly-implements: bp tempest-plugin-interface
Depends-On: I26dd32b1de8cceeaa6dc674092efec683df71889
2015-09-05 00:36:33 +00:00
Valeriy Ponomaryov
6c44bff516 Add possibility to setup password for generic driver
Make it possible to setup password for generic driver service image and
scenario tests. Also allow to just disable its usage not unsetting the password

It is useful for testing purpose when we have some problems with SSH keys
injection to Nova VMs.

Change-Id: I26dd32b1de8cceeaa6dc674092efec683df71889
Related-Bug: #1480870
2015-09-04 18:56:57 +03:00
Jenkins
e38ce9b686 Merge "Add tempest tests for "cert" based access type" 2015-09-01 08:57:49 +00:00
Jenkins
18fb649e4c Merge "Add Share instances Admin API" 2015-08-31 19:00:53 +00:00
Igor Malinovskiy
91e438b40e Add Share instances Admin API
Add admin API which allows:
- Get share instances of given share
- Get all share instances
- Get share instance details by id
- Reset share instance state
- Force-delete share instance

Add appropriate unit and tempest tests.

APIImpact
Change-Id: Ie96dfc18b491cb4d9705da4eaca5bc4ce43225ea
Partially-Implements: blueprint share-instances
2015-08-31 13:41:17 +03:00
Jenkins
99675a3e38 Merge "Tempest: wrong assertion on the number of shares created" 2015-08-28 22:42:45 +00:00
Alin Balutoiu
94a750dbe8 Tempest: wrong assertion on the number of shares created
Two of the tests targeting share listing contain a wrong assertion on the
number of existing shares. It is asserted that more than one shares exist,
while in case snapshot tests are disabled, only one share is created
during resource setup.

This patch fixes those tests by lowering the expected minimum
number of shares to 1.

Change-Id: I28c5d7d61425c20a657af25532e5495b2ebea360
Closes-Bug: #1489846
2015-08-28 14:39:22 +03:00
Jenkins
b46f92c53a Merge "Tempest: wait for access rule to be deleted" 2015-08-28 11:21:01 +00:00
Jenkins
5871b47093 Merge "Fix Tempest tests targeting user based access rules" 2015-08-28 05:26:22 +00:00
Alin Balutoiu
ddb276f629 Tempest: wait for access rule to be deleted
The tests targeting user based access rules do not wait when
deleting such access rules. As multiple tests will create access
rules for the same user, this operation will fail if the previous
rule was not deleted in time.

This patch fixes those tests by waiting for the access rule delete
operation to complete.

The wait_for_resource_deletion method is updated to work with
access rules, re-adding previously existing functionality, removed
as unused and also requested by bug/1429952.

Closes-Bug: #1489367

Change-Id: Ie48fba56668ad663be84b74cd64927e3e5d6c675
2015-08-27 11:09:57 +00:00
Alin Balutoiu
641a5e985e Fix Tempest tests targeting user based access rules
The helper method "_create_delete_ro_access_rule" used by the tests
targeting user rules will actually create ip based rules.

This can easily be fixed by using the access_type and access_to
attributes instead of the hardcoded values.

Closes-Bug: #1489359

Change-Id: Ie81537c53744f33c10a6d0e0d584a6f355e47407
2015-08-27 14:01:35 +03:00
Marc Koderer
922204cc8d Reduce dependency to tempest: exceptions
In order to be independent from changes in Tempest Manila should
use the tempest_lib as much as possible. Exception classes are
already copied to tempest_lib.

Closes-Bug: #1489299

Change-Id: I043be143d7d35614af1f6d7892b0a0bc71d73580
2015-08-27 09:07:38 +02:00
Valeriy Ponomaryov
af6814f9ca Add possibility to filter back ends by snapshot support
On last midcycle meetup was decided to make snapshots optional feature.

Features:
 - Add new boolean capability 'snapshot_support' to base share driver
   so every existing share driver will inherit it. Make value of it be
   calculated from the fact of redefinition of three main driver
   methods for snapshots 'create_snapshot', 'delete_snapshot' and
   'create_share_from_snapshot'.
- Set extra spec 'snapshot_support' with share type creation by default
  to 'True'
- Restrict deletion of extra spec 'snapshot_support' that is expected
  to exist
- Allow to redefine new extra spec 'snapshot_support'
- Restrict API 'snapshot create' for share created with
  share type that has extra spec 'snapshot_support' equal to 'False'.
- Add migration where new extra spec 'snapshot_support' is added
  to all share types that do not have it yet.

Partially implements bp snapshots-optional
Change-Id: I069d9e911c7d7a708fa518b38ed10572a45e5f42
2015-08-26 23:05:37 +03:00
Ramana Raja
3178e70cb5 Add tempest tests for "cert" based access type
Closes Bug: #1486006

Change-Id: If774eabee53c32acd37ba1c49e7b64f4383cc1bc
2015-08-26 22:57:28 +05:30
Igor Malinovskiy
15fb464e5d Add availability zones support
Rework availability zones support which was inherited from Cinder:
- Add public API extension
- Preserve AZ if creating a share from a snapshot
- Always set AZ in Share API or Share Manager
- Update db schema and create db migration
- Update appropriate unit tests

APIImpact
Partially-Implements: blueprint availability-zones

Change-Id: Iea9fbc3fea5c0128772115c028989121f397e0c5
2015-08-26 18:48:16 +03:00
Goutham Pacha Ravi
2521b16d36 Add tempest tests for REST API microversions
This patch adds tempest tests that validate Manila's versions API. Since the
versions API is the first API to be versioned in Manila, the code in this
patch makes requests to invoke the versions API with valid and invalid
version codes and evaluates responses.

Change-Id: I4046f020f0e129dab639f99390367efda9cf4551
2015-08-25 19:48:17 +00:00
vponomaryov
d1c1c009f4 Fix Neutron config setting in pre_test_hook
Change-Id: I843416f091c6804f3a14581a1992ee2f119cbd4c
Closes-Bug: #1486501
2015-08-19 13:35:07 +03:00
Ramana Raja
ab4124cc38 Enable Tempest tests for glusterfs/hdfs protocols
Currently, Manila Tempest test cases support only testing of shares of
type nfs or cifs protocol. Allow testing of shares that use glusterfs
protocol or hdfs protocol.

Closes-Bug: #1485987

Change-Id: Ie7b1e9638ad2afaf8388cb4d7cda8d04e89a3c0a
2015-08-18 16:46:12 +05:30
Jenkins
4f7a95c403 Merge "Allow Tempest to skip snapshot tests" 2015-08-17 16:58:34 +00:00
marcusvrn
6229022845 Cleanup shares created by Tempest
Currently some tests that run successfully do not clean up shares
created, wasting space on backends that run CIs.

The root cause is that shares_client used directly for creation of shares
instead of using wrapper from base which will add them to "deletion" queue.

Change-Id: I8d80a5dd37be874c852b03e204547ed43d3843d2
Closes-bug: #1483863
2015-08-12 16:23:25 -03:00
Valeriy Ponomaryov
7b244b33b7 Allow Tempest to skip snapshot tests
On last midcycle meetup was decided to make snapshots optional feature.
So, at first step, add possibility to skip all functional tests that
use snapshots.

Partially implements bp snapshots-optional

Change-Id: Ib7dd3303e3974298439b6ac543b9df628b69e6ae
2015-08-12 08:49:27 +00:00