146 Commits

Author SHA1 Message Date
Nguyen Hung Phuong
0286c2051e Remove use of unsupported TEMPEST_SERVICES variable
TEMPEST_SERVICES global variable is not supported by devstack since long back.
- I380dd20e5ed716a0bdf92aa02c3730359b8136e4
- I9c24705e494689f09a885eb0a640efd50db33fcf

Service availability of tempest known services will be set by devstack with
local check.
- I02be777bf93143d946ccbb8e9eff637bfd1928d4

This commit removes the unused TEMPEST_SERVICES setting

Change-Id: I2c6f3cd67094002c415ce9d1018c0f08d9a2a22f
Related-Bug: #1743688
2018-02-22 14:35:40 +07:00
Rodrigo Barbieri
c4449bc4e2 Enable IPv6 scenario tests in Upstream CI
This change complements the work of preceding IPv6 scenario
test changes [1,2] by enabling the settings that allow the tests
to be run in the Upstream CI.

[1] I4ca727f92618998242af18908bbbda6bb5f86303
[2] Id8b005cdd429d53a75624885fe7ca795746c3ede

Change-Id: I705b7d9cbc14d4bbc4d89b4c343205908758a5cb
Depends-on: Id8b005cdd429d53a75624885fe7ca795746c3ede
Depends-on: I4ca727f92618998242af18908bbbda6bb5f86303
2018-01-24 23:10:09 +00:00
Rodrigo Barbieri
dc93ae534c Update manila plugin to support IPv6
This patch makes necessary changes to devstack manila
plugin in order to be able to run IPv6 scenario tests.

Part of the changes included the dependency of
neutron-dynamic-routing plugin and the installation
of quagga, so routes in devstack host can be created
automatically for each router and private network
created by tempest during the tests.

Also, added a new config option "override_ip_for_nfs_access"
for manila tempest plugin that overrides the access rules used
for NFS scenario tests. This option can be set by exporting the
environment variable "OVERRIDE_IP_FOR_NFS_ACCESS" with the
intended value before running devstack.

This change is enabled by a following change and
tested alongside a change on manila-tempest-plugin
project.

Please note that we are temporarily disabling IPv6
functionality in Host-assisted Share Migration, as
the Data Service is not able to handle IPv4 + IPv6
scenarios.

Change-Id: I4ca727f92618998242af18908bbbda6bb5f86303
Needed-By: Id8b005cdd429d53a75624885fe7ca795746c3ede
2018-01-24 17:11:22 -02:00
Raissa Sarmento
797048c5ce Remove in-tree tempest plugin
Remove in-tree tempest plugin in favor of newly
created manila-tempest-plugin repository.

Change-Id: I1fb124598f38067fee469df124ee684f748c6f57
2017-12-22 11:11:38 +00:00
Raissa Sarmento
d482b16ca3 Use generic user for both zuul v2 and v3
Zuul v2 uses 'jenkins' as user, but Zuul v3 uses 'zuul'.
Using $USER solves it for both cases.

Change-Id: Ib83267ef75f91fb8f95af312d644ebe19c6307c9
2017-10-13 15:13:16 +01:00
Tom Barron
345021d7ea Remove tempest pin
Change-Id: Ia402fb79e1de5c9925f575d635b9928475095f86
2017-08-04 14:05:10 -04:00
Raissa Sarmento
07fb60155f Update location of dynamic creds in tempest tests
Tempest is migrating their dynamic credentials interface to tempest
lib as part of the effort to create stable interfaces. [0] We will
need to update its location in our code to conform with it.

[0] http://lists.openstack.org/pipermail/openstack-dev/2017-July/119934.html

Depends-On: Id8d60e4d7c22bc72c3e48bc768509ff0cc0c89d5

Change-Id: Iaa6bf908ff5d63cbbfddc7060594818c66ce607c
2017-07-24 12:57:06 +01:00
Tom Barron
39fb6cc23a CI: Update tempest commit
Manila pins the version of tempest it tests on the gate.
Update the pin to tempest verstion to date.

TrivialFix

Change-Id: I7bd51c1c5365ffd4247e46565fe7d3f74b5ef701
2017-06-07 12:14:39 -04:00
Valeriy Ponomaryov
21699451f1 [Share groups] Add scheduler filter ConsistentSnapshotFilter
That will be used for scheduling share groups based on their possibility
to create consistent snapshots.

Also apply following tempest plugin changes:
- Add new 'capability_sg_consistent_snapshot_support' tempest config
option, that will be used for creation of new share group types and used
to prove that scheduling works as expected.
- Fix some share group test attributes from 'only API involved' to
  'API and Backend are involved', because it is so indeed.

Change-Id: I05553c308ae40c4ddc2c6469ff1c1a3da36a87da
Partially-Implements BP manila-share-groups
2017-06-02 17:48:05 +03:00
Goutham Pacha Ravi
76e569a616 CI: Update tempest commit
Manila pins the version of tempest it tests on the gate.
This is antithetical to the tempest plugin mechanism. So,
our gate breaks when some plugin relies on code from tempest
that is not part of the commit manila pins in the CI.

While we are working towards a permanent fix to this problem,
let's update our pin as a workaround.

Closes-Bug: #1693832

Change-Id: Ide131229efdea5a48dc389449602f2b0abb699f7
2017-05-26 15:35:51 +00:00
Valeriy Ponomaryov
16bfc82962 Add possibility to run 'manila-api' with wsgi web servers
One of the goals for Pike [1] is to make each API service be able to
run under web servers that support WSGI applications,
such as Apache (+mod-wsgi) and Nginx (+uWSGI).

Do following to address governance requirements:
- Split existing manila/wsgi.py module into 3 modules:
  First (manila/wsgi/eventlet_server.py) is used by
  eventlet-based WSGI application approach.
  Second (manila/wsgi/wsgi.py) is used for WSGI web servers.
  And third (manila/wsgi/common.py) is common code for both.
  All three are made in cinder-like way to have alike-approach.
- Reuse common code from "oslo_service/wsgi.py" module that
  allows us to remove code duplication.
- Delete config opts that are defined by newly reused common code.
- Register new entry point that will be manila wsgi app: "manila-wsgi".
- Fix "manila/api/openstack/wsgi.py" module to be compatible
  with str/bytes handling approach used by Apache mod-wsgi plugin using
  different python versions (2/3).
- Add web server config template "devstack/apache-manila.template"
- Add devstack support where usage of this feature can be
  enabled or disabled using "MANILA_USE_MOD_WSGI" env var.
  It is set to "True" by default, because it is requirement for Pike
  release - to have it running in all CI jobs.
  Disable it only for one CI job that uses dummy driver and tests
  various manila core features that are not covered by other CI jobs.

[1] https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html

Partially-Implements BluePrint wsgi-web-servers-support
DocImpact
Change-Id: Ibdef3c6810b65a5d6f3611e2d0079c635ee523ab
2017-04-10 12:49:20 +03:00
Jenkins
7408444d0b Merge "Update to current tempest tag" 2017-04-05 11:49:45 +00:00
vponomaryov
2668da7bf0 [CI] Add support for CI jobs with custom images
Manila project uses special images for some of its share drivers.
So, add possibility to configure manila and run tempest tests using
custom image that was built with proposed changes to
'manila-image-elements' project.
Supported share drivers are 'generic' and 'container'.

Change-Id: I005d1b204ea25ced8913351d19eac282ca465fb6
2017-04-04 17:46:09 +03:00
Tom Barron
423fd16671 Update to current tempest tag
Change-Id: I8f93ecf12bbe309a4156a2583bd6f57187f36607
2017-04-01 06:17:49 -04:00
Jenkins
a7c7439ddc Merge "Remove redundant revert-to-snapshot test option" 2017-03-18 11:25:10 +00:00
Valeriy Ponomaryov
9ea065ebea Enable share groups back
This feature was disabled by default in Ocata.
Now, enable it back to ease further work and updates of this feaure.

Change-Id: Ib57b55ce77df1b8e6481057014f2e9f6d743e34d
2017-03-14 16:51:12 +02:00
Valeriy Ponomaryov
0b3fce7a04 [Grenade] Fix devstack configuration in CI hook
Commit [1] fixed approach for setting devstack config according to
changes [2] in devstack. But it started updating only 1 of 2 possible
devstack directories that are 'old' and 'new'. Where 'old' is the one
that is used only in 'grenade' job as base installation for performing
project migration. 'new' is used in all other single-devstack
installations. So, with [1] everything started working except
'grenade' job.
Fix grenade job by copying devstack config to 'old' dir if it exists.

[1] Idfa8e0cb3f446361daa1449e0854d95cfb379f46
[2] http://lists.openstack.org/pipermail/openstack-dev/2017-February/112872.html

Change-Id: Ic00a1fe296fa68a65672fa680ebd8a9336ed7cef
2017-02-27 18:46:07 +02:00
Ben Swartzlander
8e791b4e0e Change tempest tag to 15.0.0
Change-Id: Iaf03059a44d08236b6de14e4dc18f2ba459e9707
2017-02-24 15:14:41 -05:00
Ben Swartzlander
0ec5196caa Fix gate breakage caused by localrc usage
It's no longer allowed to use both localrc and local.conf
so our dsvm jobs were failing very early. See:
http://lists.openstack.org/pipermail/openstack-dev/2017-February/112872.html

Change-Id: Idfa8e0cb3f446361daa1449e0854d95cfb379f46
2017-02-24 10:04:16 -05:00
Rodrigo Barbieri
30403d05bf Remove redundant revert-to-snapshot test option
This option is used to control how revert-to-snapshot
extra-spec is used when creating a share type for several tests,
but for the same tests, it is more appropriate to either not
create a share-type (thus use the default or "don't care"
behavior) or do so based on the value of
run_revert_to_snapshot_tests.

TrivialFix

Change-Id: Ie8a0030a896634eb8deaff17ea72371419eecdea
2017-02-20 12:49:00 -03:00
Tom Barron
beda4a88ff Update tempest pin to 15.0.0
Change-Id: I10fa08d4831a59e858c393f86f680f88474c7ea3
2017-02-19 08:13:05 -05:00
Tom Barron
4cfdbea447 Update tempest pin to latest commit ref
Also updated invocation of DynamicCredentialsProvider to supply
necessary parameters.

Change-Id: Ie24bc69402964af3d756c1548aed86cc53c272c4
2017-02-10 15:04:57 +00:00
Jenkins
7c2c97d725 Merge "Disable share groups APIs by default" 2017-02-07 00:42:03 +00:00
Valeriy Ponomaryov
8ac4f0a913 Disable share groups APIs by default
as of Ocata because feature has not been completed.
Revert this change back in Pike.

Change-Id: Ibe2f7fba436e298b720a492594579d64b2e81a47
Closes-Bug: #1661639
2017-02-06 14:11:27 +02:00
yogesh
301abe6257 Improve test coverage for share migration
- Added tests to validate the migration functionality
for a replicated share.
- Added tests to validate the extend and shrink
functionality on migrated share.
- Added tests to cover the preserve snapshot
functionality on migrated share.
- Current waiter logic in migration_progress
method of dummy driver causes every migration
test to take over 4 minutes to complete.
Therefore, updated the method to cut the
overall time in half and avoid the possible timeout.

Partially-Implements: bp ocata-migration-improvements
Change-Id: Ia7c7eb7b48b464aa670073e8c0795369bd972678
2017-02-03 23:27:34 -05:00
Jenkins
40a030bf5d Merge "Enable host-assisted migration in ZFSOnLinux CI" 2017-02-02 16:02:48 +00:00
Rodrigo Barbieri
31ea6a3c14 Enable host-assisted migration in ZFSOnLinux CI
ZFSOnLinux CI is able to run host-assisted migration tests,
so enable them.

TrivialFix

Change-Id: I66578bad361867d1f698e39ceef07d94038c30c2
2017-01-31 09:02:05 -02:00
Rodrigo Barbieri
c5feb3b684 Decrease share migration periodic task interval
Improve CI execution time by decreasing the interval
to wait for next 'migration_continue' driver call.

TrivialFix

Change-Id: I6134d3b0cc05ab17837639f1bc209d5f3ed99658
2017-01-31 08:56:10 -02:00
tpsilva
0549f016aa Mountable snapshots scenario tests
This patch adds the scenario tests for mountable snapshots.

Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Lucio Seki <luciomitsuru.seki@fit-tecnologia.org.br>

Change-Id: Idb2eb5ee18ce55edb77545bcdf4df4ec4dd90135
Partially-implements: bp manila-mountable-snapshots
2017-01-25 12:42:48 -02:00
Jenkins
dbd8f5dc95 Merge "[Tempest] Add functional tests for share groups feature" 2017-01-25 07:49:25 +00:00
tpsilva
8d71932c69 Add mountable snapshots support
This new feature gives the user the ability to allow and
deny access to the snapshots, so that they could be mounted in
read-only mode to retrieve files.

APIImpact
DocImpact

Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Alyson Rosa <alyson.rosa@fit-tecnologia.org.br>
Co-Authored-By: Miriam Yumi <miriam.peixoto@fit-tecnologia.org.br>

Partially-implements: blueprint manila-mountable-snapshots
Change-Id: I65f398a05f82eef31ec317d70dfa101483b44b30
2017-01-24 17:26:45 -02:00
Andrew Kerr
1002642674 [Tempest] Add functional tests for share groups feature
Add functional tempest tests to new 'share group' feature [1].

[1] I79a80a62ae4e0015d6161edc2b93fd1f9ba69537

Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Co-Authored-By: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Partially-implements-blueprint: manila-share-groups
Depends-On: I8e29baed62355fc31caeec9c7a66eaebfcbdf184
Change-Id: I820eb959082995d961b1be992e4b2d1d1a985c1c
2017-01-23 19:30:33 +00:00
Alex Meade
d25f101ab4 Manila Share Groups
Remove the experimental consistency group APIs and
replace them with the experimental Share Group APIs.

DocImpact
APIImpact
Partially-implements-blueprint: manila-share-groups

Change-Id: I79a80a62ae4e0015d6161edc2b93fd1f9ba69537
2017-01-23 21:29:12 +02:00
Rodrigo Barbieri
53539c0e1d Share Migration Ocata Improvements
Implemented several improvements to share migration
according to spec [1].

Summary of changes:
- Snapshot restriction in API has been changed to return error only
when parameter force-host-assisted-migration is True
- Added preserve_snapshot to API and migration_check_compatibility
driver interface
- Changed all driver-assisted API parameters to be mandatory
- Added validation to prevent 'force_host_assisted_migration' to be
used alongside driver-assisted parameters
- Changed "same host" validation to reject only if the combination
of "host", "new_share_network" and "new_share_type" is the same as
the source
- Updated migration driver interfaces to support snapshots
- Updated zfsonlinux driver, defaulting preserve_snapshots to False
- Updated dummy driver to support preserve_snapshots

Spec update with latest changes since [1] merged
can be found in [2].

APIImpact
DocImpact

[1] I5717e902373d79ed0d55372afdedfaa98134c24e
[2] If02180ec3b5ae05c9ff18c9f5a054c33f13edcdf

Change-Id: I764b389816319ed0ac5178cadbf809cb632035b4
Partially-implements: blueprint ocata-migration-improvements
2017-01-20 09:45:14 -02:00
Clinton Knight
d4a379d083 Implement share revert to snapshot
This commit adds the ability for Manila to revert a
share to the latest available snapshot.

The feature is implemented in the LVM driver, for
testing purposes.

APIImpact
DocImpact
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Implements: blueprint manila-share-revert-to-snapshot
Change-Id: Id497e13070e0003db2db951526a52de6c2182cca
2017-01-17 15:06:01 -05:00
Goutham Pacha Ravi
733fcfe0fb Fix test variable injection in CI
* Migration is not a multi-backend only feature.
* Snapshot tests, Shrink tests and Quota test flags
  should have appropriate devstack variables.

Closes-Bug: 1652150

Change-Id: I7a83981045c03bcc605ec0119a4b425fb5f9e987
2016-12-27 16:39:27 -08:00
Clinton Knight
0d6db3588c Add create_share_from_snapshot_support extra spec
The snapshot_support extra spec has always meant two
things: a driver can take snapshots and create shares
from snapshots. As we add alternate snapshot semantics,
it is likely that some drivers will want to support
snapshots and some of the new semantics while being
unable to create new shares from snapshots.

This work adds a new extra spec,
create_share_from_snapshot_support, that removes the
overloading on snapshot_support. It also makes the
existing snapshot_support extra spec optional,
allowing admins to create types without setting
snapshot_support; shares created with such types
will not support snapshots.

APIImpact
DocImpact

Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Implements: blueprint add-create-share-from-snapshot-extra-spec
Change-Id: Ib0ad5fbfdf6297665c208149b08c8d21b3c232be
2016-12-21 09:57:08 -05:00
Alexey Ovchinnikov
c19dbbd2e6 Enable scenario tests for LVM and ZFSonLinux drivers
In addition to functional tests.

Also, make some corrections to scenario tests code to make it work
with shares located on localhost.

Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Co-Authored-By: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Change-Id: Ice61a7593f5e4e443cffbe78ecd71d683a9c274e
2016-12-13 12:53:03 +00:00
Rodrigo Barbieri
6168f16c25 Fix Manila service image config for 3rd party CIs
3rd party CIs cannot set this config by exporting an env var
or local.conf setting, as it is always being overriden in code.

Change-Id: Ia2a4790d3a0d03faa231b400c32f6c1928bdca1f
Closes-bug: #1649249
2016-12-12 09:14:57 -02:00
Valeriy Ponomaryov
8c5ed29ba6 [Devstack] Fix DHSS=False setup for Generic driver
Recent change to devstack [1] broke our DHSS=False CI job running
generic driver that was depending on ip route from host to
private network. So, to avoid this error use floating ip address
for connection to service Nova VMs from host machine.
Also, fix generation of second export location that should be dedicated
for service needs such as mounting share doing host-assisted
migration.

[1] If45e3fc15c050cfbac11b57c1eaf137dd7ed816f

Change-Id: Ieea992293ae02898741c939da15f0dbb4609d8b0
Closes-Bug: #1644523
Closes-Bug: #1646097
2016-12-02 14:33:56 +03:00
Valeriy Ponomaryov
899941c60e Remove fake CG support from Generic share driver
Generic share driver has fake support of consistency groups. It was
implemented only for testing purpose. Now, it is redundant, because
we have 'dummy' share driver that already used to test this behavior.

So, drop fake CG support from Generic share driver and disable
appropriate tempest tests.

Change-Id: I6ce07fd3a11cd62a3a01ba4ee7c424b839a62757
Closes-Bug: #1638994
2016-11-04 16:41:54 +02:00
Alexey Ovchinnikov
a915b75d00 Fix concurrency issues in container driver
Previously container driver failed in concurrent environments
deleting shares and share servers.
Fix it by adding shared external lock.

Change-Id: I543fd1226b3d99e06c736f4d925991d0bda7ea6d
Closes-Bug: 1613676
2016-10-20 14:52:50 +03:00
Valeriy Ponomaryov
5be1e8f3d3 [Grenade] Update devstack and pre_test_hook
Make devstack grenade code use 'newton' as base branch for migration to
master (ocata).
After switch [1] to dummy driver we need to override defaults.
So, update pre_test_hook approach for saving configuration data properly.

[1] I2b698f69a29ef983b13534b2f4d393570be4dac8

Change-Id: Ie5fa0f78b0e3caa06d34a0822a8ed60b69406f9d
Closes-Bug: #1631412
2016-10-17 13:42:26 +03:00
Jenkins
4d54f96f96 Merge "Improve Share Migration tempest tests" 2016-09-14 21:08:11 +00:00
Rodrigo Barbieri
053acc8db1 Improve Share Migration tempest tests
Improve coverage by adding tests that validate the share-type
change while also changing the driver mode.

Closes-bug: #1620800

Change-Id: I924c34aa69591754b437d75f43db91d77e73fb07
2016-09-13 13:17:07 -03:00
Jenkins
29c05e72f5 Merge "[ZFSonLinux] Fix share migration support" 2016-09-10 15:47:07 +00:00
Valeriy Ponomaryov
a5c2297ded [ZFSonLinux] Fix share migration support
Current implementation of share migration works only if we use
local shell executor. So, refactor approach for it to make work
with both shell executors - local and remote (SSH).

Change-Id: Ibe31bf52006040ea26badfa7236ed67559f7aded
Closes-Bug: #1620502
2016-09-07 18:36:55 +03:00
Jenkins
368093fa64 Merge "[ZFSonLinux] Add share migration support" 2016-09-01 13:56:54 +00:00
Valeriy Ponomaryov
754750a11d [Dummy driver] Add share migration support
Add 'fake' share migration support to 'dummy' driver
for ease of interfaces testing.

Change-Id: Ifd444e38a9678e98c156b7ce658aa1b50c13b24c
2016-09-01 05:45:27 +00:00
Valeriy Ponomaryov
976e2d58cb [ZFSonLinux] Add share migration support
Add support of share migration feature to ZFsonLinux driver.

Implements BP zfsonlinux-share-migration

Change-Id: If9e1fec9a6d4f96d2bb5c176c6249fafc7e3e891
2016-09-01 05:45:18 +00:00