Commit Graph

171 Commits

Author SHA1 Message Date
Goutham Pacha Ravi
2277c20c02 [devstack][ci] Move bgp setup to plugin
We currently use a legacy devstack-gate script
to create the bgp speaker, and peer required
to communicate to and from the tenant private
IPv6 networks.

Move this logic to the devstack plugin so that
we can invoke it automatically with the existing
devstack variable MANILA_SETUP_IPV6.

Change-Id: Iea90e3f04ae05e5783c3163bbf2d2dabd128c7b5
2020-05-01 16:51:45 +00:00
Goutham Pacha Ravi
36b1715e86 [devstack][ci] Modify firewall in ds-plugin
To set up some first party backends such as
ZFSOnLinux, CephFS via NFS gateway, Container
(where the NAS server is containerized) and LVM,
manila's devstack plugin creates a NAS server
on the devstack host.

On test machines, access to this NAS server is
firewalled from networks outside of the host's
internal network namespace (including from private
project networks that are in different network
namespaces, on the same devstack host).

We currently use a legacy devstack-gate script
to disable firewall on NFS ports; however,
anyone that installs devstack with LVM, Container,
ZFSOnLinux, CephFS-NFS drivers will need these
firewall ports to be opened to be able to mount
shares exported off their devstack host machines.

Move these firewall commands to the devstack plugin.
These commands can be invoked by setting the localrc
variable MANILA_ALLOW_NAS_SERVER_PORTS_ON_HOST to True.
The value of this variable is False by default,
to preserve existing behavior.

Change-Id: Ic9cad47662f1edf2e5c710dbe64d580bc5f01d44
2020-04-29 21:15:21 +00:00
andrebeltrami
8f8ddfd670 [ZFSonLinux] Create share from snapshot in different backends
This patch improves the operation of create share from snapshot
to support different backends/hosts for the zfsonlinux driver

Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend

Depends-On: Iab13a0961eb4a387a502246e5d4b79bc9046e04b

Change-Id: I124803734c81d3630c5147f5f3bb75724489c929
2020-04-09 21:16:06 +00:00
Douglas Viroel
d67dee1776 Enable replication tests (DHSS=True) on Dummy driver
This patch enables the replication test to run on Dummy driver
when configured with DHSS True. This change depends on subnets
implementation to properly work.

Change-Id: Ib6a44d8f0d8c56acd149b8d80549079f771ffbe0
Depends-On: Icb93fc346b71edc7bb3ac989e9751899bb80fbe5
2019-08-20 19:10:09 -03:00
Goutham Pacha Ravi
8d03a2a46e Make manila-tempest-plugin installation optional
manila-tempest-plugin can be installed with its
devstack plugin; Installing it via manila's plugin
is unnecessary. So, deprecate its installation
in the DevStack plugin.

Change-Id: I21c08069ff82b3bfb52ef7ac960183ddc866c2ee
2019-07-29 17:54:09 -07:00
Tom Barron
70e908344d Conditionally restore default route in setup_ipv6
Restoring the default IPv6 route is not needed in CI and
fails when there are multiple defaults, but it is useful
in local devstacks where multiple default routes are not
typical.

Add a variable in settings and use it to make this behavior
conditional and set it to False for the lvm job.

Closes-bug: #1836788
Change-Id: Id73de8100509ec5935641f5f35f93f482d108bcd
2019-07-24 19:09:27 +00:00
Tom Barron
d5bd526f11 [tests] Fix PYTHON3_VERSION
The pre test hook for gate tests has a faulty test
for whether the PYTHON3_VERSION variable is set and
as a result puts a blank value for this variable in
local.conf when it is not set instead of leaving it
out of local.conf altogether.  This causes jobs
running with USE_PYTHON3=True and no explict version
setting to actually run manila under python 2.7 because
that is what devstack itself defaults for pip installs
if the version is not set (or auto-detected in devstack
when local.conf does not contain the PYTHON3_VERSION
variable at all).

So correct the condition in our pre-test hook so that
it only inserts a PYTHON3_VERSION in local.conf if
that version has been supplied.

Closes-bug: #1822178
Change-Id: Ie37595bde17d18770d82fb7fd6475dcfab74a543
2019-03-28 20:02:14 +00:00
Victoria Martinez de la Cruz
bd4673100b Deploy manila with uwsgi on devstack
Switch manila to use devstack common functions
for deploying a wsgi app under uwsgi and apache.

This change aims to fulfill the community requirement
for all projects on OpenStack to switch devstack jobs
to deploy control-plane API services under uwsgi with
Apache acting as a front end proxy.

More details in https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html

We used to deploy with mod_wsgi, but mod_wsgi has many
issues when being used in devstack for development or testing.

Now we leave mod_wsgi as optional and we default to uwsgi.

Implements: bp wsgi-web-servers-support
Depends-On: https://review.openstack.org/#/c/643188/
Change-Id: I12d4c864f2ceb0f3555e32d2dc893e00dbef0d96
2019-03-14 06:25:24 +00:00
Zuul
b0c7acf871 Merge "Add manage/unmanage implementation to Container Driver" 2019-03-07 13:32:46 +00:00
Rodrigo Barbieri
d877b61c5e Add manage/unmanage of shares in DHSS=True
This patch adds Manage/Unmanage of share servers in
Manila. It also updates the Manage Share API to accept
a "share_server_id" parameter, and updates Unmanage
of Share and Snapshots API to allow unmanaging of
shares and snapshots in DHSS=True.

Managed share servers are not deleted automatically
by manila, and if a single share is unmanaged in
DHSS=True, the respective share server will not be
deleted automatically as well.

Managing share servers require that the driver
implements 2 functions:
- get_share_server_network_info: obtain IPs from
  share server.
- manage_server: perform required operations to
  manage and return dict of backend_details.

Unmanaging share servers require that the driver
overrides unmanage_server function.

The IPs obtained from the backend are validated
by the Network plugin, so ports with the exact
IPs must exist in the subnet and admin subnet
associated with the share network specified
when managing the share server (unless
StandaloneNetworkPlugin is used).

It is recommended to rename the backend resource
if possible when managing the share server, to
prevent issues with re-managing a share server
that has already been managed.

This patch bumps the API microversion to 2.49.

APIImpact
DocImpact

Depends-On: I17c74b2aa242918188eeff368232c762a4b31093
Partially-implements: bp manage-unmanage-with-share-servers
Change-Id: I108961e7436ba13550ef2b8f02079c6e599a6166
2019-03-05 16:03:29 -03:00
Rodrigo Barbieri
edf227ee27 Add manage/unmanage implementation to Container Driver
This patch implements managing and unmanaging of share
servers and shares functionality to the Container driver.

Depends-On: I452c2a99b186f53d737cb7fbd7eabfcfd9b249d6
Implements: bp container-driver-manage-unmanage-with-share-servers
Change-Id: I623c88dd1155740f0a444d2063236d91efd3bd1e
2019-03-04 18:35:06 +00:00
Goutham Pacha Ravi
22d25e834d Configure per backend availability zones in devstack
Every enabled backend gets assigned to its own AZ.
Test cases in manila-tempest-plugin already exercise
creating shares (and replicas) across AZs when
multiple AZs are available.

This is done for all back-end drivers that are not
the Generic driver. Configuring backend AZ for each
Generic driver based back end will require configuring
Nova and Cinder in a similar fashion.

Partially-implements: bp per-backend-availability-zones
Depends-On: https://review.openstack.org/#/c/630886/
Depends-On: https://review.openstack.org/#/c/629958/
Change-Id: I1b6ff535a22f10e70d379377767c8ffac3ef5286
2019-01-23 14:30:18 -08:00
Tom Barron
8f88779778 Convert dummy job to py3
Install all python packages for python3 and
run tempest with python 3.

Change-Id: If25075933b0b07a12814fda44ef5139a40c78f67
2019-01-08 19:05:53 +00:00
Tom Barron
6299ed5ed0 Fix image_name retrieval in custom-image jobs
And use a 'file://' url for the custom manila image.

See also https://review.openstack.org/#/c/623330.  These
are interdependent patches but no harm is done by merging
this one first and having 623330 depend on it since retrieval
of the image name is not working correctly anyways.

Partial-Bug: #1807969

Change-Id: Ib3a74d170f159dfbc38b5302f00800f3530f4921
2018-12-19 15:00:51 -05:00
Goutham Pacha Ravi
8c925198ae [LVM][IPv6] Quagga changes to support Bionic Beaver
Pre-create config and log directories and
update Quagga service configuration.

This patch does not add Bionic support to the gate
jobs. https://review.openstack.org/#/c/608761/ does
the job configuration changes to run with Bionic.

Needed-By: https://review.openstack.org/#/c/608761/
Change-Id: I146ab5a41c06df2b2615ebdb38aaeb972db4c1d2
2018-11-19 22:14:34 +00:00
Goutham Pacha Ravi
b1bf39412b [CI] Switch Xenial tempest jobs to Bionic Beaver
This change specifically ignores the grenade
job (manila-grenade), which is pending conversion
to zuulv3 because we need a way to set the default
python3 interpreter when running on Bionic Beaver.

Depends-On: https://review.openstack.org/#/c/605291/
Depends-On: https://review.openstack.org/#/c/605242/
Depends-On: https://review.openstack.org/#/c/605810/
Change-Id: Ic49d4ac5ac36dd3b3d909941434b2d2f104a10b9
2018-11-01 14:13:35 -07:00
Goutham Pacha Ravi
f9ae7aab05 Test share type per test suite changes
Make default share type and group type
creation optional in
manila-tempest-minimal-dsvm-dummy job.

Depends-On: https://review.openstack.org/#/c/540070/
Change-Id: I585c15d9eb4eefb843df62f7aad08838c83611e2
2018-08-06 17:05:04 +00:00
Goutham Pacha Ravi
ed95238da9 [CI] Don't set test config for API microversions if master
If the version's being bumped on manila,
the newer API changes cause test failures with
manila-tempest-plugin.

We could carefully craft a
manila-tempest-plugin changeset that fixes
any incongruous test assumptions and make the
manila change depend on the tempest changeset.
However, that may make developing the API harder
than it is today.

Change-Id: Ie97fb49b38b565dcc57332db8b42605216372aa8
Related-Bug: #1781671
2018-07-18 08:43:15 -07:00
Zuul
1c1cac3a67 Merge "Allow setting test API microversions in gate tests" 2018-07-17 18:13:49 +00:00
Goutham Pacha Ravi
2f42c984e5 Allow setting test API microversions in gate tests
Add MANILA_TEMPEST_MIN_API_MICROVERSION and
MANILA_TEMPEST_MAX_API_MICROVERSION to the ci
post_test_hook. These options default to
the values specified in the api_version_request.py
file.

Also add the API Version History to the api-ref.

Change-Id: I21b3a705986216afb0ce73ca1f92571dda757408
Closes-Bug: #1781671
2018-07-16 08:41:03 -07:00
Goutham Pacha Ravi
fb26e5211b Fix results capturing for the dummy driver
Tempest moved to running under stestr from
testr [1] and the subunit2html installation
has been altered as well [2].

[1] https://review.openstack.org/#/c/504345/
[2] https://review.openstack.org/#/c/534431/

Change-Id: I615fa6d4362f83f35e40a750bfcb3a96cc193f35
2018-07-11 15:10:33 -07:00
Goutham Pacha Ravi
a0947ab42d Fix post-execution for tempest tests
Manila's post_test_hook runs tempest after
populating tempest.conf with the requisite
options, the results from tempest are saved
prior to performing some post tasks, including
deciding to run tests again with a different
configuration if necessary
(Ex: manila-tempest-minimal-dsvm-dummy). The
result is then replayed at the end of the gate
job so it can be reported to gerrit as a
SUCCESS or a FAILURE.

Fix the mis-aligned command that saves the
test execution status to replay later.

Change-Id: I1578760697aeae6e20bf567c60fbcdee0a642bee
Closes-Bug: 1770183
2018-05-16 21:04:28 +00:00
zhangxuanyuan
e41d126452 Remove option standalone_network_plugin_ip_version
'standalone_network_plugin_ip_version' option has been
deprecated. This option has been replaced by
'network_plugin_ipv4_enabled' and
'network_plugin_ipv6_enabled' options.

Change-Id: I0251fecddac74578698bcda5d46e44ca16bd8a76
2018-03-27 06:32:26 -04:00
Zuul
4cb39b3ab0 Merge "Fetch and install manila-tempest-plugin system-wide" 2018-03-22 15:21:23 +00:00
Rishabh Dave
72f79e5c57 Fetch and install manila-tempest-plugin system-wide
And don't enlist manila-tempest-plugin for installation in venv-tempest.

Change-Id: I0844cdb48b9d415bd341c4687620412178be871b
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-03-08 17:35:30 +05:30
Nguyen Hung Phuong
0286c2051e Remove use of unsupported TEMPEST_SERVICES variable
TEMPEST_SERVICES global variable is not supported by devstack since long back.
- I380dd20e5ed716a0bdf92aa02c3730359b8136e4
- I9c24705e494689f09a885eb0a640efd50db33fcf

Service availability of tempest known services will be set by devstack with
local check.
- I02be777bf93143d946ccbb8e9eff637bfd1928d4

This commit removes the unused TEMPEST_SERVICES setting

Change-Id: I2c6f3cd67094002c415ce9d1018c0f08d9a2a22f
Related-Bug: #1743688
2018-02-22 14:35:40 +07:00
Rodrigo Barbieri
c4449bc4e2 Enable IPv6 scenario tests in Upstream CI
This change complements the work of preceding IPv6 scenario
test changes [1,2] by enabling the settings that allow the tests
to be run in the Upstream CI.

[1] I4ca727f92618998242af18908bbbda6bb5f86303
[2] Id8b005cdd429d53a75624885fe7ca795746c3ede

Change-Id: I705b7d9cbc14d4bbc4d89b4c343205908758a5cb
Depends-on: Id8b005cdd429d53a75624885fe7ca795746c3ede
Depends-on: I4ca727f92618998242af18908bbbda6bb5f86303
2018-01-24 23:10:09 +00:00
Rodrigo Barbieri
dc93ae534c Update manila plugin to support IPv6
This patch makes necessary changes to devstack manila
plugin in order to be able to run IPv6 scenario tests.

Part of the changes included the dependency of
neutron-dynamic-routing plugin and the installation
of quagga, so routes in devstack host can be created
automatically for each router and private network
created by tempest during the tests.

Also, added a new config option "override_ip_for_nfs_access"
for manila tempest plugin that overrides the access rules used
for NFS scenario tests. This option can be set by exporting the
environment variable "OVERRIDE_IP_FOR_NFS_ACCESS" with the
intended value before running devstack.

This change is enabled by a following change and
tested alongside a change on manila-tempest-plugin
project.

Please note that we are temporarily disabling IPv6
functionality in Host-assisted Share Migration, as
the Data Service is not able to handle IPv4 + IPv6
scenarios.

Change-Id: I4ca727f92618998242af18908bbbda6bb5f86303
Needed-By: Id8b005cdd429d53a75624885fe7ca795746c3ede
2018-01-24 17:11:22 -02:00
Raissa Sarmento
797048c5ce Remove in-tree tempest plugin
Remove in-tree tempest plugin in favor of newly
created manila-tempest-plugin repository.

Change-Id: I1fb124598f38067fee469df124ee684f748c6f57
2017-12-22 11:11:38 +00:00
Raissa Sarmento
d482b16ca3 Use generic user for both zuul v2 and v3
Zuul v2 uses 'jenkins' as user, but Zuul v3 uses 'zuul'.
Using $USER solves it for both cases.

Change-Id: Ib83267ef75f91fb8f95af312d644ebe19c6307c9
2017-10-13 15:13:16 +01:00
Tom Barron
345021d7ea Remove tempest pin
Change-Id: Ia402fb79e1de5c9925f575d635b9928475095f86
2017-08-04 14:05:10 -04:00
Raissa Sarmento
07fb60155f Update location of dynamic creds in tempest tests
Tempest is migrating their dynamic credentials interface to tempest
lib as part of the effort to create stable interfaces. [0] We will
need to update its location in our code to conform with it.

[0] http://lists.openstack.org/pipermail/openstack-dev/2017-July/119934.html

Depends-On: Id8d60e4d7c22bc72c3e48bc768509ff0cc0c89d5

Change-Id: Iaa6bf908ff5d63cbbfddc7060594818c66ce607c
2017-07-24 12:57:06 +01:00
Tom Barron
39fb6cc23a CI: Update tempest commit
Manila pins the version of tempest it tests on the gate.
Update the pin to tempest verstion to date.

TrivialFix

Change-Id: I7bd51c1c5365ffd4247e46565fe7d3f74b5ef701
2017-06-07 12:14:39 -04:00
Valeriy Ponomaryov
21699451f1 [Share groups] Add scheduler filter ConsistentSnapshotFilter
That will be used for scheduling share groups based on their possibility
to create consistent snapshots.

Also apply following tempest plugin changes:
- Add new 'capability_sg_consistent_snapshot_support' tempest config
option, that will be used for creation of new share group types and used
to prove that scheduling works as expected.
- Fix some share group test attributes from 'only API involved' to
  'API and Backend are involved', because it is so indeed.

Change-Id: I05553c308ae40c4ddc2c6469ff1c1a3da36a87da
Partially-Implements BP manila-share-groups
2017-06-02 17:48:05 +03:00
Goutham Pacha Ravi
76e569a616 CI: Update tempest commit
Manila pins the version of tempest it tests on the gate.
This is antithetical to the tempest plugin mechanism. So,
our gate breaks when some plugin relies on code from tempest
that is not part of the commit manila pins in the CI.

While we are working towards a permanent fix to this problem,
let's update our pin as a workaround.

Closes-Bug: #1693832

Change-Id: Ide131229efdea5a48dc389449602f2b0abb699f7
2017-05-26 15:35:51 +00:00
Valeriy Ponomaryov
16bfc82962 Add possibility to run 'manila-api' with wsgi web servers
One of the goals for Pike [1] is to make each API service be able to
run under web servers that support WSGI applications,
such as Apache (+mod-wsgi) and Nginx (+uWSGI).

Do following to address governance requirements:
- Split existing manila/wsgi.py module into 3 modules:
  First (manila/wsgi/eventlet_server.py) is used by
  eventlet-based WSGI application approach.
  Second (manila/wsgi/wsgi.py) is used for WSGI web servers.
  And third (manila/wsgi/common.py) is common code for both.
  All three are made in cinder-like way to have alike-approach.
- Reuse common code from "oslo_service/wsgi.py" module that
  allows us to remove code duplication.
- Delete config opts that are defined by newly reused common code.
- Register new entry point that will be manila wsgi app: "manila-wsgi".
- Fix "manila/api/openstack/wsgi.py" module to be compatible
  with str/bytes handling approach used by Apache mod-wsgi plugin using
  different python versions (2/3).
- Add web server config template "devstack/apache-manila.template"
- Add devstack support where usage of this feature can be
  enabled or disabled using "MANILA_USE_MOD_WSGI" env var.
  It is set to "True" by default, because it is requirement for Pike
  release - to have it running in all CI jobs.
  Disable it only for one CI job that uses dummy driver and tests
  various manila core features that are not covered by other CI jobs.

[1] https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html

Partially-Implements BluePrint wsgi-web-servers-support
DocImpact
Change-Id: Ibdef3c6810b65a5d6f3611e2d0079c635ee523ab
2017-04-10 12:49:20 +03:00
Jenkins
7408444d0b Merge "Update to current tempest tag" 2017-04-05 11:49:45 +00:00
vponomaryov
2668da7bf0 [CI] Add support for CI jobs with custom images
Manila project uses special images for some of its share drivers.
So, add possibility to configure manila and run tempest tests using
custom image that was built with proposed changes to
'manila-image-elements' project.
Supported share drivers are 'generic' and 'container'.

Change-Id: I005d1b204ea25ced8913351d19eac282ca465fb6
2017-04-04 17:46:09 +03:00
Tom Barron
423fd16671 Update to current tempest tag
Change-Id: I8f93ecf12bbe309a4156a2583bd6f57187f36607
2017-04-01 06:17:49 -04:00
Jenkins
a7c7439ddc Merge "Remove redundant revert-to-snapshot test option" 2017-03-18 11:25:10 +00:00
Valeriy Ponomaryov
9ea065ebea Enable share groups back
This feature was disabled by default in Ocata.
Now, enable it back to ease further work and updates of this feaure.

Change-Id: Ib57b55ce77df1b8e6481057014f2e9f6d743e34d
2017-03-14 16:51:12 +02:00
Valeriy Ponomaryov
0b3fce7a04 [Grenade] Fix devstack configuration in CI hook
Commit [1] fixed approach for setting devstack config according to
changes [2] in devstack. But it started updating only 1 of 2 possible
devstack directories that are 'old' and 'new'. Where 'old' is the one
that is used only in 'grenade' job as base installation for performing
project migration. 'new' is used in all other single-devstack
installations. So, with [1] everything started working except
'grenade' job.
Fix grenade job by copying devstack config to 'old' dir if it exists.

[1] Idfa8e0cb3f446361daa1449e0854d95cfb379f46
[2] http://lists.openstack.org/pipermail/openstack-dev/2017-February/112872.html

Change-Id: Ic00a1fe296fa68a65672fa680ebd8a9336ed7cef
2017-02-27 18:46:07 +02:00
Ben Swartzlander
8e791b4e0e Change tempest tag to 15.0.0
Change-Id: Iaf03059a44d08236b6de14e4dc18f2ba459e9707
2017-02-24 15:14:41 -05:00
Ben Swartzlander
0ec5196caa Fix gate breakage caused by localrc usage
It's no longer allowed to use both localrc and local.conf
so our dsvm jobs were failing very early. See:
http://lists.openstack.org/pipermail/openstack-dev/2017-February/112872.html

Change-Id: Idfa8e0cb3f446361daa1449e0854d95cfb379f46
2017-02-24 10:04:16 -05:00
Rodrigo Barbieri
30403d05bf Remove redundant revert-to-snapshot test option
This option is used to control how revert-to-snapshot
extra-spec is used when creating a share type for several tests,
but for the same tests, it is more appropriate to either not
create a share-type (thus use the default or "don't care"
behavior) or do so based on the value of
run_revert_to_snapshot_tests.

TrivialFix

Change-Id: Ie8a0030a896634eb8deaff17ea72371419eecdea
2017-02-20 12:49:00 -03:00
Tom Barron
beda4a88ff Update tempest pin to 15.0.0
Change-Id: I10fa08d4831a59e858c393f86f680f88474c7ea3
2017-02-19 08:13:05 -05:00
Tom Barron
4cfdbea447 Update tempest pin to latest commit ref
Also updated invocation of DynamicCredentialsProvider to supply
necessary parameters.

Change-Id: Ie24bc69402964af3d756c1548aed86cc53c272c4
2017-02-10 15:04:57 +00:00
Jenkins
7c2c97d725 Merge "Disable share groups APIs by default" 2017-02-07 00:42:03 +00:00
Valeriy Ponomaryov
8ac4f0a913 Disable share groups APIs by default
as of Ocata because feature has not been completed.
Revert this change back in Pike.

Change-Id: Ibe2f7fba436e298b720a492594579d64b2e81a47
Closes-Bug: #1661639
2017-02-06 14:11:27 +02:00
yogesh
301abe6257 Improve test coverage for share migration
- Added tests to validate the migration functionality
for a replicated share.
- Added tests to validate the extend and shrink
functionality on migrated share.
- Added tests to cover the preserve snapshot
functionality on migrated share.
- Current waiter logic in migration_progress
method of dummy driver causes every migration
test to take over 4 minutes to complete.
Therefore, updated the method to cut the
overall time in half and avoid the possible timeout.

Partially-Implements: bp ocata-migration-improvements
Change-Id: Ia7c7eb7b48b464aa670073e8c0795369bd972678
2017-02-03 23:27:34 -05:00