This patch adds support for migration of share servers. This
migration is performed using a two-phase approach. Administrators
are now able to request the migration of a share server within and
across backends, with the possibility of chooosing a different share
network for the destination share server.
- A new field called `task_state` was added to the share server
model in order to help the administrator to track the share server
migration steps. A new field called `source_share_server_id` was
added to link destination and source share servers.
- A new periodic task was added to track migration of share servers
and its resources.
- Two new states were added: `server_migrating` and
`server_migrating_to` to represent that share migration is in
progress.
- When performing the server migration, manila will not go to the
scheduler, instead it will provide a request spec to drivers
during migration check driver call. It'll be up to the driver
validate if there is free space to handle the share server.
- A new API called `share-server-migration-check' was added to
check the feasibility of a migration, before actually triggering
the start operation.
APIImpact
DocImpact
Partially Implements: bp share-server-migration
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Signed-off-by: Douglas Viroel <viroel@gmail.com>
When the image count is over 25, there might not get
manila-service-image, because current manila shares
creation is using novaclient to get image info, but
novaclient can only get 25 images due to pagination
of glance server, So this change is to switch to use
glanceclient instead of novaclient to get image info,
because glanceclient can iter all image info, while
novaclient is rarely maintained with stuff of image
API.
Change-Id: Id1715d0b9cb3a4aeedeb23d9b1d9924a78d18dc6
Closes-Bug: #1741425
After fixing the uwsgi installation from source that was broken[1][2],
Manila jobs started to fail since no uwsgi wasn't found in the
expected path. This fix now uses 'which' command to search for
uwsgi pathnames in the system.
[1] https://bugs.launchpad.net/devstack/+bug/1883468
[2] https://review.opendev.org/#/c/577955/
Related-Bug: #1883468
Closes-Bug: #1883715
Change-Id: I8d8b2fe07d86899c694cb73a81087d25311d30a5
If the cephfs protocol is enabled, clients
need to access the ceph daemons.
We also don't need to enable access to NFS
ports when not using NFS.
Change-Id: I077d12785f94c940716f0e479d43dbb29ddc3c3c
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
When running IPv6 tests, we disable the creation of
the public network in devstack, and handle it in the
devstack plugin. So, we'll need to configure it
in tempest in the plugin.
Change-Id: Icf5f26397e16d7fbfae77aa246ac5e35581f441e
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Devstack no longer installs this package after [1]. The driver
needs to replace the use of brctl to completely remove this
package from our test environments.
[1] https://review.opendev.org/724443/
Partial-Bug: #1876820
Change-Id: Id6094827341bf6ef8856cd4e7af11b36e9afb560
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
We currently use a legacy devstack-gate script
to create the bgp speaker, and peer required
to communicate to and from the tenant private
IPv6 networks.
Move this logic to the devstack plugin so that
we can invoke it automatically with the existing
devstack variable MANILA_SETUP_IPV6.
Change-Id: Iea90e3f04ae05e5783c3163bbf2d2dabd128c7b5
A fix was committed [1] to the python-openstackclient
project to handle VMs with multiple ports. We can
drop the workaround of setting a floating IP to the
VM's port.
[1] 013c9a4f3a
Change-Id: I67c38b4e6695c05a8272a86448ed248b492a279d
Closes-Bug: #1747721
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
To set up some first party backends such as
ZFSOnLinux, CephFS via NFS gateway, Container
(where the NAS server is containerized) and LVM,
manila's devstack plugin creates a NAS server
on the devstack host.
On test machines, access to this NAS server is
firewalled from networks outside of the host's
internal network namespace (including from private
project networks that are in different network
namespaces, on the same devstack host).
We currently use a legacy devstack-gate script
to disable firewall on NFS ports; however,
anyone that installs devstack with LVM, Container,
ZFSOnLinux, CephFS-NFS drivers will need these
firewall ports to be opened to be able to mount
shares exported off their devstack host machines.
Move these firewall commands to the devstack plugin.
These commands can be invoked by setting the localrc
variable MANILA_ALLOW_NAS_SERVER_PORTS_ON_HOST to True.
The value of this variable is False by default,
to preserve existing behavior.
Change-Id: Ic9cad47662f1edf2e5c710dbe64d580bc5f01d44
This patch enables the use scheduler creating share from
snapshot flag in order to run properly the manila tempest tests.
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
Change-Id: I15311089c45be1574857d46caa073e89424e716d
Today we either expose the CephFS back end via the CephFS native
protocol, in which case kernel NFS and Samba are not needed, or
by NFS-Ganesha, in which case Samba is not needed and kernel NFS
is incompatible (we just turn it off again later).
In the future someone could develop support for exposing CephFS via
Samba so we'd add specific checks for that then and install Samba.
Change-Id: Id65ce2536a3deb32a04a70bd0c49081297741965
It was deprecated in the Queens release, in
favor of ``data_node_access_ips``.
Change-Id: I01f24c2e0d66e1da4c30c579c02afc8f3930c8f8
Related-Bug: #1745436
configure_auth_token_middleware was replaced by
configure_keystone_authtoken_middleware in
Id0dec1ba72467cce5cacfcfdb2bc0af2bd3a3610. Manila's
DevStack plugin no longer needs to create a cache
directory to use as the signing_dir for keystone
authtoken.
Change-Id: I18fa04e6c3a832e17ade3f84d8b43b15686f06fe
manila-tempest-plugin can be installed with its
devstack plugin; Installing it via manila's plugin
is unnecessary. So, deprecate its installation
in the DevStack plugin.
Change-Id: I21c08069ff82b3bfb52ef7ac960183ddc866c2ee
Restoring the default IPv6 route is not needed in CI and
fails when there are multiple defaults, but it is useful
in local devstacks where multiple default routes are not
typical.
Add a variable in settings and use it to make this behavior
conditional and set it to False for the lvm job.
Closes-bug: #1836788
Change-Id: Id73de8100509ec5935641f5f35f93f482d108bcd
Switch manila to use devstack common functions
for deploying a wsgi app under uwsgi and apache.
This change aims to fulfill the community requirement
for all projects on OpenStack to switch devstack jobs
to deploy control-plane API services under uwsgi with
Apache acting as a front end proxy.
More details in https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
We used to deploy with mod_wsgi, but mod_wsgi has many
issues when being used in devstack for development or testing.
Now we leave mod_wsgi as optional and we default to uwsgi.
Implements: bp wsgi-web-servers-support
Depends-On: https://review.openstack.org/#/c/643188/
Change-Id: I12d4c864f2ceb0f3555e32d2dc893e00dbef0d96
Enabling tls-proxy allows devstack to
set up a tls proxy server that front-ends
interactions with the manila-api and
terminates tls connections.
Also enable tls-proxy in dummy and lvm
jobs. The dummy driver job is configured
to run the in-built wsgi server, the lvm
job is configured to use mod-wsgi.
Closes-Bug: #1816836
Change-Id: I48b0ccc082604d78242ba61bee94a45efeb2467b
Every enabled backend gets assigned to its own AZ.
Test cases in manila-tempest-plugin already exercise
creating shares (and replicas) across AZs when
multiple AZs are available.
This is done for all back-end drivers that are not
the Generic driver. Configuring backend AZ for each
Generic driver based back end will require configuring
Nova and Cinder in a similar fashion.
Partially-implements: bp per-backend-availability-zones
Depends-On: https://review.openstack.org/#/c/630886/
Depends-On: https://review.openstack.org/#/c/629958/
Change-Id: I1b6ff535a22f10e70d379377767c8ffac3ef5286
There might be a rescan for volume groups needed before checking if
the volume group is already there. Otherwise, the check for the volume
group fails which means the code tries to create the volume, but that
fails then because the volume is already there.
Here is the devstack run log:
[...]
plugin.sh:configure_backing_file:538 sudo losetup -f --show /opt/stack/data/lvm-shares-backing-file
plugin.sh:configure_backing_file:538 DEV=/dev/loop2
plugin.sh:configure_backing_file:543 sudo vgs lvm-shares
Volume group "lvm-shares" not found
Cannot process volume group lvm-shares
plugin.sh:configure_backing_file:543 sudo vgcreate lvm-shares /dev/loop2
Physical volume '/dev/loop2' is already in volume group 'lvm-shares'
Unable to add physical volume '/dev/loop2' to volume group 'lvm-shares'
/dev/loop2: physical volume not initialized.
plugin.sh:configure_backing_file:1 exit_trap
[...]
Change-Id: I0189580ae40b180249fd5846aa986052e44ce2c2
Generic driver jobs are failing because of timeouts when
establishing the initial ssh connection from manila-share
to the service VM.
Bump up the default value of the connection timeout for paramiko
client and also set the banner timeout since the failure occurred
during banner exchange. Set the two timeouts to the same value
for now. This ensures that the connection timeout is at least as
long as the banner timeout and there is no current need in manila
to control these independently.
This is more of a workaround than a real fix since a real fix
would remove the delay during banner exchange. I suspect that
the real fix will need to be in neutron/ovs though.
Change-Id: Ib5e59faaf9667b9cb5e7d4072531b7d6c3d4da39
Partial-bug: #1807216
- When the URL refers to cloning or using git repositories, use the
cloning URL (https://git.openstack.org/openstack/<project>)
- When the URL refers to the browsable version of the repository, use
the cgit frontend (https://git.openstack.org/cgit/openstack/<project>)
Change-Id: I11a8e15c6d663786465c090b40db9e20b19194e6
Pre-create config and log directories and
update Quagga service configuration.
This patch does not add Bionic support to the gate
jobs. https://review.openstack.org/#/c/608761/ does
the job configuration changes to run with Bionic.
Needed-By: https://review.openstack.org/#/c/608761/
Change-Id: I146ab5a41c06df2b2615ebdb38aaeb972db4c1d2
Avoid overwriting the log formatters, reuse
the helper available in Devstack.
Closes-Bug: #1787310
Change-Id: I25eb16ea9a6e3e98de6a7ee349df17438847d601
Make default share type and group type
creation optional in
manila-tempest-minimal-dsvm-dummy job.
Depends-On: https://review.openstack.org/#/c/540070/
Change-Id: I585c15d9eb4eefb843df62f7aad08838c83611e2
And don't enlist manila-tempest-plugin for installation in venv-tempest.
Change-Id: I0844cdb48b9d415bd341c4687620412178be871b
Signed-off-by: Rishabh Dave <ridave@redhat.com>
This patch makes necessary changes to devstack manila
plugin in order to be able to run IPv6 scenario tests.
Part of the changes included the dependency of
neutron-dynamic-routing plugin and the installation
of quagga, so routes in devstack host can be created
automatically for each router and private network
created by tempest during the tests.
Also, added a new config option "override_ip_for_nfs_access"
for manila tempest plugin that overrides the access rules used
for NFS scenario tests. This option can be set by exporting the
environment variable "OVERRIDE_IP_FOR_NFS_ACCESS" with the
intended value before running devstack.
This change is enabled by a following change and
tested alongside a change on manila-tempest-plugin
project.
Please note that we are temporarily disabling IPv6
functionality in Host-assisted Share Migration, as
the Data Service is not able to handle IPv4 + IPv6
scenarios.
Change-Id: I4ca727f92618998242af18908bbbda6bb5f86303
Needed-By: Id8b005cdd429d53a75624885fe7ca795746c3ede
This patch adds policy in code support for availability_zone,
scheduler_stats and message resources.
Change-Id: I9a79b5ececc583e85149cc920321e461e832b245
Partial-Implements: blueprint policy-in-code
Supporting dual IPv4/IPv6 backends will eventually require
multi-IP support in the first party backends. This change
enables multi-IP support (IPv4-only currently) in the LVM
driver.
Change-Id: Ib3594aa5d7751c829820fce830d87f6ceea6b049
Partial-Implements: blueprint support-ipv6-access
Currently, Manila's devstack plugin clone manilaclient. Some projects
depend on Manila but would like to install packaged manilaclient.
Clone the manilaclient only if it was marked to be used from git
Change-Id: Ia8e726b9358f6901c172a31f638a59eba523e0fa
Depends-On: I94d2f143159b60c4f8693e35fe844bc1e411e580
One of the goals for Pike [1] is to make each API service be able to
run under web servers that support WSGI applications,
such as Apache (+mod-wsgi) and Nginx (+uWSGI).
Do following to address governance requirements:
- Split existing manila/wsgi.py module into 3 modules:
First (manila/wsgi/eventlet_server.py) is used by
eventlet-based WSGI application approach.
Second (manila/wsgi/wsgi.py) is used for WSGI web servers.
And third (manila/wsgi/common.py) is common code for both.
All three are made in cinder-like way to have alike-approach.
- Reuse common code from "oslo_service/wsgi.py" module that
allows us to remove code duplication.
- Delete config opts that are defined by newly reused common code.
- Register new entry point that will be manila wsgi app: "manila-wsgi".
- Fix "manila/api/openstack/wsgi.py" module to be compatible
with str/bytes handling approach used by Apache mod-wsgi plugin using
different python versions (2/3).
- Add web server config template "devstack/apache-manila.template"
- Add devstack support where usage of this feature can be
enabled or disabled using "MANILA_USE_MOD_WSGI" env var.
It is set to "True" by default, because it is requirement for Pike
release - to have it running in all CI jobs.
Disable it only for one CI job that uses dummy driver and tests
various manila core features that are not covered by other CI jobs.
[1] https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
Partially-Implements BluePrint wsgi-web-servers-support
DocImpact
Change-Id: Ibdef3c6810b65a5d6f3611e2d0079c635ee523ab
In before, Manila devstack plugin create the service endpoint in
"extra" stage. This caused issue of other services that depends
on the manila endpoints that are not yet created. This commit
move the endpoint creation to "post-config" stage, which is the
practice of other OpenStack services
Change-Id: I3adfbf78d0792471a29c7ff979aad1418e6137d4
is running under systemd since the services are incompatible.
Skip the samba install and startup as well since there is
no current use case for running samba plus nfs-ganesha.
TrivialFix
Change-Id: I6d6fe3a3babe2f7fcd9ca9461785ac413ff4ab2c
This feature was disabled by default in Ocata.
Now, enable it back to ease further work and updates of this feaure.
Change-Id: Ib57b55ce77df1b8e6481057014f2e9f6d743e34d
Shell syntax is "else", not "else:". This fixes:
/opt/stack/manila/devstack/plugin.sh: line 647: else:: command not found
Change-Id: I1742a71a3c637c4224c15e670862bf1b47d4a95f
