MANILA_INSTALL_TEMPEST_PLUGIN_SYSTEMWIDE is an
unused variable since 3b6ea369044f4bade319bdf3334d5b855e0b74ba.
This variable only makes sense in the stable/wallaby and
older branches and is recommended to be set to False;
the correct way of getting the tempest plugin installed
would be to use the "TEMPEST_PLUGINS" devstack setting.
Change-Id: I2d94d780d879053f0fa739ba61f77f956ec84cd4
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Ubuntu Jammy was failing to spawn while using the manila flavor.
It was hitting kernel panic and complaining about memory. That
leads us to increase the flavor to be used in the manila service
image.
Depends-On: Id2d35ce592c73f0c731deb17a5d574f502fe64cb
Change-Id: Idd3e6a6fdb8cf13727cd674c76efd89b3aacfc52
RSA key is no longer enabled by default in recent openssl library.
Because of this change, RSA key does not work in recent Fedora and
CentOS Stream 9 unless it is enabled somehow.
This introduces a new MANILA_KEY_FORMAT parameter to devstack plugin
so that a different ssh key format (eg. ecdsa) can be used.
Change-Id: Ibe60da0b06cade641002d5a1650c38aa57c2d131
To build the manila service image with
Ubuntu Jammy Jellyfish (22.04 LTS) we need
a slight bump in the storage allocated.
Change-Id: I1d60834c2045c069de25a1c8979c433d4dd50684
Related-bug: #1870412
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
manila.scheduler.filter_scheduler.*Scheduler was deprecated long ago[1]
in favor of new manila.scheduler.drivers.filter.*Scheduler.
[1] 3ffb4979f328259793079a5ece6f3b79e23f5187
Change-Id: I40349cbe106f688a67043a73f8563f87f069831b
MANILA_MULTI_BACKEND has been deprecated for five years now, we should remove
it from our code base.
This variable was removed from the settings scripts along with:
MANILA_BACKEND1_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND1_NAME;
MANILA_BACKEND2_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND2_NAME.
Because they work in the same context.
Instead of them, the already implemented and in use,
MANILA_ENABLED_BACKENDS variable was placed to garantee the successful
back-end setup. The same replacement was made in the contribution
samples scripts.
Apart from this, we avoid configuring generic1 and generic2 if
another backend/s are selected.
Closes-Bug: #1898791
Closes-Bug: #1878477
Change-Id: I67036a65da9255694a00a9c8d56cfdefbdf23c05
Needed for zuulv3 jobs that use two
devstack plugins that need to be invoked
in a pre-defined order [1]
[1] https://docs.openstack.org/devstack/latest/plugins.html#plugin-interface
Change-Id: I44117debfc5b7cdef6acb706786963ec81f59db7
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
To set up some first party backends such as
ZFSOnLinux, CephFS via NFS gateway, Container
(where the NAS server is containerized) and LVM,
manila's devstack plugin creates a NAS server
on the devstack host.
On test machines, access to this NAS server is
firewalled from networks outside of the host's
internal network namespace (including from private
project networks that are in different network
namespaces, on the same devstack host).
We currently use a legacy devstack-gate script
to disable firewall on NFS ports; however,
anyone that installs devstack with LVM, Container,
ZFSOnLinux, CephFS-NFS drivers will need these
firewall ports to be opened to be able to mount
shares exported off their devstack host machines.
Move these firewall commands to the devstack plugin.
These commands can be invoked by setting the localrc
variable MANILA_ALLOW_NAS_SERVER_PORTS_ON_HOST to True.
The value of this variable is False by default,
to preserve existing behavior.
Change-Id: Ic9cad47662f1edf2e5c710dbe64d580bc5f01d44
The manila service image was recently rebuilt and is
now bigger than 2 Gibibytes so increase it to 3, which
should last us for a while.
Closes-bug: #1870412
Change-Id: Ia03e48a3f0ff352b94a0224f76f278ef45acad93
This patch enables the use scheduler creating share from
snapshot flag in order to run properly the manila tempest tests.
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
Change-Id: I15311089c45be1574857d46caa073e89424e716d
configure_auth_token_middleware was replaced by
configure_keystone_authtoken_middleware in
Id0dec1ba72467cce5cacfcfdb2bc0af2bd3a3610. Manila's
DevStack plugin no longer needs to create a cache
directory to use as the signing_dir for keystone
authtoken.
Change-Id: I18fa04e6c3a832e17ade3f84d8b43b15686f06fe
manila-tempest-plugin can be installed with its
devstack plugin; Installing it via manila's plugin
is unnecessary. So, deprecate its installation
in the DevStack plugin.
Change-Id: I21c08069ff82b3bfb52ef7ac960183ddc866c2ee
Restoring the default IPv6 route is not needed in CI and
fails when there are multiple defaults, but it is useful
in local devstacks where multiple default routes are not
typical.
Add a variable in settings and use it to make this behavior
conditional and set it to False for the lvm job.
Closes-bug: #1836788
Change-Id: Id73de8100509ec5935641f5f35f93f482d108bcd
Switch manila to use devstack common functions
for deploying a wsgi app under uwsgi and apache.
This change aims to fulfill the community requirement
for all projects on OpenStack to switch devstack jobs
to deploy control-plane API services under uwsgi with
Apache acting as a front end proxy.
More details in https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
We used to deploy with mod_wsgi, but mod_wsgi has many
issues when being used in devstack for development or testing.
Now we leave mod_wsgi as optional and we default to uwsgi.
Implements: bp wsgi-web-servers-support
Depends-On: https://review.openstack.org/#/c/643188/
Change-Id: I12d4c864f2ceb0f3555e32d2dc893e00dbef0d96
We are getting:
403 errors 'Only volume-backed servers are allowed for flavors
with zero disk' on scenario tests.
Appears to be due to this change [1] which just merged to
nova master branch. Ubuntu bionic server doc says the root disk
needs 1.5 GB so we are setting the flavor definition to require
2GB.
[1] c8e65a5eb1
Closes-bug: #1816050
Change-Id: Iba0a15b78bf75a04c1ac0e64e70634772b2dca5c
Generic driver jobs are failing because of timeouts when
establishing the initial ssh connection from manila-share
to the service VM.
Bump up the default value of the connection timeout for paramiko
client and also set the banner timeout since the failure occurred
during banner exchange. Set the two timeouts to the same value
for now. This ensures that the connection timeout is at least as
long as the banner timeout and there is no current need in manila
to control these independently.
This is more of a workaround than a real fix since a real fix
would remove the delay during banner exchange. I suspect that
the real fix will need to be in neutron/ovs though.
Change-Id: Ib5e59faaf9667b9cb5e7d4072531b7d6c3d4da39
Partial-bug: #1807216
- When the URL refers to cloning or using git repositories, use the
cloning URL (https://git.openstack.org/openstack/<project>)
- When the URL refers to the browsable version of the repository, use
the cgit frontend (https://git.openstack.org/cgit/openstack/<project>)
Change-Id: I11a8e15c6d663786465c090b40db9e20b19194e6
Make default share type and group type
creation optional in
manila-tempest-minimal-dsvm-dummy job.
Depends-On: https://review.openstack.org/#/c/540070/
Change-Id: I585c15d9eb4eefb843df62f7aad08838c83611e2
And don't enlist manila-tempest-plugin for installation in venv-tempest.
Change-Id: I0844cdb48b9d415bd341c4687620412178be871b
Signed-off-by: Rishabh Dave <ridave@redhat.com>
This patch makes necessary changes to devstack manila
plugin in order to be able to run IPv6 scenario tests.
Part of the changes included the dependency of
neutron-dynamic-routing plugin and the installation
of quagga, so routes in devstack host can be created
automatically for each router and private network
created by tempest during the tests.
Also, added a new config option "override_ip_for_nfs_access"
for manila tempest plugin that overrides the access rules used
for NFS scenario tests. This option can be set by exporting the
environment variable "OVERRIDE_IP_FOR_NFS_ACCESS" with the
intended value before running devstack.
This change is enabled by a following change and
tested alongside a change on manila-tempest-plugin
project.
Please note that we are temporarily disabling IPv6
functionality in Host-assisted Share Migration, as
the Data Service is not able to handle IPv4 + IPv6
scenarios.
Change-Id: I4ca727f92618998242af18908bbbda6bb5f86303
Needed-By: Id8b005cdd429d53a75624885fe7ca795746c3ede
Supporting dual IPv4/IPv6 backends will eventually require
multi-IP support in the first party backends. This change
enables multi-IP support (IPv4-only currently) in the LVM
driver.
Change-Id: Ib3594aa5d7751c829820fce830d87f6ceea6b049
Partial-Implements: blueprint support-ipv6-access
One of the goals for Pike [1] is to make each API service be able to
run under web servers that support WSGI applications,
such as Apache (+mod-wsgi) and Nginx (+uWSGI).
Do following to address governance requirements:
- Split existing manila/wsgi.py module into 3 modules:
First (manila/wsgi/eventlet_server.py) is used by
eventlet-based WSGI application approach.
Second (manila/wsgi/wsgi.py) is used for WSGI web servers.
And third (manila/wsgi/common.py) is common code for both.
All three are made in cinder-like way to have alike-approach.
- Reuse common code from "oslo_service/wsgi.py" module that
allows us to remove code duplication.
- Delete config opts that are defined by newly reused common code.
- Register new entry point that will be manila wsgi app: "manila-wsgi".
- Fix "manila/api/openstack/wsgi.py" module to be compatible
with str/bytes handling approach used by Apache mod-wsgi plugin using
different python versions (2/3).
- Add web server config template "devstack/apache-manila.template"
- Add devstack support where usage of this feature can be
enabled or disabled using "MANILA_USE_MOD_WSGI" env var.
It is set to "True" by default, because it is requirement for Pike
release - to have it running in all CI jobs.
Disable it only for one CI job that uses dummy driver and tests
various manila core features that are not covered by other CI jobs.
[1] https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
Partially-Implements BluePrint wsgi-web-servers-support
DocImpact
Change-Id: Ibdef3c6810b65a5d6f3611e2d0079c635ee523ab
1) Increase default RAM value in manila flavor to be able
to use manila service image with new 64bit architecture.
2) Fix test requirement on "hacking" lib to avoid pep8 breakage.
3) Fix unstable maprfs driver unit test that depends on local hostnames
and IPs.
Change-Id: Ib36113e33e3616984be5864fbaf73c1fc5d7a315
Closes-Bug: #1669412
Closes-Bug: #1659162
In addition to functional tests.
Also, make some corrections to scenario tests code to make it work
with shares located on localhost.
Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Co-Authored-By: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Change-Id: Ice61a7593f5e4e443cffbe78ecd71d683a9c274e
Whereas the bug report associated to this fix is about
Manila UI not working, this fix addresses more than this.
On the first place, it decouples Manila UI from Manila Devstack
plugin. This is a desired change since not every Manila user
needs to work with the UI, and it doesn't make sense to have
Manila UI bits on the Manila code if we can place all Manila UI
related code on the Manila UI repo and, in this case,
Manila UI Devstack plugin. This patch-set removes all the traces
of Manila UI setup from the Manila Devstack plugin.
Manila UI Devstack plugin was added in 457249be7ab3849b3479b96077a411ca37c37a95.
And, secondly, it fixes Manila UI deploy with Devstack. Enabling
manila-ui plugin on Devstack properly sets the UI component for
Manila.
Change-Id: I7a3372371e25c69d3df6d395c703a6412750effd
Closes-Bug: #1570033
They are useful for case we install several back ends with different
drivers, their modes and capabilities.
New share types have fixed names based on driver modes:
- dhss_true
- dhss_false
It is possible to define their extra specs using following env vars:
- MANILA_DHSS_TRUE_SHARE_TYPE_EXTRA_SPECS
- MANILA_DHSS_FALSE_SHARE_TYPE_EXTRA_SPECS
that defaults to existing following env var:
- MANILA_DEFAULT_SHARE_TYPE_EXTRA_SPECS
Use case in CI: we are able to use more than one predefined share_type
in tempest and rally tests.
Change-Id: I7278a897bb89f0562e1eda449b725d965cb10566
Improve coverage by adding tests that validate the share-type
change while also changing the driver mode.
Closes-bug: #1620800
Change-Id: I924c34aa69591754b437d75f43db91d77e73fb07
In Scenario job, tests consist of VMs mounting shares
provided by share servers.
Admin network is used by share migration tests and is
by default used in other tests as well. Previously it
used the same network address range as tenant networks
(used by VMs) and there was no conflict.
Recently some unknown change broke connectivity between
VMs and share server in scenario job. It seems as there
is now a conflict between those network addresses ranges.
This patch changes the admin network address range,
removing the existing conflict.
Closes-bug: #1616115
Change-Id: I5bdb70cae4eab2930f3f559a621643ad732909bb
Add new neutron plugin which enables port bind actions for network
fabrics. This driver waits until all ports are in state "active"
and sets the binding host flag. Default vNIC type is set to
"baremetal" in order to benefit from the code already in place for
Ironic. It's also possible to switch to 'normal' which assumes an
neutron agent in place.
The feature can be tested using the docker driver.
DocImpact
Co-Authored-By: Daniel Gonzalez <daniel@gonzalez-nothnagel.de>
Partially-Implements: bp manila-hpb-support
Change-Id: I3156d7468d48f84f1b46885780a2426f9b99a387
This driver exploits a Docker container as a share server instead
of a virtual machine. The container acts as a CIFS server
and shares logical volume attached to it. The volume gets created
simultaneously to the container on a host in a volume group
specified by the user.
DocImpact
Change-Id: I711813a2022d765ab7a70ed670e34f68f9cf1399
Implements: blueprint manila-container-driver
This reverts commit 466a19f18f12cb77b45aa7c7a272fae7c45c4c93.
Also this reverts commit 4f74b224fad7bf9ca665d70a66c79c98cf8d5674.
Manila has decided to remove LXD from the Mitaka release.
http://lists.openstack.org/pipermail/openstack-dev/2016-March/089522.html
Change-Id: I3c4b15e8c9d92c874be754205b0b981151809cc1
SSH is required for replication to work in the ZfsOnLinux
driver, but the gate defaults to disabling it. Make it
configurable and make sure to preinstall host key.
Change-Id: Iece2ad2e966fafe3ab4d63c82d062830ac9f3170
Closes-Bug: 1552244
This driver exploits a container as a share server instead
of a virtual machine. The container acts as a NFS/CIFS server
and shares logical volume attached to it. The volume gets created
simultaneously to the container on a host in a volume group
specified by the user. Currently NFS is limited to v3 since the
only userspace NFS server that works in a container out of the box
seems to be unfs3. Later it will be replaced with nfs-ganesha.
Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Implements: blueprint lxc-driver
Depends-On: Ib1896997f2e7a505b5bf8ec0c9e5ee35942f79a6
Change-Id: Iea73bd34b94919d77ebe17cf054dde1f580384be
Added a new service defined as data, whose purpose is to
receive requests, process data operations such as copying,
migration, backup, and send back the response after operation
has been completed.
In this patch the service has no methods, it is empty, the
operation it should perform will be added in subsequent patches.
Implements: blueprint data-copy-service
Change-Id: I31365c8ac4197541af175c82f8f18a3d11740a99
Add support for ZFS (on Linux) filesystem.
Following features are supported:
- Create/delete share
- Create/delete snapshot
- Create share from snapshot
- Extend/shrink share
- Update NFS IP-based access rules using new interface.
(SMB support planned for future).
- create/delete/update/promote share replica
ZFS-related notes:
- Any amount of ZFS zpools can be used by share driver.
- Allowed to configure default options for ZFS datasets that are used
for share creation.
- Any amount of nested datasets is allowed to be used.
- All share replicas are read-only.
- All share replicas are synched periodically, not continuously.
So, status 'in_sync' means latest sync was successful.
Time range between syncs equals to value of
config global opt 'replica_state_update_interval'.
Driver-related notes:
- Able to use remote ZFSonLinux storage as well as local.
Other made changes:
- updated driver private data DB methods removing filtering by host
as redundant operation. Replication requires some common metadata
storage and filtering by host breaks it. It is safe to do so, because
if driver gets some ID of entity then it is allowed to read its info
too.
Implements bp zfsonlinux-driver
DocImpact
Change-Id: I3ddd3767184e4843037de0ac75ff18dce709b6dc
Manila CI jobs create lots of shares and for Generic driver it means
lots of Cinder volumes. After merge of [1] to Cinder project, we started
having NoValidHostFound errors in Cinder logs, it changed default
oversubscription ratio to 1.0 from 20.0. But Manila CI requires it
be bigger than 1.0, so set it explicitly to Cinder's default value 20.0
that is used by all other driver but not LVM.
[1] I5a214462b573d00d26086512821b4f8cc1db4fc3
Change-Id: If0a6e535b92b1471572e1de98e3b44ab6a2c65c9
Closes-Bug: #1540947
Reuse code of old LVM share driver.
LVM Driver is 1st party Manila driver with NFS and CIFS support
and no share server support.
LVM Driver doesn't rely on Neutron, Nova, or Cinder. As such, it
is ideal for testing purposes and, after a period of maturation,
it should be useful in production environments.
Move generic driver's helpers to manila/share/drivers/helpers.py
and reuse them in LVM driver.
Implement 'ro' access, user access, extend_share function for
LVM driver.
Implements bp lvm-driver
Change-Id: Ia46c51ed400dbb0f1d87a758fb165ca711ed3d7c
As per the plugin documentation, for reference
http://docs.openstack.org/developer/devstack/plugins.html
the config environment variables should be added to
"devstack/settings" file
Change-Id: I676fd091c22d2bef71cb988f1163a811bf4cd1de
Start using devstack's feature 'Externally Hosted Plugins' [1], that allows
to install Manila using upstream Devstack without need to put some additional
files manually.
To enable Manila with Devstack just put following in local[rc] file:
enable_plugin manila https://github.com/openstack/manila
Above satisfies following template:
enable_plugin <NAME> <GITURL> [GITREF]
where
<NAME> is an arbitrary name. (ex: manila)
<GITURL> is a valid git url that can be cloned.
[GITREF] is an optional git ref (branch / ref / tag) that will be cloned. Defaults to master.
Plugin enables all Manila services by itself, no need to enable it manually anymore.
Implements BP add-devstack-plugin
[1] http://docs.openstack.org/developer/devstack/plugins.html#externally-hosted-plugins
Change-Id: I28eff33fc65aa741d9270d8bcb263a7fccd9baa4