5550 Commits

Author SHA1 Message Date
Zuul
dd8f1aeee8 Merge "Fix api-ref for access rules" 2021-09-28 17:46:32 +00:00
Helen Walsh
084773bf0f PowerMax Manila - Version increment for Xena
Incrementing the manila version for Xena.

Change-Id: I651f91c501960f9f51f41e44479ecb172fc182a1
2021-09-23 09:35:37 +01:00
7e9c5ec03a Add Python3 yoga unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for yoga.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: If2ba76466a7235531dfc52db37e13bcd235c8b5c
2021-09-17 16:23:23 +00:00
f9c471b608 Update master for stable/xena
Add file to the reno documentation build to show release notes for
stable/xena.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/xena.

Sem-Ver: feature
Change-Id: I4e4ea6fdedfd36bb2668d5280df6ce3dce5d587f
2021-09-17 16:23:21 +00:00
Zuul
2287838bde Merge "Don't run periodic_share_replica_update() on active replicas" 2021-09-16 20:24:58 +00:00
Zuul
aa0ba139db Merge "Optimize the query logic for share network list" 2021-09-16 17:48:55 +00:00
Goutham Pacha Ravi
1991dc7538 [doc] Add Xena max api version annotation
Change-Id: I77ec2091838100abf988b24093864490259e9ee0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-09-14 17:08:00 -07:00
haixin
de72cd4736 Optimize the query logic for share network list
1: As admin user, query share network list with specified
   "security_service_id" and "project_id" search opts, will
   get wrong result.
2: put "created_since", "created_before" search opts into database to
   Increase query speed

Closes-Bug: #1923008
Change-Id: I49e412cb6c98fcda67531ff915b3b4c3edc64476
2021-09-10 19:23:25 +00:00
Zuul
7212aa2b76 Merge "[NetApp] Add FlexGroup volume support" 2021-09-09 19:21:13 +00:00
Zuul
65f3967efa Merge "[NetApp] Share server migration through SVM migrate" 2021-09-08 18:59:36 +00:00
Felipe Rodrigues
9f3c566a10 [NetApp] Add FlexGroup volume support
The NetApp driver has been working  with FlexVol ONTAP volumes.
The driver does not support scaling FlexVol volumes higher than
100 TiB, which was a theoretical limit for the large namespace that
these containers were meant to handle. ONTAP's Flexgroup volumes
eliminate such limitations. So, added the support for provisioning
share as FlexGroup in the NetApp driver.

The FlexGroup provision is enabled by new option
`netapp_enable_flexgroup`, which will make the driver report a single
pool represeting all aggregates. The selection on which aggregates the
FlexGroup share will reside is up to ONTAP. If the administrator desires
to control that selection through Manila scheduler, it must inform the set
of aggregates that formss FlexGroup pool in the new  option
`netapp_flexgroup_pool`.

Each NetApp pool will report now the capability: `netapp_flexgroup`
informing which type the pool is.

The following operations are allowed with FlexGroup shares (DHSS
True/False and NFS/CIFS):

- Create/Delete share;
- Shrink/Extend share;
- Create/Delete snapshot;
- Revert to snapshot;
- Manage/Unmanage snapshots;
- Create from snapshot;
- Replication[1]
- Manage/Unmanage shares;

The backend with one FlexGroup pool configured will drop the consistent
snapshot support for all pools.

The driver FlexGroup support requires ONTAP version 9.8 or greater.

[1] FlexGroup is limited to one single replica for ONTAP version
lower than 9.9.1.

DocImpact

Depends-On: If525e97a5d456d6ddebb4bf9bc8ff6190c95a555
Depends-On: I646f782c3e2be5ac799254f08a248a22cb9e0358
Implements: bp netapp-flexgroup-support
Change-Id: I4f68a9bb33be85f9a22e0be4ccf673647e713459
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
2021-09-08 15:47:15 -03:00
silvacarloss
04a3db2f9a Share server migration enhancements
A series of enhancements can be performed in the share server
migration operation, and share backends might support
nondisruptive share server migrations, which was not covered
in the previous approach.

-  Skip the destination share server network allocation if the
driver is capable of reusing the source share server allocation.
Manila will switch the allocations in the migration complete phase.

- Allow share backends to reuse the share servers in case they
are able to do so in a share server migration scenario.

- Nondisruptive migration is now feasible depending on whether the
share driver supports it and if the share network parameters have
not changed.

- Share servers will be always deleted whe the share server
migration complete operation is finished.

Depends-On: I43bd3fdafea02eb8e853114a57bfb863a441f3ed
Co-Authored-By: Fabio Oliveira <fabioaurelio1269@gmail.com>
Change-Id: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
2021-09-07 09:21:32 -03:00
silvacarloss
74d5a1b2cf [NetApp] Share server migration through SVM migrate
Implements share server migration using a proper mechanism
provided by ONTAP. In case the driver identifies that the ONTAP
version matches the version where this mechanism is available,
ONTAP will automatically chose to use this instead of SVM DR.

- Implemented new methods for migrating a share server using a
new mechanism provided by ONTAP, when both source and destination
clusters have versions >= 9.10. This new migration mechanism
supports nondisruptive migrations in case there aren't network
changes in the migration.

- The NetApp now does not need to create an actual share server in
the backend prior to the migration, in case SVM Migrate is being
used.

- The NetApp ONTAP driver can now reuse network allocations from
the source share server in case a share network change wasn't
identified.

Change-Id: Idf1581d933d11280287f6801fd4aa886a627f66f
Depends-On: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
2021-09-07 09:03:32 -03:00
Zuul
c419a7bad0 Merge "Add Share Affinity/Anti-Affinity Scheduler Filters" 2021-09-03 18:28:42 +00:00
Zuul
2aeac53eaa Merge "[NetApp] Add readable replication type support" 2021-09-03 18:15:33 +00:00
Zuul
45f98a7d0a Merge "Replace retrying with tenacity" 2021-09-03 17:07:51 +00:00
Chuan Miao
7e7ec7337c Add Share Affinity/Anti-Affinity Scheduler Filters
This patch implements hard affinity and anti-affinity filter for
manila scheduler. Users can specify affinity/anti-affinity share
ids to the field "share.scheduler_hints.same_host" or
"share.scheduler_hints.different_host" in the request payload
when creating a manila share. The scheduler_hints are stored as
share metadata. The filter properties are populated from this
metadata during share migration and so filters will be applied
for share migration as well.

Both fields can be a single share UUID or multiple uuids
separated by comma. For example,

`{
    "share": {
        "scheduler_hints": {
            "same_host": "share_uuid_1,share_uuid_2",
            "different_host": "share_uuid_3"
        }
    }
}`

Implements: bp/affinity-antiaffinity-filter

Change-Id: Ic42d8a0c1d22e77ae64e0ca014607b28fd336467
Co-authored-by: Maurice Escher <maurice.escher@sap.com>
2021-09-03 08:58:46 +02:00
Felipe Rodrigues
556c361558 [NetApp] Add readable replication type support
Implement the `readable` replication type for the NetApp driver.
The driver will keep having support for the `dr` type as well, being
the driver replication type a list containing them.

The replicas for readable style are mounted, created the export and
applied the QoS. When promoting, the original active replica does
not need to be unmounted. The user just loses the write access.

The update access interface is now applying rules for non active
replicas that are readable.

Implements: bp netapp-readable-replica
Change-Id: Icc74eaecc75c3064715f91bebb994e93c0053663
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
2021-09-02 18:07:23 +00:00
Zuul
d955928947 Merge "[Optimise] Use ThredGroup to manage periodic tasks" 2021-09-02 08:36:55 +00:00
ashrod98
903aab1920 Replace retrying with tenacity
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.

Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did
  (the retry interface in manila exposed time in seconds as well)
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg - we are going to set this in the retry interface
by default.
- For retries that check a result, tenacity will raise if the
retried function raises, whereas retrying retried on all
exceptions - we haven't exposed this in the retry interface.

This patch updates all usages of retrying with tenacity.
Unit tests are added where applicable.

[1] https://github.com/jd/tenacity

Co-Authored-By: boden <bodenvmw@gmail.com>
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Closes-Bug: #1635393
Change-Id: Ia0c3fa5cd82356a33becbf57444f3db5ffbb0dd0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-09-01 18:31:38 -07:00
Zuul
13d9e2c1e0 Merge "Migrate all quota parameters to [quota] section." 2021-09-01 19:44:57 +00:00
Zuul
614389008a Merge "[doc] add since and before parameter to message-list cli" 2021-08-31 22:26:41 +00:00
Goutham Pacha Ravi
18741fe632 Fix api-ref for access rules
The key for access rule metadata in the API
request or response is just "metadata".

The "access_key" in /share-access-rules
endpoint responses doesn't need to specify
a min-version since its present in all API
versions that the endpoint itself is supported
with.

TrivialFix

Change-Id: I66ac3804f22d70ad673eb7185e5e7eb08843c811
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2021-08-31 21:10:56 +00:00
haixin
b5bc9bd9b8 [Optimise] Use ThredGroup to manage periodic tasks
Instead of manually managing periodic tasks, use
oslo.service's threadgroup.
[1]:https://docs.openstack.org/oslo.service/latest/reference/threadgroup.html
[2]:https://review.opendev.org/c/openstack/cinder/+/558416

Change-Id: I43a55f0ffff3a0dc6b9fc757869e1e3c3240716f
2021-08-31 20:48:10 +00:00
Nahim Alves de Souza
58031e3693 [NetApp] Fix list of mandatory services for CIFS share creation
To create a CIFS share, it is mandatory to configure an active
directory service, but the code was requiring to configure also
the `ldap` and `kerberos` services to be configured during the
creation of a CIFS share.

This patch fixes this, removing `ldap` and `kerberos` from the
list of mandatory security services.

Change-Id: I34020da6a929f5c7ae1c63bc0f2ef338c49dbe10
Closes-bug: #1942124
2021-08-31 15:29:39 +00:00
Zuul
b50a0bb9de Merge "Add config option reserved_share_from_snapshot_percentage." 2021-08-30 19:46:52 +00:00
haixin
fb221d05bc [doc] add since and before parameter to message-list cli
after Support query user message by timestamp has been merged.
https://review.opendev.org/c/openstack/python-manilaclient/+/708807
we should update corresponding command line manual

Change-Id: Id27bcde967717bb6b80d96c39e55bc3dd9e92e1b
2021-08-30 16:05:04 +08:00
Zuul
a07930e2c3 Merge "Add documentation for share server limits" 2021-08-29 19:03:19 +00:00
Zuul
253336cbfb Merge "[NetApp] Fixed scoped account replica delete" 2021-08-29 18:38:42 +00:00
Zuul
8e066548a4 Merge "Extend share will go through scheduler" 2021-08-29 18:38:39 +00:00
Zuul
58fdee2272 Merge "handle replica state on migration complete" 2021-08-29 18:21:47 +00:00
Nahim Alves de Souza
a745e9c7c4 [NetApp] Fixed scoped account replica delete
In the ONTAP 9.8 and older, there is a bug in the zapi call
`snapmirror-release-iter` when using the scoped account
configuration. The operation is returning success, but the release
is not occurring.

To avoid this issue, the Netapp driver was changed to release with
the `snapmirror-release` zapi call, instead of the one that has a
bug. This new zapi call requires the field `relationship-id` to be
specified and it is being retrieved from ONTAP before calling it.

A small fix, not related to the bug, was made in the zapi call to
`snapmirror-release-iter` because the `relationship-info-only`
field was in the wrong place according to the documentation.

Closes-Bug: #1934889
Change-Id: I21633d4ffe14983365b0b6239216ed5d0bbfaace
2021-08-27 18:41:36 +00:00
kpdev
6ca10003a9 Add config option reserved_share_from_snapshot_percentage.
This config option allows different value for reservation percentage,
mostly useful on the platforms, where shares can only be created from
the snapshot on the host where snapshot was taken. The lower value of
this config option against existing (reserved_share_percentage) allows
to create shares from the snapshot on the same host up to a higher
threshold even though non-snapshot/regular share create fails.
In case this config option is not set, the shares created from snapshot
will use reservation percentage value set in 'reserved_share_percentage'.
This will be useful for users who want to keep same reservation percentage
for both non-snapshot/regular and snapshot shares.

DocImpact

Closes-Bug: #1938060
Change-Id: I390da933fe92875e3c7ee40709eacacc030278dc
2021-08-27 19:10:36 +02:00
Zuul
de785fea87 Merge "Add missing [oslo_reports] options" 2021-08-27 10:28:41 +00:00
haixin
cdbc428b69 Extend share will go through scheduler
Add an force parameter to the API layer that lets the user choose
whether to go through the scheduler or not, which is boolean.and
default is False,set True means extend share directly, set False
means extend share will go through scheduler.
Add an new min_version 2.64 to extend share api. force parameter
only support min_version >= 2.64.

Closes-Bug:#1855391
Change-Id: I6da36a687a37c78a7fb7d3f252318d03d4a05133
2021-08-27 05:35:45 +00:00
Zuul
1d95424dd6 Merge "Add missing oslo.service options" 2021-08-26 23:33:18 +00:00
Zuul
f11da50917 Merge "Change cifs value from string to list for Dell manila drivers" 2021-08-26 22:55:02 +00:00
Zuul
fb154955d5 Merge "Use oslo-config-generator conf to load options from libraries" 2021-08-26 18:43:36 +00:00
Zuul
6af91d4da6 Merge "[Pure Storage] Honour the share_name_template parameter" 2021-08-24 18:11:04 +00:00
Takashi Kajinami
3f97376cd0 Add missing [oslo_reports] options
The oslo.reports library provides some options under the [oslo_reports]
section. This change ensures these parameters are rendered by
the oslo-config-generator command.

Closes-Bug: #1940733
Change-Id: I5e30ef458bccb0921ce77b2d9ef875bec72d0619
2021-08-21 08:33:47 +00:00
Takashi Kajinami
c1eaf108f8 Add missing oslo.service options
Closes-Bug: #1940734
Change-Id: I42d76c7a085e3ded04b45f89469b4b66d6a426ce
2021-08-21 08:33:32 +00:00
Sam Wan
5bd44e0be3 Change cifs value from string to list for Dell manila drivers
Dell manila drivers use string as value for 'cifs' in
dhss_mandatory_security_service_association.
This fix changes it to a list.

Change-Id: I0c64e574301baf2a41a475af3b3848cbec8d495f
Closes-Bug: #1940072
2021-08-18 09:17:12 +08:00
Takashi Kajinami
7f35a177d3 Use oslo-config-generator conf to load options from libraries
... instead of importing library modules just to load options.

Change-Id: I55270b330d1fe98817df4d419e8c26c90440e3a6
2021-08-14 23:43:22 +09:00
Zuul
f56a326da8 Merge "Add generic fuzzy matching logic to the database layer" 2021-08-13 04:05:28 +00:00
Maurice Escher
68e3783522 handle replica state on migration complete
Set share instance replica_state to 'active' after successful migration
if the share is involved in a replication setup.

Change-Id: I6cf8a08379b2f2b0fd1bcada4cd20c39adc01f47
Closes-Bug: #1927060
2021-08-12 23:06:22 +00:00
Simon Dodsley
e6e4ac26b7 [Pure Storage] Honour the share_name_template parameter
No release notes required as this driver has not yet been included
in a release and all we are doing here is honouring the standard
share naming template that is documented elsewhere.

Change-Id: Id62bbfe23319f6941f37ce5eb72b511ee1bf4ebf
2021-08-12 10:29:56 -04:00
kpdev
18668d2465 Early validate for CIFS without security service.
To create a share with protocol CIFS it is mandatory to add a security
service to the share network beforehand. If this is forgotten the share
ends up in error. Validate the security service association based on
share protocol from given driver. Raise invalid request exception if
needed association between share_network and security_service is
missing for given driver e.g. ONTAP.

DocImpact

Closes-Bug: #1900752
Change-Id: Ib7e9850e6439ee5d04f826d129afb1ab06950ce7
2021-08-05 22:44:55 +02:00
Zuul
8adba915a5 Merge "Deprecate create snapshot from share group in CephFS" 2021-08-05 18:24:03 +00:00
Zuul
82cd57c413 Merge "Use tox 3.1.1 fixes" 2021-08-02 15:46:21 +00:00
Takashi Kajinami
76cfc9405e Migrate all quota parameters to [quota] section.
Currently we register all parameters about the quota feature to
[DEFAULT] section. However it is difficult for operators to identity
that some of these parameters like reservation_expire are used in the
quota feature, based of names and descriptions of the parameters.

This change migrates all quota options to the new [quota] section,
so that operators can easily understand which parameters are used
in the quota feature.

Closes-Bug: #1934025
Change-Id: I83b5750da7083718d39efb56262a49dae0a05f48
2021-07-30 08:47:56 +09:00