20772 Commits

Author SHA1 Message Date
Zuul
8dac1ee8db Merge "Ensure backup availability zone is populated if empty" into stable/2024.1 24.2.0 2024-10-24 16:31:59 +00:00
Zuul
9591d65a94 Merge "Coerce booleans to integer values in paginate_query" into stable/2024.1 2024-10-16 15:59:37 +00:00
Zuul
dc7998fc26 Merge "Make default-types APIs compatible with V3.67" into stable/2024.1 2024-09-26 23:14:52 +00:00
Zuul
a4d7a48cc4 Merge "Dell PowerFlex: update the release notes" into stable/2024.1 2024-09-25 15:28:25 +00:00
Rajat Dhasmana
2bb2f13de3 Make default-types APIs compatible with V3.67
In microversion 3.67, we made project_id optional in the URL
for all Cinder APIs.
However, the default-types APIs (set, unset, get, list) were
implemented without the project_id in the URL to make it
ready for SRBAC and anticipating the new endpoint for cinder
without project_id.

This is causing issues while implementing the SDK support for
default-types APIs since we fetch the endpoint containing
project_id (http://127.0.0.1/volume/v3/<project_id>/default-types)
but the default-types API doesn't expect it
(http://127.0.0.1/volume/v3/default-types) resulting in 404
resource not found error.

ResourceNotFound: 404: Client Error for url:
http://127.0.0.1/volume/v3/default_types/12a1b5e507e7497db79707b0ddedf1a4,
: 404 Not Found: The resource could not be found.

This patch makes the default-types APIs consistent with the
other cinder APIs that accept URL with/without project_id.

NOTE: This patch doesn't require a microversion bump or releasenote
since the expectation with MV3.67 is that all APIs support
request irrespective of project_id in the URL and this change is
making default-types APIs consisitent with MV3.67.
Also MV3.67 is just an indication of the behavior change otherwise
supplying URLs without project ID with MV < 3.67 also works.

Change-Id: I63efc0598d501d77474588a02582f5338bb8d345
(cherry picked from commit 9afa19e9c96831b319c53fd222d7b822e52da967)
2024-09-24 13:08:31 +00:00
Eric Harney
c4c587f3bb Optimize rbd upload volume to image
Open the source volume directly rather than using a
temporary file.

Closes-Bug: #2080060
Change-Id: I5415be9ad70a0d622dcb6fcb292e7897a3195a6c
(cherry picked from commit 0fdf2e141ccf04bdf23a2ec099b7826a14b7bf9c)
2024-09-20 15:05:24 +00:00
whoami-rajat
530376bc5d Add cinder-manage command to update service_uuid
In some deployments, after an upgrade, we remove the old service
records and create new ones. This leaves the volumes with the
service_uuid pointing to the old (deleted) service and causes an
issue while purging out the deleted service records.
This patch adds a cinder-manage command to update the
service_uuid field of volumes with the following command:

``cinder-manage volume update_service``

The service_uuid of volumes associated with old service_uuid
also gets updated when cinder creates a new service.

Change-Id: I0b13c6351733b8163bcf6e73c939c375493dcba5
(cherry picked from commit edeac132a19dc05a4108c630ebdfd02de9fd92ef)
2024-09-17 14:37:54 +00:00
Zuul
65ff2ed722 Merge "PowerStore Driver - Add a unit test for connection properties for NVMeOF connector" into stable/2024.1 2024-09-13 23:01:31 +00:00
Yian Zong
906298d873 Dell PowerFlex: update the release notes
Updated the release notes for Cinder Active/Active support in
Dell PowerFlex driver. Patch is proposed directly to the branch
where the release note first occurred as per cinder policy.

Change-Id: Ibc8fd9bf807501aaae398057d6ace6e20610081e
2024-09-13 12:28:46 +00:00
Rajat Dhasmana
faf478482d Fix pep8 issues in driver docs
I noticed these pep8 failures that are unrelated to my patch

...block-storage/drivers/dell-emc-vnx-driver.rst:1083: D001 Line too long
...block-storage/drivers/hpe-3par-driver.rst:348: D001 Line too long

Patch: https://review.opendev.org/c/openstack/cinder/+/823089
Job: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_824/823089/8/check/openstack-tox-pep8/824f11a/job-output.txt

Looks no harm in fixing them.

Change-Id: I6b0186740540bab728f9e0874de4defa4c42fec8
(cherry picked from commit e95754147d8edbfd52355c9ff599e6be47f23747)
2024-09-10 13:30:20 +00:00
Michel Nederlof
70bb142aea Ensure backup availability zone is populated if empty
If the availability zone is not provided when creating the backup,
the availability zone is still None after scheduling.

So just before starting the backup, we can fill the availability_zone
property with the actual az of the backup service doing the backup.

This will also fix selecting the proper host when restoring a backup.
If the az field is empty, it would randomly pick a cinder-backup service,
while it would make sense that only the services in the zone of the backup
would have access to the backup resource.

Closes-Bug: #1942154
Change-Id: Ida930463228bb60f71bb4b80d3f193dde8398a70
(cherry picked from commit 9b7eaf2e328400b2ad00a93d8cd602ebcce63731)
2024-08-28 08:53:36 +00:00
Atsushi Kawai
3bae3445ef Hitachi: Stop frequently REST API request in test
This patch stops submitting frequently REST API request in test scripts,
to avoid a risk of failing the scripts by unexpected REST API response
from a psuedo REST API in the scripts.

Hitachi driver submits a REST API request frequently, to avoid REST API
session timeout.
it should be stopped while running test scripts, or the request bothers
REST APIs which is for cinder features, like creating volume and delete
snapshots.

The test scripts have codes to stop submitting, but one of the code does
not work by using incorrect variable.
The patch fix the variable name to stop that correctoly.

Closes-Bug: #2063317
Change-Id: I81090aee4ed6c288f7d9bbdb45d7cc849663e393
(cherry picked from commit 60b7062902274de084e8fefb4435b094b92be894)
2024-08-23 16:15:03 +00:00
cuiyeliu
2416dc0b46 PowerStore Driver - Add a unit test for connection
properties for NVMeOF connector

This NVMe connection properties support multiple portals. This patch
adds the related unit test cases.

Change-Id: Ibf79950ca4805f38bf008f35fd45aa7435efd61a
(cherry picked from commit 579efddd6234ce84bbce649d57ce743907ddb4ca)
2024-08-23 10:24:07 +00:00
Zuul
fca04de6ac Merge "Hitachi: Prevent to delete a LDEV assigned to multi objects" into stable/2024.1 2024-08-20 12:23:19 +00:00
Zuul
30049aa195 Merge "Hitachi: Fix to set correct object ID for LDEV nickname" into stable/2024.1 2024-08-20 12:06:45 +00:00
Atsushi Kawai
83399aceb0 Hitachi: Prevent to delete a LDEV assigned to multi objects
This patch prevents to delete a LDEV that is unexpectedly assigned to
two or more objects(volumes or snapshots).

In the unexpected situation, if ``delete`` command for one of objects
is run again, the data which is used by other objects is lost.

In order to prevent the data-loss, when creating an object,
the driver creates a LDEV and stores a value obtained by omitting
the hyphen from the object ID(*1) to ``LDEV nickname``.
When deleting an object, the driver compares the own object ID and
the object ID in ``LDEV nickname``, then, the object and the LDEV is
deleted only if both object IDs are same.
On the other hand, if both object IDs are not same, only the object
is deleted and the LDEV is kept, to prevent data-loss.

If format of ``LDEV nickname`` is not object ID(*2), both the object
and the LDEV is deleted without comparison, because it avoids disk
full risk, due to not deleting any LDEVs.
This patch implements only the object ID storing while creating a
snapshot and comparing IDs while deleting, because the feature to
store the object ID while creating a volume has already been
implemented.
(*1) Max length of ``LDEV nickname`` is 32 digits characters on
Hitachi storage.
(*2) 32 digits hexadecimal

Closes-Bug: #2072317
Change-Id: I7c6bd9a75dd1d7165d4f8614abb3d59fa642212d
2024-08-19 14:35:26 -04:00
Atsushi Kawai
699451383a Hitachi: Fix to set correct object ID for LDEV nickname
The patch fixes to set correct object ID for LDEV nickname.

Closes-Bug: #2071697
Change-Id: I2a2cfb44207b03fbb006d29ae958896f3f3231ed
(cherry picked from commit 1895845dcf791d37dc765503b0c3300a75b16991)
2024-08-19 14:35:10 -04:00
Zuul
4d87fa818d Merge "Fix: [Ceph] Backup Driver Python3 Encoding Issue" into stable/2024.1 2024-08-14 16:54:39 +00:00
Eric Harney
712cf414f9 Constrain mypy version on stable/2024.1
Prevent compatibility problems between the stable
branch and future versions of mypy.

(Stable-only patch.)
Change-Id: Ib2261c889d9e7f19db2874290e5a921faa490328
2024-07-26 12:17:09 +00:00
bolin.wu
da46f4d775 Fix: [Ceph] Backup Driver Python3 Encoding Issue
Using python3 the Ceph backup driver faces encoding issues.
In order to solve this we should start using bytearray for
creating a zero-filled instance with a given length.

Closes-Bug: #1925809
Change-Id: I367df9b52f787403a54fb148713d122e9bb2a54f
(cherry picked from commit c13370cdec32ae6b74f3653981519af567be40de)
2024-07-25 21:28:23 +00:00
Eric Harney
3843e3ebcf Pass with mypy 1.11.0
Include some temporary skips to pass with the
newly-released mypy 1.11.0.

Actual fixes to follow.

Change-Id: Id3990df4fbe5914cc3b5e10ccf90aa9e5c1210a5
(cherry picked from commit 834d933fa9fef6630d5df7ae278c6dad6dd82b80)
2024-07-25 21:06:31 +00:00
Zuul
6565c1fd2b Merge "HPE 3par: getWsApiVersion now requires login" into stable/2024.1 2024-07-23 17:22:02 +00:00
Brian Rosmaita
db98dc2070 CVE-2024-32498: Check for external qcow2 data file
Adds code to image_utils to check for a qcow2 external data
file, a recent feature of qemu which we do not support and
which can be used maliciously.

Advice from the qemu-img community is that it is dangerous
to call qemu-img info on untrusted files, so we copy over
the format_inspector module from Glance.  This performs basic
analysis on the image data file so we can detect problematic
images before we call qemu-img info to get all the image
attributes.  It is expected that this code will eventually be
added to oslo so it can be consumed by Glance, Cinder, and
Nova.

Because cinder itself may create qcow2 format images with a
backing file in nfs-based backends, the glance format_inspector
has been modified to optionally allow such files.  Since we are
monkeying with the format_inspector code, we also copy over
its unit tests to prevent regressions and to add tests for the
changed code.

Includes an additional fix to prevent an issue where a user
could mount a raw volume and write a qcow2 header with a larger
virtual size on it. On reattaching the volume it would have the
new larger virtual size avaialable without actually changing
the size value in cinder.  While we cannot prevent this we can
prevent the user from using this volume again, which makes this
exploit pointless.

Co-authored-by: Dan Smith <dansmith@redhat.com>
Co-authored-by: Felix Huettner <felix.huettner@mail.schwarz>

Change-Id: I65857288b797cde573e7443ac6e7e6f57fedde01
Closes-bug: #2059809
(cherry picked from commit b207be3278f3dab6cfe945bd3d307db168f6f6c7)
24.1.0
2024-07-03 11:46:13 +05:30
raghavendrat
7ab8848a29 HPE 3par: getWsApiVersion now requires login
Earlier the call to getWsApiVersion() worked without login.
Now with new wsapi of 2024, login is required.

This patch makes call to client_login() before getWsApiVersion().

Closes-Bug: #2068795
Change-Id: I30f105ee619386f52bc907f5115c08e0fafb9e42
(cherry picked from commit 1b07bee38643d4d4cf52fcb01d2e45318187b27e)
2024-07-02 08:34:44 +00:00
Eric Harney
ad9f66a715 Coerce booleans to integer values in paginate_query
(Grabbed from oslo.db's I4b90f1873 which fixes the same
issue.)

This fixes a failed request when a volume listing is requested
with sorting by bootable:asc and with a marker id specified.

It looks like we are due to replace our paginate_query() method
with the one from oslo.db, as noted in the comments above
the method.

Closes-Bug: #2027532
Change-Id: Idf75e9cc8be6d9f0be320f8109e45b9b9f93dacf
(cherry picked from commit 6d60a6b1ae4c95d4b202746caacd86293ea84620)
2024-06-25 17:41:02 +00:00
Zuul
0ff4262fba Merge "Fix broken backup_swift_service_auth=True" into stable/2024.1 2024-05-22 17:22:22 +00:00
Takashi Kajinami
966bc53c8e Fix broken backup_swift_service_auth=True
Fix the missing session argument so that swift backup driver to
resolve the TypeError raised in swift object access. The error makes
the swift back driver consistently fail during backup/restore
operation.

Closes-Bug: #2058596
Change-Id: I80f2cd614ba7277a28fa8a4a429fef983113f0fb
(cherry picked from commit 9ff29a649ee890cc6bbd7175c8f8f3d2cdd32653)
2024-05-18 05:04:22 +00:00
Jan Hartkopf
52f885b777 Ceph: Catch more failure conditions on volume backup
This fixes issues for volume backups with the Ceph driver
where failures of the first process ("rbd export-diff") were
not caught. Instead, only the return code of the second
process ("rbd import-diff") was recognized.

This change also preserves the stderr that was lost previously
in order to ease debugging.

Closes-Bug: 2031897
Co-Authored-By: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Change-Id: I53b573bfff64e7460ef34f1355d3a9d52a8879f9
Signed-off-by: Jan Hartkopf <jhartkopf@inovex.de>
(cherry picked from commit f8e13a883d961a4a5ba1cf6f557a96ebe6f540ad)
2024-04-23 19:27:09 +00:00
cuiyeliu
1aa541d024 PowerMax: Allow live migration without pool name
This change is to update the live migration ability in environments
using PowerMax. In previous 2023.1 version, the live migration fails
without a pool name.
The update adds the ability of live migration without a pool name.

Change-Id: Iad767cd516c8527136508470629236f68e0c7cc2
Closes-Bug: #2034937
(cherry picked from commit 9a470d41f4d2cd389a0f85e461d87ed5a3a664e3)
2024-03-28 07:44:10 +00:00
Zuul
bcf399027e Merge "HPE 3par: Unable to create clone of replicated vol" into stable/2024.1 24.0.0.0rc2 24.0.0 2024-03-26 06:02:17 +00:00
Yian Zong
c6ff40580b Dell PowerMax: Fix SnapVx unlink failure
Dell PowerMax SnapVx unlink fails as the linked device
is not yet fully defined.
This patch fixed the issue by checking the new configuration
'snapvx_unlink_symforce' and the linked device 'defined' status
to determine the value of 'symforce' in the payload of
SnapVx unlink restcall.

Closes-Bug: #2045230
Change-Id: I614f6aef2d4da76c417b4a143ab80e4a5f716dcd
(cherry picked from commit 03ac369854597f6c886906c1f71561894b10aa04)
2024-03-25 02:26:13 +00:00
raghavendrat
2360e80cd9 HPE 3par: Unable to create clone of replicated vol
Two possibilities of clone volume:
1] same size, online copy

Existing behaviour: start clone & return from function.
Error occur because clone is not yet complete and
code tries to create vol on secondary array.

2] size is different, offline copy

Existing behaviour: (i) create new replicated vol.
(ii) during clone operation below error occur:
Volume is involved in remote copy
(iii) Since clone operation fails, delete new replicated vol (as cleanup).

To overcome both possibilities, code changes are done.

For clone of replicated vol, create offline copy only.
Steps:
(i) Create new vol without replication.
(ii) Perform clone operation; wait till completion (offline copy).
(iii) Create vol on secondary array.

Closes-Bug: #2021941
Change-Id: I1f025542a2509e36919ece01b29064377dbbe189
(cherry picked from commit 577e2ee108a5e3ec2b727928ee4e9c8b35331339)
2024-03-22 12:17:45 +00:00
7405b8d07a Update TOX_CONSTRAINTS_FILE for stable/2024.1
Update the URL to the upper-constraints file to point to the redirect
rule on releases.openstack.org so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.

Until the requirements repository has as stable/2024.1 branch, tests will
continue to use the upper-constraints list on master.

Change-Id: Ib527d05f6bc9a4fe290f52bbfa09f1324458c40f
2024-03-18 08:48:51 +00:00
de249fe217 Update .gitreview for stable/2024.1
Change-Id: I54c649af21dae73467e1d177bcf9605b20605fcf
2024-03-18 08:48:48 +00:00
Zuul
aec1c88712 Merge "Correct releasenote bug-1951250" 24.0.0.0rc1 2024-03-15 15:07:42 +00:00
Zuul
08835cab08 Merge "Dell EMC: PowerMax - Configurable SRDF snapshots" 2024-03-15 15:07:33 +00:00
Zuul
942289ecb3 Merge "Deprecate Windows OS support" 2024-03-15 09:36:08 +00:00
Zuul
baa7c33422 Merge "Dell EMC: Deprecate volume drivers for EOL products" 2024-03-14 21:49:19 +00:00
Zuul
8475859fce Merge "reno: Update master for xena Unmaintained status" 2024-03-14 19:02:53 +00:00
Zuul
c6040dc3bc Merge "JovianDSS: Rework Open-E JovianDSS driver" 2024-03-14 18:16:22 +00:00
23069626b6 reno: Update master for xena Unmaintained status
The stable/xena branch has been deleted, so reno can't find its
release notes.  Use the xena-eom tag to indicate the end of the
Cinder project's maintenance of the Xena series.

This strategy is what we used for the yoga transition, and was
discussed at a cinder weekly meeting:
https://meetings.opendev.org/irclogs/%23openstack-meeting-alt/%23openstack-meeting-alt.2024-02-07.log.html#t2024-02-07T14:06:09

Change-Id: Iafa97d17bd5684c4a43b08f4b1a8d2b580a2cfd1
2024-03-13 22:01:48 +00:00
c0ff458b05 reno: Update master for Unmaintained branches
The stable/victoria and stable/wallaby branches have been deleted,
so reno can't find their release notes.  Use the victoria-eom and
wallaby-eom tags to indicate the end of the
Cinder project's maintenance of these series.

Also fix our override for the reno closed_branch_tag_re to include
the -eom tags. Add the changed reno.yaml file to all relevant
irrelevant-files lists.

This strategy is what we used for the yoga transition, and was
discussed at a cinder weekly meeting:
https://meetings.opendev.org/irclogs/%23openstack-meeting-alt/%23openstack-meeting-alt.2024-02-07.log.html#t2024-02-07T14:06:09

Change-Id: I505b7cc12888d2373a0550b40bb945d75bb11067
2024-03-13 12:56:42 +01:00
Takashi Kajinami
c690fe7cbd Deprecate Windows OS support
Unfortunately the Winstackers project has been retired because of no
maintainers[1][2], and OpenStack on Windows host is no longer
maintained.

[1] https://lists.openstack.org/pipermail/openstack-discuss/2022-November/031044.html
[2] https://review.opendev.org/c/openstack/governance/+/886880

Change-Id: Ic9438ac444734c397cbc814cfee82f4f60abfd74
2024-03-13 01:11:24 +00:00
Takashi Kajinami
202713dd93 Dell EMC: Deprecate volume drivers for EOL products
The following three volume drivers are no longer supported, because
the storage products are now EOL[1].

 - Dell SC Series Storage Driver (iSCSI, FC)
 - Dell VNX Storage Driver (FC, iSCSI)
 - Dell XtremeIO Storage Driver (iSCSI, FC)

This change marks these drivers unsupported, so that we can remove
these drivers after 2024.1 release.

[1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/7D7GUOI66BOD7Z3ZQWPWYYHH6VHAY4JJ/

Change-Id: I763278ec72adaf6f2ddd21edbfc687ca2f17f09a
2024-03-13 10:09:56 +09:00
Rajat Dhasmana
92afd39c12 Correct releasenote bug-1951250
This might be the reason for the releasenote job failing since
this bug number already exist in another releasenote[1].

2024-03-12 08:08:22.062615 | ubuntu-jammy | Warning, treated as error:
2024-03-12 08:08:22.062665 | ubuntu-jammy | /home/zuul/src/opendev.org/openstack/cinder/releasenotes/source/zed.rst:3:Duplicate explicit target name: "bug #1920870".
2024-03-12 08:08:22.137791 | ubuntu-jammy | ERROR

[1] https://github.com/openstack/cinder/blob/master/releasenotes/notes/bug-1920870-ibm-svf-fix-extend-issue-for-mirroring-volumes-31b1a9119c49e112.yaml

Change-Id: If3b45be606bfe318e339c8ab2df9be4162a63b40
2024-03-12 20:20:09 +00:00
Zuul
0581e8cbd6 Merge "Remove six from HPE drivers" 2024-03-07 09:36:29 +00:00
Zuul
c7f2f61307 Merge "[NetApp] LUN space allocation support" 2024-03-06 18:12:12 +00:00
Zuul
59a789dfe4 Merge "Add Cinder active-active support for Dell PowerFlex driver" 2024-03-06 17:59:40 +00:00
Jayaanand Borra
48d922cf62 [NetApp] LUN space allocation support
Space allocation is an important NetApp driver specific feature.
This needs to be set when the cinder volume is created.
This is not related to thin/thick provisioning feature of cinder
volumes.It is independent of that. It enables ONTAP to reclaim
space automatically when host deletes data.This helps ONTAP
and host to see the actual space correctly when the host
deletes data.
It also helps to keep a LUN (cinder volume) online when the
LUN (cinder volume) in ontap runs out of space and containing
volume (in ONTAP) cannot automatically grow more space.

User can configure it by using volume type extra spec.
By default Space allocation value is disabled for ONTAP LUN

netapp:space_allocation: "<is> True"  # to enable space allocation
netapp:space_allocation: "<is> False" # to disable space allocation

Blueprint: netapp-space-allocation-support
Change-Id: Ib7072f3093067ecd8ad84e396aaecec8f15c49ba
2024-03-06 03:16:05 -05:00
Zuul
b5d08a53ab Merge "Remove six from Nexenta drivers" 2024-03-05 23:10:26 +00:00