The initial cinder design[1][2][3] allowed users to create mutliattach
volumes by spcifying the ``multiattach`` parameter in the request
body of volume create operation (``--allow-multiattach`` option in
cinderclient).
This functionality changed in Queens with the introduction of
microversion 3.50[4] where we used volume types to store
the multiattach capabilities. Any volume created with a multiattach
volume type will be a multiattach volume[5].
While implementing the new functionality, we had to keep backward
compatibility with the *old way* of creating multiattach volumes.
We deprecated the ``multiattach`` (``--allow-multiattach`` on cinderclient
side) parameter in the queens release[6][7].
We also removed the support of the ``--allow-multiattach`` optional
parameter from cinderclient in the train release[8] but the API
side never removed the compatibility code to disallow functionality
of creating multiattach volumes by using the ``multiattach``
parameter (instead of a multiattach volume type).
This patch removes the support of providing the ``multiattach``
parameter in the request body of a volume create operation and will
fail with a BadRequest exception stating the reason of failure
and how it can be fixed.
[1] https://blueprints.launchpad.net/cinder/+spec/multi-attach-volume
[2] https://review.opendev.org/c/openstack/cinder/+/85847/
[3] https://review.opendev.org/c/openstack/python-cinderclient/+/85856
[4] f1bfd9790d
[5] https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html#how-to-create-a-multiattach-volume
[6] 94dbf5cce2
[7] adb141a262
[8] 3c1b417959
Depends-On: https://review.opendev.org/c/openstack/tempest/+/875372
Closes-Bug: 2008259
Change-Id: I0ece6e279048abcc04b3674108290a80eca6bd62
In this patch:
- adjusted VersionsController to return only v3
- removed cinder.api.v2.router
- adjustments to cinder.tests.unit.api.contrib to use /v3 only
- moved cinder.api.v2.snapshot_metadata (and tests) to cinder.api.v3
- moved cinder.api.v2.types (and view, tests) to cinder.api.v3
- updated versions response in api-ref
- removed unnecessary config option
- updated various sample config files
- removed experimental tempest-cinder-v2-api job
- updated some docs
- updated non-voting rally job config
Some cinder.api.v2 modules are left because the v3 classes depend on
them, but with the v2 router removed, these are unreachable via the
/v2 path.
Depends-on: https://review.opendev.org/c/openstack/rally-openstack/+/794891
(changes rally to use Block Storage API v3)
Depends-on: https://review.opendev.org/c/openstack/requirements/+/794894
(corrects regression in upper-constraint on Sphinx)
Change-Id: I2093d77db9beec7543c7524d2cd273e79dd5fd5d
Introduces microversion (MV) 3.63 to add volume type ID
in the volume details.
This change comes to fix a problem we found in Gnocchi concerning
volumes monitored data. When a volume event is created, Cinder uses
the volume type as the volume type ID. On the other hand, Ceilometer
pollsters are always updating the volume_type value in Gnocchi with the
volume type from volume details API (which uses the volume type name
value).
That situation creates a massive number of resource revisions in
Gnocchi. This MV along with the Dynamic pollster system in Ceilometer
enables operators to overcome the issue.
Closes-Bug: https://bugs.launchpad.net/cinder/+bug/1911660
Change-Id: Icb88faeb00040250a25a5630aeb312a8434ed3c8
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
We don't need an HTTP client for this, we only
need status codes. Just load that instead.
https://docs.python.org/3/library/http.html
Change-Id: Ia0a00a820fff55e25d352d2bf472054ef7074f34
This patch adds a feature by which we allow setting default volume types
for projects.
The following changes are made to achieve the feature:
1) Add 4 set of APIs, set, get, get_all, unset volume type
2) All policies (except get_all) default to system/domain/project admin
3) Preference order: project default, conf default
4) Logic to not allow deletion of default type
We validate set, get and unset APIs with keystone to verify a valid
project id is passed in the request and user has proper authorization
rights to show the project.
The policies are system/domain/project admin by default except get_all
policy which defaults to system admin.
Implements: Blueprint multiple-default-volume-types
Change-Id: Idcc949ed6adbaea0c2337fac83014998b81ff1f8
The response status should lie between 200 and 300 but the operator
used makes the condition true when response status is <= 200.
This patch corrects the same.
Change-Id: I7368eb7674642032b1313afe900b5f36f8ab0dea
Closes-Bug: #1823750
This patch modifies the delete volume type logic such that a volume type
cannot be deleted if:
1) It is the default volume type
2) The default type configured is wrong/doesn't exist
This also implies that there will exist atleast 1 volume type in the
deployment and that will be the default volume type.
This also includes following 2 changes on the default_volume_type conf option:
1) It is a mandatory field
2) default value of this config option is '__DEFAULT__'
All these changes ensure that we don't allow creating untyped volumes.
Also it is now possible to delete the '__DEFAULT__' type as it acts
as a normal type.
Change-Id: Ifa3d22305060b5913332cad89ea696bf7fd84ce1
Closes-Bug: #1886632
If a volume_type is not specified in a volume-create request, change
I4da0c13b5b3f8174a30b8557f968d6b9e641b091 (introduced in Train) sets a
default volume_type in the REST API layer. This prevents the
selection logic in cinder.volume.flows.api.create_volume.
ExtractVolumeRequestTask from being able to infer the appropriate
volume_type from the source volume, snapshot, or image metadata, and
has caused a regression where the created volume is of the default
type instead of the inferred type.
This patch removes setting the default volume_type in the REST API
and modifies the selection code in ExtractVolumeRequestTask slightly
to make sure a volume_type is always assigned in that function, and
adds and revises some tests.
Change-Id: I05915f2e32b1229ad320cd1c5748de3d63183b91
Closes-bug: #1879578
While getting volume details, if the user is a non-admin then
hide the host name.
Change-Id: Iaf0ac52d9227f9a0efbf32b1faca78c8456a84ca
Closes-Bug: #1740950
Since current code assume that resource key is existed, exception
happens when filter file does not initialized. Return empty list
to avoid KeyError exception.
Closes-Bug: #1857945
Change-Id: I2d3c0f3f8d2c02ffedc4bf5ddc45e21c2d22c3e2
The patch adds the following functionality:
* A default volume type (``__DEFAULT__``) will be
created during cinder DB migration.
* Admins can override the default type in cinder.conf
* Creating a new volume without any type (and
`default_volume_type` option is unset in cinder.conf)
will be assigned the ``__DEFAULT__`` type.
* The default volume type cannot be deleted
* All untyped volumes and snapshots will be migrated to the
``__DEFAULT__`` type.
Change-Id: I4da0c13b5b3f8174a30b8557f968d6b9e641b091
Implements: blueprint untyped-volumes-default-volume-type
Much of our code renames this at import already --
just name it "volume_utils" for consistency, and
to make code that imports other modules named "utils"
less confusing.
Change-Id: I3cdf445ac9ab89b3b4c221ed2723835e09d48a53
This collects up utils that are only used in the API code,
and moves them from cinder/utils.py to here, for better code
organization.
Change-Id: Iecd909ef4e24d5597dcccb80a671a27f361e4b4d
The previous patch to update hacking
and pycodestyle turned off some new errors to
keep the patch smaller.
Fix those errors here.
Change-Id: Ib22f63e98eefb36b9b2a8be55c15271824408d5d
This is to remove the unncessary slash when the line doesn't
exceed the length of 79, to make the code more readable.
Change-Id: I3fea5d24692be8b8575379aff7b75a5479f314ef
query_volume_filters was deprecated in Pike with
Icee6c22621489f93614f4adf071329d8d2115637 and it should be safe to
remove it now.
Change-Id: Icd311db7f88c3c274d9a362eb96519e46c7e4d17
This was a dummy extension and only used by the v1 API. This is now a
standard part of the API and does not need any extra checks to see if it
should be enabled or not. This removes the dummy extensions and checks
in the code for its presence.
Change-Id: Ia0a3fa41aa06fbbe9d2ed67a1273226ee47429f9
This patch adds jsonschema validation for below Volumes API's
* POST /v3/{project_id}/volumes
* PUT /v3/{project_id}/volumes/{volume_id}
Adding new Microversion 3.53, this will have impact on both Create
and Update Volume API
* Create volume API will not accept additional properties like
(user_id, project_id, status and attach_status).
* Update volume API will require minimum one parameter in request
body otherwise it will raise 400 error.
APIImpact :
Before 3.53 version, create volume allows additional properties,
but now it will raise 400 error.
Also before 3.53 version, update volume allows empty body {},
but now it will raise 400 error.
DocImpact
Closes-Bug: #1774155
Partial-Implements: bp json-schema-validation
Change-Id: I1829b6796cd6d3fa1040dd0fb9280bec5bdbc92e
We don't have any policy check for type show
and type list APIs. This patch adds them in code
while set default rule to empty.
Change-Id: Id2d72f1c6f24292ba354a53d2519faa348f3bb0f
Closes-Bug: #1777748
Now availability zone is highly integrated into
volume type's extra spec, it will be recognized
when creating and retyping, also we can filter
volume type by extra spec now.
Change-Id: I4e6aa7af707bd063e7edf2b0bf28e3071ad5c67a
Partial-Implements: bp support-az-in-volumetype
Snapshots and backup (create/update) APIs stores 'name' and
description' parameters with leading and trailing spaces in
database after patches [1][2] are merged.
This patch removes leading and trailing whitespaces before
validation and also before storing it in db for 'name' and
'description' parameters.
[1]https://review.openstack.org/#/c/520991/
[2]https://review.openstack.org/#/c/530098/
Closes-Bug: #1742907
Change-Id: Ib5c8d32e2c20cbacebd7b6f7281a44f94a70f7c0
This patch just marks the option as deprecated, going
forward we'll go with using a volume-type instead.
Change-Id: I00615495a6ed98a996135192ed41719b2b00d936
blueprint: multi-attach-v3-attach
This patch adds jsonschema validation for below Snapshots API's
* POST /v3/{project_id}/snapshots
* PUT /v3/{project_id}/snapshots/{snapshot_id}
* POST /v2/{project_id}/snapshots
* PUT /v2/{project_id}/snapshots/{snapshot_id}
Note: For create API 'volume_id' parameter is required, if user
passes 'volume_id' as null, the existing behavior is it returns
"itemNotFound"(404) error response. If we restrict user to accept
volume_id in uuid format only in schema validation then in that case,
if user passes 'volume_id' as null it will raise BadRequest(400)
which will not match existing behavior and also tempest test case[1]
will fail in this case. Also on master if user passes 'metadata' as
null then it is accepted. To maintain consistency we are allowing
'volume_id' and 'metadata' to be null in schema.
Made changes to unit tests to pass body as keyword argument as wsgi
calls action method [2] and passes body as keyword argument.
[1] https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_snapshots_negative.py#L42
[2] https://github.com/openstack/cinder/blob/master/cinder/api/openstack/wsgi.py#L997
Partial-Implements: bp json-schema-validation
Change-Id: I9869ea08afe3c8f8f1f63bb2ead7f63388580937
This patch adds jsonschema validation for below metadata API's
* POST /v3/{project_id}/volumes/{volume_id}/metadata
* PUT /v3/{project_id}/volumes/{volume_id}/metadata
* PUT /v3/{project_id}/volumes/{volume_id}/metadata/{key}
The reason behind applying schema for v2(update and update_all) is
that in "/cinder/tests/unit/api/v2/test_volume_metadata.py" module
from this patch, V2's unit test cases are inherited for V3 and now
we are applying schema validation for V3 because of that V2's unit
test cases responses are modified. If we remove schema validation
from V2 then we will need to add separate unit test cases for V3.
Made changes to unit tests to pass body as keyword argument as wsgi
calls action method [1] and passes body as keyword argument.
[1] https://github.com/openstack/cinder/blob/master/cinder/api/openstack/wsgi.py#L997
Partial-Implements: bp json-schema-validation
Change-Id: I2226c8533cbd1ddd274d8dd0b2d77708463896f4
This patch adds policy in code support for volume
and volume type resources.
Change-Id: I47d11a2f6423a76ca053abf075791ed70ee84b07
Partial-Implements: blueprint policy-in-code
We very often end up with merge conflicts for any patches that increment
microversions due to conflicting numbers. We can't really solve that,
but we can avoid the need to update version numbers throughout the code
by defining a constant value in one place and using that variable instead.
Change-Id: Ib3a80fee6caaabb49af097aa197f550c65d94985
With the new v3 cinder attach flow, nova calls attachment_update
and then attachment_complete. During that time the volume
has 'attaching' status. Because of that status, cinder show volume
will show all the attachments as empty, even if the volume has
attachments. Currently this will only happen during migration,
but when multi-attach is working, this will be a general issue.
The change is to the show volume flow. It will no longer look at the
volume attach status, but only at the individual attachment status's.
Any attachment that has status of attached will be returned by show.
(This will only be an issue in Queens when the new multi-attach flow
is used. There is no need for this change in Pike.)
Partially Implements: blueprint cinder-new-attach-apis
Change-Id: I8a2bff5a668ec58ee80c192cb72326f2b3599c39
Closes-Bug: 1713521
Parameters like "consistencygroup_id" "source_volid" and
"source_replica" are expected to be in UUID format but if user
passes non-UUID format value like '1', create api searches for
that particular entity in database and returns 404 NotFound.
Also If user passes any non-integer value like [](empty list),
{}(empty dict), it returns 500 error. This patch proposes to
validate uuid parameters are adhering to the uuid format otherwise
it will return 400 BadRequest.
APIImpact: If user passes non-uuid values to consistencygroup_id,
source_volid and source_replica parameters in the request body of
create volume API, then it will return 400 error instead of
404/500 error.
Closes-Bug: #1680709
Change-Id: I31ea4f378be380a783d1c0249552ded8794fc52e
There are CG related code in api, scheduler, and manager that are
no longer invoked in Pike. This is because we will force users to
migrate all existing CGs and CGsnapshots to the new generic volume
groups tables when they upgrade to Pike. CG CLI and API are still
supported in Pike. They will be re-directed to create/modify
entries in generic volume groups tables instead.
Database and versioned object related code are still kept for now
because there are still drivers referencing them.
Change-Id: Ieba87c6725f07564fd5a69674602eb3ca6200db3
There are several places in the source code where HTTP response
codes are used as numeric values.
Status codes 200, 202, 204, 300, 400, 401, 403, 404, 405, 409, 413,
415, 500, 501, 503 under api/v1 and api/v2 are replaced with symbolic
constants from six.moves.http_client thus improves code readability.
More patches will be submitted to address other status codes.
Partial-Bug: #1520159
Change-Id: I7c61122a6b043d7d238bea95ef39d8fa97817df4
Currently when Cinder calls volume.api.create(), it transfers
dict of volume type to the interface.
This patch is to get volume type object, and transfer it to
volume.api.create() interface. So that it uses object fields
instead of dict.
Change-Id: I981b5060abaff280aa34a9570c65300f467045d6
CG APIs work as follows:
* Create CG - Create only in groups table
* Modify CG - Modify in CG table if CG in CG table, otherwise modify
in groups table.
* Delete CG - Delete from CG or groups table depending on where it is
* List CG - Check both CG and groups tables
* List CG snapshots - Check both CG and groups tables
* Show CG - Check both tables
* Show CG snapshot - Check both tables
* Create CG snapshot - Create either in CG or groups table depending on
the CG.
* Create CG from source - Create in either CG or groups table
depending on the source.
* Create volume - Add volume either to CG or group
Additional notes:
* default_cgsnapshot_type is reserved for migrating CGs.
* Group APIs will only write/read in/from the groups table.
* Group APIs won't work on groups with default_cgsnapshot_type.
* Groups with default_cgsnapshot_type can only be operated by CG APIs.
* After CG tables are removed, we'll allow default_cgsnapshot_type
to be used by group APIs.
Partial-Implements: blueprint generic-volume-group
Change-Id: Idd88a5c9587023a56231de42ce59d672e9600770
This change adds a new enum and field, VolumeAttachStatus
and VolumeAttachStatusField, that will hold the constants for the
'attach_status' field of the VolumeAttachStatus object. This enum
and field are based on the base oslo.versionedobjects enum and field.
This also changes over the volume object to use the new field. Finally,
all uses of strings for comparison and assignment to this field are
changed over to use the constants defined within the enum.
Partial-Implements: bp cinder-object-fields
Change-Id: Ie727348daf425bd988425767f9dfb82da4c3baa8
Usually when we wanna retrieve volume_type from db,
we do this below:
if is_uuid_like identity:
get_volume_type_by_name(identity)
else:
get_volume_type(identity)
We can combine these functions to clean code
Change-Id: Ief1060cc63dd2bccb7495adb70c0ef9a8cdb76c0
This feature introduce 'managing' and 'error_managing'
status into managing process and 'error_managing_deleting'
status into deleting processto to fix the quota decrease
issue when some exception is raised in c-vol. If volume is
in error_managing, quota wouldn't be decreased
when deleting this volume. But we still expose
'creating','error' and 'deleting' status to user for API
compatibility.
Change-Id: I5887c5f2ded6d6a18f497a018d5bf6105bc5afd7
Closes-Bug: #1504007
Implements: blueprint refactor-volume-status-in-managing-vol
This patch allows different policy rules for create and
update volume metadata.
Change-Id: I23dabd8866a9358c41eb3e048d91011a53c41cc3
Closes-Bug: #1472042
Removed context object while logging as Cinder uses oslo.context's
RequestContext which means the context object is in scope when doing
logging.
Change-Id: I7cc434ad10967596f8354775399e0c6c92ab5570
Closes-Bug:#1500896
whoami-rajat