The initial cinder design[1][2][3] allowed users to create mutliattach
volumes by spcifying the ``multiattach`` parameter in the request
body of volume create operation (``--allow-multiattach`` option in
cinderclient).
This functionality changed in Queens with the introduction of
microversion 3.50[4] where we used volume types to store
the multiattach capabilities. Any volume created with a multiattach
volume type will be a multiattach volume[5].
While implementing the new functionality, we had to keep backward
compatibility with the *old way* of creating multiattach volumes.
We deprecated the ``multiattach`` (``--allow-multiattach`` on cinderclient
side) parameter in the queens release[6][7].
We also removed the support of the ``--allow-multiattach`` optional
parameter from cinderclient in the train release[8] but the API
side never removed the compatibility code to disallow functionality
of creating multiattach volumes by using the ``multiattach``
parameter (instead of a multiattach volume type).
This patch removes the support of providing the ``multiattach``
parameter in the request body of a volume create operation and will
fail with a BadRequest exception stating the reason of failure
and how it can be fixed.
[1] https://blueprints.launchpad.net/cinder/+spec/multi-attach-volume
[2] https://review.opendev.org/c/openstack/cinder/+/85847/
[3] https://review.opendev.org/c/openstack/python-cinderclient/+/85856
[4] f1bfd9790d
[5] https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html#how-to-create-a-multiattach-volume
[6] 94dbf5cce2
[7] adb141a262
[8] 3c1b417959
Depends-On: https://review.opendev.org/c/openstack/tempest/+/875372
Closes-Bug: 2008259
Change-Id: I0ece6e279048abcc04b3674108290a80eca6bd62
In this patch:
- adjusted VersionsController to return only v3
- removed cinder.api.v2.router
- adjustments to cinder.tests.unit.api.contrib to use /v3 only
- moved cinder.api.v2.snapshot_metadata (and tests) to cinder.api.v3
- moved cinder.api.v2.types (and view, tests) to cinder.api.v3
- updated versions response in api-ref
- removed unnecessary config option
- updated various sample config files
- removed experimental tempest-cinder-v2-api job
- updated some docs
- updated non-voting rally job config
Some cinder.api.v2 modules are left because the v3 classes depend on
them, but with the v2 router removed, these are unreachable via the
/v2 path.
Depends-on: https://review.opendev.org/c/openstack/rally-openstack/+/794891
(changes rally to use Block Storage API v3)
Depends-on: https://review.opendev.org/c/openstack/requirements/+/794894
(corrects regression in upper-constraint on Sphinx)
Change-Id: I2093d77db9beec7543c7524d2cd273e79dd5fd5d
We don't need an HTTP client for this, we only
need status codes. Just load that instead.
https://docs.python.org/3/library/http.html
Change-Id: Ia0a00a820fff55e25d352d2bf472054ef7074f34
This patch modifies the delete volume type logic such that a volume type
cannot be deleted if:
1) It is the default volume type
2) The default type configured is wrong/doesn't exist
This also implies that there will exist atleast 1 volume type in the
deployment and that will be the default volume type.
This also includes following 2 changes on the default_volume_type conf option:
1) It is a mandatory field
2) default value of this config option is '__DEFAULT__'
All these changes ensure that we don't allow creating untyped volumes.
Also it is now possible to delete the '__DEFAULT__' type as it acts
as a normal type.
Change-Id: Ifa3d22305060b5913332cad89ea696bf7fd84ce1
Closes-Bug: #1886632
If a volume_type is not specified in a volume-create request, change
I4da0c13b5b3f8174a30b8557f968d6b9e641b091 (introduced in Train) sets a
default volume_type in the REST API layer. This prevents the
selection logic in cinder.volume.flows.api.create_volume.
ExtractVolumeRequestTask from being able to infer the appropriate
volume_type from the source volume, snapshot, or image metadata, and
has caused a regression where the created volume is of the default
type instead of the inferred type.
This patch removes setting the default volume_type in the REST API
and modifies the selection code in ExtractVolumeRequestTask slightly
to make sure a volume_type is always assigned in that function, and
adds and revises some tests.
Change-Id: I05915f2e32b1229ad320cd1c5748de3d63183b91
Closes-bug: #1879578
Since current code assume that resource key is existed, exception
happens when filter file does not initialized. Return empty list
to avoid KeyError exception.
Closes-Bug: #1857945
Change-Id: I2d3c0f3f8d2c02ffedc4bf5ddc45e21c2d22c3e2
The patch adds the following functionality:
* A default volume type (``__DEFAULT__``) will be
created during cinder DB migration.
* Admins can override the default type in cinder.conf
* Creating a new volume without any type (and
`default_volume_type` option is unset in cinder.conf)
will be assigned the ``__DEFAULT__`` type.
* The default volume type cannot be deleted
* All untyped volumes and snapshots will be migrated to the
``__DEFAULT__`` type.
Change-Id: I4da0c13b5b3f8174a30b8557f968d6b9e641b091
Implements: blueprint untyped-volumes-default-volume-type
Much of our code renames this at import already --
just name it "volume_utils" for consistency, and
to make code that imports other modules named "utils"
less confusing.
Change-Id: I3cdf445ac9ab89b3b4c221ed2723835e09d48a53
This collects up utils that are only used in the API code,
and moves them from cinder/utils.py to here, for better code
organization.
Change-Id: Iecd909ef4e24d5597dcccb80a671a27f361e4b4d
query_volume_filters was deprecated in Pike with
Icee6c22621489f93614f4adf071329d8d2115637 and it should be safe to
remove it now.
Change-Id: Icd311db7f88c3c274d9a362eb96519e46c7e4d17
This was a dummy extension and only used by the v1 API. This is now a
standard part of the API and does not need any extra checks to see if it
should be enabled or not. This removes the dummy extensions and checks
in the code for its presence.
Change-Id: Ia0a3fa41aa06fbbe9d2ed67a1273226ee47429f9
This patch adds jsonschema validation for below Volumes API's
* POST /v3/{project_id}/volumes
* PUT /v3/{project_id}/volumes/{volume_id}
Adding new Microversion 3.53, this will have impact on both Create
and Update Volume API
* Create volume API will not accept additional properties like
(user_id, project_id, status and attach_status).
* Update volume API will require minimum one parameter in request
body otherwise it will raise 400 error.
APIImpact :
Before 3.53 version, create volume allows additional properties,
but now it will raise 400 error.
Also before 3.53 version, update volume allows empty body {},
but now it will raise 400 error.
DocImpact
Closes-Bug: #1774155
Partial-Implements: bp json-schema-validation
Change-Id: I1829b6796cd6d3fa1040dd0fb9280bec5bdbc92e
This patch just marks the option as deprecated, going
forward we'll go with using a volume-type instead.
Change-Id: I00615495a6ed98a996135192ed41719b2b00d936
blueprint: multi-attach-v3-attach
We very often end up with merge conflicts for any patches that increment
microversions due to conflicting numbers. We can't really solve that,
but we can avoid the need to update version numbers throughout the code
by defining a constant value in one place and using that variable instead.
Change-Id: Ib3a80fee6caaabb49af097aa197f550c65d94985
Parameters like "consistencygroup_id" "source_volid" and
"source_replica" are expected to be in UUID format but if user
passes non-UUID format value like '1', create api searches for
that particular entity in database and returns 404 NotFound.
Also If user passes any non-integer value like [](empty list),
{}(empty dict), it returns 500 error. This patch proposes to
validate uuid parameters are adhering to the uuid format otherwise
it will return 400 BadRequest.
APIImpact: If user passes non-uuid values to consistencygroup_id,
source_volid and source_replica parameters in the request body of
create volume API, then it will return 400 error instead of
404/500 error.
Closes-Bug: #1680709
Change-Id: I31ea4f378be380a783d1c0249552ded8794fc52e
There are CG related code in api, scheduler, and manager that are
no longer invoked in Pike. This is because we will force users to
migrate all existing CGs and CGsnapshots to the new generic volume
groups tables when they upgrade to Pike. CG CLI and API are still
supported in Pike. They will be re-directed to create/modify
entries in generic volume groups tables instead.
Database and versioned object related code are still kept for now
because there are still drivers referencing them.
Change-Id: Ieba87c6725f07564fd5a69674602eb3ca6200db3
There are several places in the source code where HTTP response
codes are used as numeric values.
Status codes 200, 202, 204, 300, 400, 401, 403, 404, 405, 409, 413,
415, 500, 501, 503 under api/v1 and api/v2 are replaced with symbolic
constants from six.moves.http_client thus improves code readability.
More patches will be submitted to address other status codes.
Partial-Bug: #1520159
Change-Id: I7c61122a6b043d7d238bea95ef39d8fa97817df4
Currently when Cinder calls volume.api.create(), it transfers
dict of volume type to the interface.
This patch is to get volume type object, and transfer it to
volume.api.create() interface. So that it uses object fields
instead of dict.
Change-Id: I981b5060abaff280aa34a9570c65300f467045d6
CG APIs work as follows:
* Create CG - Create only in groups table
* Modify CG - Modify in CG table if CG in CG table, otherwise modify
in groups table.
* Delete CG - Delete from CG or groups table depending on where it is
* List CG - Check both CG and groups tables
* List CG snapshots - Check both CG and groups tables
* Show CG - Check both tables
* Show CG snapshot - Check both tables
* Create CG snapshot - Create either in CG or groups table depending on
the CG.
* Create CG from source - Create in either CG or groups table
depending on the source.
* Create volume - Add volume either to CG or group
Additional notes:
* default_cgsnapshot_type is reserved for migrating CGs.
* Group APIs will only write/read in/from the groups table.
* Group APIs won't work on groups with default_cgsnapshot_type.
* Groups with default_cgsnapshot_type can only be operated by CG APIs.
* After CG tables are removed, we'll allow default_cgsnapshot_type
to be used by group APIs.
Partial-Implements: blueprint generic-volume-group
Change-Id: Idd88a5c9587023a56231de42ce59d672e9600770
Usually when we wanna retrieve volume_type from db,
we do this below:
if is_uuid_like identity:
get_volume_type_by_name(identity)
else:
get_volume_type(identity)
We can combine these functions to clean code
Change-Id: Ief1060cc63dd2bccb7495adb70c0ef9a8cdb76c0
Removed context object while logging as Cinder uses oslo.context's
RequestContext which means the context object is in scope when doing
logging.
Change-Id: I7cc434ad10967596f8354775399e0c6c92ab5570
Closes-Bug:#1500896
Refactoring code: Making volume dict to use single instruction: pop()
rather than two instructions: get() and del, giving the codes a format
that carries through.
Change-Id: Ic889dac330eac24d15bd73186fc504402cddd861
Closes-Bug:#1571548
Throughout the API code we keep catching NotFound exceptions in their
various forms and converting them to webob.exc.HTTPNotFound exceptions,
but we can leave the WSGI fault handler convert them on its own.
This patch changes current behavior and removes the exception handling
closer to the operation so that those exceptions can be handled at the
WSGI level.
This has the following benefits:
- Reduces code complexity
- Increases code readability
- Provides consistent error responses, as messages are stored on the
Exceptions.
- Prevents raising errors with only partial information (we have cases
now that were removing the UUID from the message because they used a
custom message). For example: before returned error would be "The
resource could not be found", and now we raise "Volume type encryption
for type 4e9e6d23-eed0-426d-b90a-28f87a94b6fe does not exist."
automatically.
- Reduces workload for the translation team because we remove all
unnecessary custom messages.
Change-Id: I09f98921fdc2400cc3f6056e59001100abe06920
The volume update API also allow users to update the volume'metadata.
But cinder doesn't check the length limit. It'will raise 500 error
if the metadata is longer than 255. We should check it and raise
413 at the api layer to keep the same with the volume metadata
update API.
Change-Id: I3928ffff9aab6a8071d0641fa6b40b10b1f6bc10
Closes-bug: #1598007
There is no strict boolean checking for the parameter
"multiattach" of API /volumes, so that
any invalid boolean value can be specified.
This patch adds a strict checking for it to prevent
invalid value, and adds tests for this change as well.
Change-Id: I2a5dc45a3b238a2d875d7fbf7159eb91d1f111bf
Partial-Bug: #1594261
This feature allows users to more conveniently query volume details by
filtering the volume list by certain image metadata.
For example, users can query a specific bootable volume quickly
filtering by image_name or other glance metadata.
APIImpact
1. User can use glance metadata to filter volume detail in cinder api.
The query url is like this:
"volumes/detail?glance_metadata={"image_name":"xxx"}"
2. Since microversion is implemented in M, this change will add a new
version "3.4".
DocImpact
1.Operator would need to add glance_metadata to 'query_volume_filters'
option for new functionality to work.
Change-Id: I1d276d93ad5e799401b48d2234e61c28a3aaf790
Implements: blueprint support-volume-glance-metadata-query
Cinder XML API is not tested by Tempest about for one year.
We don't know if it works or not. It was deprecated in
Mitaka release.
Implements blueprint: remove-xml-api
APIImpact
DocImpact
UpgradeImpact
Change-Id: If98db25f1f4032725444c3959acb9aad8f869802
When create a volume with image name, the error message return to
user is invalid if there are several image share the same name.
This change provides a more user-friendly message.
Change-Id: Idac6149ce6944986af9716e60738b3cd62434a5a
Closes-bug: 1565653
This adds the 'cascade' parameter to volume delete,
which deletes snapshots along with a volume in
one call.
This is done in the volume manager, and not in
a driver-optimized way, which will be a later
improvement.
Blueprint: del-vols-with-snaps
Change-Id: I33d15b76d4bd0de14c635d404b2c97096c977a58
Many changes to the Cinder REST API require changes to the consumers of the API.
For example, If we need to add a required parameter to a method that is called
by Nova, we'd need both the Nova calling code and the cinderclient that
Nova uses to change. But newer Cinder versions with the change must work with
older Nova versions, and there is no mechanism for this at the moment. Adding
microversions will solve this problem.
With microversions, the highest supported version will be negotiated by a field
in the HTTP header that is sent to the Cinder API. In the case where the field
'versions' is not sent (i.e. clients and scripts that pre-date this change),
then the lowest supported version would be used. In order to ensure that the
API consumer is explicitly asking for a microversioned API, a new endpoint v3
is added, which is identical to API version v2. This means that our new
Cinder API v3 would be the default, and consumers of the API that wished to
use a newer version could do so by using that endpoint and a microversion in
the HTTP header.
New tests for microversioned API features on endpoint /v3 should be added to
cinder/tests/unit/api/v3/ directory. Existing functionality will be tested via
the .../v2/ unit tests.
DocImpact
APIImpact
Implements: https://blueprints.launchpad.net/cinder/+spec/cinder-api-microversions
Change-Id: I48cdbbc900c2805e59ee9aebc3b1c64aed3212ae
During launch instance from Horizon, if non-admin
user selects volume as source for launching
instance, drop down list for volumes will also
show non-bootable volumes. It is wrong behaviour.
To fix this, 'bootable volume' search filter is
enabled so that only bootable volumes get
displayed in drop down list of volume.
APIImpact
DocImpact:Need to add bootable against
query_volume_filters in OpenStack Configuration
Reference
Closes-Bug: #1524450
Change-Id: If5bfbd73bbe02b13b76d7169ea16424493ac5fca
This patch changes delete API methods to use compare-and-swap updates in
order to remove potential races.
Updated methods are:
- delete
- delete_snapshot
Specs: https://review.openstack.org/232599/
Implements: blueprint cinder-volume-active-active-support
Closes-Bug: #1490944
Change-Id: Ic09825b27774fa4b0ab2b7e60577ecfb3640bcf2
The following patch updates get_volume and delete_volume
API to use volume versionedobjects. Changes were made to
be backwards compatible with older RPC clients. It only
includes changes to the core cinder code. Changes in the
drivers are left to each driver maintainer to update.
Note that this patch DOES NOT try to use
object dot notation everywhere, since it would
increase the size of the patch. Instead, it
will be done in subsequent patches.
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Co-Authored-By: Szymon Wroblewski <szymon.wroblewski@intel.com>
Change-Id: Ifb36726f8372e21d1d9825d6ab04a072e5db4a6a
Partial-Implements: blueprint cinder-objects
The following patch updates create_volume API to use
volume versionedobjects. Changes were made to be
backwards compatible with older RPC clients. It
only includes changes to the core cinder code.
Changes in the drivers are left to each driver
maintainer to update.
Note that this patch DOES NOT try to use
object dot notation everywhere, since it would
increase the size of the patch. Instead, it
will be done in subsequent patches.
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Change-Id: Ic1b0f09132f8fc500b29650abbd57f18ea8bd9dd
Partial-Implements: blueprint cinder-objects
Change-Id: Ief9c63e8bddb2b40bdef4465b9099cff33d7c3bc
ast.literal_eval doesn't change types for Integer
Booleans resulting in DBAPIError exceptions with
at least postgresql. Boolean columns need to be
explicitly cast to boolean types. This changes
the way filters are processed in the volume API.
APIImpact
Change-Id: Ice9c57fa99c17e4c87de6362bf30ed30503a1bed
Closes-Bug: #1494475
Currently when there is no limit set on a volume list query we retrieve
all volumes and then limit them locally using osapi_max_limit. Similar
thing happens when we are using the marker for next pages, we get all
volumes from that marker until the last volume and then limit it
locally.
We should be limiting it on the DB side so we only retrieve the data we
are actually going to return to the API caller.
This patch always limits the data retrieved from the DB and for the
offset to keep working as it was before we need to do the offset on the
DB side as well.
For reference some tests were performed:
On a deployment with 60,000 volumes, 370,000 volume_metadata items and
240,000 volume_glance_metadata items in cinder db. Before the patch
this will use nearly 10G memory. With the patch we will just use about
500M.
Co-Authored-By: wangxiyuan <wangxiyuan@huawei.com>
Closes-bug:#1483165i
Change-Id: Ie903e546074fe118299e8e1acfb9c88c8a10d78c
If you pass name or description parameters with more than 255
characters to create and update apis of volume and snapshot
and create api of backup, then it returns 500 error code.
Added new method validate_name_and_description() in
cinder.api.openstack.wsgi.Controllera to validate string limit and
returned 400 if limit exceeds and also removing leading or trailing
whitespaces and string containing only whitespaces.
APIImpact
1. For all above APIs 400 response will be returned.
2. Earlier it was possible to pass only whitespaces or leading-trailing
spaces to 'name' parameter.
Now it will raise 400 error if only whitespaces are passed and will
remove leading-trailing spaces if present in other cases.
Closes-Bug: 1454244
Change-Id: Iaf7159e816f69fd776a09828c3bc1d27fc9fdcdb
Raise HTTPBadRequest instead of HTTPNotFound while creating
a volume from an existing source replica with invalid replication
status 'disabled'.
APIImpact
Change-Id: If7ac01a9452dca0446088189b658340da538c427
Closes-Bug: #1471694
whoami-rajat