Commit Graph

118 Commits (master)

Author SHA1 Message Date
whoami-rajat 32f1145b7d Remove multiatttach request parameter
The initial cinder design[1][2][3] allowed users to create mutliattach
volumes by spcifying the ``multiattach`` parameter in the request
body of volume create operation (``--allow-multiattach`` option in

This functionality changed in Queens with the introduction of
microversion 3.50[4] where we used volume types to store
the multiattach capabilities. Any volume created with a multiattach
volume type will be a multiattach volume[5].

While implementing the new functionality, we had to keep backward
compatibility with the *old way* of creating multiattach volumes.
We deprecated the ``multiattach`` (``--allow-multiattach`` on cinderclient
side) parameter in the queens release[6][7].
We also removed the support of the ``--allow-multiattach`` optional
parameter from cinderclient in the train release[8] but the API
side never removed the compatibility code to disallow functionality
of creating multiattach volumes by using the ``multiattach``
parameter (instead of a multiattach volume type).

This patch removes the support of providing the ``multiattach``
parameter in the request body of a volume create operation and will
fail with a BadRequest exception stating the reason of failure
and how it can be fixed.

[4] f1bfd9790d
[6] 94dbf5cce2
[7] adb141a262
[8] 3c1b417959

Closes-Bug: 2008259

Change-Id: I0ece6e279048abcc04b3674108290a80eca6bd62
3 months ago
Brian Rosmaita e05b261af7 Remove Block Storage API v2
In this patch:
- adjusted VersionsController to return only v3
- removed cinder.api.v2.router
- adjustments to cinder.tests.unit.api.contrib to use /v3 only
- moved cinder.api.v2.snapshot_metadata (and tests) to cinder.api.v3
- moved cinder.api.v2.types (and view, tests) to cinder.api.v3
- updated versions response in api-ref
- removed unnecessary config option
- updated various sample config files
- removed experimental tempest-cinder-v2-api job
- updated some docs
- updated non-voting rally job config

Some cinder.api.v2 modules are left because the v3 classes depend on
them, but with the v2 router removed, these are unreachable via the
/v2 path.

(changes rally to use Block Storage API v3)
(corrects regression in upper-constraint on Sphinx)

Change-Id: I2093d77db9beec7543c7524d2cd273e79dd5fd5d
2 years ago
Eric Harney 6b067f2b47 Import HTTPStatus instead of http_client
We don't need an HTTP client for this, we only
need status codes.  Just load that instead.

Change-Id: Ia0a00a820fff55e25d352d2bf472054ef7074f34
3 years ago
xuanyandong a85ce6c817 Remove six of dir cinder/api/*
Replace the following items with Python 3 style code.

- six.string_types
- six.text_type
- six.moves
- six.PY3
- six.unichr

Change-Id: Ic46ec5f13a0dc1986d9c01e0d11df827d7e17bb0
3 years ago
Rajat Dhasmana e5d842eb1b Modify default/delete volume type logic
This patch modifies the delete volume type logic such that a volume type
cannot be deleted if:
1) It is the default volume type
2) The default type configured is wrong/doesn't exist

This also implies that there will exist atleast 1 volume type in the
deployment and that will be the default volume type.

This also includes following 2 changes on the default_volume_type conf option:
1) It is a mandatory field
2) default value of this config option is '__DEFAULT__'

All these changes ensure that we don't allow creating untyped volumes.

Also it is now possible to delete the '__DEFAULT__' type as it acts
as a normal type.

Change-Id: Ifa3d22305060b5913332cad89ea696bf7fd84ce1
Closes-Bug: #1886632
3 years ago
Brian Rosmaita 674c8e7286 Default volume_type set too early
If a volume_type is not specified in a volume-create request, change
I4da0c13b5b3f8174a30b8557f968d6b9e641b091 (introduced in Train) sets a
default volume_type in the REST API layer.  This prevents the
selection logic in cinder.volume.flows.api.create_volume.
ExtractVolumeRequestTask from being able to infer the appropriate
volume_type from the source volume, snapshot, or image metadata, and
has caused a regression where the created volume is of the default
type instead of the inferred type.

This patch removes setting the default volume_type in the REST API
and modifies the selection code in ExtractVolumeRequestTask slightly
to make sure a volume_type is always assigned in that function, and
adds and revises some tests.

Change-Id: I05915f2e32b1229ad320cd1c5748de3d63183b91
Closes-bug: #1879578
3 years ago
Yang Youseok fe7cf6be84 Fix KeyError exception when volume filter file does not exist
Since current code assume that resource key is existed, exception
happens when filter file does not initialized. Return empty list
to avoid KeyError exception.

Closes-Bug: #1857945
Change-Id: I2d3c0f3f8d2c02ffedc4bf5ddc45e21c2d22c3e2
3 years ago
whoami-rajat a550ade303 Untyped to Default Volume Type
The patch adds the following functionality:
* A default volume type (``__DEFAULT__``) will be
  created during cinder DB migration.
* Admins can override the default type in cinder.conf
* Creating a new volume without any type (and
  `default_volume_type` option is unset in cinder.conf)
  will be assigned the ``__DEFAULT__`` type.
* The default volume type cannot be deleted
* All untyped volumes and snapshots will be migrated to the
  ``__DEFAULT__`` type.

Change-Id: I4da0c13b5b3f8174a30b8557f968d6b9e641b091
Implements: blueprint untyped-volumes-default-volume-type
4 years ago
Eric Harney de789648e5 Rename volume/ to volume/
Much of our code renames this at import already --
just name it "volume_utils" for consistency, and
to make code that imports other modules named "utils"
less confusing.

Change-Id: I3cdf445ac9ab89b3b4c221ed2723835e09d48a53
4 years ago
Eric Harney 90f962553a Refactor API utilities into
This collects up utils that are only used in the API code,
and moves them from cinder/ to here, for better code

Change-Id: Iecd909ef4e24d5597dcccb80a671a27f361e4b4d
4 years ago
Ivan Kolodyazhny ee1ac54125 Remove deprecated query_volume_filters config option
query_volume_filters was deprecated in Pike with
Icee6c22621489f93614f4adf071329d8d2115637 and it should be safe to
remove it now.

Change-Id: Icd311db7f88c3c274d9a362eb96519e46c7e4d17
5 years ago
jeremy.zhang 3541146aa1
Remove os-image-create API extension
This was a dummy extension and only used by the v1 API. This is now a
standard part of the API and does not need any extra checks to see if it
should be enabled or not. This removes the dummy extensions and checks
in the code for its presence.

Change-Id: Ia0a3fa41aa06fbbe9d2ed67a1273226ee47429f9
5 years ago
TommyLike 558f7b00f5 Update microversion number with constants.
Replace version numbers within constants in order
to make it more readable.

Change-Id: I4c84183ef51e7ff18f0c06cea1b3925acd7dd276
5 years ago
bhagyashris 48b8973aed V3 jsonschema validation: Volumes
This patch adds jsonschema validation for below Volumes API's
* POST /v3/{project_id}/volumes
* PUT  /v3/{project_id}/volumes/{volume_id}

Adding new Microversion 3.53, this will have impact on  both Create
and Update Volume API

* Create volume API will not accept additional properties like
  (user_id, project_id, status and attach_status).
* Update volume API will require minimum one parameter in request
  body otherwise it will raise 400 error.

APIImpact :
Before 3.53 version, create volume allows additional properties,
but now it will raise 400 error.
Also before 3.53 version, update volume allows empty body {},
but now it will raise 400 error.

Closes-Bug: #1774155
Partial-Implements: bp json-schema-validation

Change-Id: I1829b6796cd6d3fa1040dd0fb9280bec5bdbc92e
5 years ago
pooja jadhav 0102a38490 Fix typo errors
This patch is to fix some typo errors in
and in log messages.


Change-Id: I402c252e2fc4221de06676dfdb1ed1883491b2ad
5 years ago
John Griffith 94dbf5cce2 Deprecate the allow-multiattach option to create
This patch just marks the option as deprecated, going
forward we'll go with using a volume-type instead.

Change-Id: I00615495a6ed98a996135192ed41719b2b00d936
blueprint: multi-attach-v3-attach
5 years ago
Sean McGinnis e918482674 Use constants for microversion values
We very often end up with merge conflicts for any patches that increment
microversions due to conflicting numbers. We can't really solve that,
but we can avoid the need to update version numbers throughout the code
by defining a constant value in one place and using that variable instead.

Change-Id: Ib3a80fee6caaabb49af097aa197f550c65d94985
6 years ago
Sean McGinnis f7187d7fd5 Clean up repl v1 volume creation from replication
There is still some cruft left over from the repl v1 implmentation
that should be removed.

Change-Id: Ia98fff46505a8474dbb28d3ffdd67e66f556eddc
6 years ago
dineshbhor 2d4a804876 Validate uuid parameters strictly for create volume API
Parameters like "consistencygroup_id" "source_volid" and
"source_replica" are expected to be in UUID format but if user
passes non-UUID format value like '1', create api searches for
that particular entity in database and returns 404 NotFound.
Also If user passes any non-integer value like [](empty list),
{}(empty dict), it returns 500 error. This patch proposes to
validate uuid parameters are adhering to the uuid format otherwise
it will return 400 BadRequest.

APIImpact: If user passes non-uuid values to consistencygroup_id,
source_volid and source_replica parameters in the request body of
create volume API, then it will return 400 error instead of
404/500 error.

Closes-Bug: #1680709
Change-Id: I31ea4f378be380a783d1c0249552ded8794fc52e
6 years ago
xing-yang c979bdac87 Remove unused CG code
There are CG related code in api, scheduler, and manager that are
no longer invoked in Pike. This is because we will force users to
migrate all existing CGs and CGsnapshots to the new generic volume
groups tables when they upgrade to Pike. CG CLI and API are still
supported in Pike. They will be re-directed to create/modify
entries in generic volume groups tables instead.

Database and versioned object related code are still kept for now
because there are still drivers referencing them.

Change-Id: Ieba87c6725f07564fd5a69674602eb3ca6200db3
6 years ago
poojajadhav 92980e4972 Extracted HTTP response codes to constants
There are several places in the source code where HTTP response
codes are used as numeric values.

Status codes 200, 202, 204, 300, 400, 401, 403, 404, 405, 409, 413,
415, 500, 501, 503 under api/v1 and api/v2 are replaced with symbolic
constants from six.moves.http_client thus improves code readability.
More patches will be submitted to address other status codes.

Partial-Bug: #1520159
Change-Id: I7c61122a6b043d7d238bea95ef39d8fa97817df4
6 years ago
Sean McGinnis a55a6b5c71 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.


Change-Id: I4c96f3590d46205c45d12ee4ead8c208e11c52c5
6 years ago
lisali 1794131293 Use ovo instead of dict
Currently when Cinder calls volume.api.create(), it transfers
dict of volume type to the interface.

This patch is to get volume type object, and transfer it to
volume.api.create() interface. So that it uses object fields
instead of dict.

Change-Id: I981b5060abaff280aa34a9570c65300f467045d6
6 years ago
xing-yang 44ebdd2252 CG API changes for migrating CGs
CG APIs work as follows:
 * Create CG - Create only in groups table
 * Modify CG - Modify in CG table if CG in CG table, otherwise modify
               in groups table.
 * Delete CG - Delete from CG or groups table depending on where it is
 * List CG - Check both CG and groups tables
 * List CG snapshots - Check both CG and groups tables
 * Show CG - Check both tables
 * Show CG snapshot - Check both tables
 * Create CG snapshot - Create either in CG or groups table depending on
                        the CG.
 * Create CG from source - Create in either CG or groups table
                           depending on the source.
 * Create volume - Add volume either to CG or group

Additional notes:
 * default_cgsnapshot_type is reserved for migrating CGs.
 * Group APIs will only write/read in/from the groups table.
 * Group APIs won't work on groups with default_cgsnapshot_type.
 * Groups with default_cgsnapshot_type can only be operated by CG APIs.
 * After CG tables are removed, we'll allow default_cgsnapshot_type
   to be used by group APIs.

Partial-Implements: blueprint generic-volume-group
Change-Id: Idd88a5c9587023a56231de42ce59d672e9600770
7 years ago
Mykhailo Dovgal d1a035fa47 Add snapshot id validation during volume creation
Return bad request response if shapshot id has a bad format.

Change-Id: Ifa216e73d62b772de7aa0ae43d8472c4284fc92b
Closes-Bug: #1626474
7 years ago
TommyLike ed6e9b0b6b Add combined function get_vol_type_by_name_or_id
Usually when we wanna retrieve volume_type from db,
we do this below:

if is_uuid_like identity:

We can combine these functions to clean code

Change-Id: Ief1060cc63dd2bccb7495adb70c0ef9a8cdb76c0
7 years ago
Jenkins 5f65ed75ba Merge "Add strict Boolean checking for volume create" 7 years ago
bhagyashris 1a495922ed Remove context object in oslo.log method
Removed context object while logging as Cinder uses oslo.context's
RequestContext which means the context object is in scope when doing

Change-Id: I7cc434ad10967596f8354775399e0c6c92ab5570
7 years ago
Jenkins 8a34a0f3da Merge "Replace functions 'Dict.get' and 'del' with 'Dict.pop'" 7 years ago
Xu Ao dced19aec3 Replace functions 'Dict.get' and 'del' with 'Dict.pop'
Refactoring code: Making volume dict to use single instruction: pop()
rather than two instructions: get() and del, giving the codes a format
that carries through.

Change-Id: Ic889dac330eac24d15bd73186fc504402cddd861
7 years ago
Gorka Eguileor c491c3ee61 Handle API NotFound exceptions at WSGI level
Throughout the API code we keep catching NotFound exceptions in their
various forms and converting them to webob.exc.HTTPNotFound exceptions,
but we can leave the WSGI fault handler convert them on its own.

This patch changes current behavior and removes the exception handling
closer to the operation so that those exceptions can be handled at the
WSGI level.

This has the following benefits:

- Reduces code complexity
- Increases code readability
- Provides consistent error responses, as messages are stored on the
- Prevents raising errors with only partial information (we have cases
  now that were removing the UUID from the message because they used a
  custom message).  For example: before returned error would be "The
  resource could not be found", and now we raise "Volume type encryption
  for type 4e9e6d23-eed0-426d-b90a-28f87a94b6fe does not exist."
- Reduces workload for the translation team because we remove all
  unnecessary custom messages.

Change-Id: I09f98921fdc2400cc3f6056e59001100abe06920
7 years ago
wangxiyuan f6aec5bb96 check the validity of metadata when update volume
The volume update API also allow users to update the volume'metadata.
But cinder doesn't check the length limit. It'will raise 500 error
if the metadata is longer than 255. We should check it and raise
413 at the api layer to keep the same with the volume metadata
update API.

Change-Id: I3928ffff9aab6a8071d0641fa6b40b10b1f6bc10
Closes-bug: #1598007
7 years ago
xiexs 505ccfd62f Add strict Boolean checking for volume create
There is no strict boolean checking for the parameter
"multiattach" of API /volumes, so that
any invalid boolean value can be specified.
This patch adds a strict checking for it to prevent
invalid value, and adds tests for this change as well.

Change-Id: I2a5dc45a3b238a2d875d7fbf7159eb91d1f111bf
Partial-Bug: #1594261
7 years ago
wanghao fca31fc95e Add ability to filter by volume_glance_metadata
This feature allows users to more conveniently query volume details by
filtering the volume list by certain image metadata.
For example, users can query a specific bootable volume quickly
filtering by image_name or other glance metadata.

1. User can use glance metadata to filter volume detail in cinder api.
   The query url is like this:

2. Since microversion is implemented in M, this change will add a new
   version "3.4".

1.Operator would need to add glance_metadata to 'query_volume_filters'
option for new functionality to work.

Change-Id: I1d276d93ad5e799401b48d2234e61c28a3aaf790
Implements: blueprint support-volume-glance-metadata-query
7 years ago
Ivan Kolodyazhny c042a05ac3 Remove XML API
Cinder XML API is not tested by Tempest about for one year.
We don't know if it works or not. It was deprecated in
Mitaka release.

Implements blueprint: remove-xml-api

Change-Id: If98db25f1f4032725444c3959acb9aad8f869802
7 years ago
Cao ShuFeng 5ab43c81e9 Fix invalid error message of volume create
When create a volume with image name, the error message return to
user is invalid if there are several image share the same name.
This change provides a more user-friendly message.

Change-Id: Idac6149ce6944986af9716e60738b3cd62434a5a
Closes-bug: 1565653
7 years ago
Eric Harney bf2b6b9d90 Delete volumes with snapshots
This adds the 'cascade' parameter to volume delete,
which deletes snapshots along with a volume in
one call.

This is done in the volume manager, and not in
a driver-optimized way, which will be a later

Blueprint: del-vols-with-snaps
Change-Id: I33d15b76d4bd0de14c635d404b2c97096c977a58
7 years ago
scottda 6b11d276d1 cinder-api-microversions code
Many changes to the Cinder REST API require changes to the consumers of the API.
For example, If we need to add a required parameter to a method that is called
 by Nova, we'd need both the Nova calling code and the cinderclient that
Nova uses to change. But newer Cinder versions with the change must work with
older Nova versions, and there is no mechanism for this at the moment. Adding
microversions will solve this problem.
With microversions, the highest supported version will be negotiated by a field
in the HTTP header that is sent to the Cinder API. In the case where the field
'versions' is not sent (i.e. clients and scripts that pre-date this change),
then the lowest supported version would be used. In order to ensure that the
API consumer is explicitly asking for a microversioned API, a new endpoint v3
is added, which is identical to API version v2. This means that our new
Cinder API v3 would be the default, and consumers of the API that wished to
use a newer version could do so by using that endpoint and a microversion in
the HTTP header.
New tests for microversioned API features on endpoint /v3 should be added to
cinder/tests/unit/api/v3/ directory. Existing functionality will be tested via
the .../v2/ unit tests.

Change-Id: I48cdbbc900c2805e59ee9aebc3b1c64aed3212ae
7 years ago
Sheel Rana 7ac7413ae3 Added 'bootable volume' filter for non-admin user
During launch instance from Horizon, if non-admin
user selects volume as source for launching
instance, drop down list for volumes will also
show non-bootable volumes. It is wrong behaviour.

To fix this, 'bootable volume' search filter is
enabled so that only bootable volumes get
displayed in drop down list of volume.


DocImpact:Need to add bootable against
query_volume_filters in OpenStack Configuration

Closes-Bug: #1524450

Change-Id: If5bfbd73bbe02b13b76d7169ea16424493ac5fca
7 years ago
Gorka Eguileor 22b4300253 Remove API races from delete methods
This patch changes delete API methods to use compare-and-swap updates in
order to remove potential races.

Updated methods are:
- delete
- delete_snapshot


Implements: blueprint cinder-volume-active-active-support
Closes-Bug: #1490944
Change-Id: Ic09825b27774fa4b0ab2b7e60577ecfb3640bcf2
8 years ago
Thang Pham dabc7dedbb Update get/delete_volume API to use versionedobjects
The following patch updates get_volume and delete_volume
API to use volume versionedobjects.  Changes were made to
be backwards compatible with older RPC clients. It only
includes changes to the core cinder code.  Changes in the
drivers are left to each driver maintainer to update.

Note that this patch DOES NOT try to use
object dot notation everywhere, since it would
increase the size of the patch.  Instead, it
will be done in subsequent patches.

Co-Authored-By: Michal Dulko <>
Co-Authored-By: Szymon Wroblewski <>
Change-Id: Ifb36726f8372e21d1d9825d6ab04a072e5db4a6a
Partial-Implements: blueprint cinder-objects
8 years ago
Thang Pham 032032f0c3 Update create_volume API to use versionedobjects
The following patch updates create_volume API to use
volume versionedobjects.  Changes were made to be
backwards compatible with older RPC clients.  It
only includes changes to the core cinder code.
Changes in the drivers are left to each driver
maintainer to update.

Note that this patch DOES NOT try to use
object dot notation everywhere, since it would
increase the size of the patch.  Instead, it
will be done in subsequent patches.

Co-Authored-By: Michal Dulko <>
Change-Id: Ic1b0f09132f8fc500b29650abbd57f18ea8bd9dd
Partial-Implements: blueprint cinder-objects

Change-Id: Ief9c63e8bddb2b40bdef4465b9099cff33d7c3bc
8 years ago
Sean McCully 96cacc6225 Use of ast for integers doesn't changes type
ast.literal_eval doesn't change types for Integer
Booleans resulting in DBAPIError exceptions with
at least postgresql. Boolean columns need to be
explicitly cast to boolean types. This changes
the way filters are processed in the volume API.

Change-Id: Ice9c57fa99c17e4c87de6362bf30ed30503a1bed
Closes-Bug: #1494475
8 years ago
Jenkins fb95d86da0 Merge "On Volume list only retrieve needed data from DB" 8 years ago
Gorka Eguileor 28879f9a78 On Volume list only retrieve needed data from DB
Currently when there is no limit set on a volume list query we retrieve
all volumes and then limit them locally using osapi_max_limit.  Similar
thing happens when we are using the marker for next pages, we get all
volumes from that marker until the last volume and then limit it

We should be limiting it on the DB side so we only retrieve the data we
are actually going to return to the API caller.

This patch always limits the data retrieved from the DB and for the
offset to keep working as it was before we need to do the offset on the
DB side as well.

For reference some tests were performed:

On a deployment with 60,000 volumes, 370,000 volume_metadata items and
240,000 volume_glance_metadata items in cinder db.  Before the patch
this will use nearly 10G memory.  With the patch we will just use about

Co-Authored-By: wangxiyuan <>
Change-Id: Ie903e546074fe118299e8e1acfb9c88c8a10d78c
8 years ago
PranaliDeore a1bb185a1f Validate name and description string
If you pass name or description parameters with more than 255
characters to create and update apis of volume and snapshot
and create api of backup, then it returns 500 error code.

Added new method validate_name_and_description() in
cinder.api.openstack.wsgi.Controllera to validate string limit and
returned 400 if limit exceeds and also removing leading or trailing
whitespaces and string containing only whitespaces.

1. For all above APIs 400 response will be returned.
2. Earlier it was possible to pass only whitespaces or leading-trailing
   spaces to 'name' parameter.
   Now it will raise 400 error if only whitespaces are passed and will
   remove leading-trailing spaces if present in other cases.

Closes-Bug: 1454244
Change-Id: Iaf7159e816f69fd776a09828c3bc1d27fc9fdcdb
8 years ago
lisali f72cf0a985 Fix error message in cinder/api/v2/
Minor fix.

Change-Id: I1269e271581cc523368b3e59fe4b8ced4febf9de
Closes-Bug: #1479598
8 years ago
Jenkins 178c5bfe9d Merge "Raise BadRequest for invalid replication status" 8 years ago
Vipin Balachandran 525e27331d Raise BadRequest for invalid replication status
Raise HTTPBadRequest instead of HTTPNotFound while creating
a volume from an existing source replica with invalid replication
status 'disabled'.


Change-Id: If7ac01a9452dca0446088189b658340da538c427
Closes-Bug: #1471694
8 years ago
Jenkins 2ee68a9c02 Merge "cinder list fails with 'name' sort key" 8 years ago