Commit Graph

599 Commits

Author SHA1 Message Date
Stephen Finucane
a32e0b2aa6 api-ref: Improve sort, pagination parameter docs
The various index APIs (e.g. 'GET /volumes') support two forms of sort
parameter: the legacy 'sort_key'/'sort_dir' parameters and the combined
'sort' parameter. We were documenting different parameters in different
APIs despite the fact all APIs support all parameters.

In a similar vein, the various index APIs support the same three
pagination parameters: 'offset', 'limit', and 'marker'. Once again, we
were documenting these inconsistently.

Correct both issues.

Change-Id: Ia3300a8852825f7da830f7b1ed37b27625e267e9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-08-28 15:56:03 +01:00
Zuul
4da12c70ac Merge "Automate generation of backups api-ref samples" 2023-07-27 08:52:53 +00:00
Zuul
a0268b0faa Merge "api-ref: Remove non-existent attribute" 2023-07-21 17:35:18 +00:00
Zuul
5728d3899f Merge "api-ref: Fix indentation, case" 2023-07-17 14:37:45 +00:00
Zuul
c191432232 Merge "Automate generation of volume transfer api-ref samples" 2023-07-17 12:34:14 +00:00
Stephen Finucane
b6c38c3001 api-ref: Fix indentation, case
A bullet list shouldn't be indented in rST and doing so causes it to be
rendered inside a blockquote, which is not what we want.

In addition, indicating versions as vXXX is more prevalent than VXXX so
correct this.

Change-Id: I3ea8e0a8ae58d5b7da9aae85b41162b2458020d5
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-07-17 11:21:56 +01:00
Toheeb
6172d2a9a0 Automate generation of backups api-ref samples
This patch adds the following tests:

1) List backup with details
2) Show backup
3) Update backup

Partially-Implements: blueprint generating-sample-files
Change-Id: I9bbfab93fb138932243d880e440c9524b5f08217
2023-07-13 14:25:52 +01:00
Desire Barine
79a9210c52 Automate generation of volume transfer api-ref samples
This patch adds the following tests:

1) Volume transfer for microversions 3.55 and above

Partially-Implements: blueprint generating-sample-files

Signed-off-by: Desire Barine <desirebarine16@gmail.com>
Change-Id: Ifdd99433563ab72d8d43a374af0377213834d46e
2023-07-08 07:56:50 +00:00
Toheeb
4b082c22ce Automate generation of backups api-ref samples
This patch adds the following tests:

1) List Backup

Partially-Implements: blueprint generating-sample-files
Change-Id: I639e2786ce0fc8d0e9422704df9cdd50d3ea84ec
2023-06-09 15:41:38 +01:00
Stephen Finucane
4b30fc1e06 api-ref: Remove non-existent attribute
Extensions don't have a namespace attribute in v3.

Change-Id: I78ff472e55d822390029e1dd3817786d9b69a742
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-06-02 14:15:21 +01:00
Zuul
77b6035e4f Merge "Automate generation of backups api-ref samples" 2023-05-25 21:17:15 +00:00
Haleema khan
ed2115a6f3 Automate generation of backups api-ref samples
This patch adds the following tests:

1) Backup create

Partially-Implements: blueprint generating-sample-files

Change-Id: Ibe820e8906e6dfdf71a9fc77f3091ccd747f033c
2023-05-23 20:42:57 +00:00
Gorka Eguileor
6df1839bdf Reject unsafe delete attachment calls
Due to how the Linux SCSI kernel driver works there are some storage
systems, such as iSCSI with shared targets, where a normal user can
access other projects' volume data connected to the same compute host
using the attachments REST API.

This affects both single and multi-pathed connections.

To prevent users from doing this, unintentionally or maliciously,
cinder-api will now reject some delete attachment requests that are
deemed unsafe.

Cinder will process the delete attachment request normally in the
following cases:

- The request comes from an OpenStack service that is sending the
  service token that has one of the roles in `service_token_roles`.
- Attachment doesn't have an instance_uuid value
- The instance for the attachment doesn't exist in Nova
- According to Nova the volume is not connected to the instance
- Nova is not using this attachment record

There are 3 operations in the actions REST API endpoint that can be used
for an attack:

- `os-terminate_connection`: Terminate volume attachment
- `os-detach`: Detach a volume
- `os-force_detach`: Force detach a volume

In this endpoint we just won't allow most requests not coming from a
service. The rules we apply are the same as for attachment delete
explained earlier, but in this case we may not have the attachment id
and be more restrictive.  This should not be a problem for normal
operations because:

- Cinder backup doesn't use the REST API but RPC calls via RabbitMQ
- Glance doesn't use this interface anymore

Checking whether it's a service or not is done at the cinder-api level
by checking that the service user that made the call has at least one of
the roles in the `service_token_roles` configuration. These roles are
retrieved from keystone by the keystone middleware using the value of
the "X-Service-Token" header.

If Cinder is configured with `service_token_roles_required = true` and
an attacker provides non-service valid credentials the service will
return a 401 error, otherwise it'll return 409 as if a normal user had
made the call without the service token.

Closes-Bug: #2004555
Change-Id: I612905a1bf4a1706cce913c0d8a6df7a240d599a
2023-05-10 19:51:33 +02:00
whoami-rajat
32f1145b7d Remove multiatttach request parameter
The initial cinder design[1][2][3] allowed users to create mutliattach
volumes by spcifying the ``multiattach`` parameter in the request
body of volume create operation (``--allow-multiattach`` option in
cinderclient).

This functionality changed in Queens with the introduction of
microversion 3.50[4] where we used volume types to store
the multiattach capabilities. Any volume created with a multiattach
volume type will be a multiattach volume[5].

While implementing the new functionality, we had to keep backward
compatibility with the *old way* of creating multiattach volumes.
We deprecated the ``multiattach`` (``--allow-multiattach`` on cinderclient
side) parameter in the queens release[6][7].
We also removed the support of the ``--allow-multiattach`` optional
parameter from cinderclient in the train release[8] but the API
side never removed the compatibility code to disallow functionality
of creating multiattach volumes by using the ``multiattach``
parameter (instead of a multiattach volume type).

This patch removes the support of providing the ``multiattach``
parameter in the request body of a volume create operation and will
fail with a BadRequest exception stating the reason of failure
and how it can be fixed.

[1] https://blueprints.launchpad.net/cinder/+spec/multi-attach-volume
[2] https://review.opendev.org/c/openstack/cinder/+/85847/
[3] https://review.opendev.org/c/openstack/python-cinderclient/+/85856
[4] f1bfd9790d
[5] https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html#how-to-create-a-multiattach-volume
[6] 94dbf5cce2
[7] adb141a262
[8] 3c1b417959

Depends-On: https://review.opendev.org/c/openstack/tempest/+/875372
Closes-Bug: 2008259

Change-Id: I0ece6e279048abcc04b3674108290a80eca6bd62
2023-03-06 08:58:59 +00:00
Stephen Finucane
1074e2c0e7 api-ref: Add docs for clusters
This API is currently undocumented, as I noticed while working on OSC
changes. There isn't anything too unusual going on here. The API was
first introduced in microversion 3.7 and was then extended with
replication-related information in 3.36. Both are accounted for here.

Change-Id: I86dc774c8d203839f4dadb13e93a416d112fc7b5
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Co-authored-by: Lucas de Oliveira <lucasmoliveira059@gmail.com>
Closes-Bug: 1885544
2022-08-31 10:02:20 +01:00
Zuul
2aa4922bdd Merge "Add support for transferring encrypted volumes" 2022-08-30 11:02:51 +00:00
Alan Bishop
d59e41fb3c Add support for transferring encrypted volumes
A new microversion 3.70 adds the ability to transfer a volume's
encryption key when transferring a volume to another project.

When the volume transfer is initiated, the volume's encryption
secret is essentially transferred to the cinder service.
- The cinder service creates a new encryption_key_id that contains
  a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
  encryption_key_id (the one owned by the cinder service).
- The volume's original encryption_key_id (owned by the volume's
  owner) is deleted.

When the transfer is accepted, the secret is transferred to the
user accepting the transfer.
- A new encryption_key_id is generated on behalf of the new user
  that contains a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
  encryption_key_id (the one owned by the user).
- The intermediate encryption_key_id owned by the cinder service
  is deleted.

When a transfer is cancelled (deleted), the same process is used
to transfer ownship back to the user that cancelled the transfer.

Implements: blueprint transfer-encrypted-volume
Change-Id: I459f06504e90025c9c0b539981d3d56a2a9394c7
2022-08-26 06:32:41 -07:00
Zuul
7fa996bb66 Merge "Update volume delete api-ref" 2022-08-20 15:44:03 +00:00
tushargite96
dc6388b6fd Update volume delete api-ref
This patch update volume delete section in api-ref
to understand preconditions more correctly

Partial-Bug: #1918932
Change-Id: Icb5b4da3423555cd2ab0f5ee6d1dc881fc782484
2022-08-18 15:44:42 +05:30
Gorka Eguileor
ef741228d8 Report tri-state shared_targets for NVMe volumes
NVMe-oF drivers that share the subsystem have the same race condition
issue that iSCSI volumes that share targets do.

The race condition is caused by AER messages that trigger automatic
rescans on the connector host side in both cases.

For iSCSI we added a feature on the Open-iSCSI project that allowed
disabling these scans, and added support for it in os-brick.

Since manual scans is a new feature that may be missing in a host's
iSCSI client, cinder has a flag in volumes to indicate when they use
shared targets.  Using that flag os-brick consumers can use the
"guard_connection" context manager to ensure race conditions don't
happen.

The race condition is prevented by os-brick using manual scans if they
are available in the iSCSI client, or a file lock if not.

The problem we face now is that we also want to use the lock for NVMe-oF
volumes that share a subsystem for multiple namespaces (there is no way
to disable automatic scans), but cinder doesn't (and shouldn't) expose
the actual storage protocol on the volume resource, so we need to
leverage the "shared_targets" parameter.

So with a single boolean value we need to encode 3 possible options:

- Don't use locks because targets/subystems are not shared
- Use locks if iSCSI client doesn't support automatic connections
- Always use locks (for example for NVMe-oF)

The only option we have is using the "None" value as well. That way we
can encode 3 different cases.

But we have an additional restriction, "True" is already taken for the
iSCSI case, because there will exist volumes in the database that
already have that value stored.

And making guard_connection always lock when shared_targets is set to
True will introduce the bottleneck from bug (#1800515).

That leaves us with the "None" value to force the use of locks.

So we end up with the following tristate for "shared_targets":

- True to use lock if iSCSI initiator doesn't support manual scans
- False means that os-brick should never lock.
- None means that os-brick should always lock.

The alternative to this encoding would be to have an online data
migration for volumes to change "True" to "None", and accept that there
could be race conditions during the rolling upgrade (because os-brick on
computes will interpret "None" as "False").

Since "in theory" Cinder was only returning True or False for the
"shared_target", we add a new microversion with number 3.69 that returns
null when the value is internally set to None.

The patch also updates the database with a migration, though it looks
like it's not necessary since the DB already allows null values, but it
seems more correct to make sure that's always the case.

This patch doesn't close but #1961102 because the os-brick patch is
needed for that.

Related-Bug: #1961102
Change-Id: I8cda6d9830f39e27ac700b1d8796fe0489fd7c0a
2022-05-24 15:13:23 +02:00
James Palmer
0e6ba48347 Change key "cluster" to "cluster_name"
According to the request documentation above this sample,
the key "cluster" should be "cluster_name".

Closes-Bug: #1970135
Signed-off-by: James Palmer <jpalmer7698@gmail.com>
Change-Id: Ifd64ba39a46cdbf7d75bc17baa2862790a7a39c1
2022-04-25 19:18:45 +00:00
Zuul
41e315d63a Merge "Prohibit volume manage to an encrypted volume type" 2022-03-21 20:07:22 +00:00
Yadiel Xuan(轩艳东)
8088dc9580 Prohibit volume manage to an encrypted volume type
Managing a volume to an encrypted volume type should not be allowed.
One reason is that there is no way for an operator to specify an
encryption key ID for the volume. Another is that we already don't
allow a volume of an encrypted type to be un-managed, so this change
will be symmetric.

Also update and correct the api-ref for this call.

Co-authored-by: Yadiel Xuan(轩艳东) <xuanyandong@inspur.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Change-Id: Ic2da41f3962c1108f974aca952bce3da6d6ac277
Closes-bug: #1944577
2022-03-21 11:48:01 -04:00
Zuul
fb9e391476 Merge "api-ref: include links on backups list and details" 2022-03-18 20:02:21 +00:00
Yikun Jiang
d69e89ea3b Support volume re-image
This patch adds volume re-image API to enable the ability to
re-image a specific volume.

Implements: blueprint add-volume-re-image-api

Co-Authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>

Change-Id: I031aae50ee82198648f46c503bba04c6e231bbe5
2022-02-24 15:23:38 +05:30
Zuul
5b89039e9e Merge "Remove the need for project_id from API endpoints" 2022-02-23 15:40:46 +00:00
Alan Bishop
31b34e91e0 Remove the need for project_id from API endpoints
Inclusion of a project_id in API URLs is now optional, and no longer
required. Removing the project_id requirement facilitates supporting
Secure RBAC's notion of system scope, in which an API method is not
associated with a specific project.

The API v3 routing is enhanced to provide duplicate routes for API
methods that traditionally required a project_id in the URL:
- The existing route for which a project_id is in the URL
- A new route for when the URL does not include a project_id

To test both routes and ensure there are no regresssions, the "API
samples" functional tests include a project_id in the URLs, and the
rest of the functional tests do not include the project_id. This is
handled by changing the 'noauth' WSGI middleware to no longer add the
project_id, and adding a new 'noauth_include_project_id' middleware
filter that implements the legacy behavior.

A new microversion V3.67 is introduced, but it only serves to inform
clients whether the project_id is optional or required. When an API
node supports mv 3.67, the project_id is optional in all API requests,
even when the request specifies a earlier microversion. See the spec
Ia44f199243be8f862520d7923007e7182b32f67d for more details on this
behavior.

Note: Much of the groundwork for this is based on manila's patch
I5127e150e8a71e621890f30dba6720b3932cf583.

DocImpact
APIImpact

Implements: blueprint project-id-optional-in-urls
Change-Id: I3729cbe1902ab4dc335451d13ed921ec236fb8fd
2022-02-08 08:32:23 -08:00
Lucas Oliveira
1f67b918ac api-ref: include links on backups list and details
This patch adds backups_links optional on parameters.yaml and
include as response parameter on v3 backups list(/backups)
and details(/backups/detail).

Closes-Bug: 1944463
Change-Id: I29d2fb4b33f508638d703659a7717edf2891527d
2022-01-24 11:43:29 -03:00
Zuul
938463e36c Merge "Reject bad img formats for uploaded encrypted vols" 2022-01-11 17:19:38 +00:00
whoami-rajat
104e3e049d Add default types info in admin docs
This patch adds information about default volume types i.e. project
default and deployment default (set via default_volume_type in cinder.conf)
and also regarding __DEFAULT__ type.

Change-Id: I47151b621003aea04082d38bd1d5cc911afab0f4
2021-11-19 05:13:46 -05:00
whoami-rajat
ba28f29254 api-ref: Add info regarding default types
This patch adds information regarding the default volume types
(including __DEFAULT__) in the api-ref.
The sections modified are volume create, volume type list, show
default volume type and delete volume type.

Change-Id: I896c809bd4ff09e2f26398a1cdaa9a91dc0f8638
2021-11-18 04:19:51 -05:00
Eric Harney
9ed7c16ef0 Snapshot in-use volumes without force flag
Introduces API microversion 3.66, which allows
snapshot creation on in-use volumes without the
force flag being passed.

Co-authored-by: Eric Harney <eharney@redhat.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Implements: blueprint fix-snapshot-create-force
Change-Id: I6d45aeab065197a85ce62740fc95306bce9dfc45
2021-08-31 14:48:32 -04:00
Zuul
d5f22c6652 Merge "Expose volume & snapshot use_quota field" 2021-08-29 08:39:44 +00:00
Zuul
244a75fd6b Merge "Correct the rest api url of import_record of backup" 2021-08-28 15:56:40 +00:00
xuanyandong
400008194b Correct the rest api url of import_record of backup
Change-Id: I16cdd1bc19bf51b6012d84a7163fa5d2a710800a
2021-08-28 02:42:14 +00:00
Gorka Eguileor
ec44fc8999 Expose volume & snapshot use_quota field
This patch adds a new REST API microversion to expose the new use_quota
DB field present in volumes and snapshots.

The field will be reported when listing and showing resources and can be
used when filtering as well.

The field is exposed in the REST API as `consumes_quota` to prevent
confusion for users and admins, since exposing it as `use_quota` may
give them the wrong impression that they can set it up for their own
purposes.

For users we say what is happening with the quota for that resource -it
consumes or doesn't consume quota-, whereas internally we express
instruction to the core code -whether to use quota or not-, hence the
difference in the naming.

APIImpact
DocImpact
Implements: blueprint temp-resources
Change-Id: I655a47fc75ddc11caf1defe984d9a66a9ad5a2e7
2021-08-26 19:04:12 +02:00
Gorka Eguileor
d29f12117e Update snapshot api microversion docs and tests
Our docs and tests are not up to date with the changes introduced to the
snapshot details introduced by the different microversion.

This patch updates the docs, samples, and tests.

Change-Id: I46fa6e1f7a5d6ad0147f17c2bafbf4fb8fa89eb8
2021-08-25 15:59:56 +02:00
Zuul
3bb14f2a9c Merge "Doc: api-ref doc update" 2021-08-20 17:18:06 +00:00
Stephan Pampel
7b9143fae9 Doc: Remove backup_id from backup import url and parameter
In the documentation for importing backups the backup_id is removed
because the backup_id is generated after the upload and the user
does not need to provide it. Therefore the request parameter
backup_id has also been removed.

Closes-Bug: #1930526
Change-Id: I49e0adea8fbd97d49f4386b8240320ea2baf3661
2021-08-09 12:03:39 +02:00
Zuul
4896811eab Merge "Update volume api microversion doc and tests" 2021-08-03 17:19:57 +00:00
Zuul
776d56ff69 Merge "Update Block Storage API v2 api-ref" 2021-07-22 05:18:17 +00:00
Zuul
196a58dae1 Merge "Remove Block Storage API v2" 2021-07-22 05:18:14 +00:00
tushargite96
98de5e08bf Doc: api-ref doc update
Volume multi-attach support was added in cinder and
nova in queens release.

Change-Id: I485e87b9a80bb4acec7f9bfde188e601663becb7
2021-07-19 14:13:42 +05:30
Brian Rosmaita
de8b3b0b00 Reject bad img formats for uploaded encrypted vols
Cinder only supports uploading volumes of encrypted volume types as
images with disk format 'raw' and container format 'bare'.  Screen
for this at the REST API layer when the request is made.

Change-Id: Ibb77b8b1be6c35c5db3b07fdc4056afd51d48782
Closes-bug: #1935688
2021-07-09 14:51:51 -04:00
Zuul
7a854c11ed Merge "Fix: Schema validation for attachment create API" 2021-06-17 19:21:57 +00:00
Rajat Dhasmana
560318c82e Fix: Schema validation for attachment create API
Currently the schema validation for attachment create assumes that
instance UUID will always be present in the request but that is
not the case when glance calls cinder for attachment.
Also there isn't any schema validation for MV3.54 which accepts
attachment mode in the request hence failing requests passing mode.
This patch removes the instance_uuid from required parameters and
adds schema validation for MV3.54.

Change-Id: I5108fd51effa4d72581654ed450d191a13e0e964
2021-06-16 11:26:21 -04:00
Brian Rosmaita
dbf78b8c28 Update Block Storage API v2 api-ref
Add a note that the v2 API was removed in Xena.

Change-Id: I3dd81d81995913e30addf5bf4ee8a4406c36632f
2021-06-04 18:18:07 -04:00
Brian Rosmaita
e05b261af7 Remove Block Storage API v2
In this patch:
- adjusted VersionsController to return only v3
- removed cinder.api.v2.router
- adjustments to cinder.tests.unit.api.contrib to use /v3 only
- moved cinder.api.v2.snapshot_metadata (and tests) to cinder.api.v3
- moved cinder.api.v2.types (and view, tests) to cinder.api.v3
- updated versions response in api-ref
- removed unnecessary config option
- updated various sample config files
- removed experimental tempest-cinder-v2-api job
- updated some docs
- updated non-voting rally job config

Some cinder.api.v2 modules are left because the v3 classes depend on
them, but with the v2 router removed, these are unreachable via the
/v2 path.

Depends-on: https://review.opendev.org/c/openstack/rally-openstack/+/794891
(changes rally to use Block Storage API v3)
Depends-on: https://review.opendev.org/c/openstack/requirements/+/794894
(corrects regression in upper-constraint on Sphinx)

Change-Id: I2093d77db9beec7543c7524d2cd273e79dd5fd5d
2021-06-04 17:21:28 -04:00
Gorka Eguileor
dab5df1747 Update volume api microversion doc and tests
We have many microversions but we but we haven't kept the samples, docs,
and tests up to date with them.

Of the many microversions this patch will focus on the ones that have
updated the fields returned for volumes updating the docs, adding the
appropriate json samples, updating the tests, and updating the
contributor docs to highlight the need to keep these up to date.

Some convenience methods have been added to facilitate the testing:

- use_versions: class decorator that allows us to run the same tests
  with different microversions, each using their respective json and
  templates.

- override_mv: method decorator to change the microversion in a single
  test.

- common_api_sample: context manager to use the base sample instead of a
  microversion specific one.

Change-Id: Ide303685a4cd1477c319b36e67a3b05c8d129126
2021-05-31 12:41:46 +02:00
Gorka Eguileor
da2fbff247 Fix functional jobs
In Change-ID I8db34f31d35375dc9656d53ac7af644c28a3d3fe we updated
api-ref samples, but functional tests didn't run so we didn't detect
that those samples changes broke our functional tests.

This patch fixes the functional tests, updates remaining cases of
key_size, and enables functional tests for api-ref changes to prevent
this from happening again.

Change-Id: I2a1f3121458c2db5e389dbcf31b34d412ffcd561
2021-05-25 16:55:05 +02:00