Commit Graph

58 Commits (master)

Author SHA1 Message Date
Eric Harney 6b067f2b47 Import HTTPStatus instead of http_client
We don't need an HTTP client for this, we only
need status codes.  Just load that instead.

Change-Id: Ia0a00a820fff55e25d352d2bf472054ef7074f34
3 years ago
xuanyandong a85ce6c817 Remove six of dir cinder/api/*
Replace the following items with Python 3 style code.

- six.string_types
- six.text_type
- six.moves
- six.PY3
- six.unichr

Change-Id: Ic46ec5f13a0dc1986d9c01e0d11df827d7e17bb0
3 years ago
Eric Harney de789648e5 Rename volume/ to volume/
Much of our code renames this at import already --
just name it "volume_utils" for consistency, and
to make code that imports other modules named "utils"
less confusing.

Change-Id: I3cdf445ac9ab89b3b4c221ed2723835e09d48a53
4 years ago
Eric Harney 90f962553a Refactor API utilities into
This collects up utils that are only used in the API code,
and moves them from cinder/ to here, for better code

Change-Id: Iecd909ef4e24d5597dcccb80a671a27f361e4b4d
4 years ago
pooja jadhav c1356eeca9 Remove leading and trailing spaces from parameters
Snapshots and backup (create/update) APIs stores 'name' and
description' parameters with leading and trailing spaces in
database after patches [1][2] are merged.

This patch removes leading and trailing whitespaces before
validation and also before storing it in db for 'name' and
'description' parameters.


Closes-Bug: #1742907
Change-Id: Ib5c8d32e2c20cbacebd7b6f7281a44f94a70f7c0
5 years ago
Neha Alhat 94460b64fb V2/V3 jsonschema validation: snapshots
This patch adds jsonschema validation for below Snapshots API's
* POST /v3/{project_id}/snapshots
* PUT  /v3/{project_id}/snapshots/{snapshot_id}
* POST /v2/{project_id}/snapshots
* PUT  /v2/{project_id}/snapshots/{snapshot_id}

Note: For create API 'volume_id' parameter is required, if user
passes 'volume_id' as null, the existing behavior is it returns
"itemNotFound"(404) error response. If we restrict user to accept
volume_id in uuid format only in schema validation then in that case,
if user passes 'volume_id' as null it will raise BadRequest(400)
which will not match existing behavior and also tempest test case[1]
will fail in this case. Also on master if user passes 'metadata' as
null then it is accepted. To maintain consistency we are allowing
'volume_id' and 'metadata' to be null in schema.

Made changes to unit tests to pass body as keyword argument as wsgi
calls action method [2] and passes body as keyword argument.

Partial-Implements: bp json-schema-validation

Change-Id: I9869ea08afe3c8f8f1f63bb2ead7f63388580937
6 years ago
poojajadhav 92980e4972 Extracted HTTP response codes to constants
There are several places in the source code where HTTP response
codes are used as numeric values.

Status codes 200, 202, 204, 300, 400, 401, 403, 404, 405, 409, 413,
415, 500, 501, 503 under api/v1 and api/v2 are replaced with symbolic
constants from six.moves.http_client thus improves code readability.
More patches will be submitted to address other status codes.

Partial-Bug: #1520159
Change-Id: I7c61122a6b043d7d238bea95ef39d8fa97817df4
6 years ago
Sean McGinnis a55a6b5c71 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.


Change-Id: I4c96f3590d46205c45d12ee4ead8c208e11c52c5
6 years ago
bhagyashris 1a495922ed Remove context object in oslo.log method
Removed context object while logging as Cinder uses oslo.context's
RequestContext which means the context object is in scope when doing

Change-Id: I7cc434ad10967596f8354775399e0c6c92ab5570
7 years ago
Jenkins 8a34a0f3da Merge "Replace functions 'Dict.get' and 'del' with 'Dict.pop'" 7 years ago
Xu Ao dced19aec3 Replace functions 'Dict.get' and 'del' with 'Dict.pop'
Refactoring code: Making volume dict to use single instruction: pop()
rather than two instructions: get() and del, giving the codes a format
that carries through.

Change-Id: Ic889dac330eac24d15bd73186fc504402cddd861
7 years ago
Gorka Eguileor c491c3ee61 Handle API NotFound exceptions at WSGI level
Throughout the API code we keep catching NotFound exceptions in their
various forms and converting them to webob.exc.HTTPNotFound exceptions,
but we can leave the WSGI fault handler convert them on its own.

This patch changes current behavior and removes the exception handling
closer to the operation so that those exceptions can be handled at the
WSGI level.

This has the following benefits:

- Reduces code complexity
- Increases code readability
- Provides consistent error responses, as messages are stored on the
- Prevents raising errors with only partial information (we have cases
  now that were removing the UUID from the message because they used a
  custom message).  For example: before returned error would be "The
  resource could not be found", and now we raise "Volume type encryption
  for type 4e9e6d23-eed0-426d-b90a-28f87a94b6fe does not exist."
- Reduces workload for the translation team because we remove all
  unnecessary custom messages.

Change-Id: I09f98921fdc2400cc3f6056e59001100abe06920
7 years ago
Ivan Kolodyazhny c042a05ac3 Remove XML API
Cinder XML API is not tested by Tempest about for one year.
We don't know if it works or not. It was deprecated in
Mitaka release.

Implements blueprint: remove-xml-api

Change-Id: If98db25f1f4032725444c3959acb9aad8f869802
7 years ago
Victor Stinner 30c0f79781 Port API v1 and v2 to Python 3
* Replace jsonutils.dumps() with jsonutils.dump_as_bytes() since
  output is used for the HTTP body, and HTTP body type is bytes, not
  Unicode. On Python 3, jsonutils.dumps() returns Unicode.
* FakeHttplibSocket: replace io.StringIO with io.BytesIO, HTTP body
  type is bytes. Encode Unicode to UTF-8.
* Fix FakeHttplibSocket.makefile() API: only the first parameter is
  mandatory. On Python 3, it's only called with the first 'mode'
* Use bytes strings to test HTTP bodies (JSON or XML).
* Replace dict.values() with list(dict.values()) to get a list on
  Python 3.
* Replace error.message with encodeutils.exception_to_unicode(error),
  exceptions loose their message attribute in Python 3.
* Update JSON serializer test in to use bytes.
* tests-py3.txt: add cinder.tests.unit.api.v1 and

Partial-Implements: blueprint cinder-python3
Change-Id: I0f0048f4a1344feaa3434cbf7ebd31e3f12d6ae4
7 years ago
Gorka Eguileor e0e73a534a Add pagination to snapshots
Currently snapshot list does not provide pagination like volume does.

This patch adds pagination for snapshots in the same way that volume
does, using marker, limit, sort_keys and sort_dirs.

APIImpact: Use marker, limit, sort_keys and sort_dirs
Implements: snapshots-and-bakcup-support-pagination-query
Implements: blueprint extend-limit-implementations
Change-Id: I21edac6d29abf7fc2f446111a7d6888db9ab6eba
8 years ago
Gorka Eguileor 433cf95e06 Adapt SnapshotController to view builder
Snapshots were not using the view builder to translate responses, unlike
other resources like volumes and backups.

This patch moves snapshots to use the view builder like the rest.

Change-Id: Iec6873aedcf126f00c81fd016acfc8db9a5e512c
8 years ago
Jenkins edd25e4688 Merge "Validate name and description string" 8 years ago
PranaliDeore a1bb185a1f Validate name and description string
If you pass name or description parameters with more than 255
characters to create and update apis of volume and snapshot
and create api of backup, then it returns 500 error code.

Added new method validate_name_and_description() in
cinder.api.openstack.wsgi.Controllera to validate string limit and
returned 400 if limit exceeds and also removing leading or trailing
whitespaces and string containing only whitespaces.

1. For all above APIs 400 response will be returned.
2. Earlier it was possible to pass only whitespaces or leading-trailing
   spaces to 'name' parameter.
   Now it will raise 400 error if only whitespaces are passed and will
   remove leading-trailing spaces if present in other cases.

Closes-Bug: 1454244
Change-Id: Iaf7159e816f69fd776a09828c3bc1d27fc9fdcdb
8 years ago
Abhijeet Malawade a5f3ef5778 Remove unused context parameter
Context parameter is removed from cinder methods
which are not using it at all.

Removed 'notification' method from volume/ as it is
not getting used anywhere.

Change-Id: I2ecfdabb876485ac0c0d1cacd6778408be061fc8
Closes-Bug: #1458844
8 years ago
Jenkins 2596c5ed1c Merge "Validate outermost request body element name consistently" 8 years ago
Jenkins f3405653cd Merge "Validate bool value using strutils.bool_from_string method" 8 years ago
Jenkins e66daeb6dc Merge "Change generic NotFound to specific exception" 8 years ago
Lena Novokshonova 1a4c2e82f1 Add notifications about snapshot.update.*
Now we have notifications about volume.update.*, but don't have
notifications about snapshot.update.*
In this patchset added this notifications.

Change-Id: Idc17516436b06871421c17cde0e2128f81613716
8 years ago
PranaliDeore cb8bce4f1b Change generic NotFound to specific exception
1. Changed exception type from NotFound to specific NotFound
   exception for better readability.
   e.g. VolumeNotFound, SnapshotNotFound etc.

2. Error messages returned to the user are not consistent across
   all apis in case of all exceptions derived from NotFound exception.
   Instead of returning custom error messages return message
   defined in exception class itself.
   NOTE: Doesn't applied to 'HostBinaryNotFound' as host information to
         the tenant should not be leaked.

3. Added explanation to HTTPNotFound wherever appropriate.
   e.g. raise webob.exc.HTTPNotFound() to
        raise webob.exc.HTTPNotFound(explanation=err.msg)

Closes-Bug: #1459958
Change-Id: Ice2c375796fac7533d82125ef97288075fa68b85
8 years ago
Abhijeet Malawade 7d203027c7 Validate outermost request body element name consistently
1. Return useful error message instead of empty error message.
ex.- Return "Missing required element '<element name>' in request body"
     instead of "Bad Request"
2. Return consistent error message in case request body element
name is not present.
3. Added 'assert_valid_body' method to raise HTTPBadRequest exception
if request body is not valid. It is not possible to modify
'is_valid_body' method itself to raise exception because V1 and v2
api's return different error code when 'is_valid_body' returns False.


Closes-bug: #1461402
Change-Id: I81764cd8517484fbee4beae2b30668c8b180d677
8 years ago
Rajesh Tailor 24fd26fc98 Validate bool value using strutils.bool_from_string method
bool value is validated with different logic scattered all over the places
1. isinstance of six.string_type or bool
2. is_valid_boolstr method

Made changes to use strutils.bool_from_string method to validate bool
value at all places as this method does all things expected above.
Changes are not made in v1 api as it is going to be deprecated in
Liberty release.

Example error message:
Invalid value for force_host_copy: Unrecognized value 'xyz', acceptable
values are: '0', '1', 'f', 'false', 'n', 'no', 'off', 'on', 't', 'true',
'y', 'yes'

ApiImpact: 400 BadRequest response will be returned in case invalid force
value is provided during volume upload to image, earlier it was ignored.

Closes-Bug: 1460575
Change-Id: Ie64d429f5860a950cda9b363a65c5282f128f10b
8 years ago
Thang Pham b8a58fb01f Switch get_all_snapshots to use objects
The following patch switches direct db calls in
volume/ get_all_snapshots to use SnapshotList

Partial-Implements: blueprint cinder-objects
Change-Id: Ifdccc81a087f9797fcfbf098d3ecea3875ad7bd9
8 years ago
Sean McGinnis 3578d9e341 Logging not using oslo.i18n guidelines
Part of multi-patch set for easier chunks.

There have been quite a few instances found where the
i18n guidelines are not being followed. I believe this
has helped lead to some of the confusion around how to
correctly do this. Other developers see this code and
assume it is an example of the correct usage.

This patch attempts to clean up most of those violations
in the existing codebase to hopefully help avoid some of
that confusion in reviews.

Some issues address:
* Correct log translation markers for different log levels
* Passing format values as arguments to call, not preformatting
* Not forcing translation via six.text_type and others

Guidelines can be found here:

Hacking checks will not be able to identify all violations of
the guidelines, but it could be useful for catching obvious ones
such as"No markers!").

Change-Id: I38f52c6408b47ccb59ec2064b360f7d4427d6830
Partial-bug: 1433216
8 years ago
Ivan Kolodyazhny 5980da5d88 Use oslo.log instead of oslo-incubator
log module was removed from oslo-incubator after oslo.log released.

Change-Id: I205d0625c502cb462919edc76d12091edcc21b1b
Implements: blueprint port-oslo-incubator-to-oslo-log
Partial-Bug: #1381563
8 years ago
Thang Pham 21cda264dd Snapshot and volume objects
Abstract volumes and volume snapshots into objects.
Get, create, and delete snapshot APIs were changed to
use new snapshot objects.  A skeleton volume object was
created, but cinder internals were not changed to use
the volume object, although volume is referenced and
used by the snapshot object.  The internals will be
changed to use volume object in a subsequent patch.

Change-Id: I387018e80c8539565e99454db65d976030002c0f
Implements: blueprint cinder-objects
8 years ago
Yuriy Nesenenko 84afca21fd Fix comments style according to the Hacking Rules
According to the PEP8(E265) there should be at least
one space before block comment.

Change-Id: Ic51f80210becc375b30f0f4e9eeb54995775c817
Partial-Bug: #1407162
8 years ago
Jay S. Bryant b6ec77e6d4 Move oslo.utils to oslo_utils namespace
This is the fourth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.

There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away.  Thus, we need patches like this one to
move to the new namespace.

This patch also updates our hacking check to ensure that no instances
of oslo.utils sneak back in.

Change-Id: I19c331a9b54c5f42c278577ecc59fe1b65e61af0
9 years ago
ChangBo Guo(gcb) 7d341e267f Use oslo.utils
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some of these outdated modules will still be
around for a while until all other oslo modules that use them have been
updated in future commits.

Change-Id: Idee8600dfe42e5977b8fb824e91afff7e9119981
9 years ago
Mike Mason abedc10343 Implementing the use of _L’x’/i18n markers
Placing the _Lx markers back into the code.  No other cleaner solution has
has been implemented. Patches will be submitted in a series of sub
directories and in a fashion that is manageable.
This is the third commit of this kind

Change-Id: I33b0bb05b1c1ea91dc5a1e93eaadceea1f23fcc9
Partial-Bug: #1384312
9 years ago
Mathieu Gagné 4aeb7256ef Improve Cinder API internal cache interface
Improve the internal caching system used by the API layer
by borrowing the implementation used by Nova.

Unlike the previous implementation, this new interface makes it clear
which resource types are added or retrieved from the cache.

This change also adds other resources to the cache:
- backups
- snapshots
- volume types

It's now possible to remove database access those extensions:
- extended_snapshot_attributes
- volume_host_attribute
- volume_mig_status_attribute
- volume_tenant_attribute

Closes-bug: #1358524
Change-Id: I1fc673d0c01c41faa98292d5813d4471b455d712
9 years ago
John Griffith d68efcc0e0 Fix variable name in api/v<n>/
Noticed in a recent bug fix that the when the show
method was copy/pasted at some point the variable name
vol was not updated.

Silly annoyance, but thought I'd go ahead and replace vol
with snapshot to make the variable name reflect what's
actually in use here.

Change-Id: I137ecc4987258450e93195cfed5e128063b92914
9 years ago
Vishvananda Ishaya 4f0d1d3613 Cache snapshots in request for extension
The extended snapshots extension makes a separate db request for
snapshots which is horribly inefficent. In addition it causes an
issue when using all_tenants=1, because the second request doesn't
pass in the filter. Fix this by caching the snapshot in the request
object like we do for volumes and retrieve the cached object in the

Change-Id: Ia943438c8d48accb7f62c43f07642c3da2fe34ab
Resolves-bug: #1358960
9 years ago
James Carey ac33ad9ec9 Use oslo.i18n
oslo.i18n provides the i18n function that were provided by
oslo-incubator's gettextutils module.

Change-Id: I1f361a8321fb02f03b4f3f3e2ef688fcf19514a3
9 years ago
Christian Berendt 4dc37abcd5 Change logging level AUDIT to INFO
According to the latest revision of the logging guidelines message
of type AUDIT should be moved to INFO.

The latest revision is available at
at the moment.

Change-Id: I2d86fa634f6424b0bb862319b9ae40d3f3b1ec7a
9 years ago
Jay S. Bryant eafcc41e82 Explicitly import _() in Cinder code
To ensure that logs are properly translated and logged to
the Cinder log files (using the Cinder message catalogs)
we need to explicitly import _() in any python files that
use the _() function.

Closes-bug: 1306275
Related-Blueprint: i18n-enablement
Change-Id: I3cf1f1d63d5f8bd8ffd007104691f914e3fd8538
9 years ago
Jenkins 5a217881fb Merge "Fix docstring for snapshots API" 9 years ago
Tom Fifield 0247469109 Fix docstring for snapshots API
For some reason, the snapshots controller class' introductory
docstring stated that it was the volume API controller.
This patch changes it to say:
"The Snapshots API controller for the OpenStack API."

Change-Id: I8c9827939bbc7acf93282db854a40501f3fea399
9 years ago
Juan Manuel Olle 5b05b372c8 Adding filter options to backup list
This patch adds filter options to backup list.
Like cinder list, now cinder backup-list accept
This blueprint is complimented with the client part

blueprint: add-filter-options-to-backup-list

Change-Id: I9e095ee44a03744eb3f319323d9130a89196614e
9 years ago
john-griffith e6a1ba5291 Handle NotFound exception in snapshots API code
Passing tests in the gate leave unhandled trace/error messages
for a VolumeNotFound issue.

These are caused by the test_volumes_snapshots_negative test
which requests a non-existent volume and the tests pass however
the log files are a bit messed up due to the unhandled exception.

This change just adds a try/catch block around the volume get calls
in the snapshot modules and raises HTTPNotFound as appropriate.

Change-Id: I2096f2da7c68ef7924fc8e69b2d5c2afea578512
Closes-Bug: #1255214
10 years ago
huangtianhua 381e717b4c Create snapshot throws 500 Internal Error
The server doesn't check whether the parameter "volume_id" is in request body.
So the 500 error has been thrown.

We should catch the KeyError and transfer the KeyError to 400(HTTPBadRequest)
instead of 500.

Change-Id: I8a1dde1fd6ed820b39995af434efacc2a27c9604
Closes-Bug: #1252179
10 years ago
Ken'ichi Ohmichi 622f10e87f Allow display_name for v2 snapshot-update
According to the comment in the source code, the snapshot-update of
v2 would be able to allow either "name" or "display_name" parameter.
And if both paramters are specified, "name" parameter is effective
over "display_name" like "description" parameter.
However "display_name" parameter is ignored now, because cinder writes
over empty dict(update_dict['display_name']).

This patch fixes the problem.

Fixes bug #1226398

Change-Id: Icd48556a26b9346f73b7c1b82ffbcdd469c4d28d
10 years ago
Luis A. Garcia 681a898101 Externalize error messages in the v2 API
This patch does more internationalization for the REST API error
messages that don't currently have it to take advantage of the new
support added by bp user-locale-api to show error messages in the locale
requested by the user through the Accept-Language HTTP header.

We only do v2 because consumers have used the response error message in
the past for error checks, so changing it in v1 too would break them.

Partially implements bp user-locale-api

Change-Id: I92780b42c125a91ab4916b7a31e4b71d306a89a1
10 years ago
xiaoxi_chen 08af981ffc Pop out 'offset' and 'limit' before use for filter
In previous code of _items() from api/v{1,2}/,
and also the _items)_ from api/v1/ didn't pop
out the 'offset' and 'limit' fields from HTTP get params
before we use such params for filter.This is the root cause
for bug #1205956

For non-admin user, since 'offset' and 'limit' is not in the
allowed_search_options, so the volumes.remove_invalid_options
will help to filter them out. As a result, it walks around this

But for admin user,the volumes.remove_invalid_options will not
try to filter the search_options.So for admin user, the 'limit'
will appear in search_options, then obviously get no result.

fixed bug #1205956

Change-Id: Ib1a66c9d104ac52d6eae18be7f06d02985d4c2fd
10 years ago
Sergey Vilgelm 67f39bf1f2 Replace FLAGS with cfg.CONF in api
Replace all the FLAGS with cfg.CONF in cinder/api
Large commit was split into several parts

Change-Id: I2114d5fc45f5299c3b2011270034a3370e0ec388
Fixes: bug #1182037
10 years ago
Michael Still b0325eb772 Convert to oslo strutils.bool_from_string.
The only functional difference here is that the value '2' is now
treated as False instead of True. I suspect that's ok.

Change-Id: I30e75470cb3c312d7f92d389995eb6d845dd2fb0
10 years ago