We don't need an HTTP client for this, we only
need status codes. Just load that instead.
https://docs.python.org/3/library/http.html
Change-Id: Ia0a00a820fff55e25d352d2bf472054ef7074f34
Much of our code renames this at import already --
just name it "volume_utils" for consistency, and
to make code that imports other modules named "utils"
less confusing.
Change-Id: I3cdf445ac9ab89b3b4c221ed2723835e09d48a53
This collects up utils that are only used in the API code,
and moves them from cinder/utils.py to here, for better code
organization.
Change-Id: Iecd909ef4e24d5597dcccb80a671a27f361e4b4d
Snapshots and backup (create/update) APIs stores 'name' and
description' parameters with leading and trailing spaces in
database after patches [1][2] are merged.
This patch removes leading and trailing whitespaces before
validation and also before storing it in db for 'name' and
'description' parameters.
[1]https://review.openstack.org/#/c/520991/
[2]https://review.openstack.org/#/c/530098/
Closes-Bug: #1742907
Change-Id: Ib5c8d32e2c20cbacebd7b6f7281a44f94a70f7c0
This patch adds jsonschema validation for below Snapshots API's
* POST /v3/{project_id}/snapshots
* PUT /v3/{project_id}/snapshots/{snapshot_id}
* POST /v2/{project_id}/snapshots
* PUT /v2/{project_id}/snapshots/{snapshot_id}
Note: For create API 'volume_id' parameter is required, if user
passes 'volume_id' as null, the existing behavior is it returns
"itemNotFound"(404) error response. If we restrict user to accept
volume_id in uuid format only in schema validation then in that case,
if user passes 'volume_id' as null it will raise BadRequest(400)
which will not match existing behavior and also tempest test case[1]
will fail in this case. Also on master if user passes 'metadata' as
null then it is accepted. To maintain consistency we are allowing
'volume_id' and 'metadata' to be null in schema.
Made changes to unit tests to pass body as keyword argument as wsgi
calls action method [2] and passes body as keyword argument.
[1] https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_snapshots_negative.py#L42
[2] https://github.com/openstack/cinder/blob/master/cinder/api/openstack/wsgi.py#L997
Partial-Implements: bp json-schema-validation
Change-Id: I9869ea08afe3c8f8f1f63bb2ead7f63388580937
There are several places in the source code where HTTP response
codes are used as numeric values.
Status codes 200, 202, 204, 300, 400, 401, 403, 404, 405, 409, 413,
415, 500, 501, 503 under api/v1 and api/v2 are replaced with symbolic
constants from six.moves.http_client thus improves code readability.
More patches will be submitted to address other status codes.
Partial-Bug: #1520159
Change-Id: I7c61122a6b043d7d238bea95ef39d8fa97817df4
Removed context object while logging as Cinder uses oslo.context's
RequestContext which means the context object is in scope when doing
logging.
Change-Id: I7cc434ad10967596f8354775399e0c6c92ab5570
Closes-Bug:#1500896
Refactoring code: Making volume dict to use single instruction: pop()
rather than two instructions: get() and del, giving the codes a format
that carries through.
Change-Id: Ic889dac330eac24d15bd73186fc504402cddd861
Closes-Bug:#1571548
Throughout the API code we keep catching NotFound exceptions in their
various forms and converting them to webob.exc.HTTPNotFound exceptions,
but we can leave the WSGI fault handler convert them on its own.
This patch changes current behavior and removes the exception handling
closer to the operation so that those exceptions can be handled at the
WSGI level.
This has the following benefits:
- Reduces code complexity
- Increases code readability
- Provides consistent error responses, as messages are stored on the
Exceptions.
- Prevents raising errors with only partial information (we have cases
now that were removing the UUID from the message because they used a
custom message). For example: before returned error would be "The
resource could not be found", and now we raise "Volume type encryption
for type 4e9e6d23-eed0-426d-b90a-28f87a94b6fe does not exist."
automatically.
- Reduces workload for the translation team because we remove all
unnecessary custom messages.
Change-Id: I09f98921fdc2400cc3f6056e59001100abe06920
Cinder XML API is not tested by Tempest about for one year.
We don't know if it works or not. It was deprecated in
Mitaka release.
Implements blueprint: remove-xml-api
APIImpact
DocImpact
UpgradeImpact
Change-Id: If98db25f1f4032725444c3959acb9aad8f869802
* Replace jsonutils.dumps() with jsonutils.dump_as_bytes() since
output is used for the HTTP body, and HTTP body type is bytes, not
Unicode. On Python 3, jsonutils.dumps() returns Unicode.
* FakeHttplibSocket: replace io.StringIO with io.BytesIO, HTTP body
type is bytes. Encode Unicode to UTF-8.
* Fix FakeHttplibSocket.makefile() API: only the first parameter is
mandatory. On Python 3, it's only called with the first 'mode'
parameter.
* Use bytes strings to test HTTP bodies (JSON or XML).
* Replace dict.values() with list(dict.values()) to get a list on
Python 3.
* Replace error.message with encodeutils.exception_to_unicode(error),
exceptions loose their message attribute in Python 3.
* Update JSON serializer test in test_wsgi.py to use bytes.
* tests-py3.txt: add cinder.tests.unit.api.v1 and
cinder.tests.unit.api.v2
Partial-Implements: blueprint cinder-python3
Change-Id: I0f0048f4a1344feaa3434cbf7ebd31e3f12d6ae4
Currently snapshot list does not provide pagination like volume does.
This patch adds pagination for snapshots in the same way that volume
does, using marker, limit, sort_keys and sort_dirs.
DocImpact
APIImpact: Use marker, limit, sort_keys and sort_dirs
Implements: snapshots-and-bakcup-support-pagination-query
Implements: blueprint extend-limit-implementations
Change-Id: I21edac6d29abf7fc2f446111a7d6888db9ab6eba
Snapshots were not using the view builder to translate responses, unlike
other resources like volumes and backups.
This patch moves snapshots to use the view builder like the rest.
Change-Id: Iec6873aedcf126f00c81fd016acfc8db9a5e512c
If you pass name or description parameters with more than 255
characters to create and update apis of volume and snapshot
and create api of backup, then it returns 500 error code.
Added new method validate_name_and_description() in
cinder.api.openstack.wsgi.Controllera to validate string limit and
returned 400 if limit exceeds and also removing leading or trailing
whitespaces and string containing only whitespaces.
APIImpact
1. For all above APIs 400 response will be returned.
2. Earlier it was possible to pass only whitespaces or leading-trailing
spaces to 'name' parameter.
Now it will raise 400 error if only whitespaces are passed and will
remove leading-trailing spaces if present in other cases.
Closes-Bug: 1454244
Change-Id: Iaf7159e816f69fd776a09828c3bc1d27fc9fdcdb
Context parameter is removed from cinder methods
which are not using it at all.
Removed 'notification' method from volume/manager.py as it is
not getting used anywhere.
Change-Id: I2ecfdabb876485ac0c0d1cacd6778408be061fc8
Closes-Bug: #1458844
Now we have notifications about volume.update.*, but don't have
notifications about snapshot.update.*
In this patchset added this notifications.
Change-Id: Idc17516436b06871421c17cde0e2128f81613716
1. Changed exception type from NotFound to specific NotFound
exception for better readability.
e.g. VolumeNotFound, SnapshotNotFound etc.
2. Error messages returned to the user are not consistent across
all apis in case of all exceptions derived from NotFound exception.
Instead of returning custom error messages return message
defined in exception class itself.
NOTE: Doesn't applied to 'HostBinaryNotFound' as host information to
the tenant should not be leaked.
3. Added explanation to HTTPNotFound wherever appropriate.
e.g. raise webob.exc.HTTPNotFound() to
raise webob.exc.HTTPNotFound(explanation=err.msg)
Closes-Bug: #1459958
Change-Id: Ice2c375796fac7533d82125ef97288075fa68b85
1. Return useful error message instead of empty error message.
ex.- Return "Missing required element '<element name>' in request body"
instead of "Bad Request"
2. Return consistent error message in case request body element
name is not present.
3. Added 'assert_valid_body' method to raise HTTPBadRequest exception
if request body is not valid. It is not possible to modify
'is_valid_body' method itself to raise exception because V1 and v2
api's return different error code when 'is_valid_body' returns False.
APIImpact
Closes-bug: #1461402
Change-Id: I81764cd8517484fbee4beae2b30668c8b180d677
bool value is validated with different logic scattered all over the places
1. isinstance of six.string_type or bool
2. is_valid_boolstr method
Made changes to use strutils.bool_from_string method to validate bool
value at all places as this method does all things expected above.
Changes are not made in v1 api as it is going to be deprecated in
Liberty release.
Example error message:
Invalid value for force_host_copy: Unrecognized value 'xyz', acceptable
values are: '0', '1', 'f', 'false', 'n', 'no', 'off', 'on', 't', 'true',
'y', 'yes'
ApiImpact: 400 BadRequest response will be returned in case invalid force
value is provided during volume upload to image, earlier it was ignored.
Closes-Bug: 1460575
Change-Id: Ie64d429f5860a950cda9b363a65c5282f128f10b
The following patch switches direct db calls in
volume/api.py get_all_snapshots to use SnapshotList
instead.
Partial-Implements: blueprint cinder-objects
Change-Id: Ifdccc81a087f9797fcfbf098d3ecea3875ad7bd9
Part of multi-patch set for easier chunks.
There have been quite a few instances found where the
i18n guidelines are not being followed. I believe this
has helped lead to some of the confusion around how to
correctly do this. Other developers see this code and
assume it is an example of the correct usage.
This patch attempts to clean up most of those violations
in the existing codebase to hopefully help avoid some of
that confusion in reviews.
Some issues address:
* Correct log translation markers for different log levels
* Passing format values as arguments to call, not preformatting
* Not forcing translation via six.text_type and others
Guidelines can be found here:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html
Hacking checks will not be able to identify all violations of
the guidelines, but it could be useful for catching obvious ones
such as LOG.info("No markers!").
Change-Id: I38f52c6408b47ccb59ec2064b360f7d4427d6830
Partial-bug: 1433216
Abstract volumes and volume snapshots into objects.
Get, create, and delete snapshot APIs were changed to
use new snapshot objects. A skeleton volume object was
created, but cinder internals were not changed to use
the volume object, although volume is referenced and
used by the snapshot object. The internals will be
changed to use volume object in a subsequent patch.
Change-Id: I387018e80c8539565e99454db65d976030002c0f
Implements: blueprint cinder-objects
According to the PEP8(E265) there should be at least
one space before block comment.
Change-Id: Ic51f80210becc375b30f0f4e9eeb54995775c817
Partial-Bug: #1407162
This is the fourth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.
There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away. Thus, we need patches like this one to
move to the new namespace.
This patch also updates our hacking check to ensure that no instances
of oslo.utils sneak back in.
Change-Id: I19c331a9b54c5f42c278577ecc59fe1b65e61af0
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some of these outdated modules will still be
around for a while until all other oslo modules that use them have been
updated in future commits.
Change-Id: Idee8600dfe42e5977b8fb824e91afff7e9119981
Placing the _Lx markers back into the code. No other cleaner solution has
has been implemented. Patches will be submitted in a series of sub
directories and in a fashion that is manageable.
This is the third commit of this kind
Change-Id: I33b0bb05b1c1ea91dc5a1e93eaadceea1f23fcc9
Partial-Bug: #1384312
Improve the internal caching system used by the API layer
by borrowing the implementation used by Nova.
Unlike the previous implementation, this new interface makes it clear
which resource types are added or retrieved from the cache.
This change also adds other resources to the cache:
- backups
- snapshots
- volume types
It's now possible to remove database access those extensions:
- extended_snapshot_attributes
- volume_host_attribute
- volume_mig_status_attribute
- volume_tenant_attribute
Closes-bug: #1358524
Change-Id: I1fc673d0c01c41faa98292d5813d4471b455d712
Noticed in a recent bug fix that the when the show
method was copy/pasted at some point the variable name
vol was not updated.
Silly annoyance, but thought I'd go ahead and replace vol
with snapshot to make the variable name reflect what's
actually in use here.
Change-Id: I137ecc4987258450e93195cfed5e128063b92914
The extended snapshots extension makes a separate db request for
snapshots which is horribly inefficent. In addition it causes an
issue when using all_tenants=1, because the second request doesn't
pass in the filter. Fix this by caching the snapshot in the request
object like we do for volumes and retrieve the cached object in the
extension.
Change-Id: Ia943438c8d48accb7f62c43f07642c3da2fe34ab
Resolves-bug: #1358960
According to the latest revision of the logging guidelines message
of type AUDIT should be moved to INFO.
The latest revision is available at https://review.openstack.org/#/c/91446/
at the moment.
Change-Id: I2d86fa634f6424b0bb862319b9ae40d3f3b1ec7a
To ensure that logs are properly translated and logged to
the Cinder log files (using the Cinder message catalogs)
we need to explicitly import _() in any python files that
use the _() function.
Closes-bug: 1306275
Related-Blueprint: i18n-enablement
Change-Id: I3cf1f1d63d5f8bd8ffd007104691f914e3fd8538
For some reason, the snapshots controller class' introductory
docstring stated that it was the volume API controller.
This patch changes it to say:
"The Snapshots API controller for the OpenStack API."
Change-Id: I8c9827939bbc7acf93282db854a40501f3fea399
This patch adds filter options to backup list.
Like cinder list, now cinder backup-list accept
parameters.
This blueprint is complimented with the client part
https://blueprints.launchpad.net/python-cinderclient
/+spec/add-filter-options-to-backup-list
blueprint: add-filter-options-to-backup-list
DocImpact
Change-Id: I9e095ee44a03744eb3f319323d9130a89196614e
Passing tests in the gate leave unhandled trace/error messages
for a VolumeNotFound issue.
These are caused by the test_volumes_snapshots_negative test
which requests a non-existent volume and the tests pass however
the log files are a bit messed up due to the unhandled exception.
This change just adds a try/catch block around the volume get calls
in the snapshot modules and raises HTTPNotFound as appropriate.
Change-Id: I2096f2da7c68ef7924fc8e69b2d5c2afea578512
Closes-Bug: #1255214
The server doesn't check whether the parameter "volume_id" is in request body.
So the 500 error has been thrown.
We should catch the KeyError and transfer the KeyError to 400(HTTPBadRequest)
instead of 500.
Change-Id: I8a1dde1fd6ed820b39995af434efacc2a27c9604
Closes-Bug: #1252179
According to the comment in the source code, the snapshot-update of
v2 would be able to allow either "name" or "display_name" parameter.
And if both paramters are specified, "name" parameter is effective
over "display_name" like "description" parameter.
However "display_name" parameter is ignored now, because cinder writes
over empty dict(update_dict['display_name']).
This patch fixes the problem.
Fixes bug #1226398
Change-Id: Icd48556a26b9346f73b7c1b82ffbcdd469c4d28d
This patch does more internationalization for the REST API error
messages that don't currently have it to take advantage of the new
support added by bp user-locale-api to show error messages in the locale
requested by the user through the Accept-Language HTTP header.
We only do v2 because consumers have used the response error message in
the past for error checks, so changing it in v1 too would break them.
Partially implements bp user-locale-api
Change-Id: I92780b42c125a91ab4916b7a31e4b71d306a89a1
In previous code of _items() from api/v{1,2}/snapshots.py,
and also the _items)_ from api/v1/volume.py.we didn't pop
out the 'offset' and 'limit' fields from HTTP get params
before we use such params for filter.This is the root cause
for bug #1205956
For non-admin user, since 'offset' and 'limit' is not in the
allowed_search_options, so the volumes.remove_invalid_options
will help to filter them out. As a result, it walks around this
bug.
But for admin user,the volumes.remove_invalid_options will not
try to filter the search_options.So for admin user, the 'limit'
will appear in search_options, then obviously get no result.
fixed bug #1205956
Change-Id: Ib1a66c9d104ac52d6eae18be7f06d02985d4c2fd
Replace all the FLAGS with cfg.CONF in cinder/api
Large commit was split into several parts
Change-Id: I2114d5fc45f5299c3b2011270034a3370e0ec388
Fixes: bug #1182037
The only functional difference here is that the value '2' is now
treated as False instead of True. I suspect that's ok.
Change-Id: I30e75470cb3c312d7f92d389995eb6d845dd2fb0
Eric Harney