This allows functions that do not accept bodies to define this in the
router file. As currently many requests will cause a 500 if a body is
supplied when the API request does not expect it.
This currently only affects the core parts of the v2 api, that is,
calls to v2/images and v2/schemas. It does not cover the "tasks" API
or the metadefs api as I was keeping this patch concise. As this
does not affect the behaviour if not included this makes no change to
the metadefs api behaviour.
DocImpact
Partial-Bug: 1475647
Change-Id: Ieb510e5516128078d40d39fd9b4f339ce64e10e7
Type errors that are encountered due to unexpected
types being passed in now get a 400 'Body format is
invalid'. This hardens the glance api from other types.
ApiImpact
DocImpact
Closes-Bug: 1476695
Closes-Bug: 1476253
Change-Id: Ieee0662f67c800b2b3c07c6a8b7877939cf9e1fe
Utilizing assertIn and assertNotIn in a consistent way rather than
assertTrue( x <not> in y) and assertFalse(x in y).
Change-Id: Ic43d107c40a5a50b852ee476b1167c21e5ad86e8
If for some reason (absent\invalid token) authentication failed during a
registry call, InternalServerError appeared as long as
exception.NotAuthenticated was not being caught in ImagesController.
This patch fixes the issue by catching the exception and raising
401 Unauthorized.
Change-Id: I72016c5c7868fdc7a72def8b6f5a05324dcee387
Closes-Bug: #1451850
In Kilo, a patch landed to allow glance for returning None values. While
this was not an issue for the base and registered properties in the
`schema-image.json` file, it did break backwards compatibility for the
custom images created in V1 that have a vulue equal to None.
In order to restore compatibility, this changes returns empty strings
for those custom properties that exist in the database but that are not
part of the V2 schema. This strategy will help migrating such properties
when updates to the image data happens and it'll be a noop for just
reads.
We can't use a schema migration because we don't know off hand which of
the properties in the database are the ones that would need to be
migrated.
While we could skip this properties entirely, this patch prefers to send
the empty string back as a way to create awareness on the fact that
there's an empty property in the database. Since we didn't use to return
these properties, we can assume they weren't being used.
Closes-bug: #1419823
Change-Id: I59bb27a892fe9485fc98a612ca0148a84123f5a2
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Change-Id: Ifa8baab33cdb3e606cf175a8c29c3a4ef6c44480
The text message earlier was not helpful
while listing members of a public image.
A more helpful message will be given to
user after this commit is accepted.
Change-Id: Ifa2df37baea50c78c622344847f2c7117c17e3f4
Closes-bug: 1434501
This patch provides the ability to 'deactivate' an image by
providing two new API calls and a new image status 'deactivated'.
Attempting to download a deactivated image will result in a
403 'Forbidden' return code. Also, image locations won't be visible
for deactivated images unless the user is admin.
All other image operations should remain unaffected.
The two new API calls are:
- POST /images/{image_id}/actions/deactivate
- POST /images/{image_id}/actions/reactivate
DocImpact
UpgradeImpact
Change-Id: I32b7cc7ce8404457a87c8c05041aa2a30152b930
Implements: bp deactivate-image
Previously, every call to policy.enforce passed an empty dictionary as
the target. This prevents operators from using tenant specific
restrictions in their policy.json files since the target will always be
an empty dictionary.
If you try to restrict some actions so an image owner (users with the
correct tenant id) can perform actions, the check categorically fails
because the target is okay is an empty dictionary. By passing the
ImageTarget instance wrapping an Image, we can properly grant access to
the image owner(s) based on tenant (e.g., owner:%(tenant)). Without this
fix, the only check that actually works in glance is a RoleCheck (e.g.,
role:admin).
Partial-bug: 1346648
Implements: blueprint pass-targets-to-policy-enforcer
Change-Id: Id914c478ca7c4dfde3f08028d8b70c623f26b6e9
Mainly to improve consistency, use range() from six.moves
renames across glance.
Behaves consistently like py2 xrange() and py3 range().
Removes unnecessary range() from glace/api/v2/images.py
Change-Id: Id21f923d05600b902f2239e25ef01716c07e74a3
HTTPInternalServerError 500 response is returned to the user while
image server is down during downloading the image. When glance tries
to download the image from the remote location (image server) which
is down, Connection refused ECONNREFUSED error is raised on the
glance server.
Raised RemoteServiceUnavailable exception from glance_store and handle
it in v1 and v2 api's to return 503 HTTPServiceUnavailable response
to user.
Note: Please refer below link to check glance_store related changes
which addresses this issue partially.
I065b9a3e8e674ea74ff8563aad99d7d022417caa
Closes-Bug: #1379798
Change-Id: I45099153e75d53b028e249fad8a4d944d38adf65
This patch changes the references of glance test cases from glance.com
and example.com to local fake http server.
This change also fixed two test cases[0][1] which could broken by
next glance_store release due to merged change [2] in http store
driver changed return value when internal 404 NotFound happend on
remote image access/query.
[0] glance.tests.unit.v1.test_api.TestGlanceAPI.
test_add_copy_from_image_authorized_upload_image_authorized
[1] glance.tests.functional.v1.test_copy_to_file.TestCopyToFile.
test_copy_from_http_nonexistent
[2]
53cb5c345f
Closes bug 1298918
Change-Id: Ie80483e600e16e02240488e083293538e5512f35
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
The change will be used to restrict client to download and delete any
file in glance-api server. The same resone and logic as what we did in
v1:
https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L429
Closes-Bug: bug/1400966
DocImpact
Note: Even this change could fully resolve the problem for Glance, but
we still need to fix this issue from glance_store perspective
separatelly due to other projects can use the lib directly.
Change-Id: I72dbead3cb2dcb87f52658ddb880e26880cc229b
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Currently, Glance's API v2 doesn't return fields whose value is None.
This, unfortunately, is wrong for a client perspective since it would
create inconsistencies between calls and images due to the lack of
fields in the response.
The API should guarantee consistency in its replies and ensure all
fields have a value, even if it's None.
NOTE: This work is part of the migration to v2. It fixes inconsistencies
in the API and improves the interaction between the client library and
Glance.
NOTE2: A follow-up patch will bump the minor API version, wait for it.
ApiImpact
DocImpact
Closes-bug: #1398314
Change-Id: Ieaddd8a686cf7361f18cb1ee83b7887cdca22bd6
1. Sync glance.openstack.common.policy up to latest
version from oslo-inc.
2. Clean useless modules which depended by policy
module and pinned gettextutils module there. For
latter one, we are going to use glance.i18n instead.
* jsonutils
* strutils
docImpact
Closes-bug: #1288178
Closes-bug: #1387973
Partial-bug: #1381870
Change-Id: I84511ab1ee600e618985448dfbfbdc26cb130370
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Args of assertEquals method in glance.tests are arranged
in wrong order. In result when test fails it shows incorrect
information about observed and actual data. It's found more
than 2000 times. Right order of arguments is "expected, actual".
Change-Id: Ifea4809f5a696b608a19e36a9ed9e5eec46c8a21
Co-Authored-By: Li Yingjun <liyingjun1988@gmail.com>
Closes-Bug: 1277104
Fix a bug with storing the request context in thread local where the
storage was not properly cleared between requests.
(This was causing the same request to always be logged.)
Closes-Bug: 1384911
Change-Id: Ie30b93900451b0f9f6fd8158457b0b66903af470
In order to improve coding practices and be aligned to a standard,
self.assertEqual(...,True) statements where replaced with
self.assertTrue and self.assertFalse.
Change-Id: I4150a343493beae03d6f71c88e84f60d14fb60e1
Closes-Bug: #1230127
This patch adds support for random access to images to the API v2. It's
possible to partially access the image data by passing the offset where
the image should be fetched from and the size of the chunk expected.
Note that not all stores support random access to image data. In such
cases, the API will return an error mentioning the selected store
doesn't support such a feature.
DocImpact:
Content-Range header is now parsed to support random access to image
data.
Implements blueprint: restartable-image-download
Change-Id: Iade692a8197a9a1d9532711a5c572c505e5e7d20
This commits removes the old `store` package from glance and adopts the
usage of the new glance.store library. The library was designed to
preserve backwards compatibility as much as possible. In fact, most of
the changes in this patch are related to function args ordering and not
function renames or workflow changes.
Some changes that are worth mentioning:
1. Glance store doesn't rely on a global config object. All config
options must be explicitly registered.
2. All store operations now accepted an optional context. This is a
fallout from the context not being required in the `Store` constructor
anymore.
3. Store drivers are behind a private package called `_drivers` and
they're not suppose to be accessed directly. Instead, functions like
`get_store_from_scheme` should be used.
4. Stores are disabled by default
5. All the store specific options are under the `glance_store` group.
DocImpact:
The old store related configuration options have been moved under the
`glance_store` section. However, the old options will go through a
deprecation path. That is, they'll still be read from the `DEFAULT`
section to give deployers enough time to update their config files.
In k-2, the deprecated options will be completely obsolete.
Closes-bug: #1291848
Implements-blueprint: create-store-package
Change-Id: Iaacc70993ad5da292b93de42bbecda73d53b19fd
Added new rule in policy.json and applied that rule to
'download_image' policy.
For example,
"restricted": "not ('test_key':(test_key)s and role:_member_)"
"download_image": "role:admin or rule:restricted"
So if 'download_image' policy is enforced then in above case only admin or
user who satisfies rule 'restricted' will able to download image. Other users
will not be able to download the image and will get 403 Forbidden response.
In addition, delete property access should be restricted for other users
so that they will not be able to delete the property of the image.
[test_key]
create = admin,member
read = admin,member,_member_
update = admin,member
delete = admin,member
Added new method to create dictionary-like mashup of image core and custom
properties.
Modified v1 and v2 api to add download restriction.
Modified logic of caching to restrict download for v1 and v2 api.
DocImpact:
Need to add new rule in policy.json
"restricted": "not ('test_key':%(test_key)s and role:_member_)"
blueprint: restrict-downloading-images-protected-properties
Change-Id: I05bad0441952150bd15b831ac1b1a0bb9ae79c74
image_size_cap should be checked and enforced on upload
Enforcement is in two places:
- on image metadata save
- during image save to backend store
Closes-Bug: 1315321
Change-Id: I45bfb360703617bc394e9e27fe17adf43b09c0e1
Co-Author: Manuel Desbonnet <manuel.desbonnet@hp.com>
This fixes the test_tag_lifecycle test, which broke with a randomized
PYTHONHASHSEED.
Previously, the test assumed that the values in the list of tag names
would remain constant. The test now sorts the results and compares to a
sorted set.
Change-Id: If5a2ef683fcf55520c7d539f1b90aefaba4739d5
Partial-bug: #1348818
When property protection feature enabled, end user couldn't uses
Glance v2 API to update or delete a property of an image successfully
which value is empty, client will receive a http 500 error and a
relevant error log could be found in glance-api service log.
Closes-bug: #1332103
Change-Id: I1f9f181cea714e6ba26388d125bb7023e7a14305
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Currently the v2 functional tests are run without the v2 registry enabled.
This means that bugs which are only seen when the v2 registry is enabled
may not be caught by the gate (eg #1308419, #1339775).
Here we enable some basic functional testing (eg image life cycle) with
the v2 registry enabled (in addition to the existing direct-to-database
mode). This should help prevent some types of bugs being introduced.
There is a plan to move some functional tests into tempest, at which
point these additional tests can be removed in favour of equivalent
tempest tests.
Change-Id: I774a277673390650874aad4334b28d1a872ea93f
In Glance API V2, currently, the user gets a 500 error when tries
to use the "changes-since" filter.This filter, which was available
on API v1 is no longer available on V2.
This patch returns to the user a 400 Bad Request with a proper message,
instead of a generic 500 Internal Server Error.
Change-Id: Icece859a7f12f6f4aa1f7bb945134a935048b315
Closes-bug: 1273975
If a user is not allowed to download an image because of a policy,
glance v2 API responds with a HTTP 200 status and no data instead
of HTTP 403 for no cache only.
The problem is that get_data implementation for notification
proxy is a generator, this situation delays the other proxies
get_data calls (including the policy proxy) for the first time
data is retrieved.
Hence, there is a delay in enforcing policy, so 200 is sent before
the API gets the chance to catch the policy exception.
DocImpact
Closes-Bug: #1326781
Change-Id: I1e50a069a6b7f9eed7160cd5908a5fa30274e227
Pep8 1.5.4 fixes several bugs in visual indent and spaces
detection. Adjust the code accordingly so that it passes
once we try to upgrade.
Change-Id: I5691ffd29edcb32fdbacd9da7ca71bfcabeeffe5
This fixes an issue where copy.deepcopy was being a called
on an ImmutableLocations object, causing a Forbidden to be
raised.
Change-Id: Ia7ac121478cf63d68bd5a3d91faa51a42afeb2fe
Closes-Bug: 1287888
This patch adds the knowledge of a virtual_size field to Glance's API
v2. The virtual_size field should respect the same rules applied to the
size field in terms of readability, access control and propagation.
Glance's API v1 has been left unmodified.
docImpact
Implements blueprint: split-image-size
Change-Id: Ie4f58ee2e4da3a6c1229840295c7f62023a95b70
Enable F841 check: local variable 'name' assigned but never used.
Make appropriate changes to files listed below.
Change-Id: I02837d4abf421dc9d85f3b01587120fd68acfa12
Apply image location selection strategy into Glance server side. Image
download handling and "direct URL" exporting will be effected by this
mechanism.
Implements bp: image-location-selection-strategy
Related-Id: I86f192aeae8e5f21a72f946552f6507654c25a6c
Change-Id: I7bd093a16db3af2b604cad22a6b6971345af82a2
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
The 'owner' property on an image is exposed in v1 but does
not appear in the v2 image response. It would be useful to
have this available.
docImpact
Implement blueprint expose-owner-in-v2
Change-Id: I90f68d57e185f4a5ee63cd97bd5597948dde33ea
Niall Bunting