Extend rest api v2 with multiple sort keys support.
Example:
/v2/images/detail?sort_key=name&sort_key=size
Changed database api which now takes sort_key param as a list instead of string
python-glanceclient support will be added in separate commit
Change-Id: Ib7a6aeb2df3bc5d23fe8e070290b5bfcab00c0f5
DocImpact
Partial-Bug: 1221274
The builtin _ should not longer be used. Instead _
should be explicitly assigned from glance.i18n.
This patch does the following:
(1) adds these explict assignments to any
glance file that was missing them,
(2) removes the defintion of _ as a builtin from
tox.ini so that a pep8 failure will occur
if _ is not defined,
(3) removes calls to install().
This removes the last direct use of gettextutils by glance.
The indirect uses, via openstack/common, will be removed
in the next sync with oslo.
Change-Id: Ie54e8a67e747e022cc01022dfad35d89686bdfc4
The following replacements were done in unit tests to have
clearer messages in case of failure:
- assertTrue(* is None) with assertIsNone
- assertTrue(* is not None) with assertIsNotNone
- assertTrue(* in *) with assertIn
- assertTrue(* not in *) with assertNotIn
- assertFalse(* in *) with assertNotIn
- assertTrue(* == *) with assertEqual
- assertTrue(* != *) with assertNotEqual
Change-Id: I0c47f991c3974e441335e71c9d26fab8a127f2ca
Replace assertEqual(None, *) with assertIsNone in glance's
tests to have more clear messages in case of failure.
Also, replace assertTrue(* is not None) with assertIsNotNone
for the same reason.
Change-Id: If41a71bd750057c7136b03bef94c04517092134c
Closes-Bug: #1280522
This patch adds the knowledge of a virtual_size field to Glance's API
v2. The virtual_size field should respect the same rules applied to the
size field in terms of readability, access control and propagation.
Glance's API v1 has been left unmodified.
docImpact
Implements blueprint: split-image-size
Change-Id: Ie4f58ee2e4da3a6c1229840295c7f62023a95b70
Assign an explicit created_at datetime value to image db fixtures,
it be used to fixes race condition in Registry v2 Client test cases.
Closes-Bug: #1272136
Change-Id: Ib9f1c7072c6ac828479c6a2e5bddd0f3ad057653
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Use common oslo database code for work with session, connection, engines
instead of glance implementations.
Work with config options modified due to new common code config.
Removed unused glance code and tests for it.
Details:
glance/cmd/manage.py
- removed load load the glance-registry config. We load sql connection
in common code now.
glance/db/sqlalchemy/api.py
- removed functions _wrap_db_error(), _is_db_connection_error(),
_get_maker(), get_engine(), _ping_listener() and config
`sql_connection_opt`. We have all these thigs in module
`glance.openstack.common.db.sqlalchemy`.
- modified functions setup_db_env(), add_cli_options(), clear_db_env().
- functions _get_session() and get_engine() now call same functions
from module `glance.openstack.common.db.sqlalchemy`.
- replaced sqlalchemy exception `sqlalchemy.exc.IntegrityError` to
database duplicate exception - `db_exc.DBDuplicateEntry`.
Removed redundant tests - we use common code , so a lot of features was
already tested in Oslo.
Global variable _ENGINE was replaced by common function get_engine().
DocImpact
New database config options came from Oslo. See Table below
-------------------------------------------------------------------
Database config options ([group] option)
-----------------------------|-------------------------------------
Glance | Oslo
-----------------------------|-------------------------------------
[DEFAULT] sql_connection | [database] connection
[DEFAULT] sql_idle_timeout | [database] idle_timeout
[DEFAULT] sql_max_retries | [database] max_retries
[DEFAULT] sql_retry_interval | [database] retry_interval
[DEFAULT] db_auto_create | NONE
[DEFAULT] sqlalchemy_debug | [database] connection_debug
[DEFAULT] use_tpool | [database] use_tpool
NONE | [database] slave_connection
NONE | [database] min_pool_size
NONE | [database] max_pool_size
NONE | [database] slave_connection
NONE | [database] min_pool_size
NONE | [database] max_pool_size
NONE | [database] max_overflow
NONE | [database] connection_trace
NONE | [database] pool_timeout
-------------------------------------------------------------------
blueprint db-use-oslo-common-code
Change-Id: I3ff976545b1a82ff8df780e34128fcaf6f892b8c
Each project should directly use the standard uuid module.
uuidutils will be deprecated/removed in this cycle.
This patch replaces every uuidutils.generate_uuid() with
str(uuid.uuid4()) and uuidutils.is_uuid_like()
with utils.is_uuid_like().
Change-Id: I43642d4f1e137c14134b3d544e367b504b9851ac
Closes-Bug: #1253497
Protect against concurrent image uploads by allowing the
initial request to proceed and succeed and subsequent
concurrent requests to fail.
We now allow the upload activation to specify what state
it expects to transition to 'active' from. If by the time
the db is updated the transition has already occured i.e.
another upload has succeeded, the upload will fail and data
will be cleaned up.
Fixes: bug 1241025
Change-Id: Ie532b61484bec660910fb9a37429397bde8ef11f
This patch adds CONF.use_user_token as a config option. When true, glance will
continue to use the user token when communicating with glance registry. When
false, glance will read admin credentials from the config and use those to
communicate with glance registry. Using admin credentials allow for
reauthentication when tokens expire and prevents requests from silently failing.
DocImpact
Fixes bug 1182536
Change-Id: Ia6b563677eb67d4069571c82b9dad3f025b6e9fb
The patch implements Registry's v2 Client.
Some notes:
* The implementation uses __getattr__ to map method's calls to remote
RPC methods.
* The client supports both bulk and single calls.
* Client is based on the base HTTPClient
About Exceptions:
* If raise_exc == True, the client will try to re-raise the remote
exception. If the server raised an unknown exception, it will be
re-raised as RPCException in the client side.
Implements blueprint registry-api-v2
Change-Id: I98757c2272c68301373c1a3af96c4d06cb63cc97
Currently, Registry's client doesn't have a version and It's being
imported from glance.registry.(__init__|client).
As part of the blueprint registry-api-v2 the registry's client will be
modified. That for, it is necessary to keep backward compatibilities
with older versions until the db-registry-driver is completed.
This code might be removed / re-factored in the future once glance's api
v1 will be migrated to use the db-registry-driver.
Change-Id: I3f27cec5b6e5f857d7aa94731305629190ae642e
DB drivers currently implement a configure_db function that is supposed
to be used for seting up the environment and getting an engine back from
it. Since it is not being used anywhere in the code but tests, I removed
it and modified tests in order to call both functions (setup_db_env and
get_engine) where/ when needed.
As pointed out in another bug, it is important to just call setup_db_env
when instantiating the db_api in order to keep it lazy. AS for
sqlalchemy get_engine will be called when a session is requested.
Fixes bug 1167770
Change-Id: I2ebeb5e8424a4857df21f7c2d5b33e8ae0100c6a
A few testcases had duplicate names, which caused
the original test to be redefined and not executed
anymore. Also one case was apparently entirely duplicate.
Raises unit test coverage by two tests.
Change-Id: Ibf3db93b7fee875877c1bac8797eb88fc9a99874
There is disagreement between the v1 api and v2 api about what an
'is_public' filter ought to mean. Version 1 uses 'is_public'
to filter whether or not image.is_public == True or False. Version 2
uses 'visibility' for this purpose. But in addition, it is theoretically
possible that 'is_public' is a valid property filter in v2.
With this change, v1 can pass is_public in as an argument to
image_get_all. V2 will continue to pass is_public through filters.
fixes bug: 1157988
fixes bug: 1154760
Change-Id: I716cb8b4f207f8526125ed360f5a24a6630c1e67
* Completely drop the legacy Glance client tool
* bin/glance is gone
* glance/client.py is gone
* Drop relevant tests
Implements bp separate-client
Change-Id: Ifcb0bd9bb537e0243aeb5daf466f46868d522986
Image data is currently uploaded into the backend store even if it is a bad
request. This patch add value validation in the deserializer so we don't set bad
data. It also does a full validation of the image metadata to make sure
everything that is needed for an image upload is set correctly before the upload
can occur.
There were currently some relevant tests that had incorrect names causing them
to not run. These have been fixed and modified.
Fixes bug 1035982
Change-Id: I6f3ef83ba7b3de3cb885fad9737fef1a03d9222c
Rather than refuse to start, it's nicer to use a sane set of default
policies when no policy file is found.
Fixes bug 1043482
Change-Id: I849737c61c0266952d931395fbc2ad3745c46f6e
Impact:
* Test function names
* Test strings and comments
* Comment in glance/registry/api/v1/members.py
Change-Id: I8cec84c4934c08905e7f8758dec3622a5261b153
The glance client offered the opportunity to override
important headers via the features parameter during image
creation or update.
Created a blacklist of unsupported features like:
content-length
content-type
x-image-meta-size
These headers should not be overriden by the client.
Fixes LP Bug #1023892
Change-Id: I14aac59e00a2672fd98f6dab221096ab5de86855
* Move RequestContext class to glance.context
* Move context middlewares to glance.api.middleware
* Update tests to reflect move
* Update paste configs
Related to bp glance-request-tracking
Change-Id: I289b546ec28c973a3022be779ce378ae2febb340
Updates the v1 Glance API and Registry controllers so that admins
can modify image membership regardless of image ownership.
The motivation for this change is to be able to test image membership
when noauth middleware (UnauthenticatedContextMiddleware or
FakeAuthMiddleware) is being used and includes the following changes:
* Added is_admin option to FakeAuthMiddleware.
* Refactors the access checks into a common function called
_check_can_access_image_members.
* API unit tests which require is_admin=False for noauth were also
refactored to use an improved FakeAuthMiddleware which supports
is_admin = False.
* Update RegistryClient to use get_status_code int for 204 status code
comparisons. Fixes issue where incorrect status code was returned.
* Updated members unit tests to support these changes.
Fixes LP Bug #1021740.
Change-Id: I9d0e4679f9c0cb37f6df7c6f90460c22c37f3fbd
Implements blueprint cfg-global-object
This iteration fixes up the remaining places where we're passing a
conf object around.
Change-Id: I3b627d093a6b7afeee129ab66b377a3b0af5439b
Move the core of glance.tests.unit.utils:FakeDB to glance.db.simple.
This 'simple' driver is an alternative to the traditional sqlalchemy
driver. Additionally, the sqlalchemy driver has been moved from
glance.db to glance.db.sqlalchemy. The simple db driver is only
available to be used by tests for now.
Related to bp refactor-db-layer
Change-Id: I9d33a433c0c03e53fb5a3491076086427ae694b3
Implements blueprint use-common-timeutils
1. Edit openstack-common.conf and import glance/openstack/common/timeutils.py
2. Replace datetime.utcnow with timeutils.utcnow
3. Replace utils.isotime with timeutils.isotime
4. Replace utils.parse_isotime with timeutils.parse_isotime
5. Replace utils.normalize_time with timeutils.normalize_time
6. Remove datetime related functions and datetime related unittests
Change-Id: I014548848f738b00c670be3d372b16f47f6dee96
Implements blueprint cfg-global-object
Begin adopting cfg's global CONF object pattern rather than passing
a conf object around everywhere.
This iterations does just enough to have each of the glance programs
initialize CONF and get the unit tests working.
Change-Id: Ia34959b3ba52cbe933d5d347f31319271b2e14f9
Fixes Bug #1007093
Mixing use of os.lseek(fileno, 0, os.SEEK_SET) with
file.seek seems to cause problems when reading data
with file.read
Change-Id: Ia96ce9bbdc93a1a8c6169d85b99187f8cd7b2fdc
Add additional information to the 'image.delete' notification payload
to simplify external usage tracking systems.
Please note that this is not a backwards-compatible change as the
old payload was simply a string consisting of the image id. This
converts the payload to a dictionary.
Change-Id: I641b7de91fcc14dc3c81ec942aeb0acf5a63d128
Adjusts our version match regex so that it detects URL's
which already have version's in them that start with
a '/'.
For example: '/v1'
Fixes LP Bug #992600.
Change-Id: I331e945c46757a01a4dd939b2098e68334b27530
* Documents the configure_via_auth parameter
* Ensures that if configure_via_auth is used (the default), that
if the management_url that is returned does not contain a version
identifier, the client class' DEFAULT_DOC_ROOT is appended
appropriately.
* Adds some log debugging statements into the base client class so
that the URL being configured/constructed is more easily identifiable
Change-Id: I307a24231b59f7a183ed669f679976cc3f8ec21c
Pull the extra methods defined in glance.registry.context:RequestContext into
glance.common.context:RequestContext. This is necessary as the new v2 API
implementation talks directly to the database.
* Partially implements bp api-2
Change-Id: Ia8c1fde511bdca69be0dcf61ce12976ace5f7dc3
Content-Length and Transfer-Encoding conflict according to the HTTP
spec. This fixes bug 981332.
This also adds the ability to test both the sendfile-present and
sendfile-absent codepaths; the sendfile-present test will be skipped on
sendfile-absent platforms.
Change-Id: Ibb017b4a43d89ffdb6b868648296603cfec7780d
Fixes bug 966240
Image names longer than 255 characters are rejected with 400 BadRequest
on creation or update.
Change-Id: I460ffee547496829cbf198b50bca564978abe7f3
* Clean up authorization vs authentication failures internally
* Remove ambiguous exception.NotAuthorized in favour of exception.Forbidden for authorization failures
* Add exception.NotAuthenticated to make authentication failures more clear
* Fixes bug 956206
Change-Id: I39ce0fcd77d4f06273040a2aa4913a9be911ceab
Fixes bug #942118
For security reasons, file:// URIs local to the glance services
should not be supported as external sources (as specified via the
x-image-meta-location or x-glance-api-copy-from headers).
Change-Id: I43763cbefba95153434c7dcdcce3765ed04e05fe
Mike Fedosin
James Carey
liyingjun
Julien Danjou
Svetlana Shturm
Sergey Nikitin
Jia Dong
Flavio Percoco
Zhi Yan Liu
Noboru arai
Boris Pavlovic
Dirk Mueller
Arnaud Legendre
Edward Hope-Morley
Alexander Gordeev
Alex Meade
Mark J. Washenberger
Zhongyue Luo
Mark Washenberger
Brian Waldon
Rongze Zhu
Lars Gellrich
Dan Prince
Mark McLoughlin
Zhongyue Luo
Derek Higgins
Brian Elliott
Jay Pipes
Mike Lundy
Josh Kearney
Eoghan Glynn