1226 Commits

Author SHA1 Message Date
Russell Bryant
aaa18a02a5 Change xattr usage to be more broadly compatible.
This patch updates the xattr image cache driver to use methods that are
more broadly compatible.  The issue here is that there is more than one
xattr Python module out there.  Specifically, there are:

    https://github.com/xattr/xattr
    http://git.k1024.org/pyxattr.git/

The previous glance code expected an interface available in the first
version.  This patch changes the code to use an interface that is common
to both of these.

This makes all of the image cache tests working against the current pyxattr
package in Fedora.

Change-Id: I1fca66cfe914364c921408b676a06387d1de5c2a
2012-01-25 12:52:45 -05:00
Russell Bryant
8ec2f81e59 Fix mixed usage of 's' and 'self'.
The first version of this patch fixed one misuse of 'self' when 's' was
expected.  There were some other uses of 'self' that I missed.  This
patch just changes these functions to use 'self' everywhere.

Change-Id: I66b74f00e5416e3e67eb60cf6c505f70134e6ddb
2012-01-25 12:51:52 -05:00
Jay Pipes
180ae8964d Adds documentation for policy files.
Implements interim-glance-authz-service

Change-Id: Ifefa4174aa45a29fd58ab40c74b5eaf6fecb4f83
2012-01-24 21:19:26 -05:00
Jenkins
971e4b6a53 Merge "Client.add_image() accepts image data as iterable." 2012-01-24 20:42:36 +00:00
Jenkins
79cc9254d9 Merge "More flexible specification of auth credentials." 2012-01-24 20:32:15 +00:00
Eoghan Glynn
7094e8926c Client.add_image() accepts image data as iterable.
Fixes bug 773562

Allow the image data to be passed to glance.Client.add_add_image()
via an iterator, in addition to the previously accepted images types
(file-like or string).

Change-Id: I1f90b9875f5610d478a5ed123fbf431501f1c2b6
2012-01-24 19:07:32 +00:00
Eoghan Glynn
6cac288a87 More flexible specification of auth credentials.
Fixes bug 853933

Add new --username|--password|--tenant|--auth_url|--auth_strategy
switches to bin/glance to allow the username, password, tenant name,
and authentication URL & strategy be specified on the command line.

Avoid needlessly falling back to keystone v2 auth after a successful:

  GET /v1.0/tokens

returns with the X-Image-Management-Url or X-Glance header set,
as opposed to X-Server-Management-Url.

Extend the keystone functional test support to ensure that the URL
returned by keystone via the X-*-Url header contains the appropriate
dynamically allocated port for the glance API service.

Ensure the underlying $OS_* environment variables do not leak into the
TestPrivateImagesCli functional tests, also explicitly exercise both
noauth and keystone strategies.

Change-Id: Iee8bf3745d65a9c57a9da803d5cf9ae5f343a159
2012-01-24 17:45:41 +00:00
Jenkins
7df7586976 Merge "glance-control need not locate a server's config file (lp#919520)" 2012-01-24 17:38:57 +00:00
Jenkins
f66ff4f2e1 Merge "glance-api fails fast if default store unsupported" 2012-01-24 17:27:37 +00:00
Jenkins
53c8ad4b9d Merge "Bug #909574: Glance does not sanity-check given image size on upload" 2012-01-23 21:49:24 +00:00
Eoghan Glynn
601fb310ef glance-api fails fast if default store unsupported
Fixes bug 852850

Ensure the glance API server will not start if the class supporting
the configured default store cannot be imported.

Change-Id: I80a97dbc0af4737dbbc1bf0491e7cef222ec3454
2012-01-23 21:15:46 +00:00
Ewan Mellor
946278d1c7 Bug #909574: Glance does not sanity-check given image size on upload
Don't allow an image size greater than 1 PiB.  Also, add an exception
block catching and reraising HTTPError inside _upload.  This way, we don't
fall into the Exception block below and log the full backtrace when the
error has been diagnosed already.

Adds unit test to verify 400 returned with supplied overlimit image size

Change-Id: Ie745dffaf2fe43fdb7643c27d087e36d76b7c0e4
2012-01-23 14:53:04 -05:00
Mark McLoughlin
3b2ef8e29e glance-control need not locate a server's config file (lp#919520)
Before converting to cfg, glance-control would use find_config_file() to
locate the servers config file if the user did not explicitly specify
it.

With the change over to cfg, we screwed that code up. However, we should
instead just allow the server to locate its own config file rather than
trying to guess what it will use.

Change-Id: I422839ae4c684ff2de3a0a96dd54c6b74ce6df97
2012-01-23 08:32:52 +00:00
Jay Pipes
0db2cfadf5 Bug#911599 - Location field wiped on update
Adds unit test to verify behaviour required:

* Location field may only be **edited** if the image is
  in a queued state, which indicates that the user issued
  a POST /images with NO location field and NO image data
* Otherwise, Location field may not be edited and may not
  be read via the API server either.

Change-Id: I42aba7bc8e2da6ac81b18b564ba096208406f893
2012-01-20 16:07:12 -05:00
Jenkins
8d83124522 Merge "Return 400 if registry returns 400." 2012-01-20 20:08:33 +00:00
Jenkins
71b0bb8d12 Merge "Add policy checking for basic image operations" 2012-01-20 19:58:25 +00:00
Kevin L. Mitchell
eb2ab3a40a Return 400 if registry returns 400.
Fixes bug 919244.  The registry would return 400 for invalid input,
which resulted in an exception.Invalid being thrown in the registry
client.  There was no exception handler for this exception, so it
got bubbled up and turned into a 500.  This should fix the problem
by adding the missing exception handlers.

Change-Id: I75ecfec1c0b0b4b3df1a8c9ace83e75d19527c93
2012-01-20 13:31:47 -06:00
Jenkins
087074729c Merge "Swallow exception on unsupported image deletion." 2012-01-20 19:27:50 +00:00
Dan Prince
2e8e628d7a Set url's on AuthBadRequest exceptions.
Fixes LP Bug #919269.

Change-Id: Ibc9c20e90e06ec4c235d07003fc01bcdf77c976f
2012-01-20 11:17:01 -05:00
Brian Waldon
eeb4d1ee29 Add policy checking for basic image operations
Partially implements bp interim-glance-authz-service.
This checks basic image operations: get_images, get_image,
add_image, modify_image, delete_image. It looks for a
policy json file next to our config files.

Change-Id: I07f29c11934c68d38a6bdadd39f9dc841b61648a
2012-01-19 16:23:13 -08:00
Eoghan Glynn
41b981c98d Swallow exception on unsupported image deletion.
Fixes LP #912897

When the remote store does not support deletion of images (e.g. HTTP)
the StoreDeleteNotSupported exception is now swallowed and logged as
opposed to it being propagated back to the glance CLI.

Also after a failed delete, we avoid falling through to the delayed
delete logic.

Change-Id: I6cc529f029750fc66a4d32e2076f29198b3d2093
2012-01-19 22:40:05 +00:00
Brian Waldon
b712949ec5 Ensure we only send a single content-type header
Fixes bug 917438

Change-Id: Ic71bd139f77ed7f291eca258c123deb30c7177ce
2012-01-19 11:32:10 -08:00
Stuart McLaren
e893b248a2 Multi-process Glance API server support.
Implements blueprint multi-process-server. Allows several Glance API
worker processes to be started, which can increase performance on machines
with more than one CPU.

Change-Id: I1cbb48945fd23afd71de3a30b80836b590c023a1
2012-01-19 09:45:21 -08:00
Jenkins
1698eb10b5 Merge "Make paste.ini file location configurable." 2012-01-19 02:01:30 +00:00
Jenkins
ee38a16a5e Merge "Set size metadata correctly for remote images." 2012-01-19 01:43:48 +00:00
Jenkins
b1abca214d Merge "Avoid the need for users to manually edit PasteDeploy config in order to switch pipelines." 2012-01-19 01:26:28 +00:00
Jenkins
bbfa3edc06 Merge "Split out paste deployment config from the core glance *.conf files into corresponding *-paste.ini files." 2012-01-19 00:55:22 +00:00
Eoghan Glynn
3956c546fb Set size metadata correctly for remote images.
Previously the size metadata was always zero'd for remote images,
which was misleading and led to issues like:

  Bug 900959

We now query the remote store (HTTP, S3 or Swift) with the
equivalent of a HTTP HEAD in order to determine the correct
image size without downloading.

Where the size is available, the metadata is set appropriately.
Otherwise it falls back to zero as before.

Change-Id: I3093eba1b2fa023348558c45febee39e68e1a08f
2012-01-18 23:20:38 +00:00
Eoghan Glynn
097ced8cb2 Make paste.ini file location configurable.
Add an optional paste_deploy.config_file variable to identify
a non-default location for the paste.ini file (the default for
/path/to/component.conf is /path/to/component-paste.ini).

Change-Id: Ibd7aefe95706fb50e4e353c14eb6dab0f948331f
2012-01-17 12:00:51 +00:00
Eoghan Glynn
5835b30cc2 Avoid the need for users to manually edit PasteDeploy config in order to switch pipelines.
Define multiple pipelines in glace-api.conf to reflect the
various supported deployment flavors (minimal, with caching,
with cache management, with keystone-based auth etc.).

Add an optional paste_deploy.flavor config variable to allow the
user select the appropriate pipeline without having to edit the
paste config (i.e. uncommenting lines as before). For example
in glance-api.conf, a setting of:

  [paste_deploy]
  flavor = keystone+caching

identifies the following pipeline in glace-api-paste.ini:

  [pipeline:glance-api-keystone+caching]
  pipeline = versionnegotiation authtoken auth-context cache apiv1app

the advantage being that the user need not be concerned with
the precise sequence of filters required to realize the QoS
they desire.

Modify the functional tests that patch configuration (i.e. the
keystone and caching tests) to use the new deployment_flavor
mechanism.

Extend the TestConfigOpts to support option groups.

Change-Id: Ide843ada11bce115b7dc650440397853c6409b03
2012-01-17 12:00:33 +00:00
Eoghan Glynn
14593a3b96 Split out paste deployment config from the core glance
*.conf files into corresponding *-paste.ini files.

Fixes LP #815208

Change-Id: I5acb8488cc25bb3c7dd735c8c2b0fe62fcea8e5e
2012-01-17 11:40:26 +00:00
Jay Pipes
b4624ec242 Fixes LP Bug#913608 - tests should be isolated
This reworks the stubs.FAKE_FILESYSTEM_ROOTDIR used in unit
tests by making each test case create and destroy its own little
sandbox in /tmp/test.*/images. Adds a base IsolatedUnitTest that
tests needing to have an isolated filesystem_store_datadir and
configuration can inherit from.

Change-Id: I396f5127c6a687da8dcef3368e7ed0912efc9b3a
2012-01-16 17:07:39 -05:00
Eoghan Glynn
6fd005465e Set correct Content-Length on cached remote images
Override the image metadata size attribute with the actual
cached image file size when setting the Content-Length header
on the response to a GET of a cache remote image.

This avoids an non-initial GET on a remote image failing when
caching is enabled, as described in this bug:

  Bug 900959

Change-Id: I7656bc971d982cce78ce1ed4cf7ea509f8bbdfa8
2012-01-16 18:50:28 +00:00
Chris Behrens
195e6670c4 Implement retries in notify_kombu
Fixes bug 915817

Copies code from nova and includes improvements pending for nova as well

Change-Id: Ib596f4bacc5f58c4507f4111cdfac273029018bc
2012-01-12 23:49:41 -08:00
Jenkins
b0181065d0 Merge "Return correct href if bind_host is 0.0.0.0." 2012-01-12 21:40:21 +00:00
Pete Zaitcev
b46401a17b Return correct href if bind_host is 0.0.0.0.
The existing code returns incorrect href= in the root document (aka
"version" document) if the default configuration is used. If a remote
system accesses Glance API, the netspec of href= contains 0.0.0.0,
which cannot be used for further remote accesses. This happens because
attempts to use local configuration or the bind address to reconstruct
URLs are futile.

The correct way resolve this problem is to use the Host: header
of the HTTP request. As a bonus, it works with IPv6, should we ever
support it.

Change-Id: I661351b679c695bd6ab310104a3246dd3069f9c6
Bug: 915621
2012-01-12 14:18:07 -07:00
Major Hayden
2be1efe281 Remove assertDictEqual for python 2.6 compatibility.
Fixes bug 915604. Removes assertDictEqual from glance tests and replaces it with assertEqual.

Change-Id: I1439516331b4e92051ccfce6abebe17a0516a52e
2012-01-12 15:14:01 -06:00
Jenkins
c2c805b707 Merge "Add notifications for sending an image" 2012-01-12 15:03:29 +00:00
Jenkins
76f24c3b68 Merge "LP Bug#912800 - Delete image remain in cache" 2012-01-11 16:21:57 +00:00
Mark McLoughlin
1590dadcb4 Add optional revision field to version number
Add a revision field to the version number to account for releases from
the stable branch, e.g. 2011.3.1.

See http://wiki.openstack.org/StableBranchRelease

Change-Id: Ibe3b69fcc7e7d09f17c8f6e68a0e7bb17afbfdce
2012-01-11 07:25:36 +00:00
Jay Pipes
878036019a LP Bug#912800 - Delete image remain in cache
Adds a functional test case that verifies that when an
image is deleted and the cache middleware is place, that
the image is properly removed from the image cache.

Change-Id: I20fc4461bd5f53528b264536b248f596aff510af
2012-01-10 20:48:04 -05:00
Alex Meade
e2f9d15e4c Add notifications for sending an image
An image.send notification is to be sent to the notifier every time an image is
transmitted from glance. This can be used to track things such as bandwidth
usage.

Addresses bug 914440

Change-Id: If8b6504c4250fa6444d17d611de43d9704ca9aae
2012-01-10 19:47:22 +00:00
Jenkins
7f4b1c5899 Merge "Split notification strategies out into modules." 2012-01-10 16:19:45 +00:00
Jenkins
883bdd8564 Merge "Fixes incorrect URI scheme for s3 backend" 2012-01-10 02:51:01 +00:00
Jenkins
e27b0e1c27 Merge "Add comments for swift options in glance-api.conf." 2012-01-10 02:41:04 +00:00
Jenkins
e4bd841b6d Merge "Bug #909533: Swift uploads through Glance using ridiculously small chunks" 2012-01-09 22:26:08 +00:00
Jenkins
85a91606b1 Merge "Remove doc/Makefile." 2012-01-09 22:21:58 +00:00
Ewan Mellor
48449171a7 Bug #909533: Swift uploads through Glance using ridiculously small chunks
Remove the use of a temporary disk buffer when streaming to Swift.  This
was added on the assumption that it was not possible to stream chunks to
Swift direct from webob.Request.body_file.  That's not true -- a simple
file-like object wrapping body_file does the job perfectly.

This removes the need for swift_store_object_buffer_dir in the config file.
It is also significantly cheaper, since there is one copy fewer.

Fix the parsing of the swift_store_large_object_size and
swift_store_large_object_chunk_size options.  These are
specified in MB in the config file, but needs to be in bytes internally,
because they are compared against the image_size parameter given to add().

Update the unit tests to match.  This includes an additional check that
put_object is called the correct number of times.  The unit tests missed the
latter problem problem because there was no check that a small object is only
uploaded in one chunk, and despite the comment to the contrary there was no
check that the right number of chunks were written in the large-object case
either.  Added these in all places where store.add is called.

Change-Id: Ieb4cf68516b53bd16d2671e49e805b26118b3671
2012-01-06 19:44:49 -05:00
Pete Zaitcev
1cff8250c6 Add Fedora clauses to the installing document.
This change resolves 2 Fedora-related todo items in installing.rst.

It also cleans up a bit:
 - fixes weird grammar
 - fixes improper list numbering
 - uses proper name for Red Hat, where space is necessary
 - replaces A/B syntax with A, B

This is my first commit and I am adding myself to Authors file
on a suggestion by Brian Waldon.

Change-Id: I398ae8150d7912e0b61375a7bdc5aab1101f34bc
2012-01-05 15:39:55 -07:00
Russell Bryant
422c1e4aa9 Remove doc/Makefile.
Use "python setup.py build_sphinx" instead.  There are some things that
the Makefile did that setup.py does not do, but they will be easy enough
to add later if anyone misses them.

Change-Id: I430dcc027bac207af515fb7d927500db18320ba5
2012-01-05 08:54:16 -05:00