This fixes LP bug #1063793.
This adds a new API config option registry_client_insecure
(default false) which optionally allows not to specify a ca
file for registry connections from the API server.
The default is false so existing behaviour is
unchanged unless this option is specified.
Adding this option makes the registry symmetric wrt the API
with which you can connect using clients with --insecure
Change-Id: I4fa462cd68afaf14f7d7de6c572980a41c0849d6
Fixes bug 1054183
Avoid AMQPChannelException: (406, "PRECONDITION_FAILED...") failures
due to a mismatch between the durability of rabbitmq exchange/queue
declared by glance and ceilometer.
Change-Id: I4e25986a1f503782e701aa1168c4eb231ff25d06
In realistic production environments, multi-processing is likely
to be enabled.
A default setting of one worker would allow the multi-processing
codepath to be followed by default, without changing the number
of processes actually servicing API requests (i.e. the single
parent process will just wait on the child).
For most functional tests, we continue to explicitly configure
zero workers, so as to avoid spawning twice as many glance-api
processes during a test run.
Change-Id: I1c327805f8fb975a3175e9a8b40aeddd21d5ef91
Swift's implementation of the S3 interface doesn't support subdomain
access to containers. It requires that the bucket be prepended to the
request path. The option 's3_store_bucket_url_format' can be set to
either 'path' or 'subdomain' (default) to control how boto forms
the bucket url.
Fixes bug 997658
Change-Id: Ia6e1e7356eef7ac2267f7738e2f4a7c70dc12eeb
One can not simply remove the hash mark in front of a commented-out
config option if there is a space following it. This patch removes
that extraneous whitespace.
Change-Id: Id084eef34a9e894de3f67b2919d55879cd9a715e
Uses config options enable_v1_api and enable_v2_api
to control which APIs have been deployed.
(Thanks markwash for all the help)
Fixes bug 1043497
Change-Id: I42dc1d88f94e61b9550c5f114ffc1abad25be7ff
* Reduce comments in paste files to just what is necessary
* Add a 'cachemanage' pipeline to complete our set of paste
deploy flavors in glance-api-paste.ini
* Add a sample [paste_deploy] section to both glance-api and
glance-registry configs
Related to bp glance-folsom-docs-cleanup.
Change-Id: I0208b12f6861b48e52260b25802c47ebf6316b9a
Since we can now configure Keystone's auth_token middleware using
our own config files, we should ship our example config to show
how this is done. This moves the auth_* variables out of
glance-api-paste.ini and glance-registry-paste.ini into
glance-api.conf and glance-registry.conf.
Fixes bug 1043490
Change-Id: Ibdaf8b2ef0f55e5e892fbd1b994dceb323f07c7b
Add conf option image_size_cap to represent what the internal
IMAGE_SIZE_CAP constant used to cover in the v1 API. The default
is reduced to 1 TB from 50 PB.
Thoroughly test this on image create and update through the v1 API.
Fixes bug 1038994
Change-Id: Ie0985b62228f8b28c005a8847049d7b68d9b959a
This adds a 'ca_file' config option that points to a local
CA cert that will be used to verify certs provided by
connecting clients. The 'ca_file' option is only used if the
server is already properly configured to to use SSL - that
means having a valid 'cert_file' and 'key_file'. If no 'ca_file'
is provided, the behavior will remain the same - the server
will still provide its cert to clients, but it will ignore
certs sent back from those clients.
Fixes bug 1032451
Change-Id: Ie48646b0fc5398ba7cda2fb627b820f533482e00
Add a swift storage option to configure a default list of admin
tenant IDs which will be granted write access on any swift
images.
This setting might be useful when a user choose not to use a global
swift admin role when managing glance images. This might also
be useful when implementing delayed delete for glance swift
multi-tenant.
Partially implements blueprint: swift-tenant-specific-storage.
Change-Id: Id65e1c8d114c9551b99f6a18c527ba26aefd341f
Updates the swift store to support multiple tenants:
* Added configuration option for swift_store_multi_tenant.
* Updated the swift connection creation logic so that in multi-tenant
mode the token and storage URL from the service catalog are used
to create swift connection.
* When in multi-tenant mode locations URL's (stored in the DB) do
not contain hard coded swift credentials.
Includes unit tests to verify multi-tenant swift storage URLs.
Partially implements blueprint: swift-tenant-specific-storage.
Change-Id: I45fc97027e6f211ac353513c2d9d6da51ccf4489
Updates the default value for the 'known_stores' config
variable to include the full list of storage classes
supported in Glance.
Also, removes the known_stores config section from the
example config file since it is arguably an advanced config
that won't get used by most users. Removing this config avoids
the overhead of maintaining internal class names in config files.
Fixes LP Bug #1008698.
Change-Id: I0117376aa4de3103410ecb1a36df6998fcd0d5b5
Certain deployments need to allow anonymous access to its
images. This patch allows a user that has failed to authenticate with
Keystone to access the API in read-only context.
* Configure through 'allow_anonymous_access' option, defaults to False
* Implements bp api-v2-anonymous-access
Change-Id: Ia8f57e54bd141a2da1ca4600d1970558fb497f67
The Glance v2 API makes use of the DB directly. As such
we should include the sql_ configs in the default glance-api.conf
file.
Fixes LP Bug #1003504.
Change-Id: I5e287dafcd65864f56789ae22bab3dcc6b353a47
- Update glance with the latest changes in swift.common.client.
- Use 5000 for port.
- Properly test v2 in tests.
- Fixes bug 980196.
- Test swift against a specific stable version like for keystone.
Change-Id: Iaff536efc92e78e51d8042010ce0eb407c4a71b2
* Allow a deployer to define the role that should give an authenticated
user admin privileges. Defaults to 'admin'
* Fixes bug 956547
Change-Id: I018b9c438a12c527833ed8e2e6c377a93fde9130
This patch implements a new notification strategy that sends
notifications to a message queue via Qpid.
Change-Id: Ie2640139dcf735e428af63e9e87b3667a2b22eb5
Implements blueprint multi-process-server. Allows several Glance API
worker processes to be started, which can increase performance on machines
with more than one CPU.
Change-Id: I1cbb48945fd23afd71de3a30b80836b590c023a1
Remove the use of a temporary disk buffer when streaming to Swift. This
was added on the assumption that it was not possible to stream chunks to
Swift direct from webob.Request.body_file. That's not true -- a simple
file-like object wrapping body_file does the job perfectly.
This removes the need for swift_store_object_buffer_dir in the config file.
It is also significantly cheaper, since there is one copy fewer.
Fix the parsing of the swift_store_large_object_size and
swift_store_large_object_chunk_size options. These are
specified in MB in the config file, but needs to be in bytes internally,
because they are compared against the image_size parameter given to add().
Update the unit tests to match. This includes an additional check that
put_object is called the correct number of times. The unit tests missed the
latter problem problem because there was no check that a small object is only
uploaded in one chunk, and despite the comment to the contrary there was no
check that the right number of chunks were written in the large-object case
either. Added these in all places where store.add is called.
Change-Id: Ieb4cf68516b53bd16d2671e49e805b26118b3671
Fixes bug #912094.
URI scheme for s3 backend should be 's3+https', not 'swift+https'.
Use s3.amazonaws.com instead of s3service.com to follow RFC2606.
Indent reason for BadStoreUri exception in s3.py.
Comments added for the following option in etc/glance-api.conf
's3_store_host'
Change-Id: Id456b287d82fe61711f0d738e61ccb599f0b5684
Fixes bug #771465.
Comments added for the following two options in
glance-api.conf
'swift_store_auth_address'
'swift_store_user'
Indent reason for BadStoreUri exception in swift.py.
Change-Id: Ife28e01a4e2344cf1e5a179752d244b68e45d348
Fixes bug 901376
Glance used a SimpleQueue which would end up with a direct queue with an
exchange named the same as the queue. This was different than Nova, which
uses a topic queue with an exchange named 'nova'.
This change makes Glance use a topic queue with a configurable exchange
name to match Nova.
Change-Id: Ia014e4c00060abc2345289a54e45bbfdc6b7e8e5
These generic factories allow us to dump the copied and pasted
app_factory and filter_factory methods in the codebase.
The main difference is the paste configuration changes from:
[app:apiv1app]
paste.app_factory = glance.api.v1:app_factory
...
[filter:cache]
paste.filter_factory = glance.api.middleware.cache:filter_factory
to this:
[app:apiv1app]
paste.app_factory = glance.common.wsgi:app_factory
glance.app_factory = glance.api.v1:API
...
[filter:cache]
paste.filter_factory = glance.common.wsgi:filter_factory
glance.filter_factory = glance.api.middleware.cache:CacheFilter
Apart from reducing code duplication, this will also allow us to have
the generic factories inject other data into the apps and filters.
Change-Id: I1d2be5630ab61d29b8948ff88d58e6e1b11c2e5f
This patch addresses the dependency proliferation in
the glance client stuff. It removes references to modules
that contain non-client-necessary libraries (like
eventlet, xattr, sqlalchemy-migrate and sqlalchemy)
by restructuring the modules slightly.
Note that the additional httplib2 dependency is because
that is used in the authentication strategy stuff in
glance.common.auth. This could be rewritten to use httplib
instead, further reducing the dependencies of the client lib.
IMPORTANT NOTE: This patch changes the default entrypoint
for the Images API router application, and therefore this
should be merged along with the packaging changes in this
branch:
https://code.launchpad.net/~jaypipes/glance/ubuntu/+merge/82318
Change-Id: I5dbc8584fb77e3e011fb6ff3532f792f5103e242
Implements blueprint custom-disk-buffer
When pushing an image large enough to be chunked into either a Swift
or S3 backend, the chunks are buffered to /tmp.
This change allows a user to define an alternate directory via the
configuration files.
Change-Id: I2660b6e5a76cbfd43834384b129550b66bb9afce
Since the version negotiation middleware was introduced, versions
pipeline app in the config is unused. Consequently, the versions
app factory is unused too.
Change-Id: I2d50432de215e51c1e52f8d68eaa0820e9b5bd0b
Implements blueprint swift-location-credentials
When the new option is enabled the location metadata
(which may include user credentials) is encrypted
before being sent to the registry server.
Change-Id: I072e3f5c81f187435b1c156627076d5fde087af5
Fixes LP Bug#879136 - keyerror: 'image' when doing nova image-list
Fixes LP Bug#819936 - New image cache breaks Glance on Windows
This patch refactors the image cache further by adding an
adaptable driver layer to the cache. The existing filesystem-based
driver that depended on python-xattr and conditional fstab support
has been moved to /glance/image_cache/drivers/xattr.py, and a new
default driver is now based on SQLite and has no special requirements.
The image cache now contains a simple interface for pruning the
cache. Instead of the logic being contained in
/glance/image_cache/pruner.py, now the prune logic is self-contained
within the ImageCache.prune() method, with pruning calling the
simple well-defined driver methods of get_least_recently_accessed()
and get_cache_size().
Adds a functional test case for the caching middleware and adds
documentation on how to configure the image cache drivers.
TODO: cache-manage middleware...
TODO: cache management docs
Change-Id: Id7ae73549d6bb39222eb7ac0427b0083fd1af3ec
Fixes LP Bug#874580 - keyerror 'location' when fetch errors
Fixes LP Bug#817570 - Make new image cache a true extension
Fixes LP Bug#872372 - Image cache has virtually no unit test coverage
* Adds unit tests for the image cache (coverage goes from 26% to 100%)
* Removes caching logic from the images controller and places it into
a removeable transparent caching middleware
* Adds a functional test case that verifies caching of an image
and subsequent cache hits
* Removes the image_cache_enabled configuration variable, since it's
now enabled by simply including the cache in the application
pipeline
* Adds a singular glance-cache.conf to etc/ that replaces the
multiple glance-pruner.conf, glance-reaper.conf and
glance-prefetcher.conf files
* Adds documentation on enabling and configuring the image cache
TODO: Add documentation on the image cache utilities, like reaper,
prefetcher, etc.
Change-Id: I58845871deee26f81ffabe1750adc472ce5b3797
Addresses bug 871803 by expounding on what configuration should
look like. Also fixes the example config files, which generally
had the authentication setup completely wrong.
Change-Id: I44b2b8bd340ca95b5a2c2e9408797b0308000a65
* Adds SSL configuration params to all client classes
* Adds unit test for bad SSL client configuration
* Refactors the registry methods to no longer need
configuration options passed, and to configure the
registry client once, on images.Controller.__init__
* Adds glance-api.conf options for SSL support in
registry client connections.
* Adds SSL CA file socket wrappers via a client auth
HTTPS connection class
* Adds server SSL support, but not functional tests
for SSL support yet. Still need to research self-signed
cert generation for tests...
* Updates documentation for bind and startup options
* Adds functional test case for secure communication
with API server stood up with SSL supprt. Note it is
not very DRY. There is some DRY cleanup to do in the
future...
TODO: Integrate options with bin/glance CLI tool
Change-Id: Ie9fcd36337cc93fd5beeabb9186ad5e93ae2a0f0
* glance-api writes out to a scrubber "queue" dir on delete
* Scrubber determines images to deleted from "queue" dir not db
Change-Id: Ia5574fc75f1a9c763bdef0f5773c2c182932b68a
Fixes LP Bug #827660 - Swift driver fails to
upload image files greater than 5GB. The swift
store driver was not using the large object
manifest methods required to upload objects
greater than 5GB into Swift.
Adds configuration options to specify the
size in MB that an image file should be considered
for large object manifest, and a configurable
chunk size. We use a disk buffer of this size,
adding chunks of the image file into Swift and
then after all chunks are saved, add the manifest
to Swift.
Change-Id: I4b4421aa1bba584ab242016a041e864ef19d0214