This patch implements a new notification strategy that sends
notifications to a message queue via Qpid.
Change-Id: Ie2640139dcf735e428af63e9e87b3667a2b22eb5
Implements blueprint multi-process-server. Allows several Glance API
worker processes to be started, which can increase performance on machines
with more than one CPU.
Change-Id: I1cbb48945fd23afd71de3a30b80836b590c023a1
Add an optional paste_deploy.config_file variable to identify
a non-default location for the paste.ini file (the default for
/path/to/component.conf is /path/to/component-paste.ini).
Change-Id: Ibd7aefe95706fb50e4e353c14eb6dab0f948331f
Define multiple pipelines in glace-api.conf to reflect the
various supported deployment flavors (minimal, with caching,
with cache management, with keystone-based auth etc.).
Add an optional paste_deploy.flavor config variable to allow the
user select the appropriate pipeline without having to edit the
paste config (i.e. uncommenting lines as before). For example
in glance-api.conf, a setting of:
[paste_deploy]
flavor = keystone+caching
identifies the following pipeline in glace-api-paste.ini:
[pipeline:glance-api-keystone+caching]
pipeline = versionnegotiation authtoken auth-context cache apiv1app
the advantage being that the user need not be concerned with
the precise sequence of filters required to realize the QoS
they desire.
Modify the functional tests that patch configuration (i.e. the
keystone and caching tests) to use the new deployment_flavor
mechanism.
Extend the TestConfigOpts to support option groups.
Change-Id: Ide843ada11bce115b7dc650440397853c6409b03
Remove the use of a temporary disk buffer when streaming to Swift. This
was added on the assumption that it was not possible to stream chunks to
Swift direct from webob.Request.body_file. That's not true -- a simple
file-like object wrapping body_file does the job perfectly.
This removes the need for swift_store_object_buffer_dir in the config file.
It is also significantly cheaper, since there is one copy fewer.
Fix the parsing of the swift_store_large_object_size and
swift_store_large_object_chunk_size options. These are
specified in MB in the config file, but needs to be in bytes internally,
because they are compared against the image_size parameter given to add().
Update the unit tests to match. This includes an additional check that
put_object is called the correct number of times. The unit tests missed the
latter problem problem because there was no check that a small object is only
uploaded in one chunk, and despite the comment to the contrary there was no
check that the right number of chunks were written in the large-object case
either. Added these in all places where store.add is called.
Change-Id: Ieb4cf68516b53bd16d2671e49e805b26118b3671
Fixes bug 901376
Glance used a SimpleQueue which would end up with a direct queue with an
exchange named the same as the queue. This was different than Nova, which
uses a topic queue with an exchange named 'nova'.
This change makes Glance use a topic queue with a configurable exchange
name to match Nova.
Change-Id: Ia014e4c00060abc2345289a54e45bbfdc6b7e8e5
These generic factories allow us to dump the copied and pasted
app_factory and filter_factory methods in the codebase.
The main difference is the paste configuration changes from:
[app:apiv1app]
paste.app_factory = glance.api.v1:app_factory
...
[filter:cache]
paste.filter_factory = glance.api.middleware.cache:filter_factory
to this:
[app:apiv1app]
paste.app_factory = glance.common.wsgi:app_factory
glance.app_factory = glance.api.v1:API
...
[filter:cache]
paste.filter_factory = glance.common.wsgi:filter_factory
glance.filter_factory = glance.api.middleware.cache:CacheFilter
Apart from reducing code duplication, this will also allow us to have
the generic factories inject other data into the apps and filters.
Change-Id: I1d2be5630ab61d29b8948ff88d58e6e1b11c2e5f
Implements blueprint custom-disk-buffer
When pushing an image large enough to be chunked into either a Swift
or S3 backend, the chunks are buffered to /tmp.
This change allows a user to define an alternate directory via the
configuration files.
Change-Id: I2660b6e5a76cbfd43834384b129550b66bb9afce
Adds documentation on managing the image cache
Adds a simple CLI program to queue an image for
caching.
Change-Id: I2b4c9708c521fdcfaf1c6b8f07505e130923c4b7
Fixes LP Bug#879136 - keyerror: 'image' when doing nova image-list
Fixes LP Bug#819936 - New image cache breaks Glance on Windows
This patch refactors the image cache further by adding an
adaptable driver layer to the cache. The existing filesystem-based
driver that depended on python-xattr and conditional fstab support
has been moved to /glance/image_cache/drivers/xattr.py, and a new
default driver is now based on SQLite and has no special requirements.
The image cache now contains a simple interface for pruning the
cache. Instead of the logic being contained in
/glance/image_cache/pruner.py, now the prune logic is self-contained
within the ImageCache.prune() method, with pruning calling the
simple well-defined driver methods of get_least_recently_accessed()
and get_cache_size().
Adds a functional test case for the caching middleware and adds
documentation on how to configure the image cache drivers.
TODO: cache-manage middleware...
TODO: cache management docs
Change-Id: Id7ae73549d6bb39222eb7ac0427b0083fd1af3ec
Fixes LP Bug#874580 - keyerror 'location' when fetch errors
Fixes LP Bug#817570 - Make new image cache a true extension
Fixes LP Bug#872372 - Image cache has virtually no unit test coverage
* Adds unit tests for the image cache (coverage goes from 26% to 100%)
* Removes caching logic from the images controller and places it into
a removeable transparent caching middleware
* Adds a functional test case that verifies caching of an image
and subsequent cache hits
* Removes the image_cache_enabled configuration variable, since it's
now enabled by simply including the cache in the application
pipeline
* Adds a singular glance-cache.conf to etc/ that replaces the
multiple glance-pruner.conf, glance-reaper.conf and
glance-prefetcher.conf files
* Adds documentation on enabling and configuring the image cache
TODO: Add documentation on the image cache utilities, like reaper,
prefetcher, etc.
Change-Id: I58845871deee26f81ffabe1750adc472ce5b3797
* Adds SSL configuration params to all client classes
* Adds unit test for bad SSL client configuration
* Refactors the registry methods to no longer need
configuration options passed, and to configure the
registry client once, on images.Controller.__init__
* Adds glance-api.conf options for SSL support in
registry client connections.
* Adds SSL CA file socket wrappers via a client auth
HTTPS connection class
* Adds server SSL support, but not functional tests
for SSL support yet. Still need to research self-signed
cert generation for tests...
* Updates documentation for bind and startup options
* Adds functional test case for secure communication
with API server stood up with SSL supprt. Note it is
not very DRY. There is some DRY cleanup to do in the
future...
TODO: Integrate options with bin/glance CLI tool
Change-Id: Ie9fcd36337cc93fd5beeabb9186ad5e93ae2a0f0
Resolves bug #828719, which can potentially cause problems depending on
what $PWD commands are executed in.
Amendment: Also update documentation accordingly.
Change-Id: Ib2ec4bb07222e51c86b7d8f8ce1dd0e6bb342a64
Fixes LP Bug #827660 - Swift driver fails to
upload image files greater than 5GB. The swift
store driver was not using the large object
manifest methods required to upload objects
greater than 5GB into Swift.
Adds configuration options to specify the
size in MB that an image file should be considered
for large object manifest, and a configurable
chunk size. We use a disk buffer of this size,
adding chunks of the image file into Swift and
then after all chunks are saved, add the manifest
to Swift.
Change-Id: I4b4421aa1bba584ab242016a041e864ef19d0214
program based on Swift's swift-init script.
Adds a crap-ton of documentation on installing Glance and
controlling Glance's server programs.
This was necessary since when glance.common.server was removed
from Glance and paste.deploy used, we lost the ability to control
server daemonization with CLI options. This adds back some of that
functionality while still using paste.deploy like Swift does.