Commit Graph

121 Commits

Author SHA1 Message Date
Brian Waldon
b7197041fe Move authtoken config out of paste
Since we can now configure Keystone's auth_token middleware using
our own config files, we should ship our example config to show
how this is done. This moves the auth_* variables out of
glance-api-paste.ini and glance-registry-paste.ini into
glance-api.conf and glance-registry.conf.

Fixes bug 1043490

Change-Id: Ibdaf8b2ef0f55e5e892fbd1b994dceb323f07c7b
2012-08-29 12:24:52 -07:00
Jenkins
3c69df5aa7 Merge "Make max image size upload configurable" 2012-08-20 17:38:08 +00:00
Brian Waldon
3a80eee8ee Make max image size upload configurable
Add conf option image_size_cap to represent what the internal
IMAGE_SIZE_CAP constant used to cover in the v1 API. The default
is reduced to 1 TB from 50 PB.

Thoroughly test this on image create and update through the v1 API.

Fixes bug 1038994

Change-Id: Ie0985b62228f8b28c005a8847049d7b68d9b959a
2012-08-20 08:01:00 -04:00
Brian Waldon
2e9a467dc8 Set proper auth middleware option for anon. access
Fixes bug 1037793

Change-Id: I89c05b5004f6a93608c0e23302f04b95b9018c29
2012-08-16 17:07:43 -04:00
Brian Waldon
54cf135061 Use container_format and disk_format as-is in v2
Rather than force a painful migration to type and format, use
container_format and disk_format as they are in the v1 API for v2
since there isn't a clear path to an interoperable alternative.

Related to bp api-2

Change-Id: If8161390abf17961885e5c02e38d635b287aeca4
2012-08-10 14:09:49 -07:00
Brian Waldon
99d5173afe Drop unfinshed parts of v2 API
The v2 API will be implemented over time as the spec develops. We
shouldn't implement anything that we aren't 100% behind. This drops
all of the things that we aren't in love with:
* Drop access records and related schemas
* Drop owner attribute from image

Related to bp api-2

Change-Id: Ieef2c141282e7018d56e79aee8f20af0542af25b
2012-08-10 10:33:38 -07:00
Brian Waldon
006254c505 Allow server-side validation of client ssl certs
This adds a 'ca_file' config option that points to a local
CA cert that will be used to verify certs provided by
connecting clients. The 'ca_file' option is only used if the
server is already properly configured to to use SSL - that
means having a valid 'cert_file' and 'key_file'. If no 'ca_file'
is provided, the behavior will remain the same - the server
will still provide its cert to clients, but it will ignore
certs sent back from those clients.

Fixes bug 1032451

Change-Id: Ie48646b0fc5398ba7cda2fb627b820f533482e00
2012-08-06 11:45:34 -07:00
Jenkins
ab8bc2ab3b Merge "Make TCP_KEEPIDLE configurable" 2012-08-03 18:15:50 +00:00
Dan Prince
8a4f3773ac Add swift_store_admin_tenants option.
Add a swift storage option to configure a default list of admin
tenant IDs which will be granted write access on any swift
images.

This setting might be useful when a user choose not to use a global
swift admin role when managing glance images. This might also
be useful when implementing delayed delete for glance swift
multi-tenant.

Partially implements blueprint: swift-tenant-specific-storage.

Change-Id: Id65e1c8d114c9551b99f6a18c527ba26aefd341f
2012-08-01 13:43:38 -04:00
Stuart McLaren
efb30dd640 Make TCP_KEEPIDLE configurable
Addresses LP bug #1031794.

Change-Id: I9a3a6a23b1bcec5229b15234512e04cc47c3ffd9
2012-08-01 15:04:40 +00:00
Dan Prince
8b2d038185 Adds multi tenant support for swift backend.
Updates the swift store to support multiple tenants:

 * Added configuration option for swift_store_multi_tenant.

 * Updated the swift connection creation logic so that in multi-tenant
   mode the token and storage URL from the service catalog are used
   to create swift connection.

 * When in multi-tenant mode locations URL's (stored in the DB) do
   not contain hard coded swift credentials.

Includes unit tests to verify multi-tenant swift storage URLs.

Partially implements blueprint: swift-tenant-specific-storage.

Change-Id: I45fc97027e6f211ac353513c2d9d6da51ccf4489
2012-07-20 10:12:02 -04:00
Brian Waldon
ce899351da Reorganize context module
* Move RequestContext class to glance.context
* Move context middlewares to glance.api.middleware
* Update tests to reflect move
* Update paste configs

Related to bp glance-request-tracking

Change-Id: I289b546ec28c973a3022be779ce378ae2febb340
2012-07-10 08:31:50 -07:00
Mark McLoughlin
c44e16a538 Don't use PasteDeploy for scrubber and cache daemons
There are no WSGI apps involved here, no need to configure pipelines with
filters etc. All we're using PasteDeploy for is to construct an object.

Well ... there are easier ways to do that! :)

Change-Id: I4166d0071738fe587572db8f2883900408e9f097
2012-06-16 17:36:55 +01:00
Mark McLoughlin
3b4c276550 Remove some unused glance-cache-queue-image code
Commit d453def removed the glance-cache-queue-image script but
neglected to remove the PasteDeploy app and config.

Change-Id: I20daf529fd9e466cdad9d3db19bc79ed16f24375
2012-06-16 17:07:32 +01:00
Dan Prince
45b64d5495 Add metadata_encryption_key to glance-cache.conf.
Fixes LP Bug #1012752.

Change-Id: I802583e2766cd1533223638259e57e44b39448eb
2012-06-13 12:30:59 -04:00
Jenkins
4f688ef429 Merge "Remove the conf passing PasteDeploy factories" 2012-06-11 20:15:33 +00:00
Dan Prince
15c204af75 Update default values for known_stores config.
Updates the default value for the 'known_stores' config
variable to include the full list of storage classes
supported in Glance.

Also, removes the known_stores config section from the
example config file since it is arguably an advanced config
that won't get used by most users. Removing this config avoids
the overhead of maintaining internal class names in config files.

Fixes LP Bug #1008698.

Change-Id: I0117376aa4de3103410ecb1a36df6998fcd0d5b5
2012-06-08 15:37:29 -04:00
Mark McLoughlin
d33ce445d3 Remove the conf passing PasteDeploy factories
Implements blueprint cfg-global-object

In glance.common.wsgi we have a paste_deploy_app() API which allows
a ConfigOpts instance to be passed the the applications and filters
constructed by PasteDeploy.

Now that we're using a global object, we don't need this anymore

Change-Id: I222d3c58308bc2f504cd802c0c8405d3baa2e49a
2012-06-08 17:36:26 +01:00
Brian Waldon
2a01e6ef75 Enable anonymous access through context middleware
Certain deployments need to allow anonymous access to its
images. This patch allows a user that has failed to authenticate with
Keystone to access the API in read-only context.

* Configure through 'allow_anonymous_access' option, defaults to False
* Implements bp api-v2-anonymous-access

Change-Id: Ia8f57e54bd141a2da1ca4600d1970558fb497f67
2012-05-28 20:44:14 +01:00
Jenkins
87e45ac659 Merge "Correct format of schema-image.json." 2012-05-23 22:45:20 +00:00
Dan Prince
a92b01fe58 Adds sql_ config settings to glance-api.conf.
The Glance v2 API makes use of the DB directly. As such
we should include the sql_ configs in the default glance-api.conf
file.

Fixes LP Bug #1003504.

Change-Id: I5e287dafcd65864f56789ae22bab3dcc6b353a47
2012-05-23 11:56:10 -04:00
Dan Prince
88f802027c Correct format of schema-image.json.
Correct the JSON format of schema-image.json. Fixes issues where
glance-api won't start due to invalid JSON in this file.

Fixes LP Bug #1003470.

Change-Id: Ibacc49f2c45e233bd6ad25b3334e013acaae8339
2012-05-23 10:48:43 -04:00
Brian Waldon
1533e6d8b8 Fix paste to correctly deploy v2 API
Corrects a typo where 'apiv1app' should be 'apiv2app'

Change-Id: I4ef29f7539a572ed65422fd548e9718e86161b5f
2012-05-22 13:23:44 -07:00
Joshua Harlow
ef475e17d5 Implements blueprint import-dynamic-stores.
Make glance more pluggable with regard to stores.

Change-Id: I7b264d1b047a321f7b60857bb73154f831b82a7b
2012-05-16 09:39:49 -07:00
Jenkins
6880a4ecf6 Merge "Update glance to pass properly tenant_name." 2012-05-15 20:28:27 +00:00
Brian Waldon
eee5fecdf6 Support custom properties in schemas for v2 API
* Add glance schema API (glance.schema:API)
* Disallow custom properties that conflict with base properties
* Implements bp api-v2-schemas

Change-Id: Ibfa617cb5edf16627627debc30149669213d4b2d
2012-05-14 08:35:23 -07:00
Dan Prince
06821eaff2 Update noauth caching pipeline to use unauth-ctx.
Updates the noauth version of the caching pipeline to use
unauthenticated-context.

Fixes LP Bug #995323.

Change-Id: Id83be95f582a4a6ebe47dbeddc8101a51988e05e
2012-05-05 22:09:19 -04:00
Brian Waldon
59aa1ef2a7 Add /v2 to sample glance-api-paste.ini
* Related to bp api-2

Change-Id: I147624128bea7e0ca84c1673857b227f0590ec78
2012-05-03 17:52:25 -07:00
Brian Waldon
eb4aaa97bd Split noauth context middleware into new class
Use UnauthenticatedContextMiddleware in the case that you want to deploy
without an auth service. ContextMiddleware is now solely for authenticated
requests.

* Rewrite ownership tests for adding/updating images through the v1 API
* Fixes bug 992859

Change-Id: I608671aac8300c9195769542708708afda991e04
2012-05-03 09:00:07 -07:00
Chmouel Boudjnah
fe3ae1d73b Update glance to pass properly tenant_name.
- Update glance with the latest changes in swift.common.client.
- Use 5000 for port.
- Properly test v2 in tests.
- Fixes bug 980196.
- Test swift against a specific stable version like for keystone.

Change-Id: Iaff536efc92e78e51d8042010ce0eb407c4a71b2
2012-05-02 13:31:09 +01:00
Joe Gordon
f0a0c4a000 Cleanup authtoken examples
* remove auth_uri - do not need when set auth_port, auth_host, auth_protocol
* remove servce_*  variables - not needed in new keystone
* update associated documentation

Change-Id: I9bf7d53f7d62371848645a242ae63f77b42319db
2012-05-01 15:38:07 -07:00
Brian Waldon
bd305fa07b Merge registry context with common context
Pull the extra methods defined in glance.registry.context:RequestContext into
glance.common.context:RequestContext. This is necessary as the new v2 API
implementation talks directly to the database.

* Partially implements bp api-2

Change-Id: Ia8c1fde511bdca69be0dcf61ce12976ace5f7dc3
2012-04-26 08:53:24 -07:00
Jay Pipes
c449d76ff2 Fixes LP #978119 - cachemanagement w/o keystone
Removes section of glance-api-paste.ini that refers to
an improper cachemanagement middleware without keystone

Change-Id: I86a1a84aac16a8345b06ce83562f4508b49dac10
2012-04-24 13:06:44 -04:00
Brian Waldon
35ed310555 Reorganize pipelines for multiple api versions
* Partially implements bp api-2

Change-Id: Ie942c54e0ba294da661de870a95d685a9b8966de
2012-04-05 08:52:12 -07:00
Juerg Haefliger
633f83109f Add support for auth version 2.
Implements version 2 authentication for the Swift backend.
Fixes bug 944946.

Change-Id: I614ad03374a2e31a945f12fb9588947022203f1f
2012-03-28 10:29:40 +02:00
Jenkins
c5ae3213a8 Merge "Remove keystone.middleware.glance_auth_token" 2012-03-19 13:17:50 +00:00
Brian Waldon
5bd453b14e Remove keystone.middleware.glance_auth_token
* The use of this middleware is no longer necessary
* Fixes bug 957501

Change-Id: I2536babe6b34aa0cb6aca2b3e768ac85800f93e7
2012-03-16 15:53:23 -07:00
Brian Waldon
9681f40130 Add policy checks for cache manage middleware
* Add checks for 'manage_image_cache' policy
* Limit 'manage_image_cache' policy to 'admin' role by default
* Fixes bug 954608

Change-Id: Ie8d79201f39db5a9d8dd0d943056e33dc8498c21
2012-03-16 14:50:37 -07:00
Brian Waldon
eff04c5612 Add admin_role option
* Allow a deployer to define the role that should give an authenticated
  user admin privileges. Defaults to 'admin'
* Fixes bug 956547

Change-Id: I018b9c438a12c527833ed8e2e6c377a93fde9130
2012-03-16 11:50:20 -07:00
Dan Prince
4d41027d21 Glance cache updates to support Keystone Essex.
Glance caching updates to support keystone service tenants:

 * Update glance-cache-manage so that it supports the same style OS_
  environment variables and CLI options for auth that bin/glance does.

 * Updates registry/client so that it supports Keystone KSL style
  service tenants. As services no longer use admin_tokens this is
  a requirement to be able to use caching w/ KSL.

Fixes LP Bug #949486.

Change-Id: I8d0e56a9ec0d20ef56ed2ce8b860d987ae159e01
2012-03-09 17:50:25 -05:00
Dan Prince
2773170d73 Update paste file to use service tenant.
Update the keystone section of api-paste.ini with example settings
for the keystone service tenant.

Remove the admin_token example lines.

Change-Id: I6169ae727a07e75879b5e2e5c4099c9688c726c6
2012-03-07 18:17:34 -05:00
Russell Bryant
2fa1ed0da1 Implement blueprint add-qpid-support.
This patch implements a new notification strategy that sends
notifications to a message queue via Qpid.

Change-Id: Ie2640139dcf735e428af63e9e87b3667a2b22eb5
2012-02-10 15:36:13 -05:00
Brian Waldon
eeb4d1ee29 Add policy checking for basic image operations
Partially implements bp interim-glance-authz-service.
This checks basic image operations: get_images, get_image,
add_image, modify_image, delete_image. It looks for a
policy json file next to our config files.

Change-Id: I07f29c11934c68d38a6bdadd39f9dc841b61648a
2012-01-19 16:23:13 -08:00
Stuart McLaren
e893b248a2 Multi-process Glance API server support.
Implements blueprint multi-process-server. Allows several Glance API
worker processes to be started, which can increase performance on machines
with more than one CPU.

Change-Id: I1cbb48945fd23afd71de3a30b80836b590c023a1
2012-01-19 09:45:21 -08:00
Eoghan Glynn
5835b30cc2 Avoid the need for users to manually edit PasteDeploy config in order to switch pipelines.
Define multiple pipelines in glace-api.conf to reflect the
various supported deployment flavors (minimal, with caching,
with cache management, with keystone-based auth etc.).

Add an optional paste_deploy.flavor config variable to allow the
user select the appropriate pipeline without having to edit the
paste config (i.e. uncommenting lines as before). For example
in glance-api.conf, a setting of:

  [paste_deploy]
  flavor = keystone+caching

identifies the following pipeline in glace-api-paste.ini:

  [pipeline:glance-api-keystone+caching]
  pipeline = versionnegotiation authtoken auth-context cache apiv1app

the advantage being that the user need not be concerned with
the precise sequence of filters required to realize the QoS
they desire.

Modify the functional tests that patch configuration (i.e. the
keystone and caching tests) to use the new deployment_flavor
mechanism.

Extend the TestConfigOpts to support option groups.

Change-Id: Ide843ada11bce115b7dc650440397853c6409b03
2012-01-17 12:00:33 +00:00
Eoghan Glynn
14593a3b96 Split out paste deployment config from the core glance
*.conf files into corresponding *-paste.ini files.

Fixes LP #815208

Change-Id: I5acb8488cc25bb3c7dd735c8c2b0fe62fcea8e5e
2012-01-17 11:40:26 +00:00
Jenkins
883bdd8564 Merge "Fixes incorrect URI scheme for s3 backend" 2012-01-10 02:51:01 +00:00
Jenkins
e27b0e1c27 Merge "Add comments for swift options in glance-api.conf." 2012-01-10 02:41:04 +00:00
Ewan Mellor
48449171a7 Bug #909533: Swift uploads through Glance using ridiculously small chunks
Remove the use of a temporary disk buffer when streaming to Swift.  This
was added on the assumption that it was not possible to stream chunks to
Swift direct from webob.Request.body_file.  That's not true -- a simple
file-like object wrapping body_file does the job perfectly.

This removes the need for swift_store_object_buffer_dir in the config file.
It is also significantly cheaper, since there is one copy fewer.

Fix the parsing of the swift_store_large_object_size and
swift_store_large_object_chunk_size options.  These are
specified in MB in the config file, but needs to be in bytes internally,
because they are compared against the image_size parameter given to add().

Update the unit tests to match.  This includes an additional check that
put_object is called the correct number of times.  The unit tests missed the
latter problem problem because there was no check that a small object is only
uploaded in one chunk, and despite the comment to the contrary there was no
check that the right number of chunks were written in the large-object case
either.  Added these in all places where store.add is called.

Change-Id: Ieb4cf68516b53bd16d2671e49e805b26118b3671
2012-01-06 19:44:49 -05:00
Hengqing Hu
d466cec357 Fixes incorrect URI scheme for s3 backend
Fixes bug #912094.

URI scheme for s3 backend should be 's3+https', not 'swift+https'.

Use s3.amazonaws.com instead of s3service.com to follow RFC2606.

Indent reason for BadStoreUri exception in s3.py.

Comments added for the following option in etc/glance-api.conf
    's3_store_host'

Change-Id: Id456b287d82fe61711f0d738e61ccb599f0b5684
2012-01-05 19:24:42 +08:00