openstack/glance
Code Issues Proposed changes

Adds lots of unit tests for verifying exceptions are raised

Browse Source 
properly with invalid or mismatched disk and container formats.

Adds documentation on disk and container formats. Updates
existing documentation to remove references to the now-gone
type column and replaces these references with disk_format
and container_format.

Reworked the validates_image() method in the registry.db.api
to be like what Rick was describing in reviews.
This commit is contained in:
jaypipes@gmail.com 2011-02-25 09:55:26 -05:00
parent 4ddd60b458
commit 79ba479ada
10 changed files with 327 additions and 87 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

99
doc/source/formats.rst Normal file
View File

@ -0,0 +1,99 @@
..
Copyright 2011 OpenStack, LLC
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
Disk and Container Formats
==========================
When adding an image to Glance, you are required to specify what the virtual
machine image's *disk format* and *container format* are.
This document explains exactly what these formats are.
Disk Format
-----------
The disk format of a virtual machine image is the format of the underlying
disk image. Virtual appliance vendors have different formats for laying out
the information contained in a virtual machine disk image.
You can set your image's container format to one of the following:
* **raw**
This is an unstructured disk image format
* **vhd**
This is the VHD disk format, a common disk format used by virtual machine
monitors from VMWare, Xen, Microsoft, VirtualBox, and others
* **vmdk**
Another common disk format supported by many common virtual machine monitors
* **vdi**
A disk format supported by VirtualBox virtual machine monitor and the QEMU
emulator
* **qcow2**
A disk format supported by the QEMU emulator that can expand dynamically and
supports Copy on Write
* **aki**
This indicates what is stored in Glance is an Amazon kernel image
* **ari**
This indicates what is stored in Glance is an Amazon ramdisk image
* **ami**
This indicates what is stored in Glance is an Amazon machine image
Container Format
----------------
The container format refers to whether the virtual machine image is in a
file format that also contains metadata about the actual virtual machine.
There are two main types of container formats: OVF and Amazon's AMI. In
addition, a virtual machine image may have no container format at all --
basically, it's just a blob of unstructured data...
You can set your image's container format to one of the following:
* **ovf**
This is the OVF container format
* **bare**
This indicates there is no container or metadata envelope for the image
* **aki**
This indicates what is stored in Glance is an Amazon kernel image
* **ari**
This indicates what is stored in Glance is an Amazon ramdisk image
* **ami**
This indicates what is stored in Glance is an Amazon machine image

4
doc/source/gettingstarted.rst
View File

@ -78,4 +78,6 @@ Glance registry servers are servers that conform to the Glance Registry API.
Glance ships with a reference implementation of a registry server that
complies with this API (``glance-registry``).
For more details on Glance's architecture see :doc:`here <architecture>`
For more details on Glance's architecture see :doc:`here <architecture>`. For
more information on what a Glance registry server is, see
:doc:`here <registries>`.

39
doc/source/glanceapi.rst
View File

@ -41,7 +41,8 @@ mapping in the following format::
{'images': [
{'uri': 'http://glance.example.com/images/1',
'name': 'Ubuntu 10.04 Plain',
'type': 'kernel',
'disk_format': 'vhd',
'container_format': 'ovf',
'size': '5368709120'}
...]}
@ -63,9 +64,10 @@ JSON-encoded mapping in the following format::
{'images': [
{'uri': 'http://glance.example.com/images/1',
'name': 'Ubuntu 10.04 Plain 5GB',
'type': 'kernel',
'disk_format': 'vhd',
'container_format': 'ovf',
'size': '5368709120',
'store': 'swift',
'location': 'swift://account:key/container/image.tar.gz.0',
'created_at': '2010-02-03 09:34:01',
'updated_at': '2010-02-03 09:34:01',
'deleted_at': '',
@ -111,14 +113,15 @@ following shows an example of the HTTP headers returned from the above
x-image-meta-uri http://glance.example.com/images/1
x-image-meta-name Ubuntu 10.04 Plain 5GB
x-image-meta-type kernel
x-image-meta-disk-format vhd
x-image-meta-container-format ovf
x-image-meta-size 5368709120
x-image-meta-store swift
x-image-meta-location swift://account:key/container/image.tar.gz.0
x-image-meta-created_at 2010-02-03 09:34:01
x-image-meta-updated_at 2010-02-03 09:34:01
x-image-meta-deleted_at
x-image-meta-status available
x-image-meta-is_public True
x-image-meta-is-public True
x-image-meta-property-distro Ubuntu 10.04 LTS
.. note::
@ -160,14 +163,15 @@ returned from the above ``GET`` request::
x-image-meta-uri http://glance.example.com/images/1
x-image-meta-name Ubuntu 10.04 Plain 5GB
x-image-meta-type kernel
x-image-meta-disk-format vhd
x-image-meta-container-format ovf
x-image-meta-size 5368709120
x-image-meta-store swift
x-image-meta-location swift://account:key/container/image.tar.gz.0
x-image-meta-created_at 2010-02-03 09:34:01
x-image-meta-updated_at 2010-02-03 09:34:01
x-image-meta-deleted_at
x-image-meta-status available
x-image-meta-is_public True
x-image-meta-is-public True
x-image-meta-property-distro Ubuntu 10.04 LTS
.. note::
@ -254,10 +258,19 @@ The list of metadata headers that Glance accepts are listed below.
store that is marked default. See the configuration option ``default_store``
for more information.
* ``x-image-meta-type``
* ``x-image-meta-disk-format``
This header is required. Valid values are one of ``kernel``, ``machine``,
``raw``, or ``ramdisk``.
This header is required. Valid values are one of ``aki``, ``ari``, ``ami``,
``raw``, ``vhd``, ``vdi``, ``qcow2``, or ``vmdk``.
For more information, see :doc:`About Disk and Container Formats <formats>`
* ``x-image-meta-container-format``
This header is required. Valid values are one of ``aki``, ``ari``, ``ami``,
``bare``, or ``ovf``.
For more information, see :doc:`About Disk and Container Formats <formats>`
* ``x-image-meta-size``
@ -271,7 +284,7 @@ The list of metadata headers that Glance accepts are listed below.
When not present, Glance will calculate the image's size based on the size
of the request body.
* ``x-image-meta-is_public``
* ``x-image-meta-is-public``
This header is optional.

1
doc/source/index.rst
View File

@ -50,6 +50,7 @@ Concepts
identifiers
registries
statuses
formats
Using Glance
============

73
doc/source/registries.rst
View File

@ -24,44 +24,9 @@ Glance REST-like API for image metadata.
Glance comes with a server program ``glance-registry`` that acts
as a reference implementation of a Glance Registry.
Using ``glance-registry``, the Glance Registry reference implementation
-----------------------------------------------------------------------
As mentioned above, ``glance-registry`` is the reference registry
server implementation that ships with Glance. It uses a SQL database
to store information about an image, and publishes this information
via an HTTP/REST-like interface.
Starting the server
*******************
Starting the Glance registry server is trivial. Simply call the program
from the command line, as the following example shows::
$> glance-registry
(5588) wsgi starting up on http://0.0.0.0:9191/
Configuring the server
**********************
There are a few options that can be supplied to the registry server when
starting it up:
* ``verbose``
Show more verbose/debugging output
* ``sql_connection``
A proper SQLAlchemy connection string as described `here <http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/connections.html?highlight=engine#sqlalchemy.create_engine>`_
* ``registry_host``
Address of the host the registry runs on. Defaults to 0.0.0.0.
* ``registry_port``
Port the registry server listens on. Defaults to 9191.
Please see the document :doc:`on Controlling Servers <controllingservers>`
for more information on starting up the Glance registry server that ships
with Glance.
Glance Registry API
-------------------
@ -81,6 +46,38 @@ The following is a brief description of the Glance API::
PUT /images/<ID> Update metadata about an existing image
DELETE /images/<ID> Remove an image's metadata from the registry
``POST /images``
----------------
The body of the request will be a JSON-encoded set of data about
the image to add to the registry. It will be in the following format::
{'image':
{'id': <ID>|None,
'name': <NAME>,
'status': <STATUS>,
'disk_format': <DISK_FORMAT>,
'container_format': <CONTAINER_FORMAT>,
'properties': [ ... ]
}
}
The request shall validate the following conditions and return a
``400 Bad request`` when any of the conditions are not met:
* ``status`` must be non-empty, and must be one of **active**, **saving**,
**queued**, or **killed**
* ``disk_format`` must be non-empty, and must be one of **ari**, **aki**,
**ami**, **raw**, **vhd**, **vdi**, **qcow2**, or **vmdk**
* ``container_format`` must be non-empty, and must be on of **ari**,
**aki**, **ami**, **bare**, or **ovf**
* If ``disk_format`` *or* ``container_format`` is **ari**, **aki**,
**ami**, then *both* ``disk_format`` and ``container_format`` must be
the same.
Examples
********

67
glance/registry/db/api.py
View File

@ -44,6 +44,10 @@ IMAGE_ATTRS = BASE_MODEL_ATTRS | set(['name', 'status', 'size',
'disk_format', 'container_format',
'is_public', 'location'])
CONTAINER_FORMATS = ['ami', 'ari', 'aki', 'bare', 'ovf']
DISK_FORMATS = ['ami', 'ari', 'aki', 'vhd', 'vmdk', 'raw', 'qcow2', 'vdi']
STATUSES = ['active', 'saving', 'queued', 'killed']
def configure_db(options):
"""
@ -145,39 +149,49 @@ def _drop_protected_attrs(model_class, values):
del values[attr]
def validate_image(values, new=True):
def validate_image(values):
"""
Validates the incoming data and raises a Invalid exception
if anything is out of order.
:param values: Mapping of image metadata to check
:param new: Is this a new record?
"""
status = values.get('status')
if not status and new:
disk_format = values.get('disk_format')
container_format = values.get('container_format')
if not status:
msg = "Image status is required."
raise exception.Invalid(msg)
if status and status not in ('active', 'queued', 'killed', 'saving'):
if not disk_format:
msg = "Image disk format is required."
raise exception.Invalid(msg)
if not container_format:
msg = "Image container format is required."
raise exception.Invalid(msg)
if status not in STATUSES:
msg = "Invalid image status '%s' for image." % status
raise exception.Invalid(msg)
disk_format = values.get('disk_format')
if not disk_format and new:
msg = "Image disk format is required."
raise exception.Invalid(msg)
if disk_format and disk_format not in ('vmdk', 'ami', 'raw', 'vhd'):
if disk_format not in DISK_FORMATS:
msg = "Invalid disk format '%s' for image." % disk_format
raise exception.Invalid(msg)
container_format = values.get('container_format')
if not container_format and new:
msg = "Image container format is required."
raise exception.Invalid(msg)
if container_format and container_format not in ('ami', 'ovf'):
if container_format not in CONTAINER_FORMATS:
msg = "Invalid container format '%s' for image." % container_format
raise exception.Invalid(msg)
if disk_format in ('aki', 'ari', 'ami') or\
container_format in ('aki', 'ari', 'ami'):
if container_format != disk_format:
msg = ("Invalid mix of disk and container formats. "
"When setting a disk or container format to "
"one of 'ami', 'ari', or 'ami', the container "
"and disk formats must match.")
raise exception.Invalid(msg)
def _image_update(context, values, image_id):
"""Used internally by image_create and image_update
@ -187,27 +201,28 @@ def _image_update(context, values, image_id):
:param image_id: If None, create the image, otherwise, find and update it
"""
# Validate the attributes before we go any further. From my investigation,
# the @validates decorator does not validate on new records, only on
# existing records, which is, well, idiotic.
validate_image(values, image_id is None)
session = get_session()
with session.begin():
_drop_protected_attrs(models.Image, values)
if 'size' in values:
values['size'] = int(values['size'])
values['is_public'] = bool(values.get('is_public', False))
properties = values.pop('properties', {})
if image_id:
image_ref = image_get(context, image_id, session=session)
else:
if 'size' in values:
values['size'] = int(values['size'])
values['is_public'] = bool(values.get('is_public', False))
properties = values.pop('properties', {})
image_ref = models.Image()
_drop_protected_attrs(models.Image, values)
image_ref.update(values)
# Validate the attributes before we go any further. From my
# investigation, the @validates decorator does not validate
# on new records, only on existing records, which is, well,
# idiotic.
validate_image(image_ref.to_dict())
image_ref.save(session=session)
_set_properties_for_image(context, image_ref, properties, session)

3
glance/registry/db/models.py
View File

@ -88,6 +88,9 @@ class ModelBase(object):
def items(self):
return self.__dict__.items()
def to_dict(self):
return self.__dict__.copy()
class Image(BASE, ModelBase):
"""Represents an image in the datastore"""

5
glance/registry/server.py
View File

@ -146,6 +146,11 @@ class Controller(wsgi.Controller):
try:
updated_image = db_api.image_update(context, id, image_data)
return dict(image=make_image_dict(updated_image))
except exception.Invalid, e:
msg = ("Failed to update image metadata. "
"Got error: %(e)s" % locals())
logger.error(msg)
return exc.HTTPBadRequest(msg)
except exception.NotFound:
raise exc.HTTPNotFound(body='Image not found',
request=req,

5
tests/stubs.py
View File

@ -408,6 +408,10 @@ def stub_out_registry_db_image_api(stubs):
def image_update(self, _context, image_id, values):
image = self.image_get(_context, image_id)
copy_image = image.copy()
copy_image.update(values)
glance.registry.db.api.validate_image(copy_image)
props = []
if 'properties' in values.keys():
@ -423,7 +427,6 @@ def stub_out_registry_db_image_api(stubs):
values['properties'] = props
image = self.image_get(_context, image_id)
image.update(values)
return image

118
tests/unit/test_api.py
View File

@ -127,6 +127,57 @@ class TestRegistryAPI(unittest.TestCase):
# Test status was updated properly
self.assertEquals('active', res_dict['image']['status'])
def test_create_image_with_bad_container_format(self):
"""Tests proper exception is raised if a bad disk_format is set"""
fixture = {'id': 3,
'name': 'fake public image',
'is_public': True,
'disk_format': 'vhd',
'container_format': 'invalid'}
req = webob.Request.blank('/images')
req.method = 'POST'
req.body = json.dumps(dict(image=fixture))
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid container format' in res.body)
def test_create_image_with_bad_disk_format(self):
"""Tests proper exception is raised if a bad disk_format is set"""
fixture = {'id': 3,
'name': 'fake public image',
'is_public': True,
'disk_format': 'invalid',
'container_format': 'ovf'}
req = webob.Request.blank('/images')
req.method = 'POST'
req.body = json.dumps(dict(image=fixture))
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid disk format' in res.body)
def test_create_image_with_mismatched_formats(self):
"""Tests that exception raised for bad matching disk and container
formats"""
fixture = {'name': 'fake public image #3',
'container_format': 'aki',
'disk_format': 'ari'}
req = webob.Request.blank('/images')
req.method = 'POST'
req.body = json.dumps(dict(image=fixture))
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid mix of disk and container formats'
in res.body)
def test_create_image_with_bad_status(self):
"""Tests proper exception is raised if a bad status is set"""
fixture = {'id': 3,
@ -143,6 +194,7 @@ class TestRegistryAPI(unittest.TestCase):
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid image status' in res.body)
def test_update_image(self):
"""Tests that the /images PUT registry API updates the image"""
@ -166,12 +218,7 @@ class TestRegistryAPI(unittest.TestCase):
def test_update_image_not_existing(self):
"""Tests proper exception is raised if attempt to update non-existing
image"""
fixture = {'id': 3,
'name': 'fake public image',
'is_public': True,
'disk_format': 'vhd',
'container_format': 'ovf',
'status': 'bad status'}
fixture = {'status': 'killed'}
req = webob.Request.blank('/images/3')
@ -182,6 +229,60 @@ class TestRegistryAPI(unittest.TestCase):
self.assertEquals(res.status_int,
webob.exc.HTTPNotFound.code)
def test_update_image_with_bad_status(self):
"""Tests that exception raised trying to set a bad status"""
fixture = {'status': 'invalid'}
req = webob.Request.blank('/images/2')
req.method = 'PUT'
req.body = json.dumps(dict(image=fixture))
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid image status' in res.body)
def test_update_image_with_bad_disk_format(self):
"""Tests that exception raised trying to set a bad disk_format"""
fixture = {'disk_format': 'invalid'}
req = webob.Request.blank('/images/2')
req.method = 'PUT'
req.body = json.dumps(dict(image=fixture))
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid disk format' in res.body)
def test_update_image_with_bad_container_format(self):
"""Tests that exception raised trying to set a bad container_format"""
fixture = {'container_format': 'invalid'}
req = webob.Request.blank('/images/2')
req.method = 'PUT'
req.body = json.dumps(dict(image=fixture))
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid container format' in res.body)
def test_update_image_with_mismatched_formats(self):
"""Tests that exception raised for bad matching disk and container
formats"""
fixture = {'container_format': 'ari'}
req = webob.Request.blank('/images/2') # Image 2 has disk format 'vhd'
req.method = 'PUT'
req.body = json.dumps(dict(image=fixture))
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid mix of disk and container formats'
in res.body)
def test_delete_image(self):
"""Tests that the /images DELETE registry API deletes the image"""
@ -261,7 +362,8 @@ class TestGlanceAPI(unittest.TestCase):
fixture_headers = {'x-image-meta-store': 'bad',
'x-image-meta-name': 'bogus',
'x-image-meta-location': 'http://example.com/image.tar.gz',
'x-image-meta-disk-format': 'invalid'}
'x-image-meta-disk-format': 'invalid',
'x-image-meta-container-format': 'ami'}
req = webob.Request.blank("/images")
req.method = 'POST'
@ -270,7 +372,7 @@ class TestGlanceAPI(unittest.TestCase):
res = req.get_response(self.api)
self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code)
self.assertTrue('Invalid disk format' in res.body)
self.assertTrue('Invalid disk format' in res.body, res.body)
def test_missing_container_format(self):
fixture_headers = {'x-image-meta-store': 'bad',