Copying from glance
This commit is contained in:
commit
6afd6df9d4
26
.gitignore
vendored
Normal file
26
.gitignore
vendored
Normal file
@ -0,0 +1,26 @@
|
||||
*.pyc
|
||||
*.swp
|
||||
*.log
|
||||
.glance-venv
|
||||
.venv
|
||||
.tox
|
||||
.coverage
|
||||
cover/*
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
glance.sqlite
|
||||
AUTHORS
|
||||
ChangeLog
|
||||
build
|
||||
doc/source/api
|
||||
dist
|
||||
*.egg
|
||||
glance.egg-info
|
||||
tests.sqlite
|
||||
glance/versioninfo
|
||||
# Files created by doc build
|
||||
doc/source/api
|
||||
|
||||
# IDE files
|
||||
.project
|
||||
.pydevproject
|
176
LICENSE
Normal file
176
LICENSE
Normal file
@ -0,0 +1,176 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
4
README.rst
Normal file
4
README.rst
Normal file
@ -0,0 +1,4 @@
|
||||
Glance Store Library
|
||||
=====================
|
||||
|
||||
Glance's stores library
|
0
glance/__init__.py
Normal file
0
glance/__init__.py
Normal file
715
glance/store/__init__.py
Normal file
715
glance/store/__init__.py
Normal file
@ -0,0 +1,715 @@
|
||||
# Copyright 2010-2011 OpenStack Foundation
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import collections
|
||||
import sys
|
||||
|
||||
from oslo.config import cfg
|
||||
|
||||
from glance.common import exception
|
||||
from glance.common import utils
|
||||
import glance.context
|
||||
import glance.domain.proxy
|
||||
from glance.openstack.common import importutils
|
||||
import glance.openstack.common.log as logging
|
||||
from glance.store import location
|
||||
from glance.store import scrubber
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
store_opts = [
|
||||
cfg.ListOpt('known_stores',
|
||||
default=[
|
||||
'glance.store.filesystem.Store',
|
||||
'glance.store.http.Store',
|
||||
'glance.store.rbd.Store',
|
||||
'glance.store.s3.Store',
|
||||
'glance.store.swift.Store',
|
||||
'glance.store.sheepdog.Store',
|
||||
'glance.store.cinder.Store',
|
||||
],
|
||||
help=_('List of which store classes and store class locations '
|
||||
'are currently known to glance at startup.')),
|
||||
cfg.StrOpt('default_store', default='file',
|
||||
help=_("Default scheme to use to store image data. The "
|
||||
"scheme must be registered by one of the stores "
|
||||
"defined by the 'known_stores' config option.")),
|
||||
cfg.StrOpt('scrubber_datadir',
|
||||
default='/var/lib/glance/scrubber',
|
||||
help=_('Directory that the scrubber will use to track '
|
||||
'information about what to delete. '
|
||||
'Make sure this is set in glance-api.conf and '
|
||||
'glance-scrubber.conf')),
|
||||
cfg.BoolOpt('delayed_delete', default=False,
|
||||
help=_('Turn on/off delayed delete.')),
|
||||
cfg.IntOpt('scrub_time', default=0,
|
||||
help=_('The amount of time in seconds to delay before '
|
||||
'performing a delete.')),
|
||||
]
|
||||
|
||||
CONF = cfg.CONF
|
||||
CONF.register_opts(store_opts)
|
||||
|
||||
|
||||
class BackendException(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class UnsupportedBackend(BackendException):
|
||||
pass
|
||||
|
||||
|
||||
class Indexable(object):
|
||||
|
||||
"""
|
||||
Wrapper that allows an iterator or filelike be treated as an indexable
|
||||
data structure. This is required in the case where the return value from
|
||||
Store.get() is passed to Store.add() when adding a Copy-From image to a
|
||||
Store where the client library relies on eventlet GreenSockets, in which
|
||||
case the data to be written is indexed over.
|
||||
"""
|
||||
|
||||
def __init__(self, wrapped, size):
|
||||
"""
|
||||
Initialize the object
|
||||
|
||||
:param wrappped: the wrapped iterator or filelike.
|
||||
:param size: the size of data available
|
||||
"""
|
||||
self.wrapped = wrapped
|
||||
self.size = int(size) if size else (wrapped.len
|
||||
if hasattr(wrapped, 'len') else 0)
|
||||
self.cursor = 0
|
||||
self.chunk = None
|
||||
|
||||
def __iter__(self):
|
||||
"""
|
||||
Delegate iteration to the wrapped instance.
|
||||
"""
|
||||
for self.chunk in self.wrapped:
|
||||
yield self.chunk
|
||||
|
||||
def __getitem__(self, i):
|
||||
"""
|
||||
Index into the next chunk (or previous chunk in the case where
|
||||
the last data returned was not fully consumed).
|
||||
|
||||
:param i: a slice-to-the-end
|
||||
"""
|
||||
start = i.start if isinstance(i, slice) else i
|
||||
if start < self.cursor:
|
||||
return self.chunk[(start - self.cursor):]
|
||||
|
||||
self.chunk = self.another()
|
||||
if self.chunk:
|
||||
self.cursor += len(self.chunk)
|
||||
|
||||
return self.chunk
|
||||
|
||||
def another(self):
|
||||
"""Implemented by subclasses to return the next element"""
|
||||
raise NotImplementedError
|
||||
|
||||
def getvalue(self):
|
||||
"""
|
||||
Return entire string value... used in testing
|
||||
"""
|
||||
return self.wrapped.getvalue()
|
||||
|
||||
def __len__(self):
|
||||
"""
|
||||
Length accessor.
|
||||
"""
|
||||
return self.size
|
||||
|
||||
|
||||
def _get_store_class(store_entry):
|
||||
store_cls = None
|
||||
try:
|
||||
LOG.debug("Attempting to import store %s", store_entry)
|
||||
store_cls = importutils.import_class(store_entry)
|
||||
except exception.NotFound:
|
||||
raise BackendException('Unable to load store. '
|
||||
'Could not find a class named %s.'
|
||||
% store_entry)
|
||||
return store_cls
|
||||
|
||||
|
||||
def create_stores():
|
||||
"""
|
||||
Registers all store modules and all schemes
|
||||
from the given config. Duplicates are not re-registered.
|
||||
"""
|
||||
store_count = 0
|
||||
store_classes = set()
|
||||
for store_entry in CONF.known_stores:
|
||||
store_entry = store_entry.strip()
|
||||
if not store_entry:
|
||||
continue
|
||||
store_cls = _get_store_class(store_entry)
|
||||
try:
|
||||
store_instance = store_cls()
|
||||
except exception.BadStoreConfiguration as e:
|
||||
LOG.warn(_("%s Skipping store driver.") % unicode(e))
|
||||
continue
|
||||
schemes = store_instance.get_schemes()
|
||||
if not schemes:
|
||||
raise BackendException('Unable to register store %s. '
|
||||
'No schemes associated with it.'
|
||||
% store_cls)
|
||||
else:
|
||||
if store_cls not in store_classes:
|
||||
LOG.debug("Registering store %s with schemes %s",
|
||||
store_cls, schemes)
|
||||
store_classes.add(store_cls)
|
||||
scheme_map = {}
|
||||
for scheme in schemes:
|
||||
loc_cls = store_instance.get_store_location_class()
|
||||
scheme_map[scheme] = {
|
||||
'store_class': store_cls,
|
||||
'location_class': loc_cls,
|
||||
}
|
||||
location.register_scheme_map(scheme_map)
|
||||
store_count += 1
|
||||
else:
|
||||
LOG.debug("Store %s already registered", store_cls)
|
||||
return store_count
|
||||
|
||||
|
||||
def verify_default_store():
|
||||
scheme = cfg.CONF.default_store
|
||||
context = glance.context.RequestContext()
|
||||
try:
|
||||
get_store_from_scheme(context, scheme)
|
||||
except exception.UnknownScheme:
|
||||
msg = _("Store for scheme %s not found") % scheme
|
||||
raise RuntimeError(msg)
|
||||
|
||||
|
||||
def get_known_schemes():
|
||||
"""Returns list of known schemes"""
|
||||
return location.SCHEME_TO_CLS_MAP.keys()
|
||||
|
||||
|
||||
def get_store_from_scheme(context, scheme, loc=None):
|
||||
"""
|
||||
Given a scheme, return the appropriate store object
|
||||
for handling that scheme.
|
||||
"""
|
||||
if scheme not in location.SCHEME_TO_CLS_MAP:
|
||||
raise exception.UnknownScheme(scheme=scheme)
|
||||
scheme_info = location.SCHEME_TO_CLS_MAP[scheme]
|
||||
store = scheme_info['store_class'](context, loc)
|
||||
return store
|
||||
|
||||
|
||||
def get_store_from_uri(context, uri, loc=None):
|
||||
"""
|
||||
Given a URI, return the store object that would handle
|
||||
operations on the URI.
|
||||
|
||||
:param uri: URI to analyze
|
||||
"""
|
||||
scheme = uri[0:uri.find('/') - 1]
|
||||
store = get_store_from_scheme(context, scheme, loc)
|
||||
return store
|
||||
|
||||
|
||||
def get_from_backend(context, uri, **kwargs):
|
||||
"""Yields chunks of data from backend specified by uri"""
|
||||
|
||||
loc = location.get_location_from_uri(uri)
|
||||
store = get_store_from_uri(context, uri, loc)
|
||||
|
||||
try:
|
||||
return store.get(loc)
|
||||
except NotImplementedError:
|
||||
raise exception.StoreGetNotSupported
|
||||
|
||||
|
||||
def get_size_from_backend(context, uri):
|
||||
"""Retrieves image size from backend specified by uri"""
|
||||
|
||||
loc = location.get_location_from_uri(uri)
|
||||
store = get_store_from_uri(context, uri, loc)
|
||||
|
||||
return store.get_size(loc)
|
||||
|
||||
|
||||
def delete_from_backend(context, uri, **kwargs):
|
||||
"""Removes chunks of data from backend specified by uri"""
|
||||
loc = location.get_location_from_uri(uri)
|
||||
store = get_store_from_uri(context, uri, loc)
|
||||
|
||||
try:
|
||||
return store.delete(loc)
|
||||
except NotImplementedError:
|
||||
raise exception.StoreDeleteNotSupported
|
||||
|
||||
|
||||
def get_store_from_location(uri):
|
||||
"""
|
||||
Given a location (assumed to be a URL), attempt to determine
|
||||
the store from the location. We use here a simple guess that
|
||||
the scheme of the parsed URL is the store...
|
||||
|
||||
:param uri: Location to check for the store
|
||||
"""
|
||||
loc = location.get_location_from_uri(uri)
|
||||
return loc.store_name
|
||||
|
||||
|
||||
def safe_delete_from_backend(context, uri, image_id, **kwargs):
|
||||
"""Given a uri, delete an image from the store."""
|
||||
try:
|
||||
return delete_from_backend(context, uri, **kwargs)
|
||||
except exception.NotFound:
|
||||
msg = _('Failed to delete image %s in store from URI')
|
||||
LOG.warn(msg % image_id)
|
||||
except exception.StoreDeleteNotSupported as e:
|
||||
LOG.warn(str(e))
|
||||
except UnsupportedBackend:
|
||||
exc_type = sys.exc_info()[0].__name__
|
||||
msg = (_('Failed to delete image %s from store (%s)') %
|
||||
(image_id, exc_type))
|
||||
LOG.error(msg)
|
||||
|
||||
|
||||
def schedule_delayed_delete_from_backend(context, uri, image_id, **kwargs):
|
||||
"""Given a uri, schedule the deletion of an image location."""
|
||||
(file_queue, _db_queue) = scrubber.get_scrub_queues()
|
||||
# NOTE(zhiyan): Defautly ask glance-api store using file based queue.
|
||||
# In future we can change it using DB based queued instead,
|
||||
# such as using image location's status to saving pending delete flag
|
||||
# when that property be added.
|
||||
file_queue.add_location(image_id, uri)
|
||||
|
||||
|
||||
def delete_image_from_backend(context, store_api, image_id, uri):
|
||||
if CONF.delayed_delete:
|
||||
store_api.schedule_delayed_delete_from_backend(context, uri, image_id)
|
||||
else:
|
||||
store_api.safe_delete_from_backend(context, uri, image_id)
|
||||
|
||||
|
||||
def check_location_metadata(val, key=''):
|
||||
if isinstance(val, dict):
|
||||
for key in val:
|
||||
check_location_metadata(val[key], key=key)
|
||||
elif isinstance(val, list):
|
||||
ndx = 0
|
||||
for v in val:
|
||||
check_location_metadata(v, key='%s[%d]' % (key, ndx))
|
||||
ndx = ndx + 1
|
||||
elif not isinstance(val, unicode):
|
||||
raise BackendException(_("The image metadata key %s has an invalid "
|
||||
"type of %s. Only dict, list, and unicode "
|
||||
"are supported.") % (key, type(val)))
|
||||
|
||||
|
||||
def store_add_to_backend(image_id, data, size, store):
|
||||
"""
|
||||
A wrapper around a call to each stores add() method. This gives glance
|
||||
a common place to check the output
|
||||
|
||||
:param image_id: The image add to which data is added
|
||||
:param data: The data to be stored
|
||||
:param size: The length of the data in bytes
|
||||
:param store: The store to which the data is being added
|
||||
:return: The url location of the file,
|
||||
the size amount of data,
|
||||
the checksum of the data
|
||||
the storage systems metadata dictionary for the location
|
||||
"""
|
||||
(location, size, checksum, metadata) = store.add(image_id, data, size)
|
||||
if metadata is not None:
|
||||
if not isinstance(metadata, dict):
|
||||
msg = (_("The storage driver %s returned invalid metadata %s"
|
||||
"This must be a dictionary type") %
|
||||
(str(store), str(metadata)))
|
||||
LOG.error(msg)
|
||||
raise BackendException(msg)
|
||||
try:
|
||||
check_location_metadata(metadata)
|
||||
except BackendException as e:
|
||||
e_msg = (_("A bad metadata structure was returned from the "
|
||||
"%s storage driver: %s. %s.") %
|
||||
(str(store), str(metadata), str(e)))
|
||||
LOG.error(e_msg)
|
||||
raise BackendException(e_msg)
|
||||
return (location, size, checksum, metadata)
|
||||
|
||||
|
||||
def add_to_backend(context, scheme, image_id, data, size):
|
||||
store = get_store_from_scheme(context, scheme)
|
||||
try:
|
||||
return store_add_to_backend(image_id, data, size, store)
|
||||
except NotImplementedError:
|
||||
raise exception.StoreAddNotSupported
|
||||
|
||||
|
||||
def set_acls(context, location_uri, public=False, read_tenants=[],
|
||||
write_tenants=[]):
|
||||
loc = location.get_location_from_uri(location_uri)
|
||||
scheme = get_store_from_location(location_uri)
|
||||
store = get_store_from_scheme(context, scheme, loc)
|
||||
try:
|
||||
store.set_acls(loc, public=public, read_tenants=read_tenants,
|
||||
write_tenants=write_tenants)
|
||||
except NotImplementedError:
|
||||
LOG.debug(_("Skipping store.set_acls... not implemented."))
|
||||
|
||||
|
||||
class ImageRepoProxy(glance.domain.proxy.Repo):
|
||||
|
||||
def __init__(self, image_repo, context, store_api):
|
||||
self.context = context
|
||||
self.store_api = store_api
|
||||
proxy_kwargs = {'context': context, 'store_api': store_api}
|
||||
super(ImageRepoProxy, self).__init__(image_repo,
|
||||
item_proxy_class=ImageProxy,
|
||||
item_proxy_kwargs=proxy_kwargs)
|
||||
|
||||
def _set_acls(self, image):
|
||||
public = image.visibility == 'public'
|
||||
member_ids = []
|
||||
if image.locations and not public:
|
||||
member_repo = image.get_member_repo()
|
||||
member_ids = [m.member_id for m in member_repo.list()]
|
||||
for location in image.locations:
|
||||
self.store_api.set_acls(self.context, location['url'], public,
|
||||
read_tenants=member_ids)
|
||||
|
||||
def add(self, image):
|
||||
result = super(ImageRepoProxy, self).add(image)
|
||||
self._set_acls(image)
|
||||
return result
|
||||
|
||||
def save(self, image):
|
||||
result = super(ImageRepoProxy, self).save(image)
|
||||
self._set_acls(image)
|
||||
return result
|
||||
|
||||
|
||||
def _check_location_uri(context, store_api, uri):
|
||||
"""
|
||||
Check if an image location uri is valid.
|
||||
|
||||
:param context: Glance request context
|
||||
:param store_api: store API module
|
||||
:param uri: location's uri string
|
||||
"""
|
||||
is_ok = True
|
||||
try:
|
||||
size = store_api.get_size_from_backend(context, uri)
|
||||
# NOTE(zhiyan): Some stores return zero when it catch exception
|
||||
is_ok = size > 0
|
||||
except (exception.UnknownScheme, exception.NotFound):
|
||||
is_ok = False
|
||||
if not is_ok:
|
||||
raise exception.BadStoreUri(_('Invalid location: %s') % uri)
|
||||
|
||||
|
||||
def _check_image_location(context, store_api, location):
|
||||
_check_location_uri(context, store_api, location['url'])
|
||||
store_api.check_location_metadata(location['metadata'])
|
||||
|
||||
|
||||
def _set_image_size(context, image, locations):
|
||||
if not image.size:
|
||||
for location in locations:
|
||||
size_from_backend = glance.store.get_size_from_backend(
|
||||
context, location['url'])
|
||||
if size_from_backend:
|
||||
# NOTE(flwang): This assumes all locations have the same size
|
||||
image.size = size_from_backend
|
||||
break
|
||||
|
||||
|
||||
class ImageFactoryProxy(glance.domain.proxy.ImageFactory):
|
||||
def __init__(self, factory, context, store_api):
|
||||
self.context = context
|
||||
self.store_api = store_api
|
||||
proxy_kwargs = {'context': context, 'store_api': store_api}
|
||||
super(ImageFactoryProxy, self).__init__(factory,
|
||||
proxy_class=ImageProxy,
|
||||
proxy_kwargs=proxy_kwargs)
|
||||
|
||||
def new_image(self, **kwargs):
|
||||
locations = kwargs.get('locations', [])
|
||||
for l in locations:
|
||||
_check_image_location(self.context, self.store_api, l)
|
||||
|
||||
if locations.count(l) > 1:
|
||||
raise exception.DuplicateLocation(location=l['url'])
|
||||
|
||||
return super(ImageFactoryProxy, self).new_image(**kwargs)
|
||||
|
||||
|
||||
class StoreLocations(collections.MutableSequence):
|
||||
"""
|
||||
The proxy for store location property. It takes responsibility for:
|
||||
1. Location uri correctness checking when adding a new location.
|
||||
2. Remove the image data from the store when a location is removed
|
||||
from an image.
|
||||
"""
|
||||
def __init__(self, image_proxy, value):
|
||||
self.image_proxy = image_proxy
|
||||
if isinstance(value, list):
|
||||
self.value = value
|
||||
else:
|
||||
self.value = list(value)
|
||||
|
||||
def append(self, location):
|
||||
# NOTE(flaper87): Insert this
|
||||
# location at the very end of
|
||||
# the value list.
|
||||
self.insert(len(self.value), location)
|
||||
|
||||
def extend(self, other):
|
||||
if isinstance(other, StoreLocations):
|
||||
locations = other.value
|
||||
else:
|
||||
locations = list(other)
|
||||
|
||||
for location in locations:
|
||||
self.append(location)
|
||||
|
||||
def insert(self, i, location):
|
||||
_check_image_location(self.image_proxy.context,
|
||||
self.image_proxy.store_api, location)
|
||||
|
||||
if location in self.value:
|
||||
raise exception.DuplicateLocation(location=location['url'])
|
||||
|
||||
self.value.insert(i, location)
|
||||
_set_image_size(self.image_proxy.context,
|
||||
self.image_proxy,
|
||||
[location])
|
||||
|
||||
def pop(self, i=-1):
|
||||
location = self.value.pop(i)
|
||||
try:
|
||||
delete_image_from_backend(self.image_proxy.context,
|
||||
self.image_proxy.store_api,
|
||||
self.image_proxy.image.image_id,
|
||||
location['url'])
|
||||
except Exception:
|
||||
self.value.insert(i, location)
|
||||
raise
|
||||
return location
|
||||
|
||||
def count(self, location):
|
||||
return self.value.count(location)
|
||||
|
||||
def index(self, location, *args):
|
||||
return self.value.index(location, *args)
|
||||
|
||||
def remove(self, location):
|
||||
if self.count(location):
|
||||
self.pop(self.index(location))
|
||||
else:
|
||||
self.value.remove(location)
|
||||
|
||||
def reverse(self):
|
||||
self.value.reverse()
|
||||
|
||||
# Mutable sequence, so not hashable
|
||||
__hash__ = None
|
||||
|
||||
def __getitem__(self, i):
|
||||
return self.value.__getitem__(i)
|
||||
|
||||
def __setitem__(self, i, location):
|
||||
_check_image_location(self.image_proxy.context,
|
||||
self.image_proxy.store_api, location)
|
||||
self.value.__setitem__(i, location)
|
||||
_set_image_size(self.image_proxy.context,
|
||||
self.image_proxy,
|
||||
[location])
|
||||
|
||||
def __delitem__(self, i):
|
||||
location = None
|
||||
try:
|
||||
location = self.value.__getitem__(i)
|
||||
except Exception:
|
||||
return self.value.__delitem__(i)
|
||||
delete_image_from_backend(self.image_proxy.context,
|
||||
self.image_proxy.store_api,
|
||||
self.image_proxy.image.image_id,
|
||||
location['url'])
|
||||
self.value.__delitem__(i)
|
||||
|
||||
def __delslice__(self, i, j):
|
||||
i = max(i, 0)
|
||||
j = max(j, 0)
|
||||
locations = []
|
||||
try:
|
||||
locations = self.value.__getslice__(i, j)
|
||||
except Exception:
|
||||
return self.value.__delslice__(i, j)
|
||||
for location in locations:
|
||||
delete_image_from_backend(self.image_proxy.context,
|
||||
self.image_proxy.store_api,
|
||||
self.image_proxy.image.image_id,
|
||||
location['url'])
|
||||
self.value.__delitem__(i)
|
||||
|
||||
def __iadd__(self, other):
|
||||
self.extend(other)
|
||||
return self
|
||||
|
||||
def __contains__(self, location):
|
||||
return location in self.value
|
||||
|
||||
def __len__(self):
|
||||
return len(self.value)
|
||||
|
||||
def __cast(self, other):
|
||||
if isinstance(other, StoreLocations):
|
||||
return other.value
|
||||
else:
|
||||
return other
|
||||
|
||||
def __cmp__(self, other):
|
||||
return cmp(self.value, self.__cast(other))
|
||||
|
||||
def __iter__(self):
|
||||
return iter(self.value)
|
||||
|
||||
|
||||
def _locations_proxy(target, attr):
|
||||
"""
|
||||
Make a location property proxy on the image object.
|
||||
|
||||
:param target: the image object on which to add the proxy
|
||||
:param attr: the property proxy we want to hook
|
||||
"""
|
||||
def get_attr(self):
|
||||
value = getattr(getattr(self, target), attr)
|
||||
return StoreLocations(self, value)
|
||||
|
||||
def set_attr(self, value):
|
||||
if not isinstance(value, (list, StoreLocations)):
|
||||
raise exception.BadStoreUri(_('Invalid locations: %s') % value)
|
||||
ori_value = getattr(getattr(self, target), attr)
|
||||
if ori_value != value:
|
||||
# NOTE(zhiyan): Enforced locations list was previously empty list.
|
||||
if len(ori_value) > 0:
|
||||
raise exception.Invalid(_('Original locations is not empty: '
|
||||
'%s') % ori_value)
|
||||
# NOTE(zhiyan): Check locations are all valid.
|
||||
for location in value:
|
||||
_check_image_location(self.context, self.store_api,
|
||||
location)
|
||||
|
||||
if value.count(location) > 1:
|
||||
raise exception.DuplicateLocation(location=location['url'])
|
||||
_set_image_size(self.context, getattr(self, target), value)
|
||||
return setattr(getattr(self, target), attr, list(value))
|
||||
|
||||
def del_attr(self):
|
||||
value = getattr(getattr(self, target), attr)
|
||||
while len(value):
|
||||
delete_image_from_backend(self.context, self.store_api,
|
||||
self.image.image_id, value[0]['url'])
|
||||
del value[0]
|
||||
setattr(getattr(self, target), attr, value)
|
||||
return delattr(getattr(self, target), attr)
|
||||
|
||||
return property(get_attr, set_attr, del_attr)
|
||||
|
||||
|
||||
class ImageProxy(glance.domain.proxy.Image):
|
||||
|
||||
locations = _locations_proxy('image', 'locations')
|
||||
|
||||
def __init__(self, image, context, store_api):
|
||||
self.image = image
|
||||
self.context = context
|
||||
self.store_api = store_api
|
||||
proxy_kwargs = {
|
||||
'context': context,
|
||||
'image': self,
|
||||
'store_api': store_api,
|
||||
}
|
||||
super(ImageProxy, self).__init__(
|
||||
image, member_repo_proxy_class=ImageMemberRepoProxy,
|
||||
member_repo_proxy_kwargs=proxy_kwargs)
|
||||
|
||||
def delete(self):
|
||||
self.image.delete()
|
||||
if self.image.locations:
|
||||
for location in self.image.locations:
|
||||
self.store_api.delete_image_from_backend(self.context,
|
||||
self.store_api,
|
||||
self.image.image_id,
|
||||
location['url'])
|
||||
|
||||
def set_data(self, data, size=None):
|
||||
if size is None:
|
||||
size = 0 # NOTE(markwash): zero -> unknown size
|
||||
location, size, checksum, loc_meta = self.store_api.add_to_backend(
|
||||
self.context, CONF.default_store,
|
||||
self.image.image_id, utils.CooperativeReader(data), size)
|
||||
self.image.locations = [{'url': location, 'metadata': loc_meta}]
|
||||
self.image.size = size
|
||||
self.image.checksum = checksum
|
||||
self.image.status = 'active'
|
||||
|
||||
def get_data(self):
|
||||
if not self.image.locations:
|
||||
raise exception.NotFound(_("No image data could be found"))
|
||||
err = None
|
||||
for loc in self.image.locations:
|
||||
try:
|
||||
data, size = self.store_api.get_from_backend(self.context,
|
||||
loc['url'])
|
||||
|
||||
return data
|
||||
except Exception as e:
|
||||
LOG.warn(_('Get image %(id)s data from %(loc)s '
|
||||
'failed: %(err)s.') % {'id': self.image.image_id,
|
||||
'loc': loc, 'err': e})
|
||||
err = e
|
||||
# tried all locations
|
||||
LOG.error(_('Glance tried all locations to get data for image %s '
|
||||
'but all have failed.') % self.image.image_id)
|
||||
raise err
|
||||
|
||||
|
||||
class ImageMemberRepoProxy(glance.domain.proxy.Repo):
|
||||
def __init__(self, repo, image, context, store_api):
|
||||
self.repo = repo
|
||||
self.image = image
|
||||
self.context = context
|
||||
self.store_api = store_api
|
||||
super(ImageMemberRepoProxy, self).__init__(repo)
|
||||
|
||||
def _set_acls(self):
|
||||
public = self.image.visibility == 'public'
|
||||
if self.image.locations and not public:
|
||||
member_ids = [m.member_id for m in self.repo.list()]
|
||||
for location in self.image.locations:
|
||||
self.store_api.set_acls(self.context, location['url'], public,
|
||||
read_tenants=member_ids)
|
||||
|
||||
def add(self, member):
|
||||
super(ImageMemberRepoProxy, self).add(member)
|
||||
self._set_acls()
|
||||
|
||||
def remove(self, member):
|
||||
super(ImageMemberRepoProxy, self).remove(member)
|
||||
self._set_acls()
|
167
glance/store/base.py
Normal file
167
glance/store/base.py
Normal file
@ -0,0 +1,167 @@
|
||||
# Copyright 2011 OpenStack Foundation
|
||||
# Copyright 2012 RedHat Inc.
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""Base class for all storage backends"""
|
||||
|
||||
from glance.common import exception
|
||||
from glance.openstack.common import importutils
|
||||
import glance.openstack.common.log as logging
|
||||
from glance.openstack.common import strutils
|
||||
from glance.openstack.common import units
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _exception_to_unicode(exc):
|
||||
try:
|
||||
return unicode(exc)
|
||||
except UnicodeError:
|
||||
try:
|
||||
return strutils.safe_decode(str(exc), errors='ignore')
|
||||
except UnicodeError:
|
||||
msg = (_("Caught '%(exception)s' exception.") %
|
||||
{"exception": exc.__class__.__name__})
|
||||
return strutils.safe_decode(msg, errors='ignore')
|
||||
|
||||
|
||||
class Store(object):
|
||||
|
||||
CHUNKSIZE = 16 * units.Mi # 16M
|
||||
|
||||
def __init__(self, context=None, location=None):
|
||||
"""
|
||||
Initialize the Store
|
||||
"""
|
||||
self.store_location_class = None
|
||||
self.context = context
|
||||
self.configure()
|
||||
|
||||
try:
|
||||
self.configure_add()
|
||||
except exception.BadStoreConfiguration as e:
|
||||
self.add = self.add_disabled
|
||||
msg = (_(u"Failed to configure store correctly: %s "
|
||||
"Disabling add method.") % _exception_to_unicode(e))
|
||||
LOG.warn(msg)
|
||||
|
||||
def configure(self):
|
||||
"""
|
||||
Configure the Store to use the stored configuration options
|
||||
Any store that needs special configuration should implement
|
||||
this method.
|
||||
"""
|
||||
pass
|
||||
|
||||
def get_schemes(self):
|
||||
"""
|
||||
Returns a tuple of schemes which this store can handle.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def get_store_location_class(self):
|
||||
"""
|
||||
Returns the store location class that is used by this store.
|
||||
"""
|
||||
if not self.store_location_class:
|
||||
class_name = "%s.StoreLocation" % (self.__module__)
|
||||
LOG.debug("Late loading location class %s", class_name)
|
||||
self.store_location_class = importutils.import_class(class_name)
|
||||
return self.store_location_class
|
||||
|
||||
def configure_add(self):
|
||||
"""
|
||||
This is like `configure` except that it's specifically for
|
||||
configuring the store to accept objects.
|
||||
|
||||
If the store was not able to successfully configure
|
||||
itself, it should raise `exception.BadStoreConfiguration`.
|
||||
"""
|
||||
pass
|
||||
|
||||
def get(self, location):
|
||||
"""
|
||||
Takes a `glance.store.location.Location` object that indicates
|
||||
where to find the image file, and returns a tuple of generator
|
||||
(for reading the image file) and image_size
|
||||
|
||||
:param location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
:raises `glance.exception.NotFound` if image does not exist
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def get_size(self, location):
|
||||
"""
|
||||
Takes a `glance.store.location.Location` object that indicates
|
||||
where to find the image file, and returns the size
|
||||
|
||||
:param location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
:raises `glance.exception.NotFound` if image does not exist
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def add_disabled(self, *args, **kwargs):
|
||||
"""
|
||||
Add method that raises an exception because the Store was
|
||||
not able to be configured properly and therefore the add()
|
||||
method would error out.
|
||||
"""
|
||||
raise exception.StoreAddDisabled
|
||||
|
||||
def add(self, image_id, image_file, image_size):
|
||||
"""
|
||||
Stores an image file with supplied identifier to the backend
|
||||
storage system and returns a tuple containing information
|
||||
about the stored image.
|
||||
|
||||
:param image_id: The opaque image identifier
|
||||
:param image_file: The image data to write, as a file-like object
|
||||
:param image_size: The size of the image data to write, in bytes
|
||||
|
||||
:retval tuple of URL in backing store, bytes written, checksum
|
||||
and a dictionary with storage system specific information
|
||||
:raises `glance.common.exception.Duplicate` if the image already
|
||||
existed
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def delete(self, location):
|
||||
"""
|
||||
Takes a `glance.store.location.Location` object that indicates
|
||||
where to find the image file to delete
|
||||
|
||||
:location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
:raises `glance.exception.NotFound` if image does not exist
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def set_acls(self, location, public=False, read_tenants=[],
|
||||
write_tenants=[]):
|
||||
"""
|
||||
Sets the read and write access control list for an image in the
|
||||
backend store.
|
||||
|
||||
:location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
:public A boolean indicating whether the image should be public.
|
||||
:read_tenants A list of tenant strings which should be granted
|
||||
read access for an image.
|
||||
:write_tenants A list of tenant strings which should be granted
|
||||
write access for an image.
|
||||
"""
|
||||
raise NotImplementedError
|
182
glance/store/cinder.py
Normal file
182
glance/store/cinder.py
Normal file
@ -0,0 +1,182 @@
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""Storage backend for Cinder"""
|
||||
|
||||
from cinderclient import exceptions as cinder_exception
|
||||
from cinderclient import service_catalog
|
||||
from cinderclient.v2 import client as cinderclient
|
||||
|
||||
from oslo.config import cfg
|
||||
|
||||
from glance.common import exception
|
||||
from glance.common import utils
|
||||
import glance.openstack.common.log as logging
|
||||
from glance.openstack.common import units
|
||||
import glance.store.base
|
||||
import glance.store.location
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
cinder_opts = [
|
||||
cfg.StrOpt('cinder_catalog_info',
|
||||
default='volume:cinder:publicURL',
|
||||
help='Info to match when looking for cinder in the service '
|
||||
'catalog. Format is : separated values of the form: '
|
||||
'<service_type>:<service_name>:<endpoint_type>'),
|
||||
cfg.StrOpt('cinder_endpoint_template',
|
||||
default=None,
|
||||
help='Override service catalog lookup with template for cinder '
|
||||
'endpoint e.g. http://localhost:8776/v1/%(project_id)s'),
|
||||
cfg.StrOpt('os_region_name',
|
||||
default=None,
|
||||
help='Region name of this node'),
|
||||
cfg.StrOpt('cinder_ca_certificates_file',
|
||||
default=None,
|
||||
help='Location of ca certicates file to use for cinder client '
|
||||
'requests.'),
|
||||
cfg.IntOpt('cinder_http_retries',
|
||||
default=3,
|
||||
help='Number of cinderclient retries on failed http calls'),
|
||||
cfg.BoolOpt('cinder_api_insecure',
|
||||
default=False,
|
||||
help='Allow to perform insecure SSL requests to cinder'),
|
||||
]
|
||||
|
||||
CONF = cfg.CONF
|
||||
CONF.register_opts(cinder_opts)
|
||||
|
||||
|
||||
def get_cinderclient(context):
|
||||
if CONF.cinder_endpoint_template:
|
||||
url = CONF.cinder_endpoint_template % context.to_dict()
|
||||
else:
|
||||
info = CONF.cinder_catalog_info
|
||||
service_type, service_name, endpoint_type = info.split(':')
|
||||
|
||||
# extract the region if set in configuration
|
||||
if CONF.os_region_name:
|
||||
attr = 'region'
|
||||
filter_value = CONF.os_region_name
|
||||
else:
|
||||
attr = None
|
||||
filter_value = None
|
||||
|
||||
# FIXME: the cinderclient ServiceCatalog object is mis-named.
|
||||
# It actually contains the entire access blob.
|
||||
# Only needed parts of the service catalog are passed in, see
|
||||
# nova/context.py.
|
||||
compat_catalog = {
|
||||
'access': {'serviceCatalog': context.service_catalog or []}}
|
||||
sc = service_catalog.ServiceCatalog(compat_catalog)
|
||||
|
||||
url = sc.url_for(attr=attr,
|
||||
filter_value=filter_value,
|
||||
service_type=service_type,
|
||||
service_name=service_name,
|
||||
endpoint_type=endpoint_type)
|
||||
|
||||
LOG.debug(_('Cinderclient connection created using URL: %s') % url)
|
||||
|
||||
c = cinderclient.Client(context.user,
|
||||
context.auth_tok,
|
||||
project_id=context.tenant,
|
||||
auth_url=url,
|
||||
insecure=CONF.cinder_api_insecure,
|
||||
retries=CONF.cinder_http_retries,
|
||||
cacert=CONF.cinder_ca_certificates_file)
|
||||
|
||||
# noauth extracts user_id:project_id from auth_token
|
||||
c.client.auth_token = context.auth_tok or '%s:%s' % (context.user,
|
||||
context.tenant)
|
||||
c.client.management_url = url
|
||||
return c
|
||||
|
||||
|
||||
class StoreLocation(glance.store.location.StoreLocation):
|
||||
|
||||
"""Class describing a Cinder URI"""
|
||||
|
||||
def process_specs(self):
|
||||
self.scheme = self.specs.get('scheme', 'cinder')
|
||||
self.volume_id = self.specs.get('volume_id')
|
||||
|
||||
def get_uri(self):
|
||||
return "cinder://%s" % self.volume_id
|
||||
|
||||
def parse_uri(self, uri):
|
||||
if not uri.startswith('cinder://'):
|
||||
reason = _("URI must start with cinder://")
|
||||
LOG.error(reason)
|
||||
raise exception.BadStoreUri(uri, reason)
|
||||
|
||||
self.scheme = 'cinder'
|
||||
self.volume_id = uri[9:]
|
||||
|
||||
if not utils.is_uuid_like(self.volume_id):
|
||||
reason = _("URI contains invalid volume ID: %s") % self.volume_id
|
||||
LOG.error(reason)
|
||||
raise exception.BadStoreUri(uri, reason)
|
||||
|
||||
|
||||
class Store(glance.store.base.Store):
|
||||
|
||||
"""Cinder backend store adapter."""
|
||||
|
||||
EXAMPLE_URL = "cinder://volume-id"
|
||||
|
||||
def get_schemes(self):
|
||||
return ('cinder',)
|
||||
|
||||
def configure_add(self):
|
||||
"""
|
||||
Configure the Store to use the stored configuration options
|
||||
Any store that needs special configuration should implement
|
||||
this method. If the store was not able to successfully configure
|
||||
itself, it should raise `exception.BadStoreConfiguration`
|
||||
"""
|
||||
|
||||
if self.context is None:
|
||||
reason = _("Cinder storage requires a context.")
|
||||
raise exception.BadStoreConfiguration(store_name="cinder",
|
||||
reason=reason)
|
||||
if self.context.service_catalog is None:
|
||||
reason = _("Cinder storage requires a service catalog.")
|
||||
raise exception.BadStoreConfiguration(store_name="cinder",
|
||||
reason=reason)
|
||||
|
||||
def get_size(self, location):
|
||||
"""
|
||||
Takes a `glance.store.location.Location` object that indicates
|
||||
where to find the image file and returns the image size
|
||||
|
||||
:param location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
:raises `glance.exception.NotFound` if image does not exist
|
||||
:rtype int
|
||||
"""
|
||||
|
||||
loc = location.store_location
|
||||
|
||||
try:
|
||||
volume = get_cinderclient(self.context).volumes.get(loc.volume_id)
|
||||
# GB unit convert to byte
|
||||
return volume.size * units.Gi
|
||||
except cinder_exception.NotFound as e:
|
||||
reason = _("Failed to get image size due to "
|
||||
"volume can not be found: %s") % self.volume_id
|
||||
LOG.error(reason)
|
||||
raise exception.NotFound(reason)
|
||||
except Exception as e:
|
||||
LOG.exception(_("Failed to get image size due to "
|
||||
"internal error: %s") % e)
|
||||
return 0
|
301
glance/store/filesystem.py
Normal file
301
glance/store/filesystem.py
Normal file
@ -0,0 +1,301 @@
|
||||
# Copyright 2010 OpenStack Foundation
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""
|
||||
A simple filesystem-backed store
|
||||
"""
|
||||
|
||||
import errno
|
||||
import hashlib
|
||||
import os
|
||||
import urlparse
|
||||
|
||||
from oslo.config import cfg
|
||||
|
||||
from glance.common import exception
|
||||
from glance.common import utils
|
||||
from glance.openstack.common import jsonutils
|
||||
import glance.openstack.common.log as logging
|
||||
import glance.store
|
||||
import glance.store.base
|
||||
import glance.store.location
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
filesystem_opts = [
|
||||
cfg.StrOpt('filesystem_store_datadir',
|
||||
help=_('Directory to which the Filesystem backend '
|
||||
'store writes images.')),
|
||||
cfg.StrOpt('filesystem_store_metadata_file',
|
||||
help=_("The path to a file which contains the "
|
||||
"metadata to be returned with any location "
|
||||
"associated with this store. The file must "
|
||||
"contain a valid JSON dict."))]
|
||||
|
||||
CONF = cfg.CONF
|
||||
CONF.register_opts(filesystem_opts)
|
||||
|
||||
|
||||
class StoreLocation(glance.store.location.StoreLocation):
|
||||
|
||||
"""Class describing a Filesystem URI"""
|
||||
|
||||
def process_specs(self):
|
||||
self.scheme = self.specs.get('scheme', 'file')
|
||||
self.path = self.specs.get('path')
|
||||
|
||||
def get_uri(self):
|
||||
return "file://%s" % self.path
|
||||
|
||||
def parse_uri(self, uri):
|
||||
"""
|
||||
Parse URLs. This method fixes an issue where credentials specified
|
||||
in the URL are interpreted differently in Python 2.6.1+ than prior
|
||||
versions of Python.
|
||||
"""
|
||||
pieces = urlparse.urlparse(uri)
|
||||
assert pieces.scheme in ('file', 'filesystem')
|
||||
self.scheme = pieces.scheme
|
||||
path = (pieces.netloc + pieces.path).strip()
|
||||
if path == '':
|
||||
reason = _("No path specified in URI: %s") % uri
|
||||
LOG.debug(reason)
|
||||
raise exception.BadStoreUri('No path specified')
|
||||
self.path = path
|
||||
|
||||
|
||||
class ChunkedFile(object):
|
||||
|
||||
"""
|
||||
We send this back to the Glance API server as
|
||||
something that can iterate over a large file
|
||||
"""
|
||||
|
||||
CHUNKSIZE = 65536
|
||||
|
||||
def __init__(self, filepath):
|
||||
self.filepath = filepath
|
||||
self.fp = open(self.filepath, 'rb')
|
||||
|
||||
def __iter__(self):
|
||||
"""Return an iterator over the image file"""
|
||||
try:
|
||||
if self.fp:
|
||||
while True:
|
||||
chunk = self.fp.read(ChunkedFile.CHUNKSIZE)
|
||||
if chunk:
|
||||
yield chunk
|
||||
else:
|
||||
break
|
||||
finally:
|
||||
self.close()
|
||||
|
||||
def close(self):
|
||||
"""Close the internal file pointer"""
|
||||
if self.fp:
|
||||
self.fp.close()
|
||||
self.fp = None
|
||||
|
||||
|
||||
class Store(glance.store.base.Store):
|
||||
|
||||
def get_schemes(self):
|
||||
return ('file', 'filesystem')
|
||||
|
||||
def configure_add(self):
|
||||
"""
|
||||
Configure the Store to use the stored configuration options
|
||||
Any store that needs special configuration should implement
|
||||
this method. If the store was not able to successfully configure
|
||||
itself, it should raise `exception.BadStoreConfiguration`
|
||||
"""
|
||||
self.datadir = CONF.filesystem_store_datadir
|
||||
if self.datadir is None:
|
||||
reason = (_("Could not find %s in configuration options.") %
|
||||
'filesystem_store_datadir')
|
||||
LOG.error(reason)
|
||||
raise exception.BadStoreConfiguration(store_name="filesystem",
|
||||
reason=reason)
|
||||
|
||||
if not os.path.exists(self.datadir):
|
||||
msg = _("Directory to write image files does not exist "
|
||||
"(%s). Creating.") % self.datadir
|
||||
LOG.info(msg)
|
||||
try:
|
||||
os.makedirs(self.datadir)
|
||||
except (IOError, OSError):
|
||||
if os.path.exists(self.datadir):
|
||||
# NOTE(markwash): If the path now exists, some other
|
||||
# process must have beat us in the race condition. But it
|
||||
# doesn't hurt, so we can safely ignore the error.
|
||||
return
|
||||
reason = _("Unable to create datadir: %s") % self.datadir
|
||||
LOG.error(reason)
|
||||
raise exception.BadStoreConfiguration(store_name="filesystem",
|
||||
reason=reason)
|
||||
|
||||
@staticmethod
|
||||
def _resolve_location(location):
|
||||
filepath = location.store_location.path
|
||||
|
||||
if not os.path.exists(filepath):
|
||||
raise exception.NotFound(_("Image file %s not found") % filepath)
|
||||
|
||||
filesize = os.path.getsize(filepath)
|
||||
return filepath, filesize
|
||||
|
||||
def _get_metadata(self):
|
||||
if CONF.filesystem_store_metadata_file is None:
|
||||
return {}
|
||||
|
||||
try:
|
||||
with open(CONF.filesystem_store_metadata_file, 'r') as fptr:
|
||||
metadata = jsonutils.load(fptr)
|
||||
glance.store.check_location_metadata(metadata)
|
||||
return metadata
|
||||
except glance.store.BackendException as bee:
|
||||
LOG.error(_('The JSON in the metadata file %s could not be used: '
|
||||
'%s An empty dictionary will be returned '
|
||||
'to the client.')
|
||||
% (CONF.filesystem_store_metadata_file, str(bee)))
|
||||
return {}
|
||||
except IOError as ioe:
|
||||
LOG.error(_('The path for the metadata file %s could not be '
|
||||
'opened: %s An empty dictionary will be returned '
|
||||
'to the client.')
|
||||
% (CONF.filesystem_store_metadata_file, ioe))
|
||||
return {}
|
||||
except Exception as ex:
|
||||
LOG.exception(_('An error occurred processing the storage systems '
|
||||
'meta data file: %s. An empty dictionary will be '
|
||||
'returned to the client.') % str(ex))
|
||||
return {}
|
||||
|
||||
def get(self, location):
|
||||
"""
|
||||
Takes a `glance.store.location.Location` object that indicates
|
||||
where to find the image file, and returns a tuple of generator
|
||||
(for reading the image file) and image_size
|
||||
|
||||
:param location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
:raises `glance.exception.NotFound` if image does not exist
|
||||
"""
|
||||
filepath, filesize = self._resolve_location(location)
|
||||
msg = _("Found image at %s. Returning in ChunkedFile.") % filepath
|
||||
LOG.debug(msg)
|
||||
return (ChunkedFile(filepath), filesize)
|
||||
|
||||
def get_size(self, location):
|
||||
"""
|
||||
Takes a `glance.store.location.Location` object that indicates
|
||||
where to find the image file and returns the image size
|
||||
|
||||
:param location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
:raises `glance.exception.NotFound` if image does not exist
|
||||
:rtype int
|
||||
"""
|
||||
filepath, filesize = self._resolve_location(location)
|
||||
msg = _("Found image at %s.") % filepath
|
||||
LOG.debug(msg)
|
||||
return filesize
|
||||
|
||||
def delete(self, location):
|
||||
"""
|
||||
Takes a `glance.store.location.Location` object that indicates
|
||||
where to find the image file to delete
|
||||
|
||||
:location `glance.store.location.Location` object, supplied
|
||||
from glance.store.location.get_location_from_uri()
|
||||
|
||||
:raises NotFound if image does not exist
|
||||
:raises Forbidden if cannot delete because of permissions
|
||||
"""
|
||||
loc = location.store_location
|
||||
fn = loc.path
|
||||
if |