Merge "Make cinder driver compatible with multiple stores"
This commit is contained in:
Zuul
Gerrit Code Review
commit
e8a183671b
@ -0,0 +1,290 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
==================================================
|
||||
Make cinder driver compatible with multiple stores
|
||||
==================================================
|
||||
|
||||
https://blueprints.launchpad.net/glance_store/+spec/multiple-cinder-backend-support
|
||||
|
||||
From Train onwards Glance is fully supporting configuring multiple glance
|
||||
stores. Glance can configure to use cinder as a store using available cinder
|
||||
driver of glance_store. Cinder makes available volume-types, which describe
|
||||
the characteristics of volumes. Currently, the cinder glance_store can only
|
||||
use one volume-type. The point of glance multi-store implemented in Train
|
||||
is to give operators the ability to expose glance stores of differing
|
||||
characteristics to end users. Even though all images will wind up in a single
|
||||
cinder installation, it is possible for operators to expose different
|
||||
categories of storage in cinder by creating different volume-types.
|
||||
So when a cinder installation exposes multiple volume-types of differing
|
||||
characteristics, what we want to do here is to be able to map different glance
|
||||
'stores' to cinder volume-types.
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
1. As of now cinder can configure different volume-types but glance will be able
|
||||
to use only one of those available volume-types. If glance store is
|
||||
set to use cinder then every time new image is created it will always be
|
||||
uploaded to default volume-type unless operator has configured
|
||||
``cinder_volume_type`` in glance-api.conf file.
|
||||
|
||||
2. If cinder store is configured to use in glance then while creating the
|
||||
image cinder store of glance creates location URL with ``cinder://`` prefix
|
||||
only. When cinder has configured multiple backends and glance has also
|
||||
configured multiple cinder stores then it is difficult to analyse
|
||||
the image is stored in which of the cinder store as the location url
|
||||
for the image as ``cinder://<image_id>``.
|
||||
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
We propose that Glance be able to expose multiple cinder stores that differ
|
||||
by what volume-type each store uses. These would be defined in the normal way
|
||||
in the glance configuration file using the ``enabled_backends`` option
|
||||
(see example below). Further, when using multiple glance stores, each cinder
|
||||
store *must* have the ``cinder_volume_type`` option set.
|
||||
|
||||
While initializing the store object, glance will validate that volume type
|
||||
set using ``cinder_volume_type`` is exist in cinder. If it's not then that
|
||||
store will be excluded by disabling 'add' and 'delete' operations. To
|
||||
connect to cinder from glance operator needs to specify
|
||||
``cinder_store_auth_address``, ``cinder_store_user_name``,
|
||||
``cinder_store_password`` and ``cinder_catalog_info`` for each of the store
|
||||
in glance-api.conf file.
|
||||
|
||||
Below are some multiple cinder store configuration examples.
|
||||
|
||||
Example 1: Fresh deployment
|
||||
|
||||
For example, if cinder has configured 2 volume types ``glance-fast`` and
|
||||
``glance-slow`` then glance configuration should look like;::
|
||||
|
||||
[DEFAULT]
|
||||
# list of enabled stores identified by their property group name
|
||||
enabled_backends = fast:cinder,slow:cinder
|
||||
|
||||
# the default store, if not set glance-api service will not start
|
||||
[glance_store]
|
||||
default_backend = fast
|
||||
|
||||
# conf props for fast store instance
|
||||
[fast]
|
||||
rootwrap_config = /etc/glance/rootwrap.conf
|
||||
cinder_volume_type = glance-fast
|
||||
description = Really fast and expensive storage
|
||||
cinder_catalog_info = volumev2::publicURL
|
||||
cinder_store_auth_address = http://localhost/identity/v3
|
||||
cinder_store_user_name = glance
|
||||
cinder_store_password = admin
|
||||
cinder_store_project_name = service
|
||||
# etc..
|
||||
|
||||
# conf props for slow store instance
|
||||
[slow]
|
||||
rootwrap_config = /etc/glance/rootwrap.conf
|
||||
cinder_volume_type = glance-slow
|
||||
description = Slower but less expensive storage
|
||||
cinder_catalog_info = volumev2::publicURL
|
||||
cinder_store_auth_address = http://localhost/identity/v3
|
||||
cinder_store_user_name = glance
|
||||
cinder_store_password = admin
|
||||
cinder_store_project_name = service
|
||||
# etc..
|
||||
|
||||
Example 2: Upgrade from single cinder store to multiple cinder stores, if
|
||||
default_volume_type is set in cinder.conf and cinder_volume_type is also set in
|
||||
glance-api.conf then administrator needs to create one store in glance where
|
||||
cinder_volume_type same as old glance configuration::
|
||||
|
||||
# cinder.conf
|
||||
The glance administrator has to find out what the default volume-type is
|
||||
in the cinder installation, so he/she needs to discuss with either cinder
|
||||
admin or cloud admin to identify default volume-type from cinder and then
|
||||
explicitly configure that as the value of ``cinder_volume_type``.
|
||||
|
||||
Example config before upgrade::
|
||||
|
||||
[glance_store]
|
||||
stores = cinder, file, http
|
||||
default_store = cinder
|
||||
cinder_state_transition_timeout = 300
|
||||
rootwrap_config = /etc/glance/rootwrap.conf
|
||||
cinder_catalog_info = volumev2::publicURL
|
||||
cinder_volume_type = glance-old
|
||||
|
||||
Example config after upgrade::
|
||||
|
||||
[DEFAULT]
|
||||
enabled_backends = old:cinder, new:cinder
|
||||
|
||||
[glance_store]
|
||||
default_backend = new
|
||||
|
||||
[new]
|
||||
rootwrap_config = /etc/glance/rootwrap.conf
|
||||
cinder_volume_type = glance-new
|
||||
description = Newly defined second (cinder) store
|
||||
cinder_catalog_info = volumev2::publicURL
|
||||
cinder_store_auth_address = http://localhost/identity/v3
|
||||
cinder_store_user_name = glance
|
||||
cinder_store_password = admin
|
||||
cinder_store_project_name = service
|
||||
# etc..
|
||||
|
||||
[old]
|
||||
rootwrap_config = /etc/glance/rootwrap.conf
|
||||
cinder_volume_type = glance-old # as per old cinder.conf
|
||||
description = Previously existing (cinder) store
|
||||
cinder_catalog_info = volumev2::publicURL
|
||||
cinder_store_auth_address = http://localhost/identity/v3
|
||||
cinder_store_user_name = glance
|
||||
cinder_store_password = admin
|
||||
cinder_store_project_name = service
|
||||
# etc..
|
||||
|
||||
Operator can decide on the basis of deployment strategy which volume type
|
||||
they wants to use by coordinating with cinder admin or cloud operator.
|
||||
|
||||
We also propose to modify location url for cinder and use
|
||||
``store identifier`` in location url so that user or operator will
|
||||
identify in which cinder store of glance image is stored. The new
|
||||
location URL should looked like ``cinder://store-name/image-id``.
|
||||
|
||||
For legacy images stored in cinder backend we will modify the lazy loading
|
||||
mechanism of glance which will update the location URL of existing images
|
||||
as per new format. The lazy loading operation is a check before
|
||||
GET API call which traverse through image location and based on location URI
|
||||
it identifies in which glance store image data is stored and updates
|
||||
that information in location metadata. This mechanism is also useful
|
||||
in a way that if in future operator decides to change the name of the
|
||||
glance store or retire one of the configured store by migrating the
|
||||
images to new stores.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
None
|
||||
|
||||
Data model impact
|
||||
-----------------
|
||||
|
||||
None
|
||||
|
||||
REST API impact
|
||||
---------------
|
||||
|
||||
None
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
|
||||
The security impact is same as it was with single store but we're just
|
||||
pointing it out here; The image-volume is stored in the configured
|
||||
project ``cinder_store_project_name`` and can be accessed with configured
|
||||
user ``cinder_store_user_name``.
|
||||
|
||||
There could be a potential risk if someone was able to get a hold of
|
||||
these credentials and access the image-volumes. Worst case is someone
|
||||
could alter the image-volumes if they had permission to perform any cinder
|
||||
operation on it such as retype, attach etc.
|
||||
|
||||
Care will have to be taken to ensure it isn't accessible by normal
|
||||
users.
|
||||
|
||||
Notifications impact
|
||||
--------------------
|
||||
|
||||
None
|
||||
|
||||
Other end user impact
|
||||
---------------------
|
||||
|
||||
None
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
After upgrade from single cinder store to use multiple cinder stores first
|
||||
image-list or first get call for image will take additional time as we are
|
||||
performing the lazy loading operation to update legacy image location url
|
||||
to use new image location urls. Subsequent get or list calls will perform
|
||||
as they were performing earlier.
|
||||
|
||||
Other deployer impact
|
||||
---------------------
|
||||
|
||||
Operators should be aware of different volume types available in cinder. They
|
||||
can either use ``type-list`` command of cinder client or coordinate with cinder
|
||||
admin and decide which volume-type of cinder should be configured in
|
||||
glance-api.conf.
|
||||
|
||||
Developer impact
|
||||
----------------
|
||||
|
||||
None
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
* whoami-rajat
|
||||
* abhishek-kekane
|
||||
|
||||
Other contributors:
|
||||
None
|
||||
|
||||
Reviewers
|
||||
---------
|
||||
|
||||
Core reviewer(s):
|
||||
|
||||
* jokke
|
||||
* rosmaita
|
||||
* smcginnis
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
* Modify cinder driver initialization to set new cinder location url
|
||||
* Modify usage of cinder location url
|
||||
* Modify lazy loading mechanism to update legacy image location URLs
|
||||
* Unit tests
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
Appropriate unit and functional tests to ensure the changes to glance function
|
||||
correctly. Aslo we could add a job which will run tests using cinder stores in
|
||||
glance.
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
We'll need to ensure that glance/glance_store docs are updated for:
|
||||
|
||||
* Usage of cinder volume types as cinder stores of glance.
|
||||
* We should also document that, if cinder store is used as glance
|
||||
backend, Only the Image Service API should be used to manipulate images.
|
||||
Manipulating image data directly via the Block Storage Service API is not
|
||||
supported and may lead to adverse consequences, including data loss.
|
||||
* How to upgrade from single cinder store to multiple cinder stores.
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
None
|
||||
Loading…
Reference in New Issue
Block a user