diff --git a/specs/victoria/approved/glance_store/multiple-cinder-backend-support.rst b/specs/victoria/approved/glance_store/multiple-cinder-backend-support.rst new file mode 100644 index 00000000..2b58e2df --- /dev/null +++ b/specs/victoria/approved/glance_store/multiple-cinder-backend-support.rst @@ -0,0 +1,290 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +================================================== +Make cinder driver compatible with multiple stores +================================================== + +https://blueprints.launchpad.net/glance_store/+spec/multiple-cinder-backend-support + +From Train onwards Glance is fully supporting configuring multiple glance +stores. Glance can configure to use cinder as a store using available cinder +driver of glance_store. Cinder makes available volume-types, which describe +the characteristics of volumes. Currently, the cinder glance_store can only +use one volume-type. The point of glance multi-store implemented in Train +is to give operators the ability to expose glance stores of differing +characteristics to end users. Even though all images will wind up in a single +cinder installation, it is possible for operators to expose different +categories of storage in cinder by creating different volume-types. +So when a cinder installation exposes multiple volume-types of differing +characteristics, what we want to do here is to be able to map different glance +'stores' to cinder volume-types. + +Problem description +=================== + +1. As of now cinder can configure different volume-types but glance will be able +to use only one of those available volume-types. If glance store is +set to use cinder then every time new image is created it will always be +uploaded to default volume-type unless operator has configured +``cinder_volume_type`` in glance-api.conf file. + +2. If cinder store is configured to use in glance then while creating the +image cinder store of glance creates location URL with ``cinder://`` prefix +only. When cinder has configured multiple backends and glance has also +configured multiple cinder stores then it is difficult to analyse +the image is stored in which of the cinder store as the location url +for the image as ``cinder://``. + + +Proposed change +=============== + +We propose that Glance be able to expose multiple cinder stores that differ +by what volume-type each store uses. These would be defined in the normal way +in the glance configuration file using the ``enabled_backends`` option +(see example below). Further, when using multiple glance stores, each cinder +store *must* have the ``cinder_volume_type`` option set. + +While initializing the store object, glance will validate that volume type +set using ``cinder_volume_type`` is exist in cinder. If it's not then that +store will be excluded by disabling 'add' and 'delete' operations. To +connect to cinder from glance operator needs to specify +``cinder_store_auth_address``, ``cinder_store_user_name``, +``cinder_store_password`` and ``cinder_catalog_info`` for each of the store +in glance-api.conf file. + +Below are some multiple cinder store configuration examples. + +Example 1: Fresh deployment + +For example, if cinder has configured 2 volume types ``glance-fast`` and +``glance-slow`` then glance configuration should look like;:: + + [DEFAULT] + # list of enabled stores identified by their property group name + enabled_backends = fast:cinder,slow:cinder + + # the default store, if not set glance-api service will not start + [glance_store] + default_backend = fast + + # conf props for fast store instance + [fast] + rootwrap_config = /etc/glance/rootwrap.conf + cinder_volume_type = glance-fast + description = Really fast and expensive storage + cinder_catalog_info = volumev2::publicURL + cinder_store_auth_address = http://localhost/identity/v3 + cinder_store_user_name = glance + cinder_store_password = admin + cinder_store_project_name = service + # etc.. + + # conf props for slow store instance + [slow] + rootwrap_config = /etc/glance/rootwrap.conf + cinder_volume_type = glance-slow + description = Slower but less expensive storage + cinder_catalog_info = volumev2::publicURL + cinder_store_auth_address = http://localhost/identity/v3 + cinder_store_user_name = glance + cinder_store_password = admin + cinder_store_project_name = service + # etc.. + +Example 2: Upgrade from single cinder store to multiple cinder stores, if +default_volume_type is set in cinder.conf and cinder_volume_type is also set in +glance-api.conf then administrator needs to create one store in glance where +cinder_volume_type same as old glance configuration:: + + # cinder.conf + The glance administrator has to find out what the default volume-type is + in the cinder installation, so he/she needs to discuss with either cinder + admin or cloud admin to identify default volume-type from cinder and then + explicitly configure that as the value of ``cinder_volume_type``. + +Example config before upgrade:: + + [glance_store] + stores = cinder, file, http + default_store = cinder + cinder_state_transition_timeout = 300 + rootwrap_config = /etc/glance/rootwrap.conf + cinder_catalog_info = volumev2::publicURL + cinder_volume_type = glance-old + +Example config after upgrade:: + + [DEFAULT] + enabled_backends = old:cinder, new:cinder + + [glance_store] + default_backend = new + + [new] + rootwrap_config = /etc/glance/rootwrap.conf + cinder_volume_type = glance-new + description = Newly defined second (cinder) store + cinder_catalog_info = volumev2::publicURL + cinder_store_auth_address = http://localhost/identity/v3 + cinder_store_user_name = glance + cinder_store_password = admin + cinder_store_project_name = service + # etc.. + + [old] + rootwrap_config = /etc/glance/rootwrap.conf + cinder_volume_type = glance-old # as per old cinder.conf + description = Previously existing (cinder) store + cinder_catalog_info = volumev2::publicURL + cinder_store_auth_address = http://localhost/identity/v3 + cinder_store_user_name = glance + cinder_store_password = admin + cinder_store_project_name = service + # etc.. + +Operator can decide on the basis of deployment strategy which volume type +they wants to use by coordinating with cinder admin or cloud operator. + +We also propose to modify location url for cinder and use +``store identifier`` in location url so that user or operator will +identify in which cinder store of glance image is stored. The new +location URL should looked like ``cinder://store-name/image-id``. + +For legacy images stored in cinder backend we will modify the lazy loading +mechanism of glance which will update the location URL of existing images +as per new format. The lazy loading operation is a check before +GET API call which traverse through image location and based on location URI +it identifies in which glance store image data is stored and updates +that information in location metadata. This mechanism is also useful +in a way that if in future operator decides to change the name of the +glance store or retire one of the configured store by migrating the +images to new stores. + +Alternatives +------------ + +None + +Data model impact +----------------- + +None + +REST API impact +--------------- + +None + +Security impact +--------------- + +The security impact is same as it was with single store but we're just +pointing it out here; The image-volume is stored in the configured +project ``cinder_store_project_name`` and can be accessed with configured +user ``cinder_store_user_name``. + +There could be a potential risk if someone was able to get a hold of +these credentials and access the image-volumes. Worst case is someone +could alter the image-volumes if they had permission to perform any cinder +operation on it such as retype, attach etc. + +Care will have to be taken to ensure it isn't accessible by normal +users. + +Notifications impact +-------------------- + +None + +Other end user impact +--------------------- + +None + +Performance Impact +------------------ + +After upgrade from single cinder store to use multiple cinder stores first +image-list or first get call for image will take additional time as we are +performing the lazy loading operation to update legacy image location url +to use new image location urls. Subsequent get or list calls will perform +as they were performing earlier. + +Other deployer impact +--------------------- + +Operators should be aware of different volume types available in cinder. They +can either use ``type-list`` command of cinder client or coordinate with cinder +admin and decide which volume-type of cinder should be configured in +glance-api.conf. + +Developer impact +---------------- + +None + + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: +* whoami-rajat +* abhishek-kekane + +Other contributors: + None + +Reviewers +--------- + +Core reviewer(s): + +* jokke +* rosmaita +* smcginnis + +Work Items +---------- + +* Modify cinder driver initialization to set new cinder location url +* Modify usage of cinder location url +* Modify lazy loading mechanism to update legacy image location URLs +* Unit tests + +Dependencies +============ + +None + + +Testing +======= + +Appropriate unit and functional tests to ensure the changes to glance function +correctly. Aslo we could add a job which will run tests using cinder stores in +glance. + +Documentation Impact +==================== + +We'll need to ensure that glance/glance_store docs are updated for: + +* Usage of cinder volume types as cinder stores of glance. +* We should also document that, if cinder store is used as glance + backend, Only the Image Service API should be used to manipulate images. + Manipulating image data directly via the Block Storage Service API is not + supported and may lead to adverse consequences, including data loss. +* How to upgrade from single cinder store to multiple cinder stores. + +References +========== + +None