Add dev doc for generic volume groups
This patch adds instructions for driver developers to add consistent group snapshot support in generic volume groups. Change-Id: I8a223e76f8a1601e636ae3905be097d1e19a94fb
This commit is contained in:
xing-yang
parent
7502e1fed3
commit
6c698bc8e3
|
|
@ -0,0 +1,365 @@
|
|||
Generic Volume Groups
|
||||
=====================
|
||||
|
||||
Introduction to generic volume groups
|
||||
-------------------------------------
|
||||
|
||||
Generic volume group support was added in cinder in the Newton release.
|
||||
There is support for creating group types and group specs, creating
|
||||
groups of volumes, and creating snapshots of groups. Detailed information
|
||||
on how to create a group type, a group, and a group snapshot can be found
|
||||
in `block storage admin guide <http://docs.openstack.org/admin-guide/blockstorage-groups.html>`_.
|
||||
|
||||
How is generic volume groups different from consistency groups in cinder?
|
||||
The consistency group feature was introduced in cinder in Juno and are
|
||||
supported by a few drivers. Currently consistency groups in cinder only
|
||||
support consistent group snapshot. It cannot be extended easily to serve
|
||||
other purposes. A tenant may want to put volumes used in the same application
|
||||
together in a group so that it is easier to manage them together, and this
|
||||
group of volumes may or may not support consistent group snapshot. Generic
|
||||
volume group is introduced to solve this problem. By decoupling the tight
|
||||
relationship between the group construct and the consistency concept,
|
||||
generic volume groups can be extended to support other features in the future.
|
||||
|
||||
Action items for drivers supporting consistency groups
|
||||
------------------------------------------------------
|
||||
|
||||
Drivers currently supporting consistency groups are in the following:
|
||||
|
||||
- Juno: EMC VNX
|
||||
|
||||
- Kilo: EMC VMAX, IBM (GPFS, Storwize, SVC, and XIV), ProphetStor, Pure
|
||||
|
||||
- Liberty: Dell Storage Center, EMC XtremIO, HPE 3Par and LeftHand
|
||||
|
||||
- Mitaka: EMC ScaleIO, NetApp Data ONTAP and E-Series, SolidFire
|
||||
|
||||
- Newton: CoprHD, FalconStor, Huawei
|
||||
|
||||
Since the addition of generic volume groups, there is plan to migrate
|
||||
consistency groups to generic volume groups. A migration command and
|
||||
changes in CG APIs to support migrating CGs to groups are developed and
|
||||
merged in Ocata [1][2]. In order to support rolling upgrade, it will take
|
||||
a couple of releases before consistency groups can be deprecated.
|
||||
|
||||
For drivers planning to add consistency groups support, the new generic
|
||||
volume group driver interfaces should be implemented instead of the CG
|
||||
interfaces.
|
||||
|
||||
For drivers already supporting consistency groups, the new generic
|
||||
volume group driver interfaces should be implemented to include the
|
||||
CG support.
|
||||
|
||||
For drivers wanting generic volume groups but not consistent group
|
||||
snapshot support, no code changes are necessary. By default, every
|
||||
cinder volume driver already supports generic volume groups since
|
||||
Newton because the support was added to the common code. Testing
|
||||
should be done for every driver to make sure this feature works properly.
|
||||
|
||||
Drivers already supporting CG are expected to add CG support to
|
||||
generic volume groups by Pike-1. This is a deadline discussed and
|
||||
agreed upon at the Ocata summit in Barcelona.
|
||||
|
||||
Group Type and Group Specs / Volume Types and Extra Specs
|
||||
---------------------------------------------------------
|
||||
|
||||
The driver interfaces for consistency groups and generic volume groups
|
||||
are very similar. One new concept introduced for generic volume groups
|
||||
is the group type. Group type is used to categorize a group just like a
|
||||
volume type is used to describe a volume. Similar to extra specs for
|
||||
a volume type, group specs are also introduced to be associated with a
|
||||
group type. Group types allow a user to create different types of groups.
|
||||
|
||||
A group can support multiple volume types and volume types are required
|
||||
as input parameters when creating a group. In addition to volume types,
|
||||
a group type is also required when creating a group.
|
||||
|
||||
Group types and volume types are created by the Cloud Administrator.
|
||||
A tenant uses the group types and volume types to create groups and
|
||||
volumes.
|
||||
|
||||
A driver can support both consistent group snapshot and a group of
|
||||
snapshots that do not maintain the write order consistency by using
|
||||
different group types. In other words, a group supporting consistent
|
||||
group snapshot is a special type of generic volume group.
|
||||
|
||||
For a group to support consistent group snapshot, the group specs in the
|
||||
corresponding group type should have the following entry::
|
||||
|
||||
{'consistent_group_snapshot_enabled': <is> True}
|
||||
|
||||
Similarly, for a volume to be in a group that supports consistent group
|
||||
snapshots, the volume type extra specs would also have the following entry::
|
||||
|
||||
{'consistent_group_snapshot_enabled': <is> True}
|
||||
|
||||
By requiring the above entry to be in both group specs and volume type
|
||||
extra specs, we can make sure the scheduler will choose a backend that
|
||||
supports the group type and volume types for a group. It is up to the driver
|
||||
to parse the group type info when creating a group, parse the volume type
|
||||
info when creating a volume, and set things up as requested.
|
||||
|
||||
Capabilities reporting
|
||||
----------------------
|
||||
The following entry is expected to be added to the stats/capabilities update
|
||||
for drivers supporting consistent group snapshot::
|
||||
|
||||
stats["consistent_group_snapshot_enabled"] = True
|
||||
|
||||
Driver methods
|
||||
--------------
|
||||
The following driver methods should to be implemented for the driver to
|
||||
support consistent group snapshot:
|
||||
|
||||
- create_group(context, group)
|
||||
|
||||
- delete_group(context, group, volumes)
|
||||
|
||||
- update_group(context, group, add_volumes=None, remove_volumes=None)
|
||||
|
||||
- create_group_from_src(context, group, volumes,
|
||||
group_snapshot=None, snapshots=None,
|
||||
source_group=None, source_vols=None)
|
||||
|
||||
- create_group_snapshot(context, group_snapshot, snapshots)
|
||||
|
||||
- delete_group_snapshot(context, group_snapshot, snapshots)
|
||||
|
||||
Here is an example that add CG capabitilty to generic volume groups [3].
|
||||
Details of driver interfaces are as follows.
|
||||
|
||||
**create_group**
|
||||
|
||||
This method creates a group. It has context and group object as input
|
||||
parameters. A group object has volume_types and group_type_id that can be used
|
||||
by the driver.
|
||||
|
||||
create_group returns model_update. model_update will be in this format:
|
||||
{'status': xxx, ......}.
|
||||
|
||||
If the status in model_update is 'error', the manager will throw
|
||||
an exception and it will be caught in the try-except block in the
|
||||
manager. If the driver throws an exception, the manager will also
|
||||
catch it in the try-except block. The group status in the db will
|
||||
be changed to 'error'.
|
||||
|
||||
For a successful operation, the driver can either build the
|
||||
model_update and return it or return None. The group status will
|
||||
be set to 'available'.
|
||||
|
||||
**delete_group**
|
||||
|
||||
This method deletes a group. It has context, group object, and a list
|
||||
of volume objects as input parameters. It returns model_update and
|
||||
volumes_model_update.
|
||||
|
||||
volumes_model_update is a list of volume dictionaries. It has to be built
|
||||
by the driver. An entry will be in this format: {'id': xxx, 'status': xxx,
|
||||
......}. model_update will be in this format: {'status': xxx, ......}.
|
||||
The driver should populate volumes_model_update and model_update
|
||||
and return them.
|
||||
|
||||
The manager will check volumes_model_update and update db accordingly
|
||||
for each volume. If the driver successfully deleted some volumes
|
||||
but failed to delete others, it should set statuses of the volumes
|
||||
accordingly so that the manager can update db correctly.
|
||||
|
||||
If the status in any entry of volumes_model_update is 'error_deleting'
|
||||
or 'error', the status in model_update will be set to the same if it
|
||||
is not already 'error_deleting' or 'error'.
|
||||
|
||||
If the status in model_update is 'error_deleting' or 'error', the
|
||||
manager will raise an exception and the status of the group will be
|
||||
set to 'error' in the db. If volumes_model_update is not returned by
|
||||
the driver, the manager will set the status of every volume in the
|
||||
group to 'error' in the except block.
|
||||
|
||||
If the driver raises an exception during the operation, it will be
|
||||
caught by the try-except block in the manager. The statuses of the
|
||||
group and all volumes in it will be set to 'error'.
|
||||
|
||||
For a successful operation, the driver can either build the
|
||||
model_update and volumes_model_update and return them or
|
||||
return None, None. The statuses of the group and all volumes
|
||||
will be set to 'deleted' after the manager deletes them from db.
|
||||
|
||||
**update_group**
|
||||
|
||||
This method adds existing volumes to a group or removes volumes
|
||||
from a group. It has context, group object, a list of volume objects
|
||||
to be added to the group, and a list of a volume objects to be
|
||||
removed from the group. It returns model_update, add_volumes_update,
|
||||
and remove_volumes_update.
|
||||
|
||||
model_update is a dictionary that the driver wants the manager
|
||||
to update upon a successful return. If None is returned, the manager
|
||||
will set the status to 'available'.
|
||||
|
||||
add_volumes_update and remove_volumes_update are lists of dictionaries
|
||||
that the driver wants the manager to update upon a successful return.
|
||||
Note that each entry requires a {'id': xxx} so that the correct
|
||||
volume entry can be updated. If None is returned, the volume will
|
||||
remain its original status.
|
||||
|
||||
If the driver throws an exception, the status of the group as well as
|
||||
those of the volumes to be added/removed will be set to 'error'.
|
||||
|
||||
**create_group_from_src**
|
||||
|
||||
This method creates a group from source. The source can be a
|
||||
group_snapshot or a source group. create_group_from_src has context,
|
||||
group object, a list of volume objects, group_snapshot object, a list
|
||||
of snapshot objects, source group object, and a list of source volume
|
||||
objects as input parameters. It returns model_update and
|
||||
volumes_model_update.
|
||||
|
||||
volumes_model_update is a list of dictionaries. It has to be built by
|
||||
the driver. An entry will be in this format: {'id': xxx, 'status': xxx,
|
||||
......}. model_update will be in this format: {'status': xxx, ......}.
|
||||
|
||||
To be consistent with other volume operations, the manager will
|
||||
assume the operation is successful if no exception is thrown by
|
||||
the driver. For a successful operation, the driver can either build
|
||||
the model_update and volumes_model_update and return them or
|
||||
return None, None.
|
||||
|
||||
**create_group_snapshot**
|
||||
|
||||
This method creates a group_snapshot. It has context, group_snapshot
|
||||
object, and a list of snapshot objects as input parameters. It returns
|
||||
model_update and snapshots_model_update.
|
||||
|
||||
snapshots_model_update is a list of dictionaries. It has to be built by the
|
||||
driver. An entry will be in this format: {'id': xxx, 'status': xxx, ......}.
|
||||
model_update will be in this format: {'status': xxx, ......}. The driver
|
||||
should populate snapshots_model_update and model_update and return them.
|
||||
|
||||
The manager will check snapshots_model_update and update db accordingly
|
||||
for each snapshot. If the driver successfully created some snapshots
|
||||
but failed to create others, it should set statuses of the snapshots
|
||||
accordingly so that the manager can update db correctly.
|
||||
|
||||
If the status in any entry of snapshots_model_update is 'error', the
|
||||
status in model_update will be set to the same if it is not already
|
||||
'error'.
|
||||
|
||||
If the status in model_update is 'error', the manager will raise an
|
||||
exception and the status of group_snapshot will be set to 'error' in
|
||||
the db. If snapshots_model_update is not returned by the driver, the
|
||||
manager will set the status of every snapshot to 'error' in the except
|
||||
block.
|
||||
|
||||
If the driver raises an exception during the operation, it will be
|
||||
caught by the try-except block in the manager and the statuses of
|
||||
group_snapshot and all snapshots will be set to 'error'.
|
||||
|
||||
For a successful operation, the driver can either build the
|
||||
model_update and snapshots_model_update and return them or
|
||||
return None, None. The statuses of group_snapshot and all snapshots
|
||||
will be set to 'available' at the end of the manager function.
|
||||
|
||||
**delete_group_snapshot**
|
||||
|
||||
This method deletes a group_snapshot. It has context, group_snapshot
|
||||
object, and a list of snapshot objects. It returns model_update and
|
||||
snapshots_model_update.
|
||||
|
||||
snapshots_model_update is a list of dictionaries. It has to be built by
|
||||
the driver. An entry will be in this format: {'id': xxx, 'status': xxx,
|
||||
......}. model_update will be in this format: {'status': xxx, ......}.
|
||||
The driver should populate snapshots_model_update and model_update
|
||||
and return them.
|
||||
|
||||
The manager will check snapshots_model_update and update db accordingly
|
||||
for each snapshot. If the driver successfully deleted some snapshots
|
||||
but failed to delete others, it should set statuses of the snapshots
|
||||
accordingly so that the manager can update db correctly.
|
||||
|
||||
If the status in any entry of snapshots_model_update is
|
||||
'error_deleting' or 'error', the status in model_update will be set to
|
||||
the same if it is not already 'error_deleting' or 'error'.
|
||||
|
||||
If the status in model_update is 'error_deleting' or 'error', the
|
||||
manager will raise an exception and the status of group_snapshot will
|
||||
be set to 'error' in the db. If snapshots_model_update is not returned
|
||||
by the driver, the manager will set the status of every snapshot to
|
||||
'error' in the except block.
|
||||
|
||||
If the driver raises an exception during the operation, it will be
|
||||
caught by the try-except block in the manager and the statuses of
|
||||
group_snapshot and all snapshots will be set to 'error'.
|
||||
|
||||
For a successful operation, the driver can either build the
|
||||
model_update and snapshots_model_update and return them or
|
||||
return None, None. The statuses of group_snapshot and all snapshots
|
||||
will be set to 'deleted' after the manager deletes them from db.
|
||||
|
||||
Migrate CGs to Generic Volume Groups
|
||||
------------------------------------
|
||||
|
||||
This section only affects drivers already supporting CGs by the
|
||||
Newton release. Drivers planning to add CG support after Newton are
|
||||
not affected.
|
||||
|
||||
A group type named default_cgsnapshot_type will be created by the
|
||||
migration script. The following command needs to be run to migrate
|
||||
migrate data and copy data from consistency groups to groups and
|
||||
from cgsnapshots to group_snapshots. Migrated consistency groups
|
||||
and cgsnapshots will be removed from the database::
|
||||
|
||||
cinder-manage db online_data_migrations
|
||||
--max_count <max>
|
||||
--ignore_state
|
||||
|
||||
max_count is optional. Default is 50.
|
||||
ignore_state is optional. Default is False.
|
||||
|
||||
After running the above migration command to migrate CGs to generic
|
||||
volume groups, CG and group APIs work as follows:
|
||||
|
||||
* Create CG only creates in the groups table.
|
||||
|
||||
* Modify CG modifies in the CG table if the CG is in the
|
||||
CG table, otherwise it modifies in the groups table.
|
||||
|
||||
* Delete CG deletes from the CG or the groups table
|
||||
depending on where the CG is.
|
||||
|
||||
* List CG checks both CG and groups tables.
|
||||
|
||||
* List CG Snapshots checks both the CG and the groups
|
||||
tables.
|
||||
|
||||
* Show CG checks both tables.
|
||||
|
||||
* Show CG Snapshot checks both tables.
|
||||
|
||||
* Create CG Snapshot creates either in the CG or the groups
|
||||
table depending on where the CG is.
|
||||
|
||||
* Create CG from Source creates in either the CG or the
|
||||
groups table depending on the source.
|
||||
|
||||
* Create Volume adds the volume either to the CG or the
|
||||
group.
|
||||
|
||||
* default_cgsnapshot_type is reserved for migrating CGs.
|
||||
|
||||
* Group APIs will only write/read in/from the groups table.
|
||||
|
||||
* Group APIs will not work on groups with default_cgsnapshot_type.
|
||||
|
||||
* Groups with default_cgsnapshot_type can only be operated by
|
||||
CG APIs.
|
||||
|
||||
* After CG tables are removed, we will allow default_cgsnapshot_type
|
||||
to be used by group APIs.
|
||||
|
||||
References
|
||||
----------
|
||||
[1] Migration script
|
||||
https://review.openstack.org/#/c/350350/
|
||||
[2] CG APIs changes for migrating CGs
|
||||
https://review.openstack.org/#/c/401839/
|
||||
[3] Example adding CG capability to generic volume groups
|
||||
https://review.openstack.org/#/c/413927/
|
||||
|
|
@ -40,6 +40,7 @@ Programming HowTos and Tutorials
|
|||
migration
|
||||
api.apache
|
||||
rolling.upgrades
|
||||
groups
|
||||
|
||||
Background Concepts for Cinder
|
||||
------------------------------
|
||||
|
|
|
|||
Loading…
Reference in New Issue