Add ability to extend 'in-use' volume

This change adds the ability to extend 'in-use' volume.

Once the volume size is extended, Nova is informed of the size change
through the external-event extension so the virt driver can perform
the appropriate actions for the host and guest to detect the new volume size.

Tempest related patches:

1. https://review.openstack.org/#/c/480746/
2. https://review.openstack.org/#/c/480778/

Depends-On: If10cffd0dc4c9879f6754ce39bee5fae1d04f474

Blueprint: extend-attached-volume
Co-Authored-By: TommyLike <tommylikehu@gmail.com>
APIImpact
Change-Id: I60c8ea9eb0bbcfe41f5f0a30ed8dc67bdcab3ebc
This commit is contained in:
Mathieu Gagné 2017-04-05 12:40:03 -04:00 committed by Matt Riedemann
parent 431d356c8a
commit 3dd842de82
14 changed files with 332 additions and 45 deletions

View File

@ -1483,6 +1483,16 @@ namespace_1:
new_size:
description: |
The new size of the volume, in gibibytes (GiB).
.. note:: Some volume backends require the storage to be in some multiple
value rather than incremental. For example, the EMC ScaleIO backend
requires storage in multiples of 8GB. There is a known limitation such
that a request to extend the size of a volume for these backends will be
rounded up to the nearest multiple but the actual physical size of the
storage will not be reflected back in the API for the volume size. For
example, a request to extend the size of an 8GB ScaleIO-backed volume
to 9GB will actually result in 16GB of physical storage but only 9GB will
be reflected in the API and counted for quota usage.
in: body
required: true
type: integer

View File

@ -15,16 +15,36 @@ Extend a volume size
.. rest_method:: POST /v3/{project_id}/volumes/{volume_id}/action
Extends the size of a volume to a requested size, in gibibytes (GiB). Specify the ``os-extend`` action in the request body.
Extends the size of a volume to a requested size, in gibibytes (GiB).
Specify the ``os-extend`` action in the request body.
Preconditions
- Volume status must be ``available``.
- Prior to microversion ``3.42`` the volume status must be ``available``.
Starting with microversion ``3.42``, attached volumes with status ``in-use``
may be able to be extended depending on policy and backend volume and
compute driver constraints in the cloud. Note that ``reserved`` is not a
valid state for extend.
- Sufficient amount of storage must exist to extend the volume.
- The user quota must have sufficient volume storage.
Postconditions
- If the request is processed successfully, the volume status will change to
``extending`` while the volume size is being extended.
- Upon successful completion of the extend operation, the volume status will
go back to its original value.
- Starting with microversion ``3.42``, when extending the size of an attached
volume, the Block Storage service will notify the Compute service that an
attached volume has been extended. The Compute service will asynchronously
process the volume size change for the related server instance. This can be
monitored using the ``GET /servers/{server_id}/os-instance-actions`` API in
the Compute service.
Troubleshooting
- An ``error_extending`` volume status indicates that the request

View File

@ -303,10 +303,12 @@ class VolumeActionsController(wsgi.Controller):
raise webob.exc.HTTPBadRequest(explanation=six.text_type(error))
return {'os-volume_upload_image': response}
@wsgi.response(http_client.ACCEPTED)
@wsgi.action('os-extend')
def _extend(self, req, id, body):
"""Extend size of volume."""
context = req.environ['cinder.context']
req_version = req.api_version_request
# Not found exception will be handled at the wsgi level
volume = self.volume_api.get(context, id)
@ -317,12 +319,13 @@ class VolumeActionsController(wsgi.Controller):
raise webob.exc.HTTPBadRequest(explanation=msg)
try:
self.volume_api.extend(context, volume, size)
if req_version.matches("3.42") and volume.status in ['in-use']:
self.volume_api.extend_attached_volume(context, volume, size)
else:
self.volume_api.extend(context, volume, size)
except exception.InvalidVolume as error:
raise webob.exc.HTTPBadRequest(explanation=error.msg)
return webob.Response(status_int=http_client.ACCEPTED)
@wsgi.action('os-update_readonly_flag')
def _volume_readonly_update(self, req, id, body):
"""Update volume readonly flag."""

View File

@ -95,6 +95,16 @@ REST_API_VERSION_HISTORY = """
* 3.39 - Add ``project_id`` admin filters support to limits.
* 3.40 - Add volume revert to its latest snapshot support.
* 3.41 - Add ``user_id`` field to snapshot list/detail and snapshot show.
* 3.42 - Add ability to extend 'in-use' volume. User should be aware of the
whole environment before using this feature because it's dependent
on several external factors below:
1. nova-compute version - needs to be the latest for Pike.
2. only the libvirt compute driver supports this currently.
3. only iscsi and fibre channel volume types are supported
on the nova side currently.
Administrator can disable this ability by updating the
'volume:extend_attached_volume' policy rule. Extend in reserved
state is intentionally NOT allowed.
"""
# The minimum and maximum versions of the API supported
@ -102,7 +112,7 @@ REST_API_VERSION_HISTORY = """
# minimum version of the API supported.
# Explicitly using /v1 or /v2 endpoints will still work
_MIN_API_VERSION = "3.0"
_MAX_API_VERSION = "3.41"
_MAX_API_VERSION = "3.42"
_LEGACY_API_VERSION1 = "1.0"
_LEGACY_API_VERSION2 = "2.0"

View File

@ -342,3 +342,18 @@ user documentation.
3.41
----
Add ``user_id`` field to snapshot list/detail and snapshot show.
3.42
----
Add ability to extend 'in-use' volume. User should be aware of the
whole environment before using this feature because it's dependent
on several external factors below:
1. nova-compute version - needs to be the latest for Pike.
2. only the libvirt compute driver supports this currently.
3. only iscsi and fibre channel volume types are supported on the
nova side currently.
Administrator can disable this ability by updating the
``volume:extend_attached_volume`` policy rule. Extend of a resered
Volume is NOT allowed.

View File

@ -88,7 +88,8 @@ NOVA_API_VERSION = "2.1"
nova_extensions = [ext for ext in
nova_client.discover_extensions(NOVA_API_VERSION)
if ext.name in ("assisted_volume_snapshots",
"list_extensions")]
"list_extensions",
"server_external_events")]
def _get_identity_endpoint_from_sc(context):
@ -103,7 +104,7 @@ def _get_identity_endpoint_from_sc(context):
raise nova_exceptions.EndpointNotFound()
def novaclient(context, privileged_user=False, timeout=None):
def novaclient(context, privileged_user=False, timeout=None, api_version=None):
"""Returns a Nova client
@param privileged_user: If True, use the account from configuration
@ -111,6 +112,7 @@ def novaclient(context, privileged_user=False, timeout=None):
options to be set in the [nova] section)
@param timeout: Number of seconds to wait for an answer before raising a
Timeout exception (None to disable)
@param api_version: api version of nova
"""
if privileged_user and CONF[NOVA_GROUP].auth_type:
@ -148,15 +150,16 @@ def novaclient(context, privileged_user=False, timeout=None):
NOVA_GROUP,
auth=n_auth)
c = nova_client.Client(api_versions.APIVersion(NOVA_API_VERSION),
session=keystone_session,
insecure=CONF[NOVA_GROUP].insecure,
timeout=timeout,
region_name=CONF[NOVA_GROUP].region_name,
endpoint_type=CONF[NOVA_GROUP].interface,
cacert=CONF[NOVA_GROUP].cafile,
global_request_id=context.global_id,
extensions=nova_extensions)
c = nova_client.Client(
api_versions.APIVersion(api_version or NOVA_API_VERSION),
session=keystone_session,
insecure=CONF[NOVA_GROUP].insecure,
timeout=timeout,
region_name=CONF[NOVA_GROUP].region_name,
endpoint_type=CONF[NOVA_GROUP].interface,
cacert=CONF[NOVA_GROUP].cafile,
global_request_id=context.global_id,
extensions=nova_extensions)
return c
@ -164,6 +167,38 @@ def novaclient(context, privileged_user=False, timeout=None):
class API(base.Base):
"""API for interacting with novaclient."""
def _get_volume_extended_event(self, server_id, volume_id):
return {'name': 'volume-extended',
'server_uuid': server_id,
'tag': volume_id}
def _send_events(self, context, events, api_version=None):
nova = novaclient(context, privileged_user=True,
api_version=api_version)
try:
response = nova.server_external_events.create(events)
except nova_exceptions.NotFound:
LOG.warning('Nova returned NotFound for events: %s.', events)
except Exception:
LOG.exception('Failed to notify nova on events: %s.', events)
else:
if not isinstance(response, list):
LOG.error('Error response returned from nova: %s.', response)
return
response_error = False
for event in response:
code = event.get('code')
if code is None:
response_error = True
continue
if code != 200:
LOG.warning(
'Nova event: %s returned with failed status.', event)
else:
LOG.info('Nova event response: %s.', event)
if response_error:
LOG.error('Error response returned from nova: %s.', response)
def has_extension(self, context, extension, timeout=None):
try:
nova_exts = novaclient(context).list_extensions.show_all()
@ -203,3 +238,9 @@ class API(base.Base):
raise exception.ServerNotFound(uuid=server_id)
except request_exceptions.Timeout:
raise exception.APITimeout(service='Nova')
def extend_volume(self, context, server_ids, volume_id):
api_version = '2.51'
events = [self._get_volume_extended_event(server_id, volume_id)
for server_id in server_ids]
self._send_events(context, events, api_version=api_version)

View File

@ -780,6 +780,7 @@ def fake_upload_volume_to_image_service(self, context, volume, metadata,
return ret
@ddt.ddt
class VolumeImageActionsTest(test.TestCase):
def setUp(self):
super(VolumeImageActionsTest, self).setUp()
@ -996,6 +997,33 @@ class VolumeImageActionsTest(test.TestCase):
id,
body)
@ddt.data({'version': '3.41',
'status': 'available'},
{'version': '3.41',
'status': 'in-use'},
{'version': '3.42',
'status': 'available'},
{'version': '3.42',
'status': 'in-use'})
@ddt.unpack
def test_extend_attached_volume(self, version, status):
vol = db.volume_create(self.context,
{'size': 1, 'project_id': fake.PROJECT_ID,
'status': status})
self.mock_object(volume_api.API, 'get', return_value=vol)
mock_extend = self.mock_object(volume_api.API, '_extend')
body = {"os-extend": {"new_size": 2}}
req = fakes.HTTPRequest.blank('/v3/%s/volumes/%s/action' %
(fake.PROJECT_ID, vol['id']))
req.api_version_request = api_version.APIVersionRequest(version)
self.controller._extend(req, vol['id'], body)
if version == '3.42' and status == 'in-use':
mock_extend.assert_called_with(req.environ['cinder.context'],
vol, 2, attached=True)
else:
mock_extend.assert_called_with(req.environ['cinder.context'],
vol, 2, attached=False)
def test_copy_volume_to_image_notimagename(self):
id = fake.VOLUME2_ID
vol = {"container_format": 'bare',

View File

@ -184,11 +184,16 @@ class NovaClientTestCase(test.TestCase):
class FakeNovaClient(object):
class ServerExternalEvents(object):
def __getattr__(self, item):
return None
class Volumes(object):
def __getattr__(self, item):
return None
def __init__(self):
self.server_external_events = self.ServerExternalEvents()
self.volumes = self.Volumes()
def create_volume_snapshot(self, *args, **kwargs):
@ -223,3 +228,24 @@ class NovaApiTestCase(test.TestCase):
'attach_id',
'new_volume_id'
)
def test_extend_volume(self):
server_ids = ['server-id-1', 'server-id-2']
with mock.patch.object(nova, 'novaclient') as mock_novaclient, \
mock.patch.object(self.novaclient.server_external_events,
'create') as mock_create_event:
mock_novaclient.return_value = self.novaclient
self.api.extend_volume(self.ctx, server_ids, 'volume_id')
mock_novaclient.assert_called_once_with(self.ctx,
privileged_user=True,
api_version='2.51')
mock_create_event.assert_called_once_with([
{'name': 'volume-extended',
'server_uuid': 'server-id-1',
'tag': 'volume_id'},
{'name': 'volume-extended',
'server_uuid': 'server-id-2',
'tag': 'volume_id'},
])

View File

@ -34,6 +34,7 @@
"volume:delete_snapshot_metadata": "",
"volume:update_snapshot_metadata": "",
"volume:extend": "",
"volume:extend_attached_volume": "",
"volume:migrate_volume": "rule:admin_api",
"volume:migrate_volume_completion": "rule:admin_api",
"volume:update_readonly_flag": "",

View File

@ -2094,6 +2094,42 @@ class VolumeTestCase(base.BaseVolumeTestCase):
'fake2': {'key3': 'value3', 'key4': 'value4'}}
self.assertEqual(expect_results, results)
@mock.patch.object(QUOTAS, 'limit_check')
@mock.patch.object(QUOTAS, 'reserve')
def test_extend_attached_volume(self, reserve, limit_check):
volume = tests_utils.create_volume(self.context, size=2,
status='available', host=CONF.host)
volume_api = cinder.volume.api.API()
self.assertRaises(exception.InvalidVolume,
volume_api._extend,
self.context,
volume, 3, attached=True)
db.volume_update(self.context, volume.id, {'status': 'in-use'})
reserve.return_value = ["RESERVATION"]
volume_api._extend(self.context, volume, 3, attached=True)
volume.refresh()
self.assertEqual('extending', volume.status)
reserve.assert_called_once_with(self.context, gigabytes=1,
project_id=volume.project_id)
limit_check.side_effect = None
reserve.side_effect = None
db.volume_update(self.context, volume.id, {'status': 'in-use'})
volume_api.scheduler_rpcapi = mock.MagicMock()
volume_api.scheduler_rpcapi.extend_volume = mock.MagicMock()
volume_api._extend(self.context, volume, 3, attached=True)
request_spec = {
'volume_properties': volume,
'volume_type': {},
'volume_id': volume.id
}
volume_api.scheduler_rpcapi.extend_volume.assert_called_once_with(
self.context, volume, 3, ["RESERVATION"], request_spec)
# clean up
self.volume.delete_volume(self.context, volume)
@mock.patch.object(QUOTAS, 'limit_check')
@mock.patch.object(QUOTAS, 'reserve')
def test_extend_volume(self, reserve, limit_check):
@ -2105,7 +2141,7 @@ class VolumeTestCase(base.BaseVolumeTestCase):
# Extend fails when status != available
self.assertRaises(exception.InvalidVolume,
volume_api.extend,
volume_api._extend,
self.context,
volume,
3)
@ -2113,21 +2149,21 @@ class VolumeTestCase(base.BaseVolumeTestCase):
db.volume_update(self.context, volume.id, {'status': 'available'})
# Extend fails when new_size < orig_size
self.assertRaises(exception.InvalidInput,
volume_api.extend,
volume_api._extend,
self.context,
volume,
1)
# Extend fails when new_size == orig_size
self.assertRaises(exception.InvalidInput,
volume_api.extend,
volume_api._extend,
self.context,
volume,
2)
# works when new_size > orig_size
reserve.return_value = ["RESERVATION"]
volume_api.extend(self.context, volume, 3)
volume_api._extend(self.context, volume, 3)
volume.refresh()
self.assertEqual('extending', volume.status)
reserve.assert_called_once_with(self.context, gigabytes=1,
@ -2141,13 +2177,14 @@ class VolumeTestCase(base.BaseVolumeTestCase):
{'reserved': 5,
'in_use': 15}})
self.assertRaises(exception.VolumeSizeExceedsAvailableQuota,
volume_api.extend, self.context,
volume_api._extend, self.context,
volume, 3)
db.volume_update(self.context, volume.id, {'status': 'available'})
limit_check.side_effect = exception.OverQuota(
overs=['per_volume_gigabytes'], quotas={'per_volume_gigabytes': 2})
self.assertRaises(exception.VolumeSizeExceedsLimit,
volume_api.extend, self.context,
volume_api._extend, self.context,
volume, 3)
# Test scheduler path
@ -2157,7 +2194,7 @@ class VolumeTestCase(base.BaseVolumeTestCase):
volume_api.scheduler_rpcapi = mock.MagicMock()
volume_api.scheduler_rpcapi.extend_volume = mock.MagicMock()
volume_api.extend(self.context, volume, 3)
volume_api._extend(self.context, volume, 3)
request_spec = {
'volume_properties': volume,
@ -2193,15 +2230,8 @@ class VolumeTestCase(base.BaseVolumeTestCase):
self.volume.driver._initialized = True
self.volume.delete_volume(self.context, volume)
def test_extend_volume_manager(self):
"""Test volume can be extended at the manager level."""
def fake_extend(volume, new_size):
volume['size'] = new_size
def _test_extend_volume_manager_fails_with_exception(self, volume):
fake_reservations = ['RESERVATION']
volume = tests_utils.create_volume(self.context, size=2,
status='creating', host=CONF.host)
self.volume.create_volume(self.context, volume)
# Test driver exception
with mock.patch.object(self.volume.driver,
@ -2215,6 +2245,16 @@ class VolumeTestCase(base.BaseVolumeTestCase):
self.assertEqual(2, volume.size)
self.assertEqual('error_extending', volume.status)
@mock.patch('cinder.compute.API')
def _test_extend_volume_manager_successful(self, volume, nova_api):
"""Test volume can be extended at the manager level."""
def fake_extend(volume, new_size):
volume['size'] = new_size
nova_extend_volume = nova_api.return_value.extend_volume
fake_reservations = ['RESERVATION']
orig_status = volume.status
# Test driver success
with mock.patch.object(self.volume.driver,
'extend_volume') as extend_volume:
@ -2225,13 +2265,60 @@ class VolumeTestCase(base.BaseVolumeTestCase):
fake_reservations)
volume.refresh()
self.assertEqual(4, volume.size)
self.assertEqual('available', volume.status)
self.assertEqual(orig_status, volume.status)
quotas_commit.assert_called_with(
self.context,
['RESERVATION'],
project_id=volume.project_id)
if orig_status == 'in-use':
instance_uuids = [
attachment.instance_uuid
for attachment in volume.volume_attachment]
nova_extend_volume.assert_called_with(
self.context, instance_uuids, volume.id)
# clean up
def test_extend_volume_manager_available_fails_with_exception(self):
volume = tests_utils.create_volume(self.context, size=2,
status='creating', host=CONF.host)
self.volume.create_volume(self.context, volume)
self._test_extend_volume_manager_fails_with_exception(volume)
self.volume.delete_volume(self.context, volume)
def test_extend_volume_manager_available_successful(self):
volume = tests_utils.create_volume(self.context, size=2,
status='creating', host=CONF.host)
self.volume.create_volume(self.context, volume)
self._test_extend_volume_manager_successful(volume)
self.volume.delete_volume(self.context, volume)
def test_extend_volume_manager_in_use_fails_with_exception(self):
volume = tests_utils.create_volume(self.context, size=2,
status='creating', host=CONF.host)
self.volume.create_volume(self.context, volume)
instance_uuid = '12345678-1234-5678-1234-567812345678'
attachment = db.volume_attach(self.context,
{'volume_id': volume.id,
'attached_host': 'fake-host'})
db.volume_attached(self.context, attachment.id, instance_uuid,
'fake-host', 'vdb')
volume.refresh()
self._test_extend_volume_manager_fails_with_exception(volume)
self.volume.detach_volume(self.context, volume.id, attachment.id)
self.volume.delete_volume(self.context, volume)
def test_extend_volume_manager_in_use_successful(self):
volume = tests_utils.create_volume(self.context, size=2,
status='creating', host=CONF.host)
self.volume.create_volume(self.context, volume)
instance_uuid = '12345678-1234-5678-1234-567812345678'
attachment = db.volume_attach(self.context,
{'volume_id': volume.id,
'attached_host': 'fake-host'})
db.volume_attached(self.context, attachment.id, instance_uuid,
'fake-host', 'vdb')
volume.refresh()
self._test_extend_volume_manager_successful(volume)
self.volume.detach_volume(self.context, volume.id, attachment.id)
self.volume.delete_volume(self.context, volume)
@mock.patch('cinder.volume.rpcapi.VolumeAPI.extend_volume')
@ -2252,7 +2339,7 @@ class VolumeTestCase(base.BaseVolumeTestCase):
self.assertEqual(100, volumes_in_use)
db.volume_update(self.context, volume.id, {'status': 'available'})
volume_api.extend(self.context, volume, 200)
volume_api._extend(self.context, volume, 200)
mock_rpc_extend.called_once_with(self.context, volume, 200, mock.ANY)
try:

View File

@ -1289,18 +1289,22 @@ class API(base.Base):
resource=volume)
return response
@wrap_check_policy
def extend(self, context, volume, new_size):
def _extend(self, context, volume, new_size, attached=False):
value = {'status': 'extending'}
expected = {'status': 'available'}
if attached:
expected = {'status': 'in-use'}
else:
expected = {'status': 'available'}
orig_status = {'status': volume.status}
def _roll_back_status():
msg = _('Could not return volume %s to available.')
status = orig_status['status']
msg = _('Could not return volume %(id)s to %(status)s.')
try:
if not volume.conditional_update(expected, value):
LOG.error(msg, volume.id)
if not volume.conditional_update(orig_status, value):
LOG.error(msg, {'id': volume.id, 'status': status})
except Exception:
LOG.exception(msg, volume.id)
LOG.exception(msg, {'id': volume.id, 'status': status})
size_increase = (int(new_size)) - volume.size
if size_increase <= 0:
@ -1312,8 +1316,11 @@ class API(base.Base):
result = volume.conditional_update(value, expected)
if not result:
msg = _('Volume %(vol_id)s status must be available '
'to extend.') % {'vol_id': volume.id}
msg = (_("Volume %(vol_id)s status must be '%(expected)s' "
"to extend, currently %(status)s.")
% {'vol_id': volume.id,
'status': volume.status,
'expected': six.text_type(expected)})
raise exception.InvalidVolume(reason=msg)
rollback = True
@ -1379,6 +1386,17 @@ class API(base.Base):
LOG.info("Extend volume request issued successfully.",
resource=volume)
@wrap_check_policy
def extend(self, context, volume, new_size):
self._extend(context, volume, new_size, attached=False)
# NOTE(tommylikehu): New method is added here so that administrator
# can enable/disable this ability by editing the policy file if the
# cloud environment doesn't allow this operation.
@wrap_check_policy
def extend_attached_volume(self, context, volume, new_size):
self._extend(context, volume, new_size, attached=True)
@wrap_check_policy
def migrate_volume(self, context, volume, host, cluster_name, force_copy,
lock_volume):

View File

@ -2549,8 +2549,22 @@ class VolumeManager(manager.CleanableManager,
return
QUOTAS.commit(context, reservations, project_id=project_id)
volume.update({'size': int(new_size), 'status': 'available'})
attachments = volume.volume_attachment
if not attachments:
orig_volume_status = 'available'
else:
orig_volume_status = 'in-use'
volume.update({'size': int(new_size), 'status': orig_volume_status})
volume.save()
if orig_volume_status == 'in-use':
nova_api = compute.API()
instance_uuids = [attachment.instance_uuid
for attachment in attachments]
nova_api.extend_volume(context, instance_uuids, volume.id)
pool = vol_utils.extract_host(volume.host, 'pool')
if pool is None:
# Legacy volume, put them into default pool

View File

@ -25,6 +25,7 @@
"volume:delete_snapshot_metadata": "rule:admin_or_owner",
"volume:update_snapshot_metadata": "rule:admin_or_owner",
"volume:extend": "rule:admin_or_owner",
"volume:extend_attached_volume": "rule:admin_or_owner",
"volume:update_readonly_flag": "rule:admin_or_owner",
"volume:retype": "rule:admin_or_owner",
"volume:update": "rule:admin_or_owner",

View File

@ -0,0 +1,13 @@
---
features:
- |
Add ability to extend ``in-use`` volume. User should be aware of the
whole environment before using this feature because it's dependent
on several external factors below:
* nova-compute version - needs to be the latest for Pike.
* only the libvirt compute driver supports this currently.
* only iscsi and fibre channel volume types are supported on the nova side currently.
Administrator can disable this ability by updating the
``volume:extend_attached_volume`` policy rule.