XenAPI: define a new image handler to use vdi streaming
With the new image handler, it creates an image proxy which will use the vdi streaming function from os-xenapi to remotely export VHD from XenServer(image upload) or import VHD to Xenerver(image download). The existing GlanceStore uses custom functionality to directly manipulate files on-disk, so it has the restriction that SR's type must be file system based: e.g. ext or nfs. The new image handler invokes APIs formally supported by XenServer to export/import VDI remotely, it can support other SR types also e.g. lvm, iscsi, etc. Note: vdi streaming would be supported by XenServer 6.5 or above. The function of image handler depends on os-xenapi 0.3.3 or above, so bump os-xenapi's version to 0.3.3 and also declare depends on the patch which bump version in openstack/requirements. Blueprint: xenapi-image-handler-option-improvement Change-Id: I0ad8e34808401ace9b85e1b937a542f4c4e61690 Depends-On: Ib8bc0f837c55839dc85df1d1f0c76b320b9d97b8
This commit is contained in:
parent
a7da8f257f
commit
f7593ded8f
@ -440,19 +440,30 @@ attached NFS or any other shared storage):
|
|||||||
|
|
||||||
sr_matching_filter = "default-sr:true"
|
sr_matching_filter = "default-sr:true"
|
||||||
|
|
||||||
Image upload in TGZ compressed format
|
Use different image handler
|
||||||
-------------------------------------
|
---------------------------
|
||||||
|
|
||||||
To start uploading ``tgz`` compressed raw disk images to the Image service,
|
We support three different implementations for glance image handler. You
|
||||||
configure ``xenapi_image_upload_handler`` by replacing ``GlanceStore`` with
|
can choose a specific image handler based on the demand:
|
||||||
``VdiThroughDevStore``.
|
|
||||||
|
* ``direct_vhd``: This image handler will call XAPI plugins to directly
|
||||||
|
process the VHD files in XenServer SR(Storage Repository). So this handler
|
||||||
|
only works when the host's SR type is file system based e.g. ext, nfs.
|
||||||
|
|
||||||
|
* ``vdi_local_dev``: This image handler uploads ``tgz`` compressed raw
|
||||||
|
disk images to the glance image service.
|
||||||
|
|
||||||
|
* ``vdi_remote_stream``: With this image handler, the image data streams
|
||||||
|
between XenServer and the glance image service. As it uses the remote
|
||||||
|
APIs supported by XAPI, this plugin works for all SR types supported by
|
||||||
|
XenServer.
|
||||||
|
|
||||||
|
``direct_vhd`` is the default image handler. If want to use a different image
|
||||||
|
handler, you can change the config setting of ``image_handler`` within the
|
||||||
|
``[xenserver]`` section. For example, the following config setting is to use
|
||||||
|
``vdi_remote_stream`` as the image handler:
|
||||||
|
|
||||||
.. code-block:: ini
|
.. code-block:: ini
|
||||||
|
|
||||||
xenapi_image_upload_handler=nova.virt.xenapi.image.vdi_through_dev.VdiThroughDevStore
|
[xenserver]
|
||||||
|
image_handler=vdi_remote_stream
|
||||||
As opposed to:
|
|
||||||
|
|
||||||
.. code-block:: ini
|
|
||||||
|
|
||||||
xenapi_image_upload_handler=nova.virt.xenapi.image.glance.GlanceStore
|
|
||||||
|
@ -72,7 +72,7 @@ os-service-types==1.2.0
|
|||||||
os-traits==0.4.0
|
os-traits==0.4.0
|
||||||
os-vif==1.7.0
|
os-vif==1.7.0
|
||||||
os-win==3.0.0
|
os-win==3.0.0
|
||||||
os-xenapi==0.3.1
|
os-xenapi==0.3.3
|
||||||
osc-lib==1.10.0
|
osc-lib==1.10.0
|
||||||
oslo.cache==1.26.0
|
oslo.cache==1.26.0
|
||||||
oslo.concurrency==3.26.0
|
oslo.concurrency==3.26.0
|
||||||
|
@ -472,7 +472,7 @@ GlanceStore.
|
|||||||
"""),
|
"""),
|
||||||
cfg.StrOpt('image_handler',
|
cfg.StrOpt('image_handler',
|
||||||
default='direct_vhd',
|
default='direct_vhd',
|
||||||
choices=('direct_vhd', 'vdi_local_dev'),
|
choices=('direct_vhd', 'vdi_local_dev', 'vdi_remote_stream'),
|
||||||
help="""
|
help="""
|
||||||
The plugin used to handle image uploads and downloads.
|
The plugin used to handle image uploads and downloads.
|
||||||
|
|
||||||
@ -488,6 +488,11 @@ the instance's VDI as a local disk to the VM where the OpenStack Compute
|
|||||||
service runs in. It uploads the raw disk to glance when creating image;
|
service runs in. It uploads the raw disk to glance when creating image;
|
||||||
When booting an instance from a glance image, it downloads the image and
|
When booting an instance from a glance image, it downloads the image and
|
||||||
streams it into the disk which is attached to the compute VM.
|
streams it into the disk which is attached to the compute VM.
|
||||||
|
* ``vdi_remote_stream``: This plugin implements an image handler which works
|
||||||
|
as a proxy between glance and XenServer. The VHD streams to XenServer via
|
||||||
|
a remote import API supplied by XAPI for image download; and for image
|
||||||
|
upload, the VHD streams from XenServer via a remote export API supplied
|
||||||
|
by XAPI. This plugin works for all SR types supported by XenServer.
|
||||||
"""),
|
"""),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
149
nova/tests/unit/virt/xenapi/image/test_vdi_stream.py
Normal file
149
nova/tests/unit/virt/xenapi/image/test_vdi_stream.py
Normal file
@ -0,0 +1,149 @@
|
|||||||
|
# Copyright 2017 Citrix System
|
||||||
|
# All Rights Reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||||
|
# not use this file except in compliance with the License. You may obtain
|
||||||
|
# a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||||
|
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||||
|
# License for the specific language governing permissions and limitations
|
||||||
|
# under the License.
|
||||||
|
|
||||||
|
import mock
|
||||||
|
|
||||||
|
from os_xenapi.client import exception as xenapi_except
|
||||||
|
from os_xenapi.client import image
|
||||||
|
|
||||||
|
from nova import context
|
||||||
|
from nova import exception
|
||||||
|
from nova.image.api import API as image_api
|
||||||
|
from nova.tests.unit.virt.xenapi import stubs
|
||||||
|
from nova.virt.xenapi.image import utils
|
||||||
|
from nova.virt.xenapi.image import vdi_stream
|
||||||
|
from nova.virt.xenapi import vm_utils
|
||||||
|
|
||||||
|
|
||||||
|
class TestVdiStreamStore(stubs.XenAPITestBaseNoDB):
|
||||||
|
def setUp(self):
|
||||||
|
super(TestVdiStreamStore, self).setUp()
|
||||||
|
self.store = vdi_stream.VdiStreamStore()
|
||||||
|
|
||||||
|
self.flags(connection_url='test_url',
|
||||||
|
image_compression_level=5,
|
||||||
|
group='xenserver')
|
||||||
|
|
||||||
|
self.session = mock.Mock()
|
||||||
|
self.context = context.RequestContext(
|
||||||
|
'user', 'project', auth_token='foobar')
|
||||||
|
self.instance = {'uuid': 'e6ad57c9-115e-4b7d-a872-63cea0ac3cf2',
|
||||||
|
'system_metadata': [],
|
||||||
|
'auto_disk_config': True,
|
||||||
|
'os_type': 'default',
|
||||||
|
'xenapi_use_agent': 'true'}
|
||||||
|
|
||||||
|
@mock.patch.object(image_api, 'download',
|
||||||
|
return_value='fake_data')
|
||||||
|
@mock.patch.object(utils, 'IterableToFileAdapter',
|
||||||
|
return_value='fake_stream')
|
||||||
|
@mock.patch.object(vm_utils, 'safe_find_sr',
|
||||||
|
return_value='fake_sr_ref')
|
||||||
|
@mock.patch.object(image, 'stream_to_vdis')
|
||||||
|
def test_download_image(self, stream_to, find_sr, to_file, download):
|
||||||
|
self.store.download_image(self.context, self.session,
|
||||||
|
self.instance, 'fake_image_uuid')
|
||||||
|
|
||||||
|
download.assert_called_once_with(self.context, 'fake_image_uuid')
|
||||||
|
to_file.assert_called_once_with('fake_data')
|
||||||
|
find_sr.assert_called_once_with(self.session)
|
||||||
|
stream_to.assert_called_once_with(self.context, self.session,
|
||||||
|
self.instance, 'test_url',
|
||||||
|
'fake_sr_ref', 'fake_stream')
|
||||||
|
|
||||||
|
@mock.patch.object(image_api, 'download',
|
||||||
|
return_value='fake_data')
|
||||||
|
@mock.patch.object(utils, 'IterableToFileAdapter',
|
||||||
|
return_value='fake_stream')
|
||||||
|
@mock.patch.object(vm_utils, 'safe_find_sr',
|
||||||
|
return_value='fake_sr_ref')
|
||||||
|
@mock.patch.object(image, 'stream_to_vdis',
|
||||||
|
side_effect=xenapi_except.OsXenApiException)
|
||||||
|
def test_download_image_exception(self, stream_to, find_sr, to_file,
|
||||||
|
download):
|
||||||
|
self.assertRaises(exception.CouldNotFetchImage,
|
||||||
|
self.store.download_image,
|
||||||
|
self.context, self.session,
|
||||||
|
self.instance, 'fake_image_uuid')
|
||||||
|
|
||||||
|
@mock.patch.object(vdi_stream.VdiStreamStore, '_get_metadata',
|
||||||
|
return_value='fake_meta_data')
|
||||||
|
@mock.patch.object(image, 'stream_from_vdis',
|
||||||
|
return_value='fake_data')
|
||||||
|
@mock.patch.object(utils, 'IterableToFileAdapter',
|
||||||
|
return_value='fake_stream')
|
||||||
|
@mock.patch.object(image_api, 'update')
|
||||||
|
def test_upload_image(self, update, to_file, to_stream, get):
|
||||||
|
fake_vdi_uuids = ['fake-vdi-uuid']
|
||||||
|
self.store.upload_image(self.context, self.session,
|
||||||
|
self.instance, 'fake_image_uuid',
|
||||||
|
fake_vdi_uuids)
|
||||||
|
|
||||||
|
get.assert_called_once_with(self.context, self.instance,
|
||||||
|
'fake_image_uuid')
|
||||||
|
to_stream.assert_called_once_with(self.context, self.session,
|
||||||
|
self.instance, 'test_url',
|
||||||
|
fake_vdi_uuids, compresslevel=5)
|
||||||
|
to_file.assert_called_once_with('fake_data')
|
||||||
|
update.assert_called_once_with(self.context, 'fake_image_uuid',
|
||||||
|
'fake_meta_data', data='fake_stream')
|
||||||
|
|
||||||
|
@mock.patch.object(vdi_stream.VdiStreamStore, '_get_metadata')
|
||||||
|
@mock.patch.object(image, 'stream_from_vdis',
|
||||||
|
side_effect=xenapi_except.OsXenApiException)
|
||||||
|
@mock.patch.object(utils, 'IterableToFileAdapter',
|
||||||
|
return_value='fake_stream')
|
||||||
|
@mock.patch.object(image_api, 'update')
|
||||||
|
def test_upload_image_exception(self, update, to_file, to_stream, get):
|
||||||
|
fake_vdi_uuids = ['fake-vdi-uuid']
|
||||||
|
self.assertRaises(exception.CouldNotUploadImage,
|
||||||
|
self.store.upload_image,
|
||||||
|
self.context, self.session,
|
||||||
|
self.instance, 'fake_image_uuid',
|
||||||
|
fake_vdi_uuids)
|
||||||
|
|
||||||
|
@mock.patch.object(image_api, 'get',
|
||||||
|
return_value={})
|
||||||
|
def test_get_metadata(self, image_get):
|
||||||
|
expect_metadata = {'disk_format': 'vhd',
|
||||||
|
'container_format': 'ovf',
|
||||||
|
'auto_disk_config': 'True',
|
||||||
|
'os_type': 'default',
|
||||||
|
'size': 0}
|
||||||
|
|
||||||
|
result = self.store._get_metadata(self.context, self.instance,
|
||||||
|
'fake_image_uuid')
|
||||||
|
|
||||||
|
self.assertEqual(result, expect_metadata)
|
||||||
|
|
||||||
|
@mock.patch.object(image_api, 'get',
|
||||||
|
return_value={})
|
||||||
|
def test_get_metadata_disabled(self, image_get):
|
||||||
|
# Verify the metadata contains auto_disk_config=disabled, when
|
||||||
|
# image_auto_disk_config is ""Disabled".
|
||||||
|
self.instance['system_metadata'] = [
|
||||||
|
{"key": "image_auto_disk_config",
|
||||||
|
"value": "Disabled"}]
|
||||||
|
|
||||||
|
expect_metadata = {'disk_format': 'vhd',
|
||||||
|
'container_format': 'ovf',
|
||||||
|
'auto_disk_config': 'disabled',
|
||||||
|
'os_type': 'default',
|
||||||
|
'size': 0}
|
||||||
|
|
||||||
|
result = self.store._get_metadata(self.context, self.instance,
|
||||||
|
'fake_image_uuid')
|
||||||
|
|
||||||
|
self.assertEqual(result, expect_metadata)
|
@ -25,7 +25,8 @@ _VDI_FORMAT_RAW = 1
|
|||||||
|
|
||||||
IMAGE_API = image.API()
|
IMAGE_API = image.API()
|
||||||
IMAGE_HANDLERS = {'direct_vhd': 'glance.GlanceStore',
|
IMAGE_HANDLERS = {'direct_vhd': 'glance.GlanceStore',
|
||||||
'vdi_local_dev': 'vdi_through_dev.VdiThroughDevStore'}
|
'vdi_local_dev': 'vdi_through_dev.VdiThroughDevStore',
|
||||||
|
'vdi_remote_stream': 'vdi_stream.VdiStreamStore'}
|
||||||
|
|
||||||
|
|
||||||
def get_image_handler(handler_name):
|
def get_image_handler(handler_name):
|
||||||
|
85
nova/virt/xenapi/image/vdi_stream.py
Normal file
85
nova/virt/xenapi/image/vdi_stream.py
Normal file
@ -0,0 +1,85 @@
|
|||||||
|
# Copyright 2017 Citrix Systems
|
||||||
|
# All Rights Reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||||
|
# not use this file except in compliance with the License. You may obtain
|
||||||
|
# a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||||
|
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||||
|
# License for the specific language governing permissions and limitations
|
||||||
|
# under the License.
|
||||||
|
|
||||||
|
""" This class will stream image data directly between glance and VDI.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from os_xenapi.client import exception as xenapi_exception
|
||||||
|
from os_xenapi.client import image as xenapi_image
|
||||||
|
from oslo_log import log as logging
|
||||||
|
|
||||||
|
import nova.conf
|
||||||
|
from nova import exception
|
||||||
|
from nova import image
|
||||||
|
from nova import utils as nova_utils
|
||||||
|
from nova.virt.xenapi.image import utils
|
||||||
|
from nova.virt.xenapi import vm_utils
|
||||||
|
|
||||||
|
CONF = nova.conf.CONF
|
||||||
|
LOG = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
IMAGE_API = image.API()
|
||||||
|
|
||||||
|
|
||||||
|
class VdiStreamStore(object):
|
||||||
|
def download_image(self, context, session, instance, image_id):
|
||||||
|
try:
|
||||||
|
host_url = CONF.xenserver.connection_url
|
||||||
|
image_data = IMAGE_API.download(context, image_id)
|
||||||
|
image_stream = utils.IterableToFileAdapter(image_data)
|
||||||
|
sr_ref = vm_utils.safe_find_sr(session)
|
||||||
|
vdis = xenapi_image.stream_to_vdis(context, session,
|
||||||
|
instance, host_url,
|
||||||
|
sr_ref, image_stream)
|
||||||
|
except xenapi_exception.OsXenApiException as e:
|
||||||
|
LOG.error("Image download failed with exception: %s", e)
|
||||||
|
raise exception.CouldNotFetchImage(image_id=image_id)
|
||||||
|
return vdis
|
||||||
|
|
||||||
|
def _get_metadata(self, context, instance, image_id):
|
||||||
|
metadata = IMAGE_API.get(context, image_id)
|
||||||
|
metadata['disk_format'] = 'vhd'
|
||||||
|
metadata['container_format'] = 'ovf'
|
||||||
|
metadata['auto_disk_config'] = str(instance['auto_disk_config'])
|
||||||
|
metadata['os_type'] = instance.get('os_type') or (
|
||||||
|
CONF.xenserver.default_os_type)
|
||||||
|
# Set size as zero, so that it will update the size in the end
|
||||||
|
# based on the uploaded image data.
|
||||||
|
metadata['size'] = 0
|
||||||
|
|
||||||
|
# Adjust the auto_disk_config value basing on instance's
|
||||||
|
# system metadata.
|
||||||
|
# TODO(mriedem): Consider adding an abstract base class for the
|
||||||
|
# various image handlers to contain common code like this.
|
||||||
|
auto_disk = nova_utils.get_auto_disk_config_from_instance(instance)
|
||||||
|
if nova_utils.is_auto_disk_config_disabled(auto_disk):
|
||||||
|
metadata['auto_disk_config'] = "disabled"
|
||||||
|
|
||||||
|
return metadata
|
||||||
|
|
||||||
|
def upload_image(self, context, session, instance, image_id, vdi_uuids):
|
||||||
|
try:
|
||||||
|
host_url = CONF.xenserver.connection_url
|
||||||
|
level = vm_utils.get_compression_level()
|
||||||
|
metadata = self._get_metadata(context, instance, image_id)
|
||||||
|
image_chunks = xenapi_image.stream_from_vdis(
|
||||||
|
context, session, instance, host_url, vdi_uuids,
|
||||||
|
compresslevel=level)
|
||||||
|
image_stream = utils.IterableToFileAdapter(image_chunks)
|
||||||
|
IMAGE_API.update(context, image_id, metadata,
|
||||||
|
data=image_stream)
|
||||||
|
except xenapi_exception.OsXenApiException as e:
|
||||||
|
LOG.error("Image upload failed with exception: %s", e)
|
||||||
|
raise exception.CouldNotUploadImage(image_id=image_id)
|
@ -23,6 +23,24 @@ features:
|
|||||||
an instance from a glance image, it downloads the image and streams it
|
an instance from a glance image, it downloads the image and streams it
|
||||||
into the disk which is attached to the compute VM.
|
into the disk which is attached to the compute VM.
|
||||||
|
|
||||||
|
* ``vdi_remote_stream``
|
||||||
|
|
||||||
|
This plugin implements an image proxy in nova compute service.
|
||||||
|
|
||||||
|
For image upload, the proxy will export a data stream for a VDI from
|
||||||
|
XenServer via the remote API supplied by XAPI; convert the stream
|
||||||
|
to the image format supported by glance; and upload the image to glance.
|
||||||
|
|
||||||
|
For image download, the proxy downloads an image stream from glance;
|
||||||
|
extracts the data stream from the image stream; and then remotely
|
||||||
|
imports the data stream to XenServer's VDI via the remote API supplied
|
||||||
|
by XAPI.
|
||||||
|
|
||||||
|
Note: Under this implementation, the image data may reside in one or
|
||||||
|
more pieces of storage of various formats on the host, but the import
|
||||||
|
and export operations interact with a single, proxied VDI object
|
||||||
|
independent of the underlying structure.
|
||||||
|
|
||||||
deprecations:
|
deprecations:
|
||||||
- |
|
- |
|
||||||
The ``image_upload_handler`` option in the ``xenserver`` conf section
|
The ``image_upload_handler`` option in the ``xenserver`` conf section
|
||||||
|
@ -58,7 +58,7 @@ os-vif!=1.8.0,>=1.7.0 # Apache-2.0
|
|||||||
os-win>=3.0.0 # Apache-2.0
|
os-win>=3.0.0 # Apache-2.0
|
||||||
castellan>=0.16.0 # Apache-2.0
|
castellan>=0.16.0 # Apache-2.0
|
||||||
microversion-parse>=0.2.1 # Apache-2.0
|
microversion-parse>=0.2.1 # Apache-2.0
|
||||||
os-xenapi>=0.3.1 # Apache-2.0
|
os-xenapi>=0.3.3 # Apache-2.0
|
||||||
tooz>=1.58.0 # Apache-2.0
|
tooz>=1.58.0 # Apache-2.0
|
||||||
cursive>=0.2.1 # Apache-2.0
|
cursive>=0.2.1 # Apache-2.0
|
||||||
pypowervm>=1.1.15 # Apache-2.0
|
pypowervm>=1.1.15 # Apache-2.0
|
||||||
|
Loading…
Reference in New Issue
Block a user