Allow unshelve to a specific host (REST API part)

This adds support to the REST API, in a new microversion, for specifying
a destination host to unshelve server action when the server
is shelved offloaded.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.

Note that the functional test changes are due to those tests using the
"latest" microversion 2.91.

Implements: blueprint unshelve-to-host
Change-Id: I9e95428c208582741e6cd99bd3260d6742fcc6b7
This commit is contained in:
René Ribaud 2022-06-13 15:22:43 +02:00
parent a263fa46f8
commit 09239fc2ea
30 changed files with 1318 additions and 57 deletions

View File

@ -1858,8 +1858,11 @@ availability_zone_state:
availability_zone_unshelve:
description: |
The availability zone name. Specifying an availability zone is only
allowed when the server status is ``SHELVED_OFFLOADED`` otherwise a
409 HTTPConflict response is returned.
allowed when the server status is ``SHELVED_OFFLOADED`` otherwise
HTTP 409 conflict response is returned.
Since microversion 2.91 ``"availability_zone":null`` allows unpinning the
instance from any availability_zone it is pinned to.
in: body
required: false
type: string
@ -3690,6 +3693,15 @@ host_status_update_rebuild:
required: false
type: string
min_version: 2.75
host_unshelve:
description: |
The destination host name. Specifying a destination host is by default only
allowed to project_admin, if it not the case HTTP 403 forbidden response
is returned.
in: body
required: false
type: string
min_version: 2.91
host_zone:
description: |
The available zone of the host.

View File

@ -121,9 +121,65 @@ Policy defaults enable only users with the administrative role or the owner of t
**Preconditions**
The server status must be ``SHELVED`` or ``SHELVED_OFFLOADED``.
Unshelving a server without parameters requires its status to be ``SHELVED`` or ``SHELVED_OFFLOADED``.
Unshelving a server with availability_zone and/or host parameters requires its status to be only ``SHELVED_OFFLOADED`` otherwise HTTP 409 conflict response is returned.
If a server is locked, you must have administrator privileges to unshelve the server.
As of ``microversion 2.91``, you can unshelve to a specific compute node if you have PROJECT_ADMIN privileges.
This microversion also gives the ability to pin a server to an availability_zone and to unpin a server
from any availability_zone.
When a server is pinned to an availability_zone, the server move operations will keep the server in that
availability_zone. However, when the server is not pinned to any availability_zone, the move operations can
move the server to nodes in different availability_zones.
The behavior according to unshelve parameters will follow the below table.
+----------+---------------------------+----------+--------------------------------+
| Boot | AZ (1) | Host (1) | Result |
+==========+===========================+==========+================================+
| No AZ | No AZ or AZ=null | No | Free scheduling (2) |
+----------+---------------------------+----------+--------------------------------+
| No AZ | No AZ or AZ=null | Host1 | Schedule to Host1. |
| | | | Server remains unpinned. |
+----------+---------------------------+----------+--------------------------------+
| No AZ | AZ="AZ1" | No | Schedule to any host in "AZ1". |
| | | | Server is pined to "AZ1". |
+----------+---------------------------+----------+--------------------------------+
| No AZ | AZ="AZ1" | Host1 | Verify Host1 is in "AZ1", |
| | | | then schedule to Host1, |
| | | | otherwise reject the request. |
| | | | Server is pined to "AZ1". |
+----------+---------------------------+----------+--------------------------------+
| AZ1 | No AZ | No | Schedule to any host in "AZ1". |
| | | | Server remains pined to "AZ1". |
+----------+---------------------------+----------+--------------------------------+
| AZ1 | AZ=null | No | Free scheduling (2). |
| | | | Server is unpinned. |
+----------+---------------------------+----------+--------------------------------+
| AZ1 | No AZ | Host1 | Verify Host1 is in "AZ1", |
| | | | then schedule to Host1, |
| | | | otherwise reject the request. |
| | | | Server remains pined to "AZ1". |
+----------+---------------------------+----------+--------------------------------+
| AZ1 | AZ=null | Host1 | Schedule to Host1. |
| | | | Server is unpinned. |
+----------+---------------------------+----------+--------------------------------+
| AZ1 | AZ="AZ2" | No | Schedule to any host in "AZ2". |
| | | | Server is pined to "AZ2". |
+----------+---------------------------+----------+--------------------------------+
| AZ1 | AZ="AZ2" | Host1 | Verify Host1 is in "AZ2" then |
| | | | schedule to Host1, |
| | | | otherwise reject the request. |
| | | | Server is pined to "AZ2". |
+----------+---------------------------+----------+--------------------------------+
(1) Unshelve body parameters
(2) Schedule to any host available.
If the server is locked, you must have administrator privileges to unshelve the server.
**Asynchronous Postconditions**
@ -147,11 +203,30 @@ Request
{"unshelve": null} or {"unshelve": {"availability_zone": <string>}}.
A request body of {"unshelve": {}} is not allowed.
.. note:: Since microversion 2.91, allowed request body schema are
- {"unshelve": null} (Keep compatibility with previous microversions)
or
- {"unshelve": {"availability_zone": <string>}} (Unshelve and pin server to availability_zone)
- {"unshelve": {"availability_zone": null}} (Unshelve and unpin server from any availability zone)
- {"unshelve": {"host": <fqdn>}}
- {"unshelve": {"availability_zone": <string>, "host": <fqdn>}}
- {"unshelve": {"availability_zone": null, "host": <fqdn>}}
Everything else is not allowed, examples:
- {"unshelve": {}}
- {"unshelve": {"host": <fqdn>, "host": <fqdn>}}
- {"unshelve": {"foo": <string>}}
.. rest_parameters:: parameters.yaml
- server_id: server_id_path
- unshelve: unshelve
- availability_zone: availability_zone_unshelve
- host: host_unshelve
|
@ -162,9 +237,22 @@ Request
**Example Unshelve server (unshelve Action) (v2.77)**
.. literalinclude:: ../../doc/api_samples/os-shelve/v2.77/os-unshelve.json
.. literalinclude:: ../../doc/api_samples/os-shelve/v2.77/os-unshelve-az.json
:language: javascript
**Examples Unshelve server (unshelve Action) (v2.91)**
.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-host.json
:language: javascript
.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-az-host.json
:language: javascript
.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-host-and-unpin-az.json
:language: javascript
.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-unpin-az.json
:language: javascript
Response
--------

View File

@ -0,0 +1,6 @@
{
"unshelve": {
"availability_zone": "nova",
"host": "host01"
}
}

View File

@ -0,0 +1,6 @@
{
"unshelve": {
"availability_zone": null,
"host": "host01"
}
}

View File

@ -0,0 +1,5 @@
{
"unshelve": {
"host": "host01"
}
}

View File

@ -0,0 +1,5 @@
{
"unshelve": {
"availability_zone": null
}
}

View File

@ -19,7 +19,7 @@
}
],
"status": "CURRENT",
"version": "2.90",
"version": "2.91",
"min_version": "2.1",
"updated": "2013-07-23T11:33:21Z"
}

View File

@ -22,7 +22,7 @@
}
],
"status": "CURRENT",
"version": "2.90",
"version": "2.91",
"min_version": "2.1",
"updated": "2013-07-23T11:33:21Z"
}

View File

@ -247,6 +247,8 @@ REST_API_VERSION_HISTORY = """REST API Version History:
updating or rebuilding an instance. The
``OS-EXT-SRV-ATTR:hostname`` attribute is now returned in various
server responses regardless of policy configuration.
* 2.91 - Add support to unshelve instance to a specific host and
to pin/unpin AZ.
"""
# The minimum and maximum versions of the API supported
@ -255,7 +257,7 @@ REST_API_VERSION_HISTORY = """REST API Version History:
# Note(cyeoh): This only applies for the v2.1 API once microversions
# support is fully merged. It does not affect the V2 API.
_MIN_API_VERSION = '2.1'
_MAX_API_VERSION = '2.90'
_MAX_API_VERSION = '2.91'
DEFAULT_API_VERSION = _MIN_API_VERSION
# Almost all proxy APIs which are related to network, images and baremetal

View File

@ -1202,3 +1202,12 @@ hostname based on the display name.
In addition, the ``OS-EXT-SRV-ATTR:hostname`` field for all server
responses is now visible to all users. Previously this was an admin-only field.
.. _microversion 2.91:
2.91
----
Add support to unshelve instance to a specific host.
Add support to pin a server to an availability zone or unpin a server from any availability zone.

View File

@ -15,7 +15,7 @@
from nova.api.validation import parameter_types
# NOTE(brinzhang): For older microversion there will be no change as
# schema is applied only for >2.77 with unshelve a server API.
# schema is applied only for version < 2.91 with unshelve a server API.
# Anything working in old version keep working as it is.
unshelve_v277 = {
'type': 'object',
@ -35,3 +35,55 @@ unshelve_v277 = {
'required': ['unshelve'],
'additionalProperties': False,
}
# NOTE(rribaud):
# schema is applied only for version >= 2.91 with unshelve a server API.
# Add host parameter to specify to unshelve to this specific host.
#
# Schema has been redefined for better clarity instead of extend 2.77.
#
# API can be called with the following body:
#
# - {"unshelve": null} (Keep compatibility with previous microversions)
#
# or
#
# - {"unshelve": {"availability_zone": <string>}}
# - {"unshelve": {"availability_zone": null}} (Unpin availability zone)
# - {"unshelve": {"host": <fqdn>}}
# - {"unshelve": {"availability_zone": <string>, "host": <fqdn>}}
# - {"unshelve": {"availability_zone": null, "host": <fqdn>}}
#
#
# Everything else is not allowed, examples:
#
# - {"unshelve": {}}
# - {"unshelve": {"host": <fqdn>, "host": <fqdn>}}
# - {"unshelve": {"foo": <string>}}
unshelve_v291 = {
"type": "object",
"properties": {
"unshelve": {
"oneOf": [
{
"type": ["object"],
"properties": {
"availability_zone": {
"oneOf": [
{"type": ["null"]},
{"type": "string"}]
},
"host": {
"type": "string"
}
},
"additionalProperties": False
},
{"type": ["null"]}
]
}
},
"required": ["unshelve"],
"additionalProperties": False
}

View File

@ -68,7 +68,6 @@ class ShelveController(wsgi.Controller):
context.can(shelve_policies.POLICY_ROOT % 'shelve_offload',
target={'user_id': instance.user_id,
'project_id': instance.project_id})
try:
self.compute_api.shelve_offload(context, instance)
except exception.InstanceIsLocked as e:
@ -87,33 +86,59 @@ class ShelveController(wsgi.Controller):
# In microversion 2.77 we support specifying 'availability_zone' to
# unshelve a server. But before 2.77 there is no request body
# schema validation (because of body=null).
@validation.schema(shelve_schemas.unshelve_v277, min_version='2.77')
@validation.schema(
shelve_schemas.unshelve_v277,
min_version='2.77',
max_version='2.90'
)
# In microversion 2.91 we support specifying 'host' to
# unshelve an instance to a specific hostself.
# 'availability_zone' = None is supported as well to unpin the
# availability zone of an instance bonded to this availability_zone
@validation.schema(shelve_schemas.unshelve_v291, min_version='2.91')
def _unshelve(self, req, id, body):
"""Restore an instance from shelved mode."""
context = req.environ["nova.context"]
instance = common.get_instance(self.compute_api, context, id)
context.can(shelve_policies.POLICY_ROOT % 'unshelve',
target={'project_id': instance.project_id})
context.can(
shelve_policies.POLICY_ROOT % 'unshelve',
target={'project_id': instance.project_id}
)
unshelve_args = {}
unshelve_dict = body['unshelve']
support_az = api_version_request.is_supported(req, '2.77')
if support_az and unshelve_dict:
unshelve_args['new_az'] = unshelve_dict['availability_zone']
unshelve_dict = body.get('unshelve')
support_az = api_version_request.is_supported(
req, '2.77')
support_host = api_version_request.is_supported(
req, '2.91')
if unshelve_dict:
if support_az and 'availability_zone' in unshelve_dict:
unshelve_args['new_az'] = (
unshelve_dict['availability_zone']
)
if support_host:
unshelve_args['host'] = unshelve_dict.get('host')
try:
self.compute_api.unshelve(context, instance, **unshelve_args)
except (exception.InstanceIsLocked,
exception.UnshelveInstanceInvalidState,
exception.MismatchVolumeAZException) as e:
self.compute_api.unshelve(
context,
instance,
**unshelve_args,
)
except (
exception.InstanceIsLocked,
exception.UnshelveInstanceInvalidState,
exception.UnshelveHostNotInAZ,
exception.MismatchVolumeAZException,
) as e:
raise exc.HTTPConflict(explanation=e.format_message())
except exception.InstanceInvalidState as state_error:
common.raise_http_conflict_for_instance_invalid_state(state_error,
'unshelve',
id)
common.raise_http_conflict_for_instance_invalid_state(
state_error, 'unshelve', id)
except (
exception.InvalidRequest,
exception.ExtendedResourceRequestOldCompute,
exception.ComputeHostNotFound,
) as e:
raise exc.HTTPBadRequest(explanation=e.format_message())

View File

@ -75,6 +75,7 @@ from nova.objects import quotas as quotas_obj
from nova.objects import service as service_obj
from nova.pci import request as pci_request
from nova.policies import servers as servers_policies
from nova.policies import shelve as shelve_policies
import nova.policy
from nova import profiler
from nova import rpc
@ -4504,6 +4505,14 @@ class API:
# host is requested, so we have to see if it exists and does not
# contradict with the AZ of the instance
if host:
# Make sure only admin can unshelve to a specific host.
context.can(
shelve_policies.POLICY_ROOT % 'unshelve_to_host',
target={
'user_id': instance.user_id,
'project_id': instance.project_id
}
)
# Ensure that the requested host exists otherwise raise
# a ComputeHostNotFound exception
objects.ComputeNode.get_first_node_by_host_for_old_compat(

View File

@ -44,6 +44,18 @@ shelve_policies = [
}
],
scope_types=['project']),
policy.DocumentedRuleDefault(
name=POLICY_ROOT % 'unshelve_to_host',
check_str=base.PROJECT_ADMIN,
description="Unshelve (restore) shelve offloaded server to a "
"specific host",
operations=[
{
'method': 'POST',
'path': '/servers/{server_id}/action (unshelve)'
}
],
scope_types=['project']),
policy.DocumentedRuleDefault(
name=POLICY_ROOT % 'shelve_offload',
check_str=base.PROJECT_ADMIN,

View File

@ -0,0 +1,5 @@
{
"%(action)s": {
"availability_zone": "%(availability_zone)s"
}
}

View File

@ -1,5 +1,3 @@
{
"%(action)s": {
"availability_zone": "%(availability_zone)s"
}
"unshelve": null
}

View File

@ -0,0 +1,6 @@
{
"%(action)s": {
"availability_zone": "%(availability_zone)s",
"host": "%(host)s"
}
}

View File

@ -0,0 +1,5 @@
{
"%(action)s": {
"availability_zone": "%(availability_zone)s"
}
}

View File

@ -0,0 +1,6 @@
{
"%(action)s": {
"availability_zone": null,
"host": "%(host)s"
}
}

View File

@ -0,0 +1,5 @@
{
"%(action)s": {
"host": "%(host)s"
}
}

View File

@ -0,0 +1,5 @@
{
"%(action)s": {
"availability_zone": null
}
}

View File

@ -15,10 +15,25 @@
import nova.conf
from nova import objects
from nova.tests.functional.api_sample_tests import test_servers
from oslo_utils.fixture import uuidsentinel
from unittest import mock
CONF = nova.conf.CONF
fake_aggregate = {
'deleted': 0,
'deleted_at': None,
'created_at': None,
'updated_at': None,
'id': 123,
'uuid': uuidsentinel.fake_aggregate,
'name': 'us-west',
'hosts': ['host01'],
'metadetails': {'availability_zone': 'us-west'},
}
class ShelveJsonTest(test_servers.ServersSampleBase):
# The 'os_compute_api:os-shelve:shelve_offload' policy is admin-only
@ -30,9 +45,11 @@ class ShelveJsonTest(test_servers.ServersSampleBase):
# Don't offload instance, so we can test the offload call.
CONF.set_override('shelved_offload_time', -1)
def _test_server_action(self, uuid, template, action):
def _test_server_action(self, uuid, template, action, subs=None):
subs = subs or {}
subs.update({'action': action})
response = self._do_post('servers/%s/action' % uuid,
template, {'action': action})
template, subs)
self.assertEqual(202, response.status_code)
self.assertEqual("", response.text)
@ -51,26 +68,288 @@ class ShelveJsonTest(test_servers.ServersSampleBase):
self._test_server_action(uuid, 'os-unshelve', 'unshelve')
class UnshelveJson277Test(test_servers.ServersSampleBase):
class UnshelveJson277Test(ShelveJsonTest):
ADMIN_API = False
sample_dir = "os-shelve"
microversion = '2.77'
scenarios = [('v2_77', {'api_major_version': 'v2.1'})]
def setUp(self):
super(UnshelveJson277Test, self).setUp()
# Almost all next tests require the instance to be shelve offloaded.
# So shelve offload the instance and skip the shelve_offload_test
# below.
CONF.set_override('shelved_offload_time', 0)
def test_shelve_offload(self):
# Skip this test as the instance is already shelve offloaded.
pass
def test_unshelve_with_az(self):
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
self._test_server_action(
uuid,
'os-unshelve-az',
'unshelve',
subs={"availability_zone": "us-west"}
)
class UnshelveJson291Test(UnshelveJson277Test):
ADMIN_API = True
sample_dir = "os-shelve"
microversion = '2.91'
scenarios = [('v2_91', {'api_major_version': 'v2.1'})]
def _test_server_action_invalid(
self, uuid, template, action, subs=None, msg=None):
subs = subs or {}
subs.update({'action': action})
response = self._do_post('servers/%s/action' % uuid,
template, subs)
self.assertEqual(400, response.status_code)
self.assertIn(msg, response.text)
def test_unshelve_with_non_valid_host(self):
"""Ensure an exception rise if host is invalid and
a http 400 error
"""
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
self._test_server_action_invalid(
uuid, 'os-unshelve-host',
'unshelve',
subs={'host': 'host01'},
msg='Compute host host01 could not be found.')
@mock.patch('nova.objects.aggregate._get_by_host_from_db')
@mock.patch('nova.objects.ComputeNodeList.get_all_by_host')
def test_unshelve_with_valid_host(
self, compute_node_get_all_by_host, mock_api_get_by_host):
"""Ensure we can unshelve to a host
"""
# Put compute in the correct az
mock_api_get_by_host.return_value = [fake_aggregate]
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
fake_computes = objects.ComputeNodeList(
objects=[
objects.ComputeNode(
host='host01',
uuid=uuidsentinel.host1,
hypervisor_hostname='host01')
]
)
compute_node_get_all_by_host.return_value = fake_computes
self._test_server_action(
uuid,
'os-unshelve-host',
'unshelve',
subs={'host': 'host01'}
)
@mock.patch('nova.objects.aggregate._get_by_host_from_db')
@mock.patch('nova.objects.ComputeNodeList.get_all_by_host')
def test_unshelve_with_az_and_host(
self, compute_node_get_all_by_host, mock_api_get_by_host):
"""Ensure we can unshelve to a host and az
"""
# Put compute in the correct az
mock_api_get_by_host.return_value = [fake_aggregate]
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
fake_computes = objects.ComputeNodeList(
objects=[
objects.ComputeNode(
host='host01',
uuid=uuidsentinel.host1,
hypervisor_hostname='host01')
]
)
compute_node_get_all_by_host.return_value = fake_computes
self._test_server_action(
uuid,
'os-unshelve-host',
'unshelve',
subs={'host': 'host01', 'availability_zone': 'us-west'},
)
@mock.patch('nova.objects.aggregate._get_by_host_from_db')
@mock.patch('nova.objects.ComputeNodeList.get_all_by_host')
def test_unshelve_with_unpin_az_and_host(
self, compute_node_get_all_by_host, mock_api_get_by_host):
"""Ensure we can unshelve to a host and az
"""
# Put compute in the correct az
mock_api_get_by_host.return_value = [fake_aggregate]
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
fake_computes = objects.ComputeNodeList(
objects=[
objects.ComputeNode(
host='host01',
uuid=uuidsentinel.host1,
hypervisor_hostname='host01')
]
)
compute_node_get_all_by_host.return_value = fake_computes
self._test_server_action(
uuid,
'os-unshelve-host-and-unpin-az',
'unshelve',
subs={'host': 'host01'},
)
@mock.patch('nova.objects.aggregate._get_by_host_from_db')
@mock.patch('nova.objects.ComputeNodeList.get_all_by_host')
def test_unshelve_with_unpin_az(
self, compute_node_get_all_by_host, mock_api_get_by_host):
"""Ensure we can unpin an az
"""
# Put compute in the correct az
mock_api_get_by_host.return_value = [fake_aggregate]
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
fake_computes = objects.ComputeNodeList(
objects=[
objects.ComputeNode(
host='host01',
uuid=uuidsentinel.host1,
hypervisor_hostname='host01')
]
)
compute_node_get_all_by_host.return_value = fake_computes
self._test_server_action(
uuid,
'os-unshelve-unpin-az',
'unshelve',
subs={'host': 'host01'},
)
class UnshelveJson291NonAdminTest(UnshelveJson291Test):
# Use non admin api credentials.
ADMIN_API = False
sample_dir = "os-shelve"
microversion = '2.91'
scenarios = [('v2_91', {'api_major_version': 'v2.1'})]
def _test_server_action_invalid(self, uuid, template, action, subs=None):
subs = subs or {}
subs.update({'action': action})
response = self._do_post('servers/%s/action' % uuid,
template, subs)
self.assertEqual(403, response.status_code)
self.assertIn(
"Policy doesn\'t allow os_compute_api:os-shelve:unshelve_to_host" +
" to be performed.", response.text)
def _test_server_action(self, uuid, template, action, subs=None):
subs = subs or {}
subs.update({'action': action})
response = self._do_post('servers/%s/action' % uuid,
template, subs)
self.assertEqual(202, response.status_code)
self.assertEqual("", response.text)
self.assertEqual('', response.text)
def test_unshelve_with_az(self):
def test_unshelve_with_non_valid_host(self):
"""Ensure an exception rise if user is not admin.
a http 403 error
"""
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
self._test_server_action(uuid, 'os-unshelve', 'unshelve',
subs={"availability_zone": "us-west"})
self._test_server_action_invalid(
uuid,
'os-unshelve-host',
'unshelve',
subs={'host': 'host01'}
)
@mock.patch('nova.objects.aggregate._get_by_host_from_db')
@mock.patch('nova.objects.ComputeNodeList.get_all_by_host')
def test_unshelve_with_unpin_az_and_host(
self, compute_node_get_all_by_host, mock_api_get_by_host):
# Put compute in the correct az
mock_api_get_by_host.return_value = [fake_aggregate]
def test_unshelve_no_az(self):
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
self._test_server_action(uuid, 'os-unshelve-null', 'unshelve')
fake_computes = objects.ComputeNodeList(
objects=[
objects.ComputeNode(
host='host01',
uuid=uuidsentinel.host1,
hypervisor_hostname='host01')
]
)
compute_node_get_all_by_host.return_value = fake_computes
self._test_server_action_invalid(
uuid,
'os-unshelve-host-and-unpin-az',
'unshelve',
subs={'host': 'host01'},
)
@mock.patch('nova.objects.aggregate._get_by_host_from_db')
@mock.patch('nova.objects.ComputeNodeList.get_all_by_host')
def test_unshelve_with_valid_host(
self, compute_node_get_all_by_host, mock_api_get_by_host):
# Put compute in the correct az
mock_api_get_by_host.return_value = [fake_aggregate]
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
fake_computes = objects.ComputeNodeList(
objects=[
objects.ComputeNode(
host='host01',
uuid=uuidsentinel.host1,
hypervisor_hostname='host01')
]
)
compute_node_get_all_by_host.return_value = fake_computes
self._test_server_action_invalid(
uuid,
'os-unshelve-host',
'unshelve',
subs={'host': 'host01'}
)
@mock.patch('nova.objects.aggregate._get_by_host_from_db')
@mock.patch('nova.objects.ComputeNodeList.get_all_by_host')
def test_unshelve_with_az_and_host(
self, compute_node_get_all_by_host, mock_api_get_by_host):
"""Ensure we can unshelve to a host and az
"""
# Put compute in the correct az
mock_api_get_by_host.return_value = [fake_aggregate]
uuid = self._post_server()
self._test_server_action(uuid, 'os-shelve', 'shelve')
fake_computes = objects.ComputeNodeList(
objects=[
objects.ComputeNode(
host='host01',
uuid=uuidsentinel.host1,
hypervisor_hostname='host01')
]
)
compute_node_get_all_by_host.return_value = fake_computes
self._test_server_action_invalid(
uuid,
'os-unshelve-host',
'unshelve',
subs={'host': 'host01', 'availability_zone': 'us-west'},
)

View File

@ -10,12 +10,16 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from nova.api.openstack.compute import hosts
from nova.compute import instance_actions
from nova import context
from nova import objects
from nova import test
from nova.tests import fixtures as nova_fixtures
from nova.tests.functional.api import client as api_client
from nova.tests.functional import fixtures as func_fixtures
from nova.tests.functional import integrated_helpers
from nova.tests.unit.api.openstack import fakes
class TestAvailabilityZoneScheduling(
@ -36,6 +40,9 @@ class TestAvailabilityZoneScheduling(
self.api = api_fixture.admin_api
self.api.microversion = 'latest'
self.controller = hosts.HostController()
self.req = fakes.HTTPRequest.blank('', use_admin_context=True)
self.start_service('conductor')
self.start_service('scheduler')
@ -68,18 +75,18 @@ class TestAvailabilityZoneScheduling(
self.api.api_post(
'/os-aggregates/%s/action' % aggregate['id'], add_host_body)
def _create_server(self, name):
def _create_server(self, name, zone=None):
# Create a server, it doesn't matter which host it ends up in.
server = super(TestAvailabilityZoneScheduling, self)._create_server(
flavor_id=self.flavor1,
networks='none',)
original_host = server['OS-EXT-SRV-ATTR:host']
# Assert the server has the AZ set (not None or 'nova').
expected_zone = 'zone1' if original_host == 'host1' else 'zone2'
self.assertEqual(expected_zone, server['OS-EXT-AZ:availability_zone'])
networks='none',
az=zone,
)
return server
def _assert_instance_az(self, server, expected_zone):
def _assert_instance_az_and_host(
self, server, expected_zone, expected_host=None):
# Check AZ
# Check the API.
self.assertEqual(expected_zone, server['OS-EXT-AZ:availability_zone'])
# Check the DB.
@ -88,6 +95,51 @@ class TestAvailabilityZoneScheduling(
ctxt, self.cell_mappings[test.CELL1_NAME]) as cctxt:
instance = objects.Instance.get_by_uuid(cctxt, server['id'])
self.assertEqual(expected_zone, instance.availability_zone)
# Check host
if expected_host:
self.assertEqual(expected_host, server['OS-EXT-SRV-ATTR:host'])
def _assert_request_spec_az(self, ctxt, server, az):
request_spec = objects.RequestSpec.get_by_instance_uuid(
ctxt, server['id'])
self.assertEqual(request_spec.availability_zone, az)
def _assert_server_with_az_unshelved_to_specified_az(self, server, az):
"""Ensure a server with an az constraints is unshelved in the
corresponding az.
"""
host_to_disable = 'host1' if az == 'zone1' else 'host2'
self._shelve_server(server, expected_state='SHELVED_OFFLOADED')
compute_service_id = self.api.get_services(
host=host_to_disable, binary='nova-compute')[0]['id']
self.api.put_service(compute_service_id, {'status': 'disabled'})
req = {
'unshelve': None
}
self.api.post_server_action(server['id'], req)
server = self._wait_for_action_fail_completion(
server, instance_actions.UNSHELVE, 'schedule_instances')
self.assertIn('Error', server['result'])
self.assertIn('No valid host', server['details'])
def _shelve_unshelve_server(self, ctxt, server, req):
self._shelve_server(server, expected_state='SHELVED_OFFLOADED')
self.api.post_server_action(server['id'], req)
server = self._wait_for_server_parameter(
server,
{'status': 'ACTIVE', },
)
return self.api.get_server(server['id'])
def other_az_than(self, az):
return 'zone2' if az == 'zone1' else 'zone1'
def other_host_than(self, host):
return 'host2' if host == 'host1' else 'host1'
def test_live_migrate_implicit_az(self):
"""Tests live migration of an instance with an implicit AZ.
@ -111,7 +163,8 @@ class TestAvailabilityZoneScheduling(
still not restricted to its current zone even if it says it is in one.
"""
server = self._create_server('test_live_migrate_implicit_az')
original_host = server['OS-EXT-SRV-ATTR:host']
original_az = server['OS-EXT-AZ:availability_zone']
expected_zone = self.other_az_than(original_az)
# Attempt to live migrate the instance; again, we don't specify a host
# because there are only two hosts so the scheduler would only be able
@ -132,8 +185,379 @@ class TestAvailabilityZoneScheduling(
# the database because the API will return the AZ from the host
# aggregate if instance.host is not None.
server = self.api.get_server(server['id'])
expected_zone = 'zone2' if original_host == 'host1' else 'zone1'
self._assert_instance_az(server, expected_zone)
self._assert_instance_az_and_host(server, expected_zone)
def test_create_server(self):
"""Create a server without an AZ constraint and make sure asking a new
request spec will not have the request_spec.availability_zone set.
"""
ctxt = context.get_admin_context()
server = self._create_server('server01')
self._assert_request_spec_az(ctxt, server, None)
def test_create_server_to_zone(self):
"""Create a server with an AZ constraint and make sure asking a new
request spec will have the request_spec.availability_zone to the
required zone.
"""
ctxt = context.get_admin_context()
server = self._create_server('server01', 'zone2')
server = self.api.get_server(server['id'])
self._assert_instance_az_and_host(server, 'zone2')
self._assert_request_spec_az(ctxt, server, 'zone2')
def test_cold_migrate_cross_az(self):
"""Test a cold migration cross AZ.
"""
server = self._create_server('server01')
original_host = server['OS-EXT-SRV-ATTR:host']
original_az = server['OS-EXT-AZ:availability_zone']
expected_host = self.other_host_than(original_host)
expected_zone = self.other_az_than(original_az)
self._migrate_server(server)
self._confirm_resize(server)
server = self.api.get_server(server['id'])
self._assert_instance_az_and_host(server, expected_zone, expected_host)
# Next tests attempt to check the following behavior
# +----------+---------------------------+-------+----------------------------+
# | Boot | Unshelve after offload AZ | Host | Result |
# +==========+===========================+=======+============================+
# | No AZ | No AZ or AZ=null | No | Free scheduling, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
# | No AZ | No AZ or AZ=null | Host1 | Schedule to host1, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
# | No AZ | AZ="AZ1" | No | Schedule to AZ1, |
# | | | | reqspec.AZ="AZ1" |
# +----------+---------------------------+-------+----------------------------+
# | No AZ | AZ="AZ1" | Host1 | Verify that host1 in AZ1, |
# | | | | or (1). Schedule to |
# | | | | host1, reqspec.AZ="AZ1" |
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | No AZ | No | Schedule to AZ1, |
# | | | | reqspec.AZ="AZ1" |
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ=null | No | Free scheduling, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | No AZ | Host1 | If host1 is in AZ1, |
# | | | | then schedule to host1, |
# | | | | reqspec.AZ="AZ1", otherwise|
# | | | | reject the request (1) |
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ=null | Host1 | Schedule to host1, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ="AZ2" | No | Schedule to AZ2, |
# | | | | reqspec.AZ="AZ2" |
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ="AZ2" | Host1 | If host1 in AZ2 then |
# | | | | schedule to host1, |
# | | | | reqspec.AZ="AZ2", |
# | | | | otherwise reject (1) |
# +----------+---------------------------+-------+----------------------------+
#
# (1) Check at the api and return an error.
#
#
# +----------+---------------------------+-------+----------------------------+
# | No AZ | No AZ or AZ=null | No | Free scheduling, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_server_without_az_contraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01')
req = {
'unshelve': None
}
self._shelve_unshelve_server(ctxt, server, req)
self._assert_request_spec_az(ctxt, server, None)
def test_unshelve_unpin_az_server_without_az_contraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01')
req = {
'unshelve': {'availability_zone': None}
}
self._shelve_unshelve_server(ctxt, server, req)
self._assert_request_spec_az(ctxt, server, None)
# +----------+---------------------------+-------+----------------------------+
# | No AZ | No AZ or AZ=null | Host1 | Schedule to host1, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_host_server_without_az_contraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01')
original_host = server['OS-EXT-SRV-ATTR:host']
original_az = server['OS-EXT-AZ:availability_zone']
dest_hostname = self.other_host_than(original_host)
expected_zone = self.other_az_than(original_az)
req = {
'unshelve': {'host': dest_hostname}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, expected_zone, dest_hostname)
self._assert_request_spec_az(ctxt, server, None)
def test_unshelve_to_host_and_unpin_server_without_az_contraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01')
original_host = server['OS-EXT-SRV-ATTR:host']
original_az = server['OS-EXT-AZ:availability_zone']
dest_hostname = self.other_host_than(original_host)
expected_zone = self.other_az_than(original_az)
req = {
'unshelve': {
'host': dest_hostname,
'availability_zone': None,
}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, expected_zone, dest_hostname)
self._assert_request_spec_az(ctxt, server, None)
# +----------+---------------------------+-------+----------------------------+
# | No AZ | AZ="AZ1" | No | Schedule to AZ1, |
# | | | | reqspec.AZ="AZ1" |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_az_server_without_az_constraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01')
original_host = server['OS-EXT-SRV-ATTR:host']
original_az = server['OS-EXT-AZ:availability_zone']
dest_hostname = 'host2' if original_host == 'host1' else 'host1'
dest_az = self.other_az_than(original_az)
req = {
'unshelve': {'availability_zone': dest_az}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, dest_az, dest_hostname)
self._assert_request_spec_az(ctxt, server, dest_az)
self._assert_server_with_az_unshelved_to_specified_az(
server, dest_az)
# +----------+---------------------------+-------+----------------------------+
# | No AZ | AZ="AZ1" | Host1 | Verify that host1 in AZ1, |
# | | | | or (3). Schedule to |
# | | | | host1, reqspec.AZ="AZ1" |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_az_and_host_server_without_az_constraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01')
original_host = server['OS-EXT-SRV-ATTR:host']
original_az = server['OS-EXT-AZ:availability_zone']
dest_hostname = 'host2' if original_host == 'host1' else 'host1'
dest_az = self.other_az_than(original_az)
req = {
'unshelve': {'host': dest_hostname, 'availability_zone': dest_az}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, dest_az, dest_hostname)
self._assert_request_spec_az(ctxt, server, dest_az)
self._assert_server_with_az_unshelved_to_specified_az(
server, dest_az)
def test_unshelve_to_wrong_az_and_host_server_without_az_constraint(self):
server = self._create_server('server01')
original_host = server['OS-EXT-SRV-ATTR:host']
original_az = server['OS-EXT-AZ:availability_zone']
dest_hostname = 'host2' if original_host == 'host1' else 'host1'
req = {
'unshelve': {'host': dest_hostname,
'availability_zone': original_az}
}
self._shelve_server(server, expected_state='SHELVED_OFFLOADED')
exc = self.assertRaises(
api_client.OpenStackApiException,
self.api.post_server_action,
server['id'],
req
)
self.assertEqual(409, exc.response.status_code)
self.assertIn(
'Host \\\"{}\\\" is not in the availability zone \\\"{}\\\".'
.format(dest_hostname, original_az),
exc.response.text
)
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | No AZ | No | Schedule to AZ1, |
# | | | | reqspec.AZ="AZ1" |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_a_server_with_az_contraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01', 'zone2')
req = {
'unshelve': None
}
self._shelve_unshelve_server(ctxt, server, req)
self._assert_request_spec_az(ctxt, server, 'zone2')
self._assert_server_with_az_unshelved_to_specified_az(
server, 'zone2')
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ=null | No | Free scheduling, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_unpin_az_a_server_with_az_constraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01', 'zone2')
req = {
'unshelve': {'availability_zone': None}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_request_spec_az(ctxt, server, None)
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | No AZ | Host1 | If host1 is in AZ1, |
# | | | | then schedule to host1, |
# | | | | reqspec.AZ="AZ1", otherwise|
# | | | | reject the request (3) |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_host_server_with_az_contraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01', 'zone1')
req = {
'unshelve': {'host': 'host1'}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, 'zone1', 'host1')
self._assert_request_spec_az(ctxt, server, 'zone1')
self._assert_server_with_az_unshelved_to_specified_az(
server, 'zone1')
def test_unshelve_to_host_wrong_az_server_with_az_contraint(self):
server = self._create_server('server01', 'zone1')
req = {
'unshelve': {'host': 'host2'}
}
self._shelve_server(server, expected_state='SHELVED_OFFLOADED')
exc = self.assertRaises(
api_client.OpenStackApiException,
self.api.post_server_action,
server['id'],
req
)
self.assertEqual(409, exc.response.status_code)
self.assertIn(
'Host \\\"host2\\\" is not in the availability '
'zone \\\"zone1\\\".',
exc.response.text
)
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ=null | Host1 | Schedule to host1, |
# | | | | reqspec.AZ=None |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_host_and_unpin_server_with_az_contraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01', 'zone1')
req = {
'unshelve': {'host': 'host2',
'availability_zone': None,
}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, 'zone2', 'host2')
self._assert_request_spec_az(ctxt, server, None)
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ="AZ2" | No | Schedule to AZ2, |
# | | | | reqspec.AZ="AZ2" |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_az_a_server_with_az_constraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01', 'zone1')
req = {
'unshelve': {'availability_zone': 'zone2'}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, 'zone2', 'host2')
self._assert_request_spec_az(ctxt, server, 'zone2')
self._assert_server_with_az_unshelved_to_specified_az(
server, 'zone2')
# +----------+---------------------------+-------+----------------------------+
# | AZ1 | AZ="AZ2" | Host1 | If host1 in AZ2 then |
# | | | | schedule to host1, |
# | | | | reqspec.AZ="AZ2", |
# | | | | otherwise reject (3) |
# +----------+---------------------------+-------+----------------------------+
def test_unshelve_to_host_and_az_a_server_with_az_constraint(self):
ctxt = context.get_admin_context()
server = self._create_server('server01', 'zone1')
req = {
'unshelve': {'host': 'host2',
'availability_zone': 'zone2',
}
}
server = self._shelve_unshelve_server(ctxt, server, req)
self._assert_instance_az_and_host(server, 'zone2', 'host2')
self._assert_request_spec_az(ctxt, server, 'zone2')
self._assert_server_with_az_unshelved_to_specified_az(
server, 'zone2')
def test_unshelve_to_host_and_wrong_az_a_server_with_az_constraint(self):
server = self._create_server('server01', 'zone1')
req = {
'unshelve': {'host': 'host2',
'availability_zone': 'zone1',
}
}
self._shelve_server(server, expected_state='SHELVED_OFFLOADED')
exc = self.assertRaises(
api_client.OpenStackApiException,
self.api.post_server_action,
server['id'],
req
)
self.assertEqual(409, exc.response.status_code)
self.assertIn(
'Host \\\"host2\\\" is not in the availability '
'zone \\\"zone1\\\".',
exc.response.text
)
def test_resize_revert_across_azs(self):
"""Creates two compute service hosts in separate AZs. Creates a server
@ -152,9 +576,9 @@ class TestAvailabilityZoneScheduling(
# Now the server should be in the other AZ.
new_zone = 'zone2' if original_host == 'host1' else 'zone1'
self._assert_instance_az(server, new_zone