Add support for translating CPU policy extra specs, image meta
Map 'hw:cpu_policy' and 'hw:cpu_thread_policy' as follows: hw:cpu_policy dedicated -> resources:PCPU=${flavor.vcpus} shared -> resources:VCPU=${flavor.vcpus} hw:cpu_thread_policy isolate -> trait:HW_CPU_HYPERTHREADING:forbidden require -> trait:HW_CPU_HYPERTHREADING:required prefer -> (none, handled later during scheduling) Ditto for the 'hw_cpu_policy' and 'hw_cpu_thread_policy' image metadata equivalents. In addition, increment the requested 'resources:PCPU' by 1 if the 'hw:emulator_threads_policy' extra spec is present and set to 'isolate'. The scheduler will attempt to get PCPUs allocations and fall back to VCPUs if that fails. This is okay because the NUMA fitting code from the 'hardware' module used by both the 'NUMATopology' filter and libvirt driver protects us. That code doesn't know anything about PCPUs or VCPUs but rather cares about the 'NUMATopology.pcpuset' field, (starting in change I492803eaacc34c69af073689f9159449557919db), which can be set to different values depending on whether this is Train with new-style config, Train with old-style config, or Stein: - For Train compute nodes with new-style config, 'NUMATopology.pcpuset' will be explictly set to the value of '[compute] cpu_dedicated_set' or, if only '[compute] cpu_dedicated_set' is configured, 'None' (it's nullable) by the virt driver so the calls to 'hardware.numa_fit_instance_to_host' in the 'NUMATopologyFilter' or virt driver will fail if it can't actually fit. - For Train compute nodes with old-style config, 'NUMATopology.pcpuset' will be set to the same value as 'NUMATopology.cpuset' by the virt driver. - For Stein compute nodes, 'NUMATopology.pcpuset' will be unset and we'll detect this in 'hardware.numa_fit_instance_to_host' and simply set it to the same value as 'NUMATopology.cpuset'. Part of blueprint cpu-resources Change-Id: Ie38aa625dff543b5980fd437ad2febeba3b50079 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This commit is contained in:
parent
d6c96436d6
commit
278ab01c32
|
@ -221,6 +221,31 @@ Related options:
|
|||
* ``compute_driver`` (libvirt)
|
||||
* ``[libvirt]/images_type`` (rbd)
|
||||
* ``instances_path``
|
||||
"""),
|
||||
|
||||
cfg.BoolOpt(
|
||||
'disable_fallback_pcpu_query',
|
||||
default=False,
|
||||
deprecated_for_removal=True,
|
||||
deprecated_since='20.0.0',
|
||||
help="""
|
||||
Disable fallback request for VCPU allocations when using pinned instances.
|
||||
|
||||
Starting in Train, compute nodes using the libvirt virt driver can report
|
||||
``PCPU`` inventory and will use this for pinned instances. The scheduler will
|
||||
automatically translate requests using the legacy CPU pinning-related flavor
|
||||
extra specs, ``hw:cpu_policy`` and ``hw:cpu_thread_policy``, their image
|
||||
metadata property equivalents, and the emulator threads pinning flavor extra
|
||||
spec, ``hw:emulator_threads_policy``, to new placement requests. However,
|
||||
compute nodes require additional configuration in order to report ``PCPU``
|
||||
inventory and this configuration may not be present immediately after an
|
||||
upgrade. To ensure pinned instances can be created without this additional
|
||||
configuration, the scheduler will make a second request to placement for
|
||||
old-style ``VCPU``-based allocations and fallback to these allocation
|
||||
candidates if necessary. This has a slight performance impact and is not
|
||||
necessary on new or upgraded deployments where the new configuration has been
|
||||
set on all hosts. By setting this option, the second lookup is disabled and the
|
||||
scheduler will only request ``PCPU``-based allocations.
|
||||
"""),
|
||||
]
|
||||
|
||||
|
|
|
@ -151,18 +151,46 @@ class SchedulerManager(manager.Manager):
|
|||
raise exception.NoValidHost(reason=e.message)
|
||||
|
||||
resources = utils.resources_from_request_spec(
|
||||
ctxt, spec_obj, self.driver.host_manager)
|
||||
ctxt, spec_obj, self.driver.host_manager,
|
||||
enable_pinning_translate=True)
|
||||
res = self.placement_client.get_allocation_candidates(ctxt,
|
||||
resources)
|
||||
if res is None:
|
||||
# We have to handle the case that we failed to connect to the
|
||||
# Placement service and the safe_connect decorator on
|
||||
# get_allocation_candidates returns None.
|
||||
alloc_reqs, provider_summaries, allocation_request_version = (
|
||||
None, None, None)
|
||||
else:
|
||||
(alloc_reqs, provider_summaries,
|
||||
allocation_request_version) = res
|
||||
res = None, None, None
|
||||
|
||||
alloc_reqs, provider_summaries, allocation_request_version = res
|
||||
alloc_reqs = alloc_reqs or []
|
||||
provider_summaries = provider_summaries or {}
|
||||
|
||||
# if the user requested pinned CPUs, we make a second query to
|
||||
# placement for allocation candidates using VCPUs instead of PCPUs.
|
||||
# This is necessary because users might not have modified all (or
|
||||
# any) of their compute nodes meaning said compute nodes will not
|
||||
# be reporting PCPUs yet. This is okay to do because the
|
||||
# NUMATopologyFilter (scheduler) or virt driver (compute node) will
|
||||
# weed out hosts that are actually using new style configuration
|
||||
# but simply don't have enough free PCPUs (or any PCPUs).
|
||||
# TODO(stephenfin): Remove when we drop support for 'vcpu_pin_set'
|
||||
if (resources.cpu_pinning_requested and
|
||||
not CONF.workarounds.disable_fallback_pcpu_query):
|
||||
LOG.debug('Requesting fallback allocation candidates with '
|
||||
'VCPU instead of PCPU')
|
||||
resources = utils.resources_from_request_spec(
|
||||
ctxt, spec_obj, self.driver.host_manager,
|
||||
enable_pinning_translate=False)
|
||||
res = self.placement_client.get_allocation_candidates(
|
||||
ctxt, resources)
|
||||
if res:
|
||||
# merge the allocation requests and provider summaries from
|
||||
# the two requests together
|
||||
alloc_reqs_fallback, provider_summaries_fallback, _ = res
|
||||
|
||||
alloc_reqs.extend(alloc_reqs_fallback)
|
||||
provider_summaries.update(provider_summaries_fallback)
|
||||
|
||||
if not alloc_reqs:
|
||||
LOG.info("Got no allocation candidates from the Placement "
|
||||
"API. This could be due to insufficient resources "
|
||||
|
|
|
@ -20,6 +20,7 @@ import sys
|
|||
import traceback
|
||||
|
||||
import os_resource_classes as orc
|
||||
import os_traits
|
||||
from oslo_log import log as logging
|
||||
from oslo_serialization import jsonutils
|
||||
from six.moves.urllib import parse
|
||||
|
@ -32,10 +33,11 @@ from nova import exception
|
|||
from nova.i18n import _
|
||||
from nova import objects
|
||||
from nova.objects import base as obj_base
|
||||
from nova.objects import fields as obj_fields
|
||||
from nova.objects import instance as obj_instance
|
||||
from nova import rpc
|
||||
from nova.scheduler.filters import utils as filters_utils
|
||||
import nova.virt.hardware as hw
|
||||
from nova.virt import hardware
|
||||
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
@ -55,7 +57,7 @@ class ResourceRequest(object):
|
|||
XS_KEYPAT = re.compile(r"^(%s)([1-9][0-9]*)?:(.*)$" %
|
||||
'|'.join((XS_RES_PREFIX, XS_TRAIT_PREFIX)))
|
||||
|
||||
def __init__(self, request_spec):
|
||||
def __init__(self, request_spec, enable_pinning_translate=True):
|
||||
"""Create a new instance of ResourceRequest from a RequestSpec.
|
||||
|
||||
Examines the flavor, flavor extra specs, and (optional) image metadata
|
||||
|
@ -80,6 +82,8 @@ class ResourceRequest(object):
|
|||
overridden by flavor extra specs.
|
||||
|
||||
:param request_spec: An instance of ``objects.RequestSpec``.
|
||||
:param enable_pinning_translate: True if the CPU policy extra specs
|
||||
should be translated to placement resources and traits.
|
||||
"""
|
||||
# { ident: RequestGroup }
|
||||
self._rg_by_id = {}
|
||||
|
@ -91,8 +95,11 @@ class ResourceRequest(object):
|
|||
# TODO(efried): Handle member_of[$N], which will need to be reconciled
|
||||
# with destination.aggregates handling in resources_from_request_spec
|
||||
|
||||
image = (request_spec.image if 'image' in request_spec
|
||||
else objects.ImageMeta(properties=objects.ImageMetaProps()))
|
||||
# request_spec.image is nullable
|
||||
if 'image' in request_spec and request_spec.image:
|
||||
image = request_spec.image
|
||||
else:
|
||||
image = objects.ImageMeta(properties=objects.ImageMetaProps())
|
||||
|
||||
# Parse the flavor extra specs
|
||||
self._process_extra_specs(request_spec.flavor)
|
||||
|
@ -102,12 +109,21 @@ class ResourceRequest(object):
|
|||
# Now parse the (optional) image metadata
|
||||
self._process_image_meta(image)
|
||||
|
||||
# TODO(stephenfin): Remove this parameter once we drop support for
|
||||
# 'vcpu_pin_set'
|
||||
self.cpu_pinning_requested = False
|
||||
|
||||
if enable_pinning_translate:
|
||||
# Next up, let's handle those pesky CPU pinning policies
|
||||
self._translate_pinning_policies(request_spec.flavor, image)
|
||||
|
||||
# Finally, parse the flavor itself, though we'll only use these fields
|
||||
# if they don't conflict with something already provided by the flavor
|
||||
# extra specs. These are all added to the unnumbered request group.
|
||||
merged_resources = self.merged_resources()
|
||||
|
||||
if orc.VCPU not in merged_resources:
|
||||
if (orc.VCPU not in merged_resources and
|
||||
orc.PCPU not in merged_resources):
|
||||
self._add_resource(None, orc.VCPU, request_spec.vcpus)
|
||||
|
||||
if orc.MEMORY_MB not in merged_resources:
|
||||
|
@ -173,7 +189,7 @@ class ResourceRequest(object):
|
|||
# NOTE(aspiers): In theory this could raise FlavorImageConflict,
|
||||
# but we already check it in the API layer, so that should never
|
||||
# happen.
|
||||
if not hw.get_mem_encryption_constraint(flavor, image):
|
||||
if not hardware.get_mem_encryption_constraint(flavor, image):
|
||||
# No memory encryption required, so no further action required.
|
||||
return
|
||||
|
||||
|
@ -185,7 +201,7 @@ class ResourceRequest(object):
|
|||
"""When the hw:pmem extra spec is present, require hosts which can
|
||||
provide enough vpmem resources.
|
||||
"""
|
||||
vpmem_labels = hw.get_vpmems(flavor)
|
||||
vpmem_labels = hardware.get_vpmems(flavor)
|
||||
if not vpmem_labels:
|
||||
# No vpmems required
|
||||
return
|
||||
|
@ -199,6 +215,54 @@ class ResourceRequest(object):
|
|||
LOG.debug("Added resource %s=%d to requested resources",
|
||||
resource_class, amount)
|
||||
|
||||
def _translate_pinning_policies(self, flavor, image):
|
||||
"""Translate the legacy pinning policies to resource requests."""
|
||||
# NOTE(stephenfin): These can raise exceptions but these have already
|
||||
# been validated by 'nova.virt.hardware.numa_get_constraints' in the
|
||||
# API layer (see change I06fad233006c7bab14749a51ffa226c3801f951b).
|
||||
# This call also handles conflicts between explicit VCPU/PCPU
|
||||
# requests and implicit 'hw:cpu_policy'-based requests, mismatches
|
||||
# between the number of CPUs in the flavor and explicit VCPU/PCPU
|
||||
# requests, etc.
|
||||
cpu_policy = hardware.get_cpu_policy_constraint(
|
||||
flavor, image)
|
||||
cpu_thread_policy = hardware.get_cpu_thread_policy_constraint(
|
||||
flavor, image)
|
||||
emul_thread_policy = hardware.get_emulator_thread_policy_constraint(
|
||||
flavor)
|
||||
|
||||
# We don't need to worry about handling 'SHARED' - that will result in
|
||||
# VCPUs which we include by default
|
||||
if cpu_policy == obj_fields.CPUAllocationPolicy.DEDICATED:
|
||||
# TODO(stephenfin): Remove when we drop support for 'vcpu_pin_set'
|
||||
self.cpu_pinning_requested = True
|
||||
|
||||
# Switch VCPU -> PCPU
|
||||
cpus = flavor.vcpus
|
||||
|
||||
LOG.debug('Translating request for %(vcpu_rc)s=%(cpus)d to '
|
||||
'%(vcpu_rc)s=0,%(pcpu_rc)s=%(cpus)d',
|
||||
{'vcpu_rc': orc.VCPU, 'pcpu_rc': orc.PCPU,
|
||||
'cpus': cpus})
|
||||
|
||||
if emul_thread_policy == 'isolate':
|
||||
cpus += 1
|
||||
|
||||
LOG.debug('Adding additional %(pcpu_rc)s to account for '
|
||||
'emulator threads', {'pcpu_rc': orc.PCPU})
|
||||
|
||||
self._add_resource(None, orc.PCPU, cpus)
|
||||
|
||||
trait = {
|
||||
obj_fields.CPUThreadAllocationPolicy.ISOLATE: 'forbidden',
|
||||
obj_fields.CPUThreadAllocationPolicy.REQUIRE: 'required',
|
||||
}.get(cpu_thread_policy)
|
||||
if trait:
|
||||
LOG.debug('Adding %(trait)s=%(value)s trait',
|
||||
{'trait': os_traits.HW_CPU_HYPERTHREADING,
|
||||
'value': trait})
|
||||
self._add_trait(None, os_traits.HW_CPU_HYPERTHREADING, trait)
|
||||
|
||||
@property
|
||||
def group_policy(self):
|
||||
return self._group_policy
|
||||
|
@ -463,18 +527,21 @@ def resources_from_flavor(instance, flavor):
|
|||
return res_req.merged_resources()
|
||||
|
||||
|
||||
def resources_from_request_spec(ctxt, spec_obj, host_manager):
|
||||
def resources_from_request_spec(ctxt, spec_obj, host_manager,
|
||||
enable_pinning_translate=True):
|
||||
"""Given a RequestSpec object, returns a ResourceRequest of the resources,
|
||||
traits, and aggregates it represents.
|
||||
|
||||
:param context: The request context.
|
||||
:param spec_obj: A RequestSpec object.
|
||||
:param host_manager: A HostManager object.
|
||||
:param enable_pinning_translate: True if the CPU policy extra specs should
|
||||
be translated to placement resources and traits.
|
||||
|
||||
:return: A ResourceRequest object.
|
||||
:raises NoValidHost: If the specified host/node is not found in the DB.
|
||||
"""
|
||||
res_req = ResourceRequest(spec_obj)
|
||||
res_req = ResourceRequest(spec_obj, enable_pinning_translate)
|
||||
|
||||
requested_resources = (spec_obj.requested_resources
|
||||
if 'requested_resources' in spec_obj and
|
||||
|
|
|
@ -64,17 +64,20 @@ class ServersTestBase(base.ServersTestBase):
|
|||
self.useFixture(fixtures.MockPatch(
|
||||
'nova.privsep.utils.supports_direct_io',
|
||||
return_value=True))
|
||||
self.useFixture(fixtures.MockPatch(
|
||||
'nova.virt.libvirt.host.Host.get_online_cpus',
|
||||
return_value=set(range(16))))
|
||||
|
||||
# Mock the 'get_connection' function, as we're going to need to provide
|
||||
# custom capabilities for each test
|
||||
_p = mock.patch('nova.virt.libvirt.host.Host.get_connection')
|
||||
self.mock_conn = _p.start()
|
||||
self.addCleanup(_p.stop)
|
||||
# As above, mock the 'get_arch' function as we may need to provide
|
||||
# different host architectures during some tests.
|
||||
|
||||
# Mock the 'get_arch' function as we may need to provide different host
|
||||
# architectures during some tests. We default to x86_64
|
||||
_a = mock.patch('nova.virt.libvirt.utils.get_arch')
|
||||
self.mock_arch = _a.start()
|
||||
# Default to X86_64
|
||||
self.mock_arch.return_value = obj_fields.Architecture.X86_64
|
||||
self.addCleanup(_a.stop)
|
||||
|
||||
|
@ -96,6 +99,11 @@ class ServersTestBase(base.ServersTestBase):
|
|||
def _get_connection(self, host_info, pci_info=None,
|
||||
libvirt_version=fakelibvirt.FAKE_LIBVIRT_VERSION,
|
||||
mdev_info=None):
|
||||
# sanity check
|
||||
self.assertGreater(16, host_info.cpus,
|
||||
"Host.get_online_cpus is only accounting for 16 CPUs but you're "
|
||||
"requesting %d; change the mock or your test" % host_info.cpus)
|
||||
|
||||
fake_connection = fakelibvirt.Connection(
|
||||
'qemu:///system',
|
||||
version=libvirt_version,
|
||||
|
|
|
@ -53,6 +53,7 @@ class NUMAServersTestBase(base.ServersTestBase):
|
|||
class NUMAServersTest(NUMAServersTestBase):
|
||||
|
||||
def _run_build_test(self, flavor_id, end_status='ACTIVE',
|
||||
filter_called_on_error=True,
|
||||
expected_usage=None):
|
||||
|
||||
# NOTE(bhagyashris): Always use host as 'compute1' so that it's
|
||||
|
@ -87,15 +88,20 @@ class NUMAServersTest(NUMAServersTestBase):
|
|||
self.assertIn(created_server_id, server_ids)
|
||||
|
||||
# Validate the quota usage
|
||||
if end_status == 'ACTIVE':
|
||||
if filter_called_on_error and end_status == 'ACTIVE':
|
||||
quota_details = self.api.get_quota_detail()
|
||||
expected_core_usages = expected_usage.get(
|
||||
'VCPU', expected_usage.get('PCPU', 0))
|
||||
self.assertEqual(expected_core_usages,
|
||||
quota_details['cores']['in_use'])
|
||||
|
||||
# Validate that NUMATopologyFilter has been called
|
||||
self.assertTrue(self.mock_filter.called)
|
||||
# Validate that NUMATopologyFilter has been called or not called,
|
||||
# depending on whether this is expected to make it past placement or
|
||||
# not (hint: if it's a lack of VCPU/PCPU resources, it won't)
|
||||
if filter_called_on_error:
|
||||
self.assertTrue(self.mock_filter.called)
|
||||
else:
|
||||
self.assertFalse(self.mock_filter.called)
|
||||
|
||||
found_server = self._wait_for_state_change(found_server, 'BUILD')
|
||||
|
||||
|
@ -151,25 +157,28 @@ class NUMAServersTest(NUMAServersTestBase):
|
|||
|
||||
self._run_build_test(flavor_id, end_status='ERROR')
|
||||
|
||||
def test_create_server_with_pinning(self):
|
||||
def test_create_server_with_legacy_pinning_policy(self):
|
||||
"""Create a server using the legacy 'hw:cpu_policy' extra spec.
|
||||
|
||||
This should pass and result in a guest NUMA topology with pinned CPUs.
|
||||
"""
|
||||
|
||||
self.flags(cpu_dedicated_set='0-9', cpu_shared_set=None,
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set=None)
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=1, cpu_sockets=1,
|
||||
cpu_cores=5, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
fake_connection = self._get_connection(host_info=host_info)
|
||||
self.mock_conn.return_value = fake_connection
|
||||
|
||||
# Create a flavor
|
||||
extra_spec = {
|
||||
'hw:cpu_policy': 'dedicated',
|
||||
'hw:cpu_thread_policy': 'prefer',
|
||||
}
|
||||
flavor_id = self._create_flavor(vcpu=5, extra_spec=extra_spec)
|
||||
expected_usage = {'DISK_GB': 20, 'MEMORY_MB': 2048, 'VCPU': 5}
|
||||
expected_usage = {'DISK_GB': 20, 'MEMORY_MB': 2048, 'PCPU': 5}
|
||||
|
||||
server = self._run_build_test(flavor_id, expected_usage=expected_usage)
|
||||
|
||||
|
@ -178,11 +187,65 @@ class NUMAServersTest(NUMAServersTestBase):
|
|||
self.assertEqual(1, len(inst.numa_topology.cells))
|
||||
self.assertEqual(5, inst.numa_topology.cells[0].cpu_topology.cores)
|
||||
|
||||
def test_create_server_with_pinning_quota_fails(self):
|
||||
def test_create_server_with_legacy_pinning_policy_old_configuration(self):
|
||||
"""Create a server using the legacy extra spec and configuration.
|
||||
|
||||
This should pass and result in a guest NUMA topology with pinned CPUs,
|
||||
though we'll still be consuming VCPUs (which would in theory be fixed
|
||||
during a later reshape).
|
||||
"""
|
||||
|
||||
self.flags(cpu_dedicated_set=None, cpu_shared_set=None,
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set='0-7')
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
fake_connection = self._get_connection(host_info=host_info)
|
||||
self.mock_conn.return_value = fake_connection
|
||||
|
||||
extra_spec = {
|
||||
'hw:cpu_policy': 'dedicated',
|
||||
'hw:cpu_thread_policy': 'prefer',
|
||||
}
|
||||
flavor_id = self._create_flavor(extra_spec=extra_spec)
|
||||
expected_usage = {'DISK_GB': 20, 'MEMORY_MB': 2048, 'VCPU': 2}
|
||||
|
||||
self._run_build_test(flavor_id, expected_usage=expected_usage)
|
||||
|
||||
def test_create_server_with_legacy_pinning_policy_fails(self):
|
||||
"""Create a pinned instance on a host with no PCPUs.
|
||||
|
||||
This should fail because we're translating the extra spec and the host
|
||||
isn't reporting the PCPUs we need.
|
||||
"""
|
||||
|
||||
self.flags(cpu_shared_set='0-9', cpu_dedicated_set=None,
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set=None)
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=1, cpu_sockets=1,
|
||||
cpu_cores=5, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
fake_connection = self._get_connection(host_info=host_info)
|
||||
self.mock_conn.return_value = fake_connection
|
||||
|
||||
extra_spec = {
|
||||
'hw:cpu_policy': 'dedicated',
|
||||
'hw:cpu_thread_policy': 'prefer',
|
||||
}
|
||||
flavor_id = self._create_flavor(vcpu=5, extra_spec=extra_spec)
|
||||
self._run_build_test(flavor_id, end_status='ERROR')
|
||||
|
||||
def test_create_server_with_legacy_pinning_policy_quota_fails(self):
|
||||
"""Create a pinned instance on a host with PCPUs but not enough quota.
|
||||
|
||||
This should fail because the quota request should fail.
|
||||
"""
|
||||
self.flags(cpu_dedicated_set='0-7', cpu_shared_set=None,
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set=None)
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
|
@ -213,12 +276,101 @@ class NUMAServersTest(NUMAServersTestBase):
|
|||
self.api.post_server, post)
|
||||
self.assertEqual(403, ex.response.status_code)
|
||||
|
||||
def test_resize_unpinned_to_pinned(self):
|
||||
def test_create_server_with_pcpu(self):
|
||||
"""Create a server using an explicit 'resources:PCPU' request.
|
||||
|
||||
This should pass and result in a guest NUMA topology with pinned CPUs.
|
||||
"""
|
||||
|
||||
self.flags(cpu_dedicated_set='0-7', cpu_shared_set=None,
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set=None)
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
fake_connection = self._get_connection(host_info=host_info)
|
||||
self.mock_conn.return_value = fake_connection
|
||||
|
||||
extra_spec = {'resources:PCPU': '2'}
|
||||
flavor_id = self._create_flavor(vcpu=2, extra_spec=extra_spec)
|
||||
expected_usage = {'DISK_GB': 20, 'MEMORY_MB': 2048, 'PCPU': 2}
|
||||
|
||||
server = self._run_build_test(flavor_id, expected_usage=expected_usage)
|
||||
|
||||
ctx = nova_context.get_admin_context()
|
||||
inst = objects.Instance.get_by_uuid(ctx, server['id'])
|
||||
self.assertEqual(1, len(inst.numa_topology.cells))
|
||||
self.assertEqual(1, inst.numa_topology.cells[0].cpu_topology.cores)
|
||||
self.assertEqual(2, inst.numa_topology.cells[0].cpu_topology.threads)
|
||||
|
||||
def test_create_server_with_pcpu_fails(self):
|
||||
"""Create a pinned instance on a host with no PCPUs.
|
||||
|
||||
This should fail because we're explicitly requesting PCPUs and the host
|
||||
isn't reporting them.
|
||||
"""
|
||||
|
||||
self.flags(cpu_shared_set='0-9', cpu_dedicated_set=None,
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set=None)
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=1, cpu_sockets=1,
|
||||
cpu_cores=5, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
fake_connection = self._get_connection(host_info=host_info)
|
||||
self.mock_conn.return_value = fake_connection
|
||||
|
||||
extra_spec = {'resources:PCPU': 2}
|
||||
flavor_id = self._create_flavor(vcpu=2, extra_spec=extra_spec)
|
||||
self._run_build_test(flavor_id, end_status='ERROR',
|
||||
filter_called_on_error=False)
|
||||
|
||||
def test_create_server_with_pcpu_quota_fails(self):
|
||||
"""Create a pinned instance on a host with PCPUs but not enough quota.
|
||||
|
||||
This should fail because the quota request should fail.
|
||||
"""
|
||||
self.flags(cpu_dedicated_set='0-7', cpu_shared_set=None,
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set=None)
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
fake_connection = self._get_connection(host_info=host_info)
|
||||
self.mock_conn.return_value = fake_connection
|
||||
|
||||
extra_spec = {'resources:PCPU': '2'}
|
||||
flavor_id = self._create_flavor(vcpu=2, extra_spec=extra_spec)
|
||||
|
||||
# Update the core quota less than we requested
|
||||
self.api.update_quota({'cores': 1})
|
||||
|
||||
# NOTE(bhagyashris): Always use host as 'compute1' so that it's
|
||||
# possible to get resource provider information for verifying
|
||||
# compute usages. This host name 'compute1' is hard coded in
|
||||
# Connection class in fakelibvirt.py.
|
||||
# TODO(stephenfin): Remove the hardcoded limit, possibly overridding
|
||||
# 'start_service' to make sure there isn't a mismatch
|
||||
self.compute = self.start_service('compute', host='compute1')
|
||||
|
||||
post = {'server': self._build_server(flavor_id)}
|
||||
|
||||
ex = self.assertRaises(client.OpenStackApiException,
|
||||
self.api.post_server, post)
|
||||
self.assertEqual(403, ex.response.status_code)
|
||||
|
||||
def test_resize_vcpu_to_pcpu(self):
|
||||
"""Create an unpinned instance and resize it to a flavor with pinning.
|
||||
|
||||
This should pass and result in a guest NUMA topology with pinned CPUs.
|
||||
"""
|
||||
|
||||
self.flags(cpu_dedicated_set='0-3', cpu_shared_set='4-7',
|
||||
group='compute')
|
||||
self.flags(vcpu_pin_set=None)
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
|
@ -260,13 +412,11 @@ class NUMAServersTest(NUMAServersTestBase):
|
|||
original_host = server['OS-EXT-SRV-ATTR:host']
|
||||
|
||||
for host, compute_rp_uuid in self.compute_rp_uuids.items():
|
||||
# TODO(stephenfin): Both of these should report PCPU when we
|
||||
# support that
|
||||
if host == original_host: # the host with the instance
|
||||
expected_usage = {'VCPU': 2, 'DISK_GB': 20,
|
||||
expected_usage = {'VCPU': 2, 'PCPU': 0, 'DISK_GB': 20,
|
||||
'MEMORY_MB': 2048}
|
||||
else: # the other host
|
||||
expected_usage = {'VCPU': 0, 'DISK_GB': 0,
|
||||
expected_usage = {'VCPU': 0, 'PCPU': 0, 'DISK_GB': 0,
|
||||
'MEMORY_MB': 0}
|
||||
|
||||
compute_usage = self.placement_api.get(
|
||||
|
@ -299,16 +449,15 @@ class NUMAServersTest(NUMAServersTestBase):
|
|||
# resource usage has been updated
|
||||
|
||||
for host, compute_rp_uuid in self.compute_rp_uuids.items():
|
||||
# TODO(stephenfin): This should use PCPU when we support those
|
||||
if host == original_host:
|
||||
# the host that had the instance should still have allocations
|
||||
# since the resize hasn't been confirmed
|
||||
expected_usage = {'VCPU': 2, 'DISK_GB': 20,
|
||||
expected_usage = {'VCPU': 2, 'PCPU': 0, 'DISK_GB': 20,
|
||||
'MEMORY_MB': 2048}
|
||||
else:
|
||||
# the other host should have the new allocations replete with
|
||||
# PCPUs
|
||||
expected_usage = {'VCPU': 2, 'DISK_GB': 20,
|
||||
expected_usage = {'VCPU': 0, 'PCPU': 2, 'DISK_GB': 20,
|
||||
'MEMORY_MB': 2048}
|
||||
|
||||
compute_usage = self.placement_api.get(
|
||||
|
@ -329,16 +478,15 @@ class NUMAServersTest(NUMAServersTestBase):
|
|||
server = self._wait_for_state_change(server, 'ACTIVE')
|
||||
|
||||
for host, compute_rp_uuid in self.compute_rp_uuids.items():
|
||||
# TODO(stephenfin): This should use PCPU when we support those
|
||||
if host == original_host:
|
||||
# the host that had the instance should no longer have
|
||||
# alocations since the resize has been confirmed
|
||||
expected_usage = {'VCPU': 0, 'DISK_GB': 0,
|
||||
expected_usage = {'VCPU': 0, 'PCPU': 0, 'DISK_GB': 0,
|
||||
'MEMORY_MB': 0}
|
||||
else:
|
||||
# the other host should still have the new allocations replete
|
||||
# with PCPUs
|
||||
expected_usage = {'VCPU': 2, 'DISK_GB': 20,
|
||||
expected_usage = {'VCPU': 0, 'PCPU': 2, 'DISK_GB': 20,
|
||||
'MEMORY_MB': 2048}
|
||||
|
||||
compute_usage = self.placement_api.get(
|
||||
|
|
|
@ -285,6 +285,8 @@ class PCIServersTest(_PCIServersTestBase):
|
|||
assigned pci device.
|
||||
"""
|
||||
|
||||
self.flags(cpu_dedicated_set='0-7', group='compute')
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
|
@ -306,6 +308,8 @@ class PCIServersTest(_PCIServersTestBase):
|
|||
memory resources from one NUMA node and a PCI device from another.
|
||||
"""
|
||||
|
||||
self.flags(cpu_dedicated_set='0-7', group='compute')
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
|
@ -355,6 +359,8 @@ class PCIServersWithNUMAPoliciesTest(_PCIServersTestBase):
|
|||
NUMA policies are in use.
|
||||
"""
|
||||
|
||||
self.flags(cpu_dedicated_set='0-7', group='compute')
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
|
|
|
@ -46,6 +46,8 @@ class RealTimeServersTest(base.ServersTestBase):
|
|||
self.api.post_server, {'server': server})
|
||||
|
||||
def test_success(self):
|
||||
self.flags(cpu_dedicated_set='0-7', group='compute')
|
||||
|
||||
host_info = fakelibvirt.HostInfo(cpu_nodes=2, cpu_sockets=1,
|
||||
cpu_cores=2, cpu_threads=2,
|
||||
kB_mem=15740000)
|
||||
|
|
|
@ -25,6 +25,8 @@ from nova.scheduler import driver
|
|||
from nova.scheduler import host_manager
|
||||
|
||||
|
||||
# TODO(stephenfin): Rework these so they're functions instead of global
|
||||
# variables that can be mutated
|
||||
NUMA_TOPOLOGY = objects.NUMATopology(cells=[
|
||||
objects.NUMACell(
|
||||
id=0,
|
||||
|
@ -164,19 +166,22 @@ COMPUTE_NODES = [
|
|||
host='fake', hypervisor_hostname='fake-hyp'),
|
||||
]
|
||||
|
||||
ALLOC_REQS = [
|
||||
{
|
||||
'allocations': {
|
||||
cn.uuid: {
|
||||
'resources': {
|
||||
'VCPU': 1,
|
||||
'MEMORY_MB': 512,
|
||||
'DISK_GB': 512,
|
||||
},
|
||||
|
||||
def get_fake_alloc_reqs():
|
||||
return [
|
||||
{
|
||||
'allocations': {
|
||||
cn.uuid: {
|
||||
'resources': {
|
||||
'VCPU': 1,
|
||||
'MEMORY_MB': 512,
|
||||
'DISK_GB': 512,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
} for cn in COMPUTE_NODES
|
||||
]
|
||||
} for cn in COMPUTE_NODES
|
||||
]
|
||||
|
||||
|
||||
RESOURCE_PROVIDERS = [
|
||||
dict(
|
||||
|
|
|
@ -88,10 +88,13 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
fake_spec = objects.RequestSpec()
|
||||
fake_spec.instance_uuid = uuids.instance
|
||||
fake_version = "9.42"
|
||||
place_res = (fakes.ALLOC_REQS, mock.sentinel.p_sums, fake_version)
|
||||
mock_p_sums = mock.Mock()
|
||||
fake_alloc_reqs = fakes.get_fake_alloc_reqs()
|
||||
place_res = (fake_alloc_reqs, mock_p_sums, fake_version)
|
||||
mock_get_ac.return_value = place_res
|
||||
mock_rfrs.return_value.cpu_pinning_requested = False
|
||||
expected_alloc_reqs_by_rp_uuid = {
|
||||
cn.uuid: [fakes.ALLOC_REQS[x]]
|
||||
cn.uuid: [fake_alloc_reqs[x]]
|
||||
for x, cn in enumerate(fakes.COMPUTE_NODES)
|
||||
}
|
||||
with mock.patch.object(self.manager.driver, 'select_destinations'
|
||||
|
@ -102,7 +105,7 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
select_destinations.assert_called_once_with(
|
||||
self.context, fake_spec,
|
||||
[fake_spec.instance_uuid], expected_alloc_reqs_by_rp_uuid,
|
||||
mock.sentinel.p_sums, fake_version, False)
|
||||
mock_p_sums, fake_version, False)
|
||||
mock_get_ac.assert_called_once_with(
|
||||
self.context, mock_rfrs.return_value)
|
||||
|
||||
|
@ -114,7 +117,7 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
return_objects=True, return_alternates=True)
|
||||
select_destinations.assert_called_once_with(None, fake_spec,
|
||||
[fake_spec.instance_uuid], expected_alloc_reqs_by_rp_uuid,
|
||||
mock.sentinel.p_sums, fake_version, True)
|
||||
mock_p_sums, fake_version, True)
|
||||
|
||||
@mock.patch('nova.scheduler.request_filter.process_reqspec')
|
||||
@mock.patch('nova.scheduler.utils.resources_from_request_spec')
|
||||
|
@ -125,10 +128,13 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
fake_spec = objects.RequestSpec()
|
||||
fake_spec.instance_uuid = uuids.instance
|
||||
fake_version = "9.42"
|
||||
place_res = (fakes.ALLOC_REQS, mock.sentinel.p_sums, fake_version)
|
||||
mock_p_sums = mock.Mock()
|
||||
fake_alloc_reqs = fakes.get_fake_alloc_reqs()
|
||||
place_res = (fake_alloc_reqs, mock_p_sums, fake_version)
|
||||
mock_get_ac.return_value = place_res
|
||||
mock_rfrs.return_value.cpu_pinning_requested = False
|
||||
expected_alloc_reqs_by_rp_uuid = {
|
||||
cn.uuid: [fakes.ALLOC_REQS[x]]
|
||||
cn.uuid: [fake_alloc_reqs[x]]
|
||||
for x, cn in enumerate(fakes.COMPUTE_NODES)
|
||||
}
|
||||
with mock.patch.object(self.manager.driver, 'select_destinations'
|
||||
|
@ -148,7 +154,7 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
# driver should have been called with True for return_alternates.
|
||||
select_destinations.assert_called_once_with(None, fake_spec,
|
||||
[fake_spec.instance_uuid], expected_alloc_reqs_by_rp_uuid,
|
||||
mock.sentinel.p_sums, fake_version, True)
|
||||
mock_p_sums, fake_version, True)
|
||||
|
||||
# Now pass False for return objects, but keep return_alternates as
|
||||
# True. Verify that the manager converted the Selection object back
|
||||
|
@ -164,7 +170,7 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
# return_alternates as False.
|
||||
select_destinations.assert_called_once_with(None, fake_spec,
|
||||
[fake_spec.instance_uuid], expected_alloc_reqs_by_rp_uuid,
|
||||
mock.sentinel.p_sums, fake_version, False)
|
||||
mock_p_sums, fake_version, False)
|
||||
|
||||
@mock.patch('nova.scheduler.request_filter.process_reqspec')
|
||||
@mock.patch('nova.scheduler.utils.resources_from_request_spec')
|
||||
|
@ -176,6 +182,7 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
fake_spec.instance_uuid = uuids.instance
|
||||
place_res = get_allocation_candidates_response
|
||||
mock_get_ac.return_value = place_res
|
||||
mock_rfrs.return_value.cpu_pinning_requested = False
|
||||
with mock.patch.object(self.manager.driver, 'select_destinations'
|
||||
) as select_destinations:
|
||||
self.assertRaises(messaging.rpc.dispatcher.ExpectedException,
|
||||
|
@ -236,13 +243,17 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
@mock.patch('nova.scheduler.utils.resources_from_request_spec')
|
||||
@mock.patch('nova.scheduler.client.report.SchedulerReportClient.'
|
||||
'get_allocation_candidates')
|
||||
def test_select_destination_with_4_3_client(self, mock_get_ac, mock_rfrs,
|
||||
mock_process):
|
||||
def test_select_destination_with_4_3_client(
|
||||
self, mock_get_ac, mock_rfrs, mock_process,
|
||||
cpu_pinning_requested=False):
|
||||
fake_spec = objects.RequestSpec()
|
||||
place_res = (fakes.ALLOC_REQS, mock.sentinel.p_sums, "42.0")
|
||||
mock_p_sums = mock.Mock()
|
||||
fake_alloc_reqs = fakes.get_fake_alloc_reqs()
|
||||
place_res = (fake_alloc_reqs, mock_p_sums, "42.0")
|
||||
mock_get_ac.return_value = place_res
|
||||
mock_rfrs.return_value.cpu_pinning_requested = cpu_pinning_requested
|
||||
expected_alloc_reqs_by_rp_uuid = {
|
||||
cn.uuid: [fakes.ALLOC_REQS[x]]
|
||||
cn.uuid: [fake_alloc_reqs[x]]
|
||||
for x, cn in enumerate(fakes.COMPUTE_NODES)
|
||||
}
|
||||
with mock.patch.object(self.manager.driver, 'select_destinations'
|
||||
|
@ -251,10 +262,78 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
mock_process.assert_called_once_with(self.context, fake_spec)
|
||||
select_destinations.assert_called_once_with(self.context,
|
||||
fake_spec, None, expected_alloc_reqs_by_rp_uuid,
|
||||
mock.sentinel.p_sums, "42.0", False)
|
||||
mock_p_sums, "42.0", False)
|
||||
mock_rfrs.assert_called_once_with(
|
||||
self.context, fake_spec, mock.ANY,
|
||||
enable_pinning_translate=True)
|
||||
mock_get_ac.assert_called_once_with(
|
||||
self.context, mock_rfrs.return_value)
|
||||
|
||||
@mock.patch('nova.scheduler.manager.LOG.debug')
|
||||
@mock.patch('nova.scheduler.request_filter.process_reqspec')
|
||||
@mock.patch('nova.scheduler.utils.resources_from_request_spec')
|
||||
@mock.patch('nova.scheduler.client.report.SchedulerReportClient.'
|
||||
'get_allocation_candidates')
|
||||
def test_select_destination_with_pcpu_fallback(
|
||||
self, mock_get_ac, mock_rfrs, mock_process, mock_log):
|
||||
"""Check that we make a second request to placement if we've got a PCPU
|
||||
request.
|
||||
"""
|
||||
self.flags(disable_fallback_pcpu_query=False, group='workarounds')
|
||||
|
||||
# mock the result from placement. In reality, the two calls we expect
|
||||
# would return two different results, but we don't care about that. All
|
||||
# we want to check is that it _was_ called twice
|
||||
fake_spec = objects.RequestSpec()
|
||||
mock_p_sums = mock.Mock()
|
||||
fake_alloc_reqs = fakes.get_fake_alloc_reqs()
|
||||
place_res = (fake_alloc_reqs, mock_p_sums, "42.0")
|
||||
mock_get_ac.return_value = place_res
|
||||
|
||||
pcpu_rreq = mock.Mock()
|
||||
pcpu_rreq.cpu_pinning_requested = True
|
||||
vcpu_rreq = mock.Mock()
|
||||
mock_rfrs.side_effect = [pcpu_rreq, vcpu_rreq]
|
||||
|
||||
# as above, the two allocation requests against each compute node would
|
||||
# be different in reality, and not all compute nodes might have two
|
||||
# allocation requests, but that doesn't matter for this simple test
|
||||
expected_alloc_reqs_by_rp_uuid = {
|
||||
cn.uuid: [fake_alloc_reqs[x], fake_alloc_reqs[x]]
|
||||
for x, cn in enumerate(fakes.COMPUTE_NODES)
|
||||
}
|
||||
|
||||
with mock.patch.object(self.manager.driver, 'select_destinations'
|
||||
) as select_destinations:
|
||||
self.manager.select_destinations(self.context, spec_obj=fake_spec)
|
||||
select_destinations.assert_called_once_with(self.context,
|
||||
fake_spec, None, expected_alloc_reqs_by_rp_uuid,
|
||||
mock_p_sums, "42.0", False)
|
||||
|
||||
mock_process.assert_called_once_with(self.context, fake_spec)
|
||||
mock_log.assert_called_with(
|
||||
'Requesting fallback allocation candidates with VCPU instead of '
|
||||
'PCPU')
|
||||
mock_rfrs.assert_has_calls([
|
||||
mock.call(self.context, fake_spec, mock.ANY,
|
||||
enable_pinning_translate=True),
|
||||
mock.call(self.context, fake_spec, mock.ANY,
|
||||
enable_pinning_translate=False),
|
||||
])
|
||||
mock_get_ac.assert_has_calls([
|
||||
mock.call(self.context, pcpu_rreq),
|
||||
mock.call(self.context, vcpu_rreq),
|
||||
])
|
||||
|
||||
def test_select_destination_with_pcpu_fallback_disabled(self):
|
||||
"""Check that we do not make a second request to placement if we've
|
||||
been told not to, even though we've got a PCPU instance.
|
||||
"""
|
||||
self.flags(disable_fallback_pcpu_query=True, group='workarounds')
|
||||
|
||||
self.test_select_destination_with_4_3_client(
|
||||
cpu_pinning_requested=True)
|
||||
|
||||
# TODO(sbauza): Remove that test once the API v4 is removed
|
||||
@mock.patch('nova.scheduler.request_filter.process_reqspec')
|
||||
@mock.patch('nova.scheduler.utils.resources_from_request_spec')
|
||||
|
@ -266,10 +345,13 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
fake_spec = objects.RequestSpec()
|
||||
fake_spec.instance_uuid = uuids.instance
|
||||
from_primitives.return_value = fake_spec
|
||||
place_res = (fakes.ALLOC_REQS, mock.sentinel.p_sums, "42.0")
|
||||
mock_p_sums = mock.Mock()
|
||||
fake_alloc_reqs = fakes.get_fake_alloc_reqs()
|
||||
place_res = (fake_alloc_reqs, mock_p_sums, "42.0")
|
||||
mock_get_ac.return_value = place_res
|
||||
mock_rfrs.return_value.cpu_pinning_requested = False
|
||||
expected_alloc_reqs_by_rp_uuid = {
|
||||
cn.uuid: [fakes.ALLOC_REQS[x]]
|
||||
cn.uuid: [fake_alloc_reqs[x]]
|
||||
for x, cn in enumerate(fakes.COMPUTE_NODES)
|
||||
}
|
||||
with mock.patch.object(self.manager.driver, 'select_destinations'
|
||||
|
@ -282,7 +364,7 @@ class SchedulerManagerTestCase(test.NoDBTestCase):
|
|||
select_destinations.assert_called_once_with(
|
||||
self.context, fake_spec,
|
||||
[fake_spec.instance_uuid], expected_alloc_reqs_by_rp_uuid,
|
||||
mock.sentinel.p_sums, "42.0", False)
|
||||
mock_p_sums, "42.0", False)
|
||||
mock_get_ac.assert_called_once_with(
|
||||
self.context, mock_rfrs.return_value)
|
||||
|
||||
|
|
|
@ -919,6 +919,58 @@ class TestUtils(TestUtilsBase):
|
|||
)
|
||||
self.assertEqual(expected_querystring, rr.to_querystring())
|
||||
|
||||
def _test_resource_request_init_with_legacy_extra_specs(self):
|
||||
flavor = objects.Flavor(
|
||||
vcpus=1, memory_mb=1024, root_gb=10, ephemeral_gb=5, swap=0,
|
||||
extra_specs={
|
||||
'hw:cpu_policy': 'dedicated',
|
||||
'hw:cpu_thread_policy': 'isolate',
|
||||
'hw:emulator_threads_policy': 'isolate',
|
||||
})
|
||||
|
||||
return objects.RequestSpec(flavor=flavor, is_bfv=False)
|
||||
|
||||
def test_resource_request_init_with_legacy_extra_specs(self):
|
||||
expected = FakeResourceRequest()
|
||||
expected._rg_by_id[None] = objects.RequestGroup(
|
||||
use_same_provider=False,
|
||||
resources={
|
||||
# we should have two PCPUs, one due to hw:cpu_policy and the
|
||||
# other due to hw:cpu_thread_policy
|
||||
'PCPU': 2,
|
||||
'MEMORY_MB': 1024,
|
||||
'DISK_GB': 15,
|
||||
},
|
||||
forbidden_traits={
|
||||
# we should forbid hyperthreading due to hw:cpu_thread_policy
|
||||
'HW_CPU_HYPERTHREADING',
|
||||
},
|
||||
)
|
||||
rs = self._test_resource_request_init_with_legacy_extra_specs()
|
||||
rr = utils.ResourceRequest(rs)
|
||||
self.assertResourceRequestsEqual(expected, rr)
|
||||
self.assertTrue(rr.cpu_pinning_requested)
|
||||
|
||||
def test_resource_request_init_with_legacy_extra_specs_no_translate(self):
|
||||
expected = FakeResourceRequest()
|
||||
expected._rg_by_id[None] = objects.RequestGroup(
|
||||
use_same_provider=False,
|
||||
resources={
|
||||
# we should have a VCPU despite hw:cpu_policy because
|
||||
# enable_pinning_translate=False
|
||||
'VCPU': 1,
|
||||
'MEMORY_MB': 1024,
|
||||
'DISK_GB': 15,
|
||||
},
|
||||
# we should not require hyperthreading despite hw:cpu_thread_policy
|
||||
# because enable_pinning_translate=False
|
||||
forbidden_traits=set(),
|
||||
)
|
||||
rs = self._test_resource_request_init_with_legacy_extra_specs()
|
||||
rr = utils.ResourceRequest(rs, enable_pinning_translate=False)
|
||||
self.assertResourceRequestsEqual(expected, rr)
|
||||
self.assertFalse(rr.cpu_pinning_requested)
|
||||
|
||||
def test_resource_request_init_with_image_props(self):
|
||||
flavor = objects.Flavor(
|
||||
vcpus=1, memory_mb=1024, root_gb=10, ephemeral_gb=5, swap=0)
|
||||
|
@ -945,6 +997,58 @@ class TestUtils(TestUtilsBase):
|
|||
rr = utils.ResourceRequest(rs)
|
||||
self.assertResourceRequestsEqual(expected, rr)
|
||||
|
||||
def _test_resource_request_init_with_legacy_image_props(self):
|
||||
flavor = objects.Flavor(
|
||||
vcpus=1, memory_mb=1024, root_gb=10, ephemeral_gb=5, swap=0)
|
||||
image = objects.ImageMeta.from_dict({
|
||||
'properties': {
|
||||
'hw_cpu_policy': 'dedicated',
|
||||
'hw_cpu_thread_policy': 'require',
|
||||
},
|
||||
'id': 'c8b1790e-a07d-4971-b137-44f2432936cd',
|
||||
})
|
||||
return objects.RequestSpec(flavor=flavor, image=image, is_bfv=False)
|
||||
|
||||
def test_resource_request_init_with_legacy_image_props(self):
|
||||
expected = FakeResourceRequest()
|
||||
expected._rg_by_id[None] = objects.RequestGroup(
|
||||
use_same_provider=False,
|
||||
resources={
|
||||
# we should have a PCPU due to hw_cpu_policy
|
||||
'PCPU': 1,
|
||||
'MEMORY_MB': 1024,
|
||||
'DISK_GB': 15,
|
||||
},
|
||||
required_traits={
|
||||
# we should require hyperthreading due to hw_cpu_thread_policy
|
||||
'HW_CPU_HYPERTHREADING',
|
||||
},
|
||||
)
|
||||
rs = self._test_resource_request_init_with_legacy_image_props()
|
||||
rr = utils.ResourceRequest(rs)
|
||||
self.assertResourceRequestsEqual(expected, rr)
|
||||
self.assertTrue(rr.cpu_pinning_requested)
|
||||
|
||||
def test_resource_request_init_with_legacy_image_props_no_translate(self):
|
||||
expected = FakeResourceRequest()
|
||||
expected._rg_by_id[None] = objects.RequestGroup(
|
||||
use_same_provider=False,
|
||||
resources={
|
||||
# we should have a VCPU despite hw_cpu_policy because
|
||||
# enable_pinning_translate=False
|
||||
'VCPU': 1,
|
||||
'MEMORY_MB': 1024,
|
||||
'DISK_GB': 15,
|
||||
},
|
||||
# we should not require hyperthreading despite hw_cpu_thread_policy
|
||||
# because enable_pinning_translate=False
|
||||
required_traits=set(),
|
||||
)
|
||||
rs = self._test_resource_request_init_with_legacy_image_props()
|
||||
rr = utils.ResourceRequest(rs, enable_pinning_translate=False)
|
||||
self.assertResourceRequestsEqual(expected, rr)
|
||||
self.assertFalse(rr.cpu_pinning_requested)
|
||||
|
||||
def test_resource_request_init_is_bfv(self):
|
||||
flavor = objects.Flavor(
|
||||
vcpus=1, memory_mb=1024, root_gb=10, ephemeral_gb=5, swap=1555)
|
||||
|
|
|
@ -1330,6 +1330,109 @@ class NUMATopologyTest(test.NoDBTestCase):
|
|||
id=0, cpuset=set([0, 1, 2, 3]), memory=2048),
|
||||
]),
|
||||
},
|
||||
{
|
||||
# We request PCPUs explicitly
|
||||
"flavor": objects.Flavor(vcpus=4, memory_mb=2048,
|
||||
extra_specs={
|
||||
"resources:PCPU": "4",
|
||||
}),
|
||||
"image": {
|
||||
"properties": {}
|
||||
},
|
||||
"expect": objects.InstanceNUMATopology(
|
||||
cells=[
|
||||
objects.InstanceNUMACell(
|
||||
id=0, cpuset=set([0, 1, 2, 3]), memory=2048,
|
||||
cpu_policy=fields.CPUAllocationPolicy.DEDICATED,
|
||||
)]),
|
||||
},
|
||||
{
|
||||
# We request the HW_CPU_HYPERTHREADING trait explicitly
|
||||
"flavor": objects.Flavor(vcpus=4, memory_mb=2048,
|
||||
extra_specs={
|
||||
"resources:PCPU": "4",
|
||||
"trait:HW_CPU_HYPERTHREADING": "forbidden",
|
||||
}),
|
||||
"image": {
|
||||
"properties": {}
|
||||
},
|
||||
"expect": objects.InstanceNUMATopology(
|
||||
cells=[
|
||||
objects.InstanceNUMACell(
|
||||
id=0, cpuset=set([0, 1, 2, 3]), memory=2048,
|
||||
cpu_policy=fields.CPUAllocationPolicy.DEDICATED,
|
||||
cpu_thread_policy=
|
||||
fields.CPUThreadAllocationPolicy.ISOLATE,
|
||||
)]),
|
||||
},
|
||||
{
|
||||
# Requesting both implicit and explicit PCPUs
|
||||
"flavor": objects.Flavor(vcpus=4, memory_mb=2048,
|
||||
extra_specs={
|
||||
"hw:cpu_policy": fields.CPUAllocationPolicy.DEDICATED,
|
||||
"resources:PCPU": "4",
|
||||
}),
|
||||
"image": {"properties": {}},
|
||||
"expect": exception.InvalidRequest,
|
||||
},
|
||||
{
|
||||
# Requesting both PCPUs and VCPUs
|
||||
"flavor": objects.Flavor(vcpus=4, memory_mb=2048,
|
||||
extra_specs={
|
||||
"resources:PCPU": "2",
|
||||
"resources:VCPU": "2",
|
||||
}),
|
||||
"image": {"properties": {}},
|
||||
"expect": exception.InvalidRequest,
|
||||
},
|
||||
{
|
||||
# Mismatch between PCPU requests and flavor.vcpus
|
||||
"flavor": objects.Flavor(vcpus=4, memory_mb=2048,
|
||||
extra_specs={
|
||||
"resources:PCPU": "5",
|
||||
}),
|
||||
"image": {"properties": {}},
|
||||
"expect": exception.InvalidRequest,
|
||||
},
|
||||
{
|
||||
# Mismatch between PCPU requests and flavor.vcpus with
|
||||
# 'isolate' emulator thread policy
|
||||
"flavor": objects.Flavor(vcpus=4, memory_mb=2048,
|
||||
extra_specs={
|
||||
"hw:emulator_threads_policy": "isolate",
|
||||
"resources:PCPU": "4",
|
||||
}),
|
||||
"image": {"properties": {}},
|
||||
"expect": exception.InvalidRequest,
|
||||
},
|
||||
{
|
||||
# Mismatch between implicit and explicit HW_CPU_HYPERTHREADING
|
||||
# trait (flavor)
|
||||
"flavor": objects.Flavor(vcpus=4, memory_mb=2048,
|
||||
extra_specs={
|
||||
"hw:cpu_thread_policy":
|
||||
fields.CPUThreadAllocationPolicy.ISOLATE,
|
||||
"trait:HW_CPU_HYPERTHREADING": "required",
|
||||
}),
|
||||
"image": {"properties": {}},
|
||||
"expect": exception.InvalidRequest,
|
||||
},
|
||||
{
|
||||
# Mismatch between implicit and explicit HW_CPU_HYPERTHREADING
|
||||
# trait (image)
|
||||
"flavor": objects.Flavor(vcpus=4, name='foo', memory_mb=2048,
|
||||
extra_specs={
|
||||
"hw:cpu_policy": fields.CPUAllocationPolicy.DEDICATED,
|
||||
"hw:cpu_thread_policy":
|
||||
fields.CPUThreadAllocationPolicy.ISOLATE,
|
||||
}),
|
||||
"image": {
|
||||
"properties": {
|
||||
"trait:HW_CPU_HYPERTHREADING": "required",
|
||||
}
|
||||
},
|
||||
"expect": exception.InvalidRequest,
|
||||
},
|
||||
]
|
||||
|
||||
for testitem in testdata:
|
||||
|
|
|
@ -16,7 +16,10 @@ import collections
|
|||
import fractions
|
||||
import itertools
|
||||
import math
|
||||
import re
|
||||
|
||||
import os_resource_classes as orc
|
||||
import os_traits
|
||||
from oslo_log import log as logging
|
||||
from oslo_utils import strutils
|
||||
from oslo_utils import units
|
||||
|
@ -1475,7 +1478,8 @@ def _get_numa_node_count_constraint(flavor, image_meta):
|
|||
return int(nodes) if nodes else nodes
|
||||
|
||||
|
||||
def _get_cpu_policy_constraint(flavor, image_meta):
|
||||
# NOTE(stephenfin): This must be public as it's used elsewhere
|
||||
def get_cpu_policy_constraint(flavor, image_meta):
|
||||
# type: (objects.Flavor, objects.ImageMeta) -> Optional[str]
|
||||
"""Validate and return the requested CPU policy.
|
||||
|
||||
|
@ -1511,12 +1515,13 @@ def _get_cpu_policy_constraint(flavor, image_meta):
|
|||
elif image_policy == fields.CPUAllocationPolicy.DEDICATED:
|
||||
cpu_policy = image_policy
|
||||
else:
|
||||
cpu_policy = fields.CPUAllocationPolicy.SHARED
|
||||
cpu_policy = None
|
||||
|
||||
return cpu_policy
|
||||
|
||||
|
||||
def _get_cpu_thread_policy_constraint(flavor, image_meta):
|
||||
# NOTE(stephenfin): This must be public as it's used elsewhere
|
||||
def get_cpu_thread_policy_constraint(flavor, image_meta):
|
||||
# type: (objects.Flavor, objects.ImageMeta) -> Optional[str]
|
||||
"""Validate and return the requested CPU thread policy.
|
||||
|
||||
|
@ -1615,6 +1620,40 @@ def is_realtime_enabled(flavor):
|
|||
return strutils.bool_from_string(flavor_rt)
|
||||
|
||||
|
||||
def _get_vcpu_pcpu_resources(flavor):
|
||||
# type: (objects.Flavor) -> Tuple[bool, bool]
|
||||
requested_vcpu = 0
|
||||
requested_pcpu = 0
|
||||
|
||||
for key, val in flavor.get('extra_specs', {}).items():
|
||||
if re.match('resources([1-9][0-9]*)?:%s' % orc.VCPU, key):
|
||||
try:
|
||||
requested_vcpu += int(val)
|
||||
except ValueError:
|
||||
# this is handled elsewhere
|
||||
pass
|
||||
if re.match('resources([1-9][0-9]*)?:%s' % orc.PCPU, key):
|
||||
try:
|
||||
requested_pcpu += int(val)
|
||||
except ValueError:
|
||||
# this is handled elsewhere
|
||||
pass
|
||||
|
||||
return (requested_vcpu, requested_pcpu)
|
||||
|
||||
|
||||
def _get_hyperthreading_trait(flavor, image_meta):
|
||||
# type: (objects.Flavor, objects.ImageMeta) -> Optional[str]
|
||||
for key, val in flavor.get('extra_specs', {}).items():
|
||||
if re.match('trait([1-9][0-9]*)?:%s' % os_traits.HW_CPU_HYPERTHREADING,
|
||||
key):
|
||||
return val
|
||||
|
||||
if os_traits.HW_CPU_HYPERTHREADING in image_meta.properties.get(
|
||||
'traits_required', []):
|
||||
return 'required'
|
||||
|
||||
|
||||
def _get_realtime_constraint(flavor, image_meta):
|
||||
# type: (objects.Flavor, objects.ImageMeta) -> Optional[str]
|
||||
"""Validate and return the requested realtime CPU mask.
|
||||
|
@ -1721,6 +1760,8 @@ def numa_get_constraints(flavor, image_meta):
|
|||
invalid value in image or flavor.
|
||||
:raises: exception.InvalidCPUThreadAllocationPolicy if policy is defined
|
||||
with invalid value in image or flavor.
|
||||
:raises: exception.InvalidRequest if there is a conflict between explicitly
|
||||
and implicitly requested resources of hyperthreading traits
|
||||
:returns: objects.InstanceNUMATopology, or None
|
||||
"""
|
||||
numa_topology = None
|
||||
|
@ -1756,14 +1797,69 @@ def numa_get_constraints(flavor, image_meta):
|
|||
for c in numa_topology.cells:
|
||||
setattr(c, 'pagesize', pagesize)
|
||||
|
||||
cpu_policy = _get_cpu_policy_constraint(flavor, image_meta)
|
||||
cpu_thread_policy = _get_cpu_thread_policy_constraint(flavor, image_meta)
|
||||
cpu_policy = get_cpu_policy_constraint(flavor, image_meta)
|
||||
cpu_thread_policy = get_cpu_thread_policy_constraint(flavor, image_meta)
|
||||
rt_mask = _get_realtime_constraint(flavor, image_meta)
|
||||
emu_threads_policy = get_emulator_thread_policy_constraint(flavor)
|
||||
|
||||
# handle explicit VCPU/PCPU resource requests and the HW_CPU_HYPERTHREADING
|
||||
# trait
|
||||
|
||||
requested_vcpus, requested_pcpus = _get_vcpu_pcpu_resources(flavor)
|
||||
|
||||
if cpu_policy and (requested_vcpus or requested_pcpus):
|
||||
# TODO(stephenfin): Make these custom exceptions
|
||||
raise exception.InvalidRequest(
|
||||
"It is not possible to use the 'resources:VCPU' or "
|
||||
"'resources:PCPU' extra specs in combination with the "
|
||||
"'hw:cpu_policy' extra spec or 'hw_cpu_policy' image metadata "
|
||||
"property; use one or the other")
|
||||
|
||||
if requested_vcpus and requested_pcpus:
|
||||
raise exception.InvalidRequest(
|
||||
"It is not possible to specify both 'resources:VCPU' and "
|
||||
"'resources:PCPU' extra specs; use one or the other")
|
||||
|
||||
if requested_pcpus:
|
||||
if (emu_threads_policy == fields.CPUEmulatorThreadsPolicy.ISOLATE and
|
||||
flavor.vcpus + 1 != requested_pcpus):
|
||||
raise exception.InvalidRequest(
|
||||
"You have requested 'hw:emulator_threads_policy=isolate' but "
|
||||
"have not requested sufficient PCPUs to handle this policy; "
|
||||
"you must allocate exactly flavor.vcpus + 1 PCPUs.")
|
||||
|
||||
if (emu_threads_policy != fields.CPUEmulatorThreadsPolicy.ISOLATE and
|
||||
flavor.vcpus != requested_pcpus):
|
||||
raise exception.InvalidRequest(
|
||||
"There is a mismatch between the number of PCPUs requested "
|
||||
"via 'resourcesNN:PCPU' and the flavor); you must allocate "
|
||||
"exactly flavor.vcpus PCPUs")
|
||||
|
||||
cpu_policy = fields.CPUAllocationPolicy.DEDICATED
|
||||
|
||||
if requested_vcpus:
|
||||
# NOTE(stephenfin): It would be nice if we could error out if
|
||||
# flavor.vcpus != resources:PCPU, but that would be a breaking change.
|
||||
# Better to wait until we remove flavor.vcpus or something
|
||||
cpu_policy = fields.CPUAllocationPolicy.SHARED
|
||||
|
||||
hyperthreading_trait = _get_hyperthreading_trait(flavor, image_meta)
|
||||
|
||||
if cpu_thread_policy and hyperthreading_trait:
|
||||
raise exception.InvalidRequest(
|
||||
"It is not possible to use the 'trait:HW_CPU_HYPERTHREADING' "
|
||||
"extra spec in combination with the 'hw:cpu_thread_policy' "
|
||||
"extra spec or 'hw_cpu_thread_policy' image metadata property; "
|
||||
"use one or the other")
|
||||
|
||||
if hyperthreading_trait == 'forbidden':
|
||||
cpu_thread_policy = fields.CPUThreadAllocationPolicy.ISOLATE
|
||||
elif hyperthreading_trait == 'required':
|
||||
cpu_thread_policy = fields.CPUThreadAllocationPolicy.REQUIRE
|
||||
|
||||
# sanity checks
|
||||
|
||||
if cpu_policy == fields.CPUAllocationPolicy.SHARED:
|
||||
if cpu_policy in (fields.CPUAllocationPolicy.SHARED, None):
|
||||
if cpu_thread_policy:
|
||||
raise exception.CPUThreadPolicyConfigurationInvalid()
|
||||
|
||||
|
|
|
@ -3,15 +3,36 @@ features:
|
|||
- |
|
||||
Compute nodes using the libvirt driver can now report ``PCPU`` inventory.
|
||||
This is consumed by instances with dedicated (pinned) CPUs. This can be
|
||||
configured using the ``[compute] cpu_dedicated_set`` config option. A
|
||||
legacy path using the now deprecated ``vcpu_pin_set`` config option is
|
||||
provided to assist with upgrades. Refer to the help text of the ``[compute]
|
||||
cpu_dedicated_set``, ``[compute] cpu_shared_set`` and ``vcpu_pin_set``
|
||||
config options for more information.
|
||||
configured using the ``[compute] cpu_dedicated_set`` config option. The
|
||||
scheduler will automatically translate the legacy ``hw:cpu_policy`` flavor
|
||||
extra spec or ``hw_cpu_policy`` image metadata property to ``PCPU``
|
||||
requests, falling back to ``VCPU`` requests only if no ``PCPU`` candidates
|
||||
are found. Refer to the help text of the ``[compute] cpu_dedicated_set``,
|
||||
``[compute] cpu_shared_set`` and ``vcpu_pin_set`` config options for more
|
||||
information.
|
||||
- |
|
||||
Compute nodes using the libvirt driver will now report the
|
||||
``HW_CPU_HYPERTHREADING`` trait if the host has hyperthreading. The
|
||||
scheduler will automatically translate the legacy ``hw:cpu_thread_policy``
|
||||
flavor extra spec or ``hw_cpu_thread_policy`` image metadata property to
|
||||
either require or forbid this trait.
|
||||
- |
|
||||
A new configuration option, ``[compute] cpu_dedicated_set``, has been
|
||||
added. This can be used to configure the host CPUs that should be used for
|
||||
``PCPU`` inventory.
|
||||
- |
|
||||
A new configuration option, ``[workarounds] disable_fallback_pcpu_query``,
|
||||
has been added. When creating or moving pinned instances, the scheduler will
|
||||
attempt to provide a ``PCPU``-based allocation, but can also fallback to a legacy
|
||||
``VCPU``-based allocation. This fallback behavior is enabled by
|
||||
default to ensure it is possible to upgrade without having to modify compute
|
||||
node configuration but it results in an additional request for allocation
|
||||
candidates from placement. This can have a slight performance impact and is
|
||||
unnecessary on new or upgraded deployments where the compute nodes have been
|
||||
correctly configured to report ``PCPU`` inventory. The ``[workarounds]
|
||||
disable_fallback_pcpu_query`` config option can be used to disable this
|
||||
fallback allocation candidate request, meaning only ``PCPU``-based
|
||||
allocation candidates will be retrieved.
|
||||
deprecations:
|
||||
- |
|
||||
The ``vcpu_pin_set`` configuration option has been deprecated. You should
|
||||
|
|
Loading…
Reference in New Issue