openstack
/
nova
Code Issues Proposed changes

Merge "Detect maximum number of SEV guests automatically"

This commit is contained in:
Zuul 2023-11-23 18:13:25 +00:00 committed by Gerrit Code Review
parent 670ba207bc 03055de176
commit 1738b52c30
9 changed files with 140 additions and 63 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
doc/source/admin/sev.rst
View File

@ -91,32 +91,31 @@ steps:
needs to track how many slots are available and used in order to
avoid attempting to exceed that limit in the hardware.
At the time of writing (September 2019), work is in progress to
allow QEMU and libvirt to expose the number of slots available on
SEV hardware; however until this is finished and released, it will
not be possible for Nova to programmatically detect the correct
value.
Since version 8.0.0, libvirt exposes maximun mumber of SEV guests
which can run concurrently in its host, so the limit is automatically
detected using this feature.
So this configuration option serves as a stop-gap, allowing the
cloud operator the option of providing this value manually. It may
later be demoted to a fallback value for cases where the limit
cannot be detected programmatically, or even removed altogether when
Nova's minimum QEMU version guarantees that it can always be
detected.
However in case an older version of libvirt is used, it is not possible for
Nova to programmatically detect the correct value and Nova imposes no limit.
So this configuration option serves as a stop-gap, allowing the cloud
operator the option of providing this value manually.
This option also allows the cloud operator to set the limit lower than
the actual hard limit.
.. note::
When deciding whether to use the default of ``None`` or manually
impose a limit, operators should carefully weigh the benefits
vs. the risk. The benefits of using the default are a) immediate
convenience since nothing needs to be done now, and b) convenience
later when upgrading compute hosts to future versions of Nova,
since again nothing will need to be done for the correct limit to
be automatically imposed. However the risk is that until
auto-detection is implemented, users may be able to attempt to
launch guests with encrypted memory on hosts which have already
reached the maximum number of guests simultaneously running with
encrypted memory. This risk may be mitigated by other limitations
If libvirt older than 8.0.0 is used, operators should carefully weigh
the benefits vs. the risk when deciding whether to use the default of
``None`` or manually impose a limit.
The benefits of using the default are a) immediate convenience since
nothing needs to be done now, and b) convenience later when upgrading
compute hosts to future versions of libvirt, since again nothing will
need to be done for the correct limit to be automatically imposed.
However the risk is that until auto-detection is implemented, users may
be able to attempt to launch guests with encrypted memory on hosts which
have already reached the maximum number of guests simultaneously running
with encrypted memory. This risk may be mitigated by other limitations
which operators can impose, for example if the smallest RAM
footprint of any flavor imposes a maximum number of simultaneously
running guests which is less than or equal to the SEV limit.
@ -221,16 +220,6 @@ features:
include using ``hw_disk_bus=scsi`` with
``hw_scsi_model=virtio-scsi`` , or ``hw_disk_bus=sata``.
- QEMU and libvirt cannot yet expose the number of slots available for
encrypted guests in the memory controller on SEV hardware. Until
this is implemented, it is not possible for Nova to programmatically
detect the correct value. As a short-term workaround, operators can
optionally manually specify the upper limit of SEV guests for each
compute host, via the new
:oslo.config:option:`libvirt.num_memory_encrypted_guests`
configuration option :ref:`described above
<num_memory_encrypted_guests>`.
Permanent limitations
~~~~~~~~~~~~~~~~~~~~~

17
nova/conf/libvirt.py
View File

@ -854,14 +854,15 @@ The option may be reused for other equivalent technologies in the
future. If the machine does not support memory encryption, the option
will be ignored and inventory will be set to 0.
If the machine does support memory encryption, *for now* a value of
``None`` means an effectively unlimited inventory, i.e. no limit will
be imposed by Nova on the number of SEV guests which can be launched,
even though the underlying hardware will enforce its own limit.
However it is expected that in the future, auto-detection of the
inventory from the hardware will become possible, at which point
``None`` will cause auto-detection to automatically impose the correct
limit.
If the machine does support memory encryption and this option is not set,
the driver detects maximum number of SEV guests from the libvirt API which
is available since v8.0.0. Setting this option overrides the detected limit,
unless the given value is not larger than the detected limit.
On the other hand, if an older version of libvirt is used, ``None`` means
an effectively unlimited inventory, i.e. no limit will be imposed by Nova
on the number of SEV guests which can be launched, even though the underlying
hardware will enforce its own limit.
.. note::

10
nova/tests/fixtures/libvirt.py vendored
View File

@ -1990,6 +1990,16 @@ class Connection(object):
_domain_capability_features_with_SEV_unsupported = \
_domain_capability_features_with_SEV.replace('yes', 'no')
_domain_capability_features_with_SEV_max_guests = ''' <features>
<gic supported='no'/>
<sev supported='yes'>
<cbitpos>47</cbitpos>
<reducedPhysBits>1</reducedPhysBits>
<maxGuests>100</maxGuests>
<maxESGuests>15</maxESGuests>
</sev>
</features>'''
def getCapabilities(self):
"""Return spoofed capabilities."""
numa_topology = self.host_info.numa_topology

32
nova/tests/unit/virt/libvirt/test_driver.py
View File

@ -29368,7 +29368,7 @@ class TestLibvirtSEVUnsupported(TestLibvirtSEV):
@mock.patch.object(vc, '_domain_capability_features',
new=vc._domain_capability_features_with_SEV)
class TestLibvirtSEVSupported(TestLibvirtSEV):
class TestLibvirtSEVSupportedNoMaxGuests(TestLibvirtSEV):
"""Libvirt driver tests for when AMD SEV support is present."""
@test.patch_exists(SEV_KERNEL_PARAM_FILE, True)
@test.patch_open(SEV_KERNEL_PARAM_FILE, "1\n")
@ -29389,6 +29389,36 @@ class TestLibvirtSEVSupported(TestLibvirtSEV):
self.assertEqual(0, self.driver._get_memory_encrypted_slots())
@mock.patch.object(vc, '_domain_capability_features',
new=vc._domain_capability_features_with_SEV_max_guests)
class TestLibvirtSEVSupportedMaxGuests(TestLibvirtSEV):
"""Libvirt driver tests for when AMD SEV support is present."""
@test.patch_exists(SEV_KERNEL_PARAM_FILE, True)
@test.patch_open(SEV_KERNEL_PARAM_FILE, "1\n")
@mock.patch.object(libvirt_driver.LOG, 'warning')
def test_get_mem_encrypted_slots_no_override(self, mock_log):
self.assertEqual(100, self.driver._get_memory_encrypted_slots())
mock_log.assert_not_called()
@test.patch_exists(SEV_KERNEL_PARAM_FILE, True)
@test.patch_open(SEV_KERNEL_PARAM_FILE, "1\n")
@mock.patch.object(libvirt_driver.LOG, 'warning')
def test_get_mem_encrypted_slots_overlide_more(self, mock_log):
self.flags(num_memory_encrypted_guests=120, group='libvirt')
self.assertEqual(100, self.driver._get_memory_encrypted_slots())
mock_log.assert_called_with(
'Host is configured with libvirt.num_memory_encrypted_guests '
'set to %d, but supports only %d.', 120, 100)
@test.patch_exists(SEV_KERNEL_PARAM_FILE, True)
@test.patch_open(SEV_KERNEL_PARAM_FILE, "1\n")
@mock.patch.object(libvirt_driver.LOG, 'warning')
def test_get_mem_encrypted_slots_override_less(self, mock_log):
self.flags(num_memory_encrypted_guests=80, group='libvirt')
self.assertEqual(80, self.driver._get_memory_encrypted_slots())
mock_log.assert_not_called()
class LibvirtPMEMNamespaceTests(test.NoDBTestCase):
def setUp(self):

18
nova/tests/unit/virt/libvirt/test_host.py
View File

@ -890,6 +890,8 @@ class HostTestCase(test.NoDBTestCase):
if supported:
self.assertEqual(47, sev.cbitpos)
self.assertEqual(1, sev.reduced_phys_bits)
self.assertIsNone(sev.max_guests)
self.assertIsNone(sev.max_es_guests)
@mock.patch.object(
fakelibvirt.virConnect, '_domain_capability_features', new=
@ -904,6 +906,22 @@ class HostTestCase(test.NoDBTestCase):
def test_get_domain_capabilities_sev_supported(self):
self._test_get_domain_capabilities_sev(True)
@mock.patch.object(
fakelibvirt.virConnect, '_domain_capability_features', new=
fakelibvirt.virConnect._domain_capability_features_with_SEV_max_guests)
def test_get_domain_capabilities_sev_max_guests(self):
caps = self._test_get_domain_capabilities()
self.assertEqual(vconfig.LibvirtConfigDomainCaps, type(caps))
features = caps.features
self.assertEqual(1, len(features))
sev = features[0]
self.assertEqual(vconfig.LibvirtConfigDomainCapsFeatureSev, type(sev))
self.assertTrue(sev.supported)
self.assertEqual(47, sev.cbitpos)
self.assertEqual(1, sev.reduced_phys_bits)
self.assertEqual(100, sev.max_guests)
self.assertEqual(15, sev.max_es_guests)
@mock.patch.object(fakelibvirt.virConnect, "getHostname")
def test_get_hostname_caching(self, mock_hostname):
mock_hostname.return_value = "foo"

6
nova/virt/libvirt/config.py
View File

@ -310,6 +310,8 @@ class LibvirtConfigDomainCapsFeatureSev(LibvirtConfigObject):
self.supported = False
self.cbitpos = None
self.reduced_phys_bits = None
self.max_guests = None
self.max_es_guests = None
def parse_dom(self, xmldoc):
super(LibvirtConfigDomainCapsFeatureSev, self).parse_dom(xmldoc)
@ -322,6 +324,10 @@ class LibvirtConfigDomainCapsFeatureSev(LibvirtConfigObject):
self.reduced_phys_bits = int(c.text)
elif c.tag == 'cbitpos':
self.cbitpos = int(c.text)
elif c.tag == 'maxGuests':
self.max_guests = int(c.text)
elif c.tag == 'maxESGuests':
self.max_es_guests = int(c.text)
class LibvirtConfigDomainCapsOS(LibvirtConfigObject):

41
nova/virt/libvirt/driver.py
View File

@ -9009,33 +9009,30 @@ class LibvirtDriver(driver.ComputeDriver):
resources[rc].add(resource_obj)
def _get_memory_encrypted_slots(self):
slots = CONF.libvirt.num_memory_encrypted_guests
conf_slots = CONF.libvirt.num_memory_encrypted_guests
if not self._host.supports_amd_sev:
if slots and slots > 0:
if conf_slots and conf_slots > 0:
LOG.warning("Host is configured with "
"libvirt.num_memory_encrypted_guests set to "
"%d, but is not SEV-capable.", slots)
"%d, but is not SEV-capable.", conf_slots)
return 0
# NOTE(aspiers): Auto-detection of the number of available
# slots for AMD SEV is not yet possible, so honor the
# configured value, or impose no limit if this is not
# specified. This does incur a risk that if operators don't
# read the instructions and configure the maximum correctly,
# the maximum could be exceeded resulting in SEV guests
# failing at launch-time. However at least SEV guests will
# launch until the maximum, and when auto-detection code is
# added later, an upgrade will magically fix the issue.
#
# Note also that the configured value can be 0 on an
# SEV-capable host, since there might conceivably be good
# reasons for the operator to want to disable SEV even when
# it's available (e.g. due to performance impact, or
# implementation bugs which may surface later).
if slots is not None:
return slots
else:
return db_const.MAX_INT
slots = db_const.MAX_INT
# NOTE(tkajinam): Current nova supports SEV only so we ignore SEV-ES
if self._host.max_sev_guests is not None:
slots = self._host.max_sev_guests
if conf_slots is not None:
if conf_slots > slots:
LOG.warning("Host is configured with "
"libvirt.num_memory_encrypted_guests set to %d, "
"but supports only %d.", conf_slots, slots)
slots = min(slots, conf_slots)
LOG.debug("Available memory encrypted slots: %d", slots)
return slots
@property
def static_traits(self) -> ty.Dict[str, bool]:

20
nova/virt/libvirt/host.py
View File

@ -162,6 +162,8 @@ class Host(object):
# kernel, QEMU, and/or libvirt. These are determined on demand and
# memoized by various properties below
self._supports_amd_sev: ty.Optional[bool] = None
self._max_sev_guests: ty.Optional[int] = None
self._max_sev_es_guests: ty.Optional[int] = None
self._supports_uefi: ty.Optional[bool] = None
self._supports_secure_boot: ty.Optional[bool] = None
@ -1836,11 +1838,29 @@ class Host(object):
if feature_is_sev and feature.supported:
LOG.info("AMD SEV support detected")
self._supports_amd_sev = True
self._max_sev_guests = feature.max_guests
self._max_sev_es_guests = feature.max_es_guests
return self._supports_amd_sev
LOG.debug("No AMD SEV support detected for any (arch, machine_type)")
return self._supports_amd_sev
@property
def max_sev_guests(self) -> ty.Optional[int]:
"""Determine maximum number of guests with AMD SEV.
"""
if not self.supports_amd_sev:
return None
return self._max_sev_guests
@property
def max_sev_es_guests(self) -> ty.Optional[int]:
"""Determine maximum number of guests with AMD SEV-ES.
"""
if not self.supports_amd_sev:
return None
return self._max_sev_es_guests
@property
def supports_remote_managed_ports(self) -> bool:
"""Determine if the host supports remote managed ports.

6
releasenotes/notes/max-sev-guests-e3da9adb3f75fbb8.yaml Normal file
View File

@ -0,0 +1,6 @@
---
features:
- |
Now the libvirt driver is capable to detect maximum number of guests with
memory encrypted which can run concurrently in its compute host using
the new fields in libvirt API available since version 8.0.0.