Browse Source

libvirt: Lift the restriction of choices for `cpu_model_extra_flags`

Commit 6b601b7 (libvirt: Allow to specify granular CPU feature flags)
added support for allowing to specify individual CPU feature flags, but
restricted the options only to "PCID" (refer to its commit message for
why).

In this change we lift the restriction of choices, and allow to specify
multiple CPU feature flags for all three CPU modes for the libvirt
driver: 'custom', 'host-model', and 'host-passthrough'.

For example:

     [libvirt]
     cpu_mode = custom
     cpu_model = IvyBridge
     cpu_model_extra_flags = pcid, vmx, pdpe1gb

This will allow additional use cases such as:

  - Ability to use 1GB huge pages with models that don't provide it
    (such as Intel "Haswell" variants) as one use case for extra flags:

        cpu_mode = custom
        cpu_model = Haswell-noTSX-IBRS
        cpu_model_extra_flags = pdpe1gb

  - Nested Virtualization -- an operator can specify the Intel 'vmx' (or
    AMD 'svm') flags for the level-1 Nova guest CPU models.  (Assuming
    the 'nested' flag is enabled on the level-0 / bare-metal kernel.)

(A future Nova patch will also allow ability to remove CPU flags.)

Change-Id: I9a862619f379057bb48cb85a84dfc50d763030a6
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
BluePrint: libvirt-cpu-model-extra-flags
tags/18.0.0.0b2
Kashyap Chamarthy 1 year ago
parent
commit
cc27a2007f
4 changed files with 85 additions and 50 deletions
  1. +39
    -22
      nova/conf/libvirt.py
  2. +41
    -6
      nova/tests/unit/virt/libvirt/test_driver.py
  3. +0
    -19
      nova/virt/libvirt/driver.py
  4. +5
    -3
      releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml

+ 39
- 22
nova/conf/libvirt.py View File

@@ -530,28 +530,49 @@ would result in an error and the instance launch will fail.
cfg.ListOpt(
'cpu_model_extra_flags',
item_type=types.String(
choices=['pcid'],
ignore_case=True,
),
default=[],
help="""
This allows specifying granular CPU feature flags when specifying CPU
This allows specifying granular CPU feature flags when configuring CPU
models. For example, to explicitly specify the ``pcid``
(Process-Context ID, an Intel processor feature) flag to the "IvyBridge"
virtual CPU model::
(Process-Context ID, an Intel processor feature -- which is now required
to address the guest performance degradation as a result of applying the
"Meltdown" CVE fixes to certain Intel CPU models) flag to the
"IvyBridge" virtual CPU model::

[libvirt]
cpu_mode = custom
cpu_model = IvyBridge
cpu_model_extra_flags = pcid

Currently, the choice is restricted to only one option: ``pcid`` (the
option is case-insensitive, so ``PCID`` is also valid). This flag is
now required to address the guest performance degradation as a result of
applying the "Meltdown" CVE fixes on certain Intel CPU models.
To specify multiple CPU flags (e.g. the Intel ``VMX`` to expose the
virtualization extensions to the guest, or ``pdpe1gb`` to configure 1GB
huge pages for CPU models that do not provide it):

Note that when using this config attribute to set the 'PCID' CPU flag,
not all virtual (i.e. libvirt / QEMU) CPU models need it:
[libvirt]
cpu_mode = custom
cpu_model = Haswell-noTSX-IBRS
cpu_model_extra_flags = PCID, VMX, pdpe1gb

As it can be noticed from above, the ``cpu_model_extra_flags`` config
attribute is case insensitive. And specifying extra flags is valid in
combination with all the three possible values for ``cpu_mode``:
``custom`` (this also requires an explicit ``cpu_model`` to be
specified), ``host-model``, or ``host-passthrough``. A valid example
for allowing extra CPU flags even for ``host-passthrough`` mode is that
sometimes QEMU may disable certain CPU features -- e.g. Intel's
"invtsc", Invariable Time Stamp Counter, CPU flag. And if you need to
expose that CPU flag to the Nova instance, the you need to explicitly
ask for it.

The possible values for ``cpu_model_extra_flags`` depends on the CPU
model in use. Refer to ``/usr/share/libvirt/cpu_map.xml`` possible CPU
feature flags for a given CPU model.

Note that when using this config attribute to set the 'PCID' CPU flag
with the ``custom`` CPU mode, not all virtual (i.e. libvirt / QEMU) CPU
models need it:

* The only virtual CPU models that include the 'PCID' capability are
Intel "Haswell", "Broadwell", and "Skylake" variants.
@@ -561,18 +582,14 @@ not all virtual (i.e. libvirt / QEMU) CPU models need it:
even if the host CPUs by the same name include it. I.e. 'PCID' needs
to be explicitly specified when using the said virtual CPU models.

For now, the ``cpu_model_extra_flags`` config attribute is valid only in
combination with ``cpu_mode`` + ``cpu_model`` options.

Besides ``custom``, the libvirt driver has two other CPU modes: The
default, ``host-model``, tells it to do the right thing with respect to
handling 'PCID' CPU flag for the guest -- *assuming* you are running
updated processor microcode, host and guest kernel, libvirt, and QEMU.
The other mode, ``host-passthrough``, checks if 'PCID' is available in
the hardware, and if so directly passes it through to the Nova guests.
Thus, in context of 'PCID', with either of these CPU modes
(``host-model`` or ``host-passthrough``), there is no need to use the
``cpu_model_extra_flags``.
The libvirt driver's default CPU mode, ``host-model``, will do the right
thing with respect to handling 'PCID' CPU flag for the guest --
*assuming* you are running updated processor microcode, host and guest
kernel, libvirt, and QEMU. The other mode, ``host-passthrough``, checks
if 'PCID' is available in the hardware, and if so directly passes it
through to the Nova guests. Thus, in context of 'PCID', with either of
these CPU modes (``host-model`` or ``host-passthrough``), there is no
need to use the ``cpu_model_extra_flags``.

Related options:


+ 41
- 6
nova/tests/unit/virt/libvirt/test_driver.py View File

@@ -6371,6 +6371,35 @@ class LibvirtConnTestCase(test.NoDBTestCase,
self.assertEqual(1, conf.cpu.threads)
mock_warn.assert_not_called()

@mock.patch.object(libvirt_driver.LOG, 'warning')
def test_get_guest_cpu_config_custom_with_multiple_extra_flags(self,
mock_warn):
drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), True)
instance_ref = objects.Instance(**self.test_instance)
image_meta = objects.ImageMeta.from_dict(self.test_image_meta)

self.flags(cpu_mode="custom",
cpu_model="IvyBridge",
cpu_model_extra_flags=['pcid', 'vmx'],
group='libvirt')
disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
instance_ref,
image_meta)
conf = drvr._get_guest_config(instance_ref,
_fake_network_info(self, 1),
image_meta, disk_info)
features = [feature.name for feature in conf.cpu.features]
self.assertIsInstance(conf.cpu,
vconfig.LibvirtConfigGuestCPU)
self.assertEqual(conf.cpu.mode, "custom")
self.assertEqual(conf.cpu.model, "IvyBridge")
self.assertIn("pcid", features)
self.assertIn("vmx", features)
self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
self.assertEqual(conf.cpu.cores, 1)
self.assertEqual(conf.cpu.threads, 1)
mock_warn.assert_not_called()

@mock.patch.object(libvirt_driver.LOG, 'warning')
def test_get_guest_cpu_config_host_model_with_extra_flags(self,
mock_warn):
@@ -6379,7 +6408,7 @@ class LibvirtConnTestCase(test.NoDBTestCase,
image_meta = objects.ImageMeta.from_dict(self.test_image_meta)

self.flags(cpu_mode="host-model",
cpu_model_extra_flags="pcid",
cpu_model_extra_flags="pdpe1gb",
group='libvirt')
disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
instance_ref,
@@ -6387,14 +6416,17 @@ class LibvirtConnTestCase(test.NoDBTestCase,
conf = drvr._get_guest_config(instance_ref,
_fake_network_info(self, 1),
image_meta, disk_info)
features = [feature.name for feature in conf.cpu.features]
self.assertIsInstance(conf.cpu,
vconfig.LibvirtConfigGuestCPU)
self.assertEqual(conf.cpu.mode, "host-model")
self.assertEqual(len(conf.cpu.features), 0)
self.assertIn("pdpe1gb", features)
self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
self.assertEqual(conf.cpu.cores, 1)
self.assertEqual(conf.cpu.threads, 1)
self.assertTrue(mock_warn.called)
# For 'host-model', it is now valid to use 'extra_flags';
# assert that no warning is thrown
mock_warn.assert_not_called()

@mock.patch.object(libvirt_driver.LOG, 'warning')
def test_get_guest_cpu_config_host_passthrough_with_extra_flags(self,
@@ -6404,7 +6436,7 @@ class LibvirtConnTestCase(test.NoDBTestCase,
image_meta = objects.ImageMeta.from_dict(self.test_image_meta)

self.flags(cpu_mode="host-passthrough",
cpu_model_extra_flags="pcid",
cpu_model_extra_flags="invtsc",
group='libvirt')
disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
instance_ref,
@@ -6412,14 +6444,17 @@ class LibvirtConnTestCase(test.NoDBTestCase,
conf = drvr._get_guest_config(instance_ref,
_fake_network_info(self, 1),
image_meta, disk_info)
features = [feature.name for feature in conf.cpu.features]
self.assertIsInstance(conf.cpu,
vconfig.LibvirtConfigGuestCPU)
self.assertEqual(conf.cpu.mode, "host-passthrough")
self.assertEqual(len(conf.cpu.features), 0)
self.assertIn("invtsc", features)
self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
self.assertEqual(conf.cpu.cores, 1)
self.assertEqual(conf.cpu.threads, 1)
self.assertTrue(mock_warn.called)
# We have lifted the restriction for 'host-passthrough' as well;
# so here too, assert that no warning is thrown
mock_warn.assert_not_called()

def test_get_guest_cpu_topology(self):
instance_ref = objects.Instance(**self.test_instance)

+ 0
- 19
nova/virt/libvirt/driver.py View File

@@ -3815,25 +3815,6 @@ class LibvirtDriver(driver.ComputeDriver):
msg = _("A CPU model name should not be set when a "
"host CPU model is requested")
raise exception.Invalid(msg)
# FIXME (kchamart): We're intentionally restricting the choices
# (in the conf/libvirt.py) for 'extra_flags` to just 'PCID', to
# address the immediate guest performance degradation caused by
# "Meltdown" CVE fixes on certain Intel CPU models. In a future
# patch, we will:
# (a) Remove the restriction of choices for 'extra_flags',
# allowing to add / remove additional CPU flags, as it will
# make way for other useful features.
# (b) Remove the below check for "host-model", as it is a
# valid configuration to supply additional CPU flags to it.
# (c) Revisit and fix the warnings / exception handling for
# different combinations of CPU modes and 'extra_flags'.
elif ((mode == "host-model" or mode == "host-passthrough") and
extra_flags):
extra_flags = []
LOG.warning("Setting extra CPU flags is only valid in "
"combination with a custom CPU model. Refer "
"to the 'nova.conf' documentation for "
"'[libvirt]/cpu_model_extra_flags'")

LOG.debug("CPU mode '%(mode)s' model '%(model)s' was chosen, "
"with extra flags: '%(extra_flags)s'",

+ 5
- 3
releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml View File

@@ -3,9 +3,11 @@ features:
- |
The libvirt driver now allows specifying individual CPU feature
flags for guests, via a new configuration attribute
``[libvirt]/cpu_model_extra_flags`` -- only with ``custom`` as the
``[libvirt]/cpu_model``. Refer to its documentation in
``nova.conf`` for usage details.
``[libvirt]/cpu_model_extra_flags`` -- this is valid in combination
with all the three possible values for ``[libvirt]/cpu_mode``:
``custom``, ``host-model``, or ``host-passthrough``. The
``cpu_model_extra_flags`` also allows specifying multiple CPU flags.
Refer to its documentation in ``nova.conf`` for usage details.

One of the motivations for this is to alleviate the performance
degradation (caused as a result of applying the "Meltdown" CVE

Loading…
Cancel
Save