Browse Source

libvirt: Lift the restriction of choices for `cpu_model_extra_flags`

Commit 6b601b7 (libvirt: Allow to specify granular CPU feature flags)
added support for allowing to specify individual CPU feature flags, but
restricted the options only to "PCID" (refer to its commit message for
why).

In this change we lift the restriction of choices, and allow to specify
multiple CPU feature flags for all three CPU modes for the libvirt
driver: 'custom', 'host-model', and 'host-passthrough'.

For example:

     [libvirt]
     cpu_mode = custom
     cpu_model = IvyBridge
     cpu_model_extra_flags = pcid, vmx, pdpe1gb

This will allow additional use cases such as:

  - Ability to use 1GB huge pages with models that don't provide it
    (such as Intel "Haswell" variants) as one use case for extra flags:

        cpu_mode = custom
        cpu_model = Haswell-noTSX-IBRS
        cpu_model_extra_flags = pdpe1gb

  - Nested Virtualization -- an operator can specify the Intel 'vmx' (or
    AMD 'svm') flags for the level-1 Nova guest CPU models.  (Assuming
    the 'nested' flag is enabled on the level-0 / bare-metal kernel.)

(A future Nova patch will also allow ability to remove CPU flags.)

Change-Id: I9a862619f379057bb48cb85a84dfc50d763030a6
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
BluePrint: libvirt-cpu-model-extra-flags
Kashyap Chamarthy 1 year ago
parent
commit
cc27a2007f

+ 39
- 22
nova/conf/libvirt.py View File

@@ -530,28 +530,49 @@ would result in an error and the instance launch will fail.
530 530
     cfg.ListOpt(
531 531
         'cpu_model_extra_flags',
532 532
         item_type=types.String(
533
-            choices=['pcid'],
534 533
             ignore_case=True,
535 534
         ),
536 535
         default=[],
537 536
         help="""
538
-This allows specifying granular CPU feature flags when specifying CPU
537
+This allows specifying granular CPU feature flags when configuring CPU
539 538
 models.  For example, to explicitly specify the ``pcid``
540
-(Process-Context ID, an Intel processor feature) flag to the "IvyBridge"
541
-virtual CPU model::
539
+(Process-Context ID, an Intel processor feature -- which is now required
540
+to address the guest performance degradation as a result of applying the
541
+"Meltdown" CVE fixes to certain Intel CPU models) flag to the
542
+"IvyBridge" virtual CPU model::
542 543
 
543 544
     [libvirt]
544 545
     cpu_mode = custom
545 546
     cpu_model = IvyBridge
546 547
     cpu_model_extra_flags = pcid
547 548
 
548
-Currently, the choice is restricted to only one option: ``pcid`` (the
549
-option is case-insensitive, so ``PCID`` is also valid).  This flag is
550
-now required to address the guest performance degradation as a result of
551
-applying the "Meltdown" CVE fixes on certain Intel CPU models.
549
+To specify multiple CPU flags (e.g. the Intel ``VMX`` to expose the
550
+virtualization extensions to the guest, or ``pdpe1gb`` to configure 1GB
551
+huge pages for CPU models that do not provide it):
552 552
 
553
-Note that when using this config attribute to set the 'PCID' CPU flag,
554
-not all virtual (i.e. libvirt / QEMU) CPU models need it:
553
+    [libvirt]
554
+    cpu_mode = custom
555
+    cpu_model = Haswell-noTSX-IBRS
556
+    cpu_model_extra_flags = PCID, VMX, pdpe1gb
557
+
558
+As it can be noticed from above, the ``cpu_model_extra_flags`` config
559
+attribute is case insensitive.  And specifying extra flags is valid in
560
+combination with all the three possible values for ``cpu_mode``:
561
+``custom`` (this also requires an explicit ``cpu_model`` to be
562
+specified), ``host-model``, or ``host-passthrough``.  A valid example
563
+for allowing extra CPU flags even for ``host-passthrough`` mode is that
564
+sometimes QEMU may disable certain CPU features -- e.g. Intel's
565
+"invtsc", Invariable Time Stamp Counter, CPU flag.  And if you need to
566
+expose that CPU flag to the Nova instance, the you need to explicitly
567
+ask for it.
568
+
569
+The possible values for ``cpu_model_extra_flags`` depends on the CPU
570
+model in use.  Refer to ``/usr/share/libvirt/cpu_map.xml`` possible CPU
571
+feature flags for a given CPU model.
572
+
573
+Note that when using this config attribute to set the 'PCID' CPU flag
574
+with the ``custom`` CPU mode, not all virtual (i.e. libvirt / QEMU) CPU
575
+models need it:
555 576
 
556 577
 * The only virtual CPU models that include the 'PCID' capability are
557 578
   Intel "Haswell", "Broadwell", and "Skylake" variants.
@@ -561,18 +582,14 @@ not all virtual (i.e. libvirt / QEMU) CPU models need it:
561 582
   even if the host CPUs by the same name include it.  I.e.  'PCID' needs
562 583
   to be explicitly specified when using the said virtual CPU models.
563 584
 
564
-For now, the ``cpu_model_extra_flags`` config attribute is valid only in
565
-combination with ``cpu_mode`` + ``cpu_model`` options.
566
-
567
-Besides ``custom``, the libvirt driver has two other CPU modes: The
568
-default, ``host-model``, tells it to do the right thing with respect to
569
-handling 'PCID' CPU flag for the guest -- *assuming* you are running
570
-updated processor microcode, host and guest kernel, libvirt, and QEMU.
571
-The other mode, ``host-passthrough``, checks if 'PCID' is available in
572
-the hardware, and if so directly passes it through to the Nova guests.
573
-Thus, in context of 'PCID', with either of these CPU modes
574
-(``host-model`` or ``host-passthrough``), there is no need to use the
575
-``cpu_model_extra_flags``.
585
+The libvirt driver's default CPU mode, ``host-model``, will do the right
586
+thing with respect to handling 'PCID' CPU flag for the guest --
587
+*assuming* you are running updated processor microcode, host and guest
588
+kernel, libvirt, and QEMU.  The other mode, ``host-passthrough``, checks
589
+if 'PCID' is available in the hardware, and if so directly passes it
590
+through to the Nova guests.  Thus, in context of 'PCID', with either of
591
+these CPU modes (``host-model`` or ``host-passthrough``), there is no
592
+need to use the ``cpu_model_extra_flags``.
576 593
 
577 594
 Related options:
578 595
 

+ 41
- 6
nova/tests/unit/virt/libvirt/test_driver.py View File

@@ -6371,6 +6371,35 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6371 6371
         self.assertEqual(1, conf.cpu.threads)
6372 6372
         mock_warn.assert_not_called()
6373 6373
 
6374
+    @mock.patch.object(libvirt_driver.LOG, 'warning')
6375
+    def test_get_guest_cpu_config_custom_with_multiple_extra_flags(self,
6376
+            mock_warn):
6377
+        drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), True)
6378
+        instance_ref = objects.Instance(**self.test_instance)
6379
+        image_meta = objects.ImageMeta.from_dict(self.test_image_meta)
6380
+
6381
+        self.flags(cpu_mode="custom",
6382
+                   cpu_model="IvyBridge",
6383
+                   cpu_model_extra_flags=['pcid', 'vmx'],
6384
+                   group='libvirt')
6385
+        disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
6386
+                                            instance_ref,
6387
+                                            image_meta)
6388
+        conf = drvr._get_guest_config(instance_ref,
6389
+                                      _fake_network_info(self, 1),
6390
+                                      image_meta, disk_info)
6391
+        features = [feature.name for feature in conf.cpu.features]
6392
+        self.assertIsInstance(conf.cpu,
6393
+                              vconfig.LibvirtConfigGuestCPU)
6394
+        self.assertEqual(conf.cpu.mode, "custom")
6395
+        self.assertEqual(conf.cpu.model, "IvyBridge")
6396
+        self.assertIn("pcid", features)
6397
+        self.assertIn("vmx", features)
6398
+        self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
6399
+        self.assertEqual(conf.cpu.cores, 1)
6400
+        self.assertEqual(conf.cpu.threads, 1)
6401
+        mock_warn.assert_not_called()
6402
+
6374 6403
     @mock.patch.object(libvirt_driver.LOG, 'warning')
6375 6404
     def test_get_guest_cpu_config_host_model_with_extra_flags(self,
6376 6405
             mock_warn):
@@ -6379,7 +6408,7 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6379 6408
         image_meta = objects.ImageMeta.from_dict(self.test_image_meta)
6380 6409
 
6381 6410
         self.flags(cpu_mode="host-model",
6382
-                   cpu_model_extra_flags="pcid",
6411
+                   cpu_model_extra_flags="pdpe1gb",
6383 6412
                    group='libvirt')
6384 6413
         disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
6385 6414
                                             instance_ref,
@@ -6387,14 +6416,17 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6387 6416
         conf = drvr._get_guest_config(instance_ref,
6388 6417
                                       _fake_network_info(self, 1),
6389 6418
                                       image_meta, disk_info)
6419
+        features = [feature.name for feature in conf.cpu.features]
6390 6420
         self.assertIsInstance(conf.cpu,
6391 6421
                               vconfig.LibvirtConfigGuestCPU)
6392 6422
         self.assertEqual(conf.cpu.mode, "host-model")
6393
-        self.assertEqual(len(conf.cpu.features), 0)
6423
+        self.assertIn("pdpe1gb", features)
6394 6424
         self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
6395 6425
         self.assertEqual(conf.cpu.cores, 1)
6396 6426
         self.assertEqual(conf.cpu.threads, 1)
6397
-        self.assertTrue(mock_warn.called)
6427
+        # For 'host-model', it is now valid to use 'extra_flags';
6428
+        # assert that no warning is thrown
6429
+        mock_warn.assert_not_called()
6398 6430
 
6399 6431
     @mock.patch.object(libvirt_driver.LOG, 'warning')
6400 6432
     def test_get_guest_cpu_config_host_passthrough_with_extra_flags(self,
@@ -6404,7 +6436,7 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6404 6436
         image_meta = objects.ImageMeta.from_dict(self.test_image_meta)
6405 6437
 
6406 6438
         self.flags(cpu_mode="host-passthrough",
6407
-                   cpu_model_extra_flags="pcid",
6439
+                   cpu_model_extra_flags="invtsc",
6408 6440
                    group='libvirt')
6409 6441
         disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
6410 6442
                                             instance_ref,
@@ -6412,14 +6444,17 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6412 6444
         conf = drvr._get_guest_config(instance_ref,
6413 6445
                                       _fake_network_info(self, 1),
6414 6446
                                       image_meta, disk_info)
6447
+        features = [feature.name for feature in conf.cpu.features]
6415 6448
         self.assertIsInstance(conf.cpu,
6416 6449
                               vconfig.LibvirtConfigGuestCPU)
6417 6450
         self.assertEqual(conf.cpu.mode, "host-passthrough")
6418
-        self.assertEqual(len(conf.cpu.features), 0)
6451
+        self.assertIn("invtsc", features)
6419 6452
         self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
6420 6453
         self.assertEqual(conf.cpu.cores, 1)
6421 6454
         self.assertEqual(conf.cpu.threads, 1)
6422
-        self.assertTrue(mock_warn.called)
6455
+        # We have lifted the restriction for 'host-passthrough' as well;
6456
+        # so here too, assert that no warning is thrown
6457
+        mock_warn.assert_not_called()
6423 6458
 
6424 6459
     def test_get_guest_cpu_topology(self):
6425 6460
         instance_ref = objects.Instance(**self.test_instance)

+ 0
- 19
nova/virt/libvirt/driver.py View File

@@ -3815,25 +3815,6 @@ class LibvirtDriver(driver.ComputeDriver):
3815 3815
             msg = _("A CPU model name should not be set when a "
3816 3816
                     "host CPU model is requested")
3817 3817
             raise exception.Invalid(msg)
3818
-        # FIXME (kchamart): We're intentionally restricting the choices
3819
-        # (in the conf/libvirt.py) for 'extra_flags` to just 'PCID', to
3820
-        # address the immediate guest performance degradation caused by
3821
-        # "Meltdown" CVE fixes on certain Intel CPU models.  In a future
3822
-        # patch, we will:
3823
-        # (a) Remove the restriction of choices for 'extra_flags',
3824
-        #     allowing to add / remove additional CPU flags, as it will
3825
-        #     make way for other useful features.
3826
-        # (b) Remove the below check for "host-model", as it is a
3827
-        #     valid configuration to supply additional CPU flags to it.
3828
-        # (c) Revisit and fix the warnings / exception handling for
3829
-        #     different combinations of CPU modes and 'extra_flags'.
3830
-        elif ((mode == "host-model" or mode == "host-passthrough") and
3831
-              extra_flags):
3832
-            extra_flags = []
3833
-            LOG.warning("Setting extra CPU flags is only valid in "
3834
-                        "combination with a custom CPU model. Refer "
3835
-                        "to the 'nova.conf' documentation for "
3836
-                        "'[libvirt]/cpu_model_extra_flags'")
3837 3818
 
3838 3819
         LOG.debug("CPU mode '%(mode)s' model '%(model)s' was chosen, "
3839 3820
                   "with extra flags: '%(extra_flags)s'",

+ 5
- 3
releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml View File

@@ -3,9 +3,11 @@ features:
3 3
   - |
4 4
     The libvirt driver now allows specifying individual CPU feature
5 5
     flags for guests, via a new configuration attribute
6
-    ``[libvirt]/cpu_model_extra_flags`` -- only with ``custom`` as the
7
-    ``[libvirt]/cpu_model``.  Refer to its documentation in
8
-    ``nova.conf`` for usage details.
6
+    ``[libvirt]/cpu_model_extra_flags`` -- this is valid in combination
7
+    with all the three possible values for ``[libvirt]/cpu_mode``:
8
+    ``custom``, ``host-model``, or ``host-passthrough``.  The
9
+    ``cpu_model_extra_flags`` also allows specifying multiple CPU flags.
10
+    Refer to its documentation in ``nova.conf`` for usage details.
9 11
 
10 12
     One of the motivations for this is to alleviate the performance
11 13
     degradation (caused as a result of applying the "Meltdown" CVE

Loading…
Cancel
Save