Merge "docs: Add SEV guide"
This commit is contained in:
		@@ -525,271 +525,6 @@ See `the KVM documentation
 | 
			
		||||
<https://www.linux-kvm.org/page/Nested_Guests#Limitations>`_ for more
 | 
			
		||||
information on these limitations.
 | 
			
		||||
 | 
			
		||||
.. _amd-sev:
 | 
			
		||||
 | 
			
		||||
AMD SEV (Secure Encrypted Virtualization)
 | 
			
		||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 | 
			
		||||
 | 
			
		||||
`Secure Encrypted Virtualization (SEV)`__ is a technology from AMD which
 | 
			
		||||
enables the memory for a VM to be encrypted with a key unique to the VM.
 | 
			
		||||
SEV is particularly applicable to cloud computing since it can reduce the
 | 
			
		||||
amount of trust VMs need to place in the hypervisor and administrator of
 | 
			
		||||
their host system.
 | 
			
		||||
 | 
			
		||||
__ https://developer.amd.com/sev/
 | 
			
		||||
 | 
			
		||||
Nova supports SEV from the Train release onwards.
 | 
			
		||||
 | 
			
		||||
Requirements for SEV
 | 
			
		||||
--------------------
 | 
			
		||||
 | 
			
		||||
First the operator will need to ensure the following prerequisites are met:
 | 
			
		||||
 | 
			
		||||
- At least one of the Nova compute hosts must be AMD hardware capable
 | 
			
		||||
  of supporting SEV.  It is entirely possible for the compute plane to
 | 
			
		||||
  be a mix of hardware which can and cannot support SEV, although as
 | 
			
		||||
  per the section on `Permanent limitations`_ below, the maximum
 | 
			
		||||
  number of simultaneously running guests with SEV will be limited by
 | 
			
		||||
  the quantity and quality of SEV-capable hardware available.
 | 
			
		||||
 | 
			
		||||
- An appropriately configured software stack on those compute hosts,
 | 
			
		||||
  so that the various layers are all SEV ready:
 | 
			
		||||
 | 
			
		||||
  - kernel >= 4.16
 | 
			
		||||
  - QEMU >= 2.12
 | 
			
		||||
  - libvirt >= 4.5
 | 
			
		||||
  - ovmf >= commit 75b7aa9528bd 2018-07-06
 | 
			
		||||
 | 
			
		||||
.. _deploying-sev-capable-infrastructure:
 | 
			
		||||
 | 
			
		||||
Deploying SEV-capable infrastructure
 | 
			
		||||
------------------------------------
 | 
			
		||||
 | 
			
		||||
In order for users to be able to use SEV, the operator will need to
 | 
			
		||||
perform the following steps:
 | 
			
		||||
 | 
			
		||||
- Ensure that sufficient memory is reserved on the SEV compute hosts
 | 
			
		||||
  for host-level services to function correctly at all times.  This is
 | 
			
		||||
  particularly important when hosting SEV-enabled guests, since they
 | 
			
		||||
  pin pages in RAM, preventing any memory overcommit which may be in
 | 
			
		||||
  normal operation on other compute hosts.
 | 
			
		||||
 | 
			
		||||
  It is `recommended`__ to achieve this by configuring an ``rlimit`` at
 | 
			
		||||
  the ``/machine.slice`` top-level ``cgroup`` on the host, with all VMs
 | 
			
		||||
  placed inside that.  (For extreme detail, see `this discussion on the
 | 
			
		||||
  spec`__.)
 | 
			
		||||
 | 
			
		||||
  __ http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html#memory-reservation-solutions
 | 
			
		||||
  __ https://review.opendev.org/#/c/641994/2/specs/train/approved/amd-sev-libvirt-support.rst@167
 | 
			
		||||
 | 
			
		||||
  An alternative approach is to configure the
 | 
			
		||||
  :oslo.config:option:`reserved_host_memory_mb` option in the
 | 
			
		||||
  ``[DEFAULT]`` section of :file:`nova.conf`, based on the expected
 | 
			
		||||
  maximum number of SEV guests simultaneously running on the host, and
 | 
			
		||||
  the details provided in `an earlier version of the AMD SEV spec`__
 | 
			
		||||
  regarding memory region sizes, which cover how to calculate it
 | 
			
		||||
  correctly.
 | 
			
		||||
 | 
			
		||||
  __ https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/amd-sev-libvirt-support.html#proposed-change
 | 
			
		||||
 | 
			
		||||
  See `the Memory Locking and Accounting section of the AMD SEV spec`__
 | 
			
		||||
  and `previous discussion for further details`__.
 | 
			
		||||
 | 
			
		||||
  __ http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html#memory-locking-and-accounting
 | 
			
		||||
  __ https://review.opendev.org/#/c/641994/2/specs/train/approved/amd-sev-libvirt-support.rst@167
 | 
			
		||||
 | 
			
		||||
- A cloud administrator will need to define one or more SEV-enabled
 | 
			
		||||
  flavors :ref:`as described in the user guide
 | 
			
		||||
  <extra-specs-memory-encryption>`, unless it is sufficient for users
 | 
			
		||||
  to define SEV-enabled images.
 | 
			
		||||
 | 
			
		||||
Additionally the cloud operator should consider the following optional
 | 
			
		||||
steps:
 | 
			
		||||
 | 
			
		||||
.. _num_memory_encrypted_guests:
 | 
			
		||||
 | 
			
		||||
- Configure the :oslo.config:option:`libvirt.num_memory_encrypted_guests`
 | 
			
		||||
  option in :file:`nova.conf` to represent the number of guests an SEV
 | 
			
		||||
  compute node can host concurrently with memory encrypted at the
 | 
			
		||||
  hardware level.  For example:
 | 
			
		||||
 | 
			
		||||
  .. code-block:: ini
 | 
			
		||||
 | 
			
		||||
     [libvirt]
 | 
			
		||||
     num_memory_encrypted_guests = 15
 | 
			
		||||
 | 
			
		||||
  This option exists because on AMD SEV-capable hardware, the memory
 | 
			
		||||
  controller has a fixed number of slots for holding encryption keys,
 | 
			
		||||
  one per guest.  For example, at the time of writing, earlier
 | 
			
		||||
  generations of hardware only have 15 slots, thereby limiting the
 | 
			
		||||
  number of SEV guests which can be run concurrently to 15.  Nova
 | 
			
		||||
  needs to track how many slots are available and used in order to
 | 
			
		||||
  avoid attempting to exceed that limit in the hardware.
 | 
			
		||||
 | 
			
		||||
  At the time of writing (September 2019), work is in progress to
 | 
			
		||||
  allow QEMU and libvirt to expose the number of slots available on
 | 
			
		||||
  SEV hardware; however until this is finished and released, it will
 | 
			
		||||
  not be possible for Nova to programmatically detect the correct
 | 
			
		||||
  value.
 | 
			
		||||
 | 
			
		||||
  So this configuration option serves as a stop-gap, allowing the
 | 
			
		||||
  cloud operator the option of providing this value manually.  It may
 | 
			
		||||
  later be demoted to a fallback value for cases where the limit
 | 
			
		||||
  cannot be detected programmatically, or even removed altogether when
 | 
			
		||||
  Nova's minimum QEMU version guarantees that it can always be
 | 
			
		||||
  detected.
 | 
			
		||||
 | 
			
		||||
  .. note::
 | 
			
		||||
 | 
			
		||||
     When deciding whether to use the default of ``None`` or manually
 | 
			
		||||
     impose a limit, operators should carefully weigh the benefits
 | 
			
		||||
     vs. the risk.  The benefits of using the default are a) immediate
 | 
			
		||||
     convenience since nothing needs to be done now, and b) convenience
 | 
			
		||||
     later when upgrading compute hosts to future versions of Nova,
 | 
			
		||||
     since again nothing will need to be done for the correct limit to
 | 
			
		||||
     be automatically imposed.  However the risk is that until
 | 
			
		||||
     auto-detection is implemented, users may be able to attempt to
 | 
			
		||||
     launch guests with encrypted memory on hosts which have already
 | 
			
		||||
     reached the maximum number of guests simultaneously running with
 | 
			
		||||
     encrypted memory.  This risk may be mitigated by other limitations
 | 
			
		||||
     which operators can impose, for example if the smallest RAM
 | 
			
		||||
     footprint of any flavor imposes a maximum number of simultaneously
 | 
			
		||||
     running guests which is less than or equal to the SEV limit.
 | 
			
		||||
 | 
			
		||||
- Configure :oslo.config:option:`libvirt.hw_machine_type` on all
 | 
			
		||||
  SEV-capable compute hosts to include ``x86_64=q35``, so that all
 | 
			
		||||
  x86_64 images use the ``q35`` machine type by default.  (Currently
 | 
			
		||||
  Nova defaults to the ``pc`` machine type for the ``x86_64``
 | 
			
		||||
  architecture, although `it is expected that this will change in the
 | 
			
		||||
  future`__.)
 | 
			
		||||
 | 
			
		||||
  Changing the default from ``pc`` to ``q35`` makes the creation and
 | 
			
		||||
  configuration of images by users more convenient by removing the
 | 
			
		||||
  need for the ``hw_machine_type`` property to be set to ``q35`` on
 | 
			
		||||
  every image for which SEV booting is desired.
 | 
			
		||||
 | 
			
		||||
  .. caution::
 | 
			
		||||
 | 
			
		||||
     Consider carefully whether to set this option.  It is
 | 
			
		||||
     particularly important since a limitation of the implementation
 | 
			
		||||
     prevents the user from receiving an error message with a helpful
 | 
			
		||||
     explanation if they try to boot an SEV guest when neither this
 | 
			
		||||
     configuration option nor the image property are set to select
 | 
			
		||||
     a ``q35`` machine type.
 | 
			
		||||
 | 
			
		||||
     On the other hand, setting it to ``q35`` may have other
 | 
			
		||||
     undesirable side-effects on other images which were expecting to
 | 
			
		||||
     be booted with ``pc``, so it is suggested to set it on a single
 | 
			
		||||
     compute node or aggregate, and perform careful testing of typical
 | 
			
		||||
     images before rolling out the setting to all SEV-capable compute
 | 
			
		||||
     hosts.
 | 
			
		||||
 | 
			
		||||
  __ https://bugs.launchpad.net/nova/+bug/1780138
 | 
			
		||||
 | 
			
		||||
Launching SEV instances
 | 
			
		||||
-----------------------
 | 
			
		||||
 | 
			
		||||
Once an operator has covered the above steps, users can launch SEV
 | 
			
		||||
instances either by requesting a flavor for which the operator set the
 | 
			
		||||
``hw:mem_encryption`` extra spec to ``True``, or by using an image
 | 
			
		||||
with the ``hw_mem_encryption`` property set to ``True``.
 | 
			
		||||
 | 
			
		||||
These do not inherently cause a preference for SEV-capable hardware,
 | 
			
		||||
but for now SEV is the only way of fulfilling the requirement for
 | 
			
		||||
memory encryption.  However in the future, support for other
 | 
			
		||||
hardware-level guest memory encryption technology such as Intel MKTME
 | 
			
		||||
may be added.  If a guest specifically needs to be booted using SEV
 | 
			
		||||
rather than any other memory encryption technology, it is possible to
 | 
			
		||||
ensure this by adding ``trait:HW_CPU_X86_AMD_SEV=required`` to the
 | 
			
		||||
flavor extra specs or image properties.
 | 
			
		||||
 | 
			
		||||
In all cases, SEV instances can only be booted from images which have
 | 
			
		||||
the ``hw_firmware_type`` property set to ``uefi``, and only when the
 | 
			
		||||
machine type is set to ``q35``.  This can be set per image by setting
 | 
			
		||||
the image property ``hw_machine_type=q35``, or per compute node by
 | 
			
		||||
the operator via :oslo.config:option:`libvirt.hw_machine_type` as
 | 
			
		||||
explained above.
 | 
			
		||||
 | 
			
		||||
Impermanent limitations
 | 
			
		||||
-----------------------
 | 
			
		||||
 | 
			
		||||
The following limitations may be removed in the future as the
 | 
			
		||||
hardware, firmware, and various layers of software receive new
 | 
			
		||||
features:
 | 
			
		||||
 | 
			
		||||
- SEV-encrypted VMs cannot yet be live-migrated or suspended,
 | 
			
		||||
  therefore they will need to be fully shut down before migrating off
 | 
			
		||||
  an SEV host, e.g. if maintenance is required on the host.
 | 
			
		||||
 | 
			
		||||
- SEV-encrypted VMs cannot contain directly accessible host devices
 | 
			
		||||
  (PCI passthrough).  So for example mdev vGPU support will not
 | 
			
		||||
  currently work.  However technologies based on `vhost-user`__ should
 | 
			
		||||
  work fine.
 | 
			
		||||
 | 
			
		||||
  __ https://wiki.qemu.org/Features/VirtioVhostUser
 | 
			
		||||
 | 
			
		||||
- The boot disk of SEV-encrypted VMs can only be ``virtio``.
 | 
			
		||||
  (``virtio-blk`` is typically the default for libvirt disks on x86,
 | 
			
		||||
  but can also be explicitly set e.g. via the image property
 | 
			
		||||
  ``hw_disk_bus=virtio``). Valid alternatives for the disk
 | 
			
		||||
  include using ``hw_disk_bus=scsi`` with
 | 
			
		||||
  ``hw_scsi_model=virtio-scsi`` , or ``hw_disk_bus=sata``.
 | 
			
		||||
 | 
			
		||||
- QEMU and libvirt cannot yet expose the number of slots available for
 | 
			
		||||
  encrypted guests in the memory controller on SEV hardware.  Until
 | 
			
		||||
  this is implemented, it is not possible for Nova to programmatically
 | 
			
		||||
  detect the correct value.  As a short-term workaround, operators can
 | 
			
		||||
  optionally manually specify the upper limit of SEV guests for each
 | 
			
		||||
  compute host, via the new
 | 
			
		||||
  :oslo.config:option:`libvirt.num_memory_encrypted_guests`
 | 
			
		||||
  configuration option :ref:`described above
 | 
			
		||||
  <num_memory_encrypted_guests>`.
 | 
			
		||||
 | 
			
		||||
Permanent limitations
 | 
			
		||||
---------------------
 | 
			
		||||
 | 
			
		||||
The following limitations are expected long-term:
 | 
			
		||||
 | 
			
		||||
- The number of SEV guests allowed to run concurrently will always be
 | 
			
		||||
  limited.  `On the first generation of EPYC machines it will be
 | 
			
		||||
  limited to 15 guests`__; however this limit becomes much higher with
 | 
			
		||||
  the second generation (Rome).
 | 
			
		||||
 | 
			
		||||
  __ https://www.redhat.com/archives/libvir-list/2019-January/msg00652.html
 | 
			
		||||
 | 
			
		||||
- The operating system running in an encrypted virtual machine must
 | 
			
		||||
  contain SEV support.
 | 
			
		||||
 | 
			
		||||
Non-limitations
 | 
			
		||||
---------------
 | 
			
		||||
 | 
			
		||||
For the sake of eliminating any doubt, the following actions are *not*
 | 
			
		||||
expected to be limited when SEV encryption is used:
 | 
			
		||||
 | 
			
		||||
- Cold migration or shelve, since they power off the VM before the
 | 
			
		||||
  operation at which point there is no encrypted memory (although this
 | 
			
		||||
  could change since there is work underway to add support for `PMEM
 | 
			
		||||
  <https://pmem.io/>`_)
 | 
			
		||||
 | 
			
		||||
- Snapshot, since it only snapshots the disk
 | 
			
		||||
 | 
			
		||||
- ``nova evacuate`` (despite the name, more akin to resurrection than
 | 
			
		||||
  evacuation), since this is only initiated when the VM is no longer
 | 
			
		||||
  running
 | 
			
		||||
 | 
			
		||||
- Attaching any volumes, as long as they do not require attaching via
 | 
			
		||||
  an IDE bus
 | 
			
		||||
 | 
			
		||||
- Use of spice / VNC / serial / RDP consoles
 | 
			
		||||
 | 
			
		||||
- `VM guest virtual NUMA (a.k.a. vNUMA)
 | 
			
		||||
  <https://www.suse.com/documentation/sles-12/singlehtml/article_vt_best_practices/article_vt_best_practices.html#sec.vt.best.perf.numa.vmguest>`_
 | 
			
		||||
 | 
			
		||||
For further technical details, see `the nova spec for SEV support`__.
 | 
			
		||||
 | 
			
		||||
__ http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
Guest agent support
 | 
			
		||||
~~~~~~~~~~~~~~~~~~~
 | 
			
		||||
 | 
			
		||||
 
 | 
			
		||||
@@ -116,6 +116,7 @@ instance for these kind of workloads.
 | 
			
		||||
   emulated-tpm
 | 
			
		||||
   uefi
 | 
			
		||||
   secure-boot
 | 
			
		||||
   sev
 | 
			
		||||
   managing-resource-providers
 | 
			
		||||
   resource-limits
 | 
			
		||||
 | 
			
		||||
 
 | 
			
		||||
							
								
								
									
										272
									
								
								doc/source/admin/sev.rst
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										272
									
								
								doc/source/admin/sev.rst
									
									
									
									
									
										Normal file
									
								
							@@ -0,0 +1,272 @@
 | 
			
		||||
.. _amd-sev:
 | 
			
		||||
 | 
			
		||||
AMD SEV (Secure Encrypted Virtualization)
 | 
			
		||||
=========================================
 | 
			
		||||
 | 
			
		||||
.. versionadded:: 20.0.0 (Train)
 | 
			
		||||
 | 
			
		||||
`Secure Encrypted Virtualization (SEV)`__ is a technology from AMD which
 | 
			
		||||
enables the memory for a VM to be encrypted with a key unique to the VM.
 | 
			
		||||
SEV is particularly applicable to cloud computing since it can reduce the
 | 
			
		||||
amount of trust VMs need to place in the hypervisor and administrator of
 | 
			
		||||
their host system.
 | 
			
		||||
 | 
			
		||||
.. __: https://developer.amd.com/sev/
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
.. _deploying-sev-capable-infrastructure:
 | 
			
		||||
 | 
			
		||||
Enabling SEV
 | 
			
		||||
------------
 | 
			
		||||
 | 
			
		||||
First the operator will need to ensure the following prerequisites are met:
 | 
			
		||||
 | 
			
		||||
- Currently SEV is only supported when using the libvirt compute driver with a
 | 
			
		||||
  :oslo.config:option:`libvirt.virt_type` of ``kvm`` or ``qemu``.
 | 
			
		||||
 | 
			
		||||
- At least one of the Nova compute hosts must be AMD hardware capable
 | 
			
		||||
  of supporting SEV.  It is entirely possible for the compute plane to
 | 
			
		||||
  be a mix of hardware which can and cannot support SEV, although as
 | 
			
		||||
  per the section on `Permanent limitations`_ below, the maximum
 | 
			
		||||
  number of simultaneously running guests with SEV will be limited by
 | 
			
		||||
  the quantity and quality of SEV-capable hardware available.
 | 
			
		||||
 | 
			
		||||
In order for users to be able to use SEV, the operator will need to
 | 
			
		||||
perform the following steps:
 | 
			
		||||
 | 
			
		||||
- Ensure that sufficient memory is reserved on the SEV compute hosts
 | 
			
		||||
  for host-level services to function correctly at all times.  This is
 | 
			
		||||
  particularly important when hosting SEV-enabled guests, since they
 | 
			
		||||
  pin pages in RAM, preventing any memory overcommit which may be in
 | 
			
		||||
  normal operation on other compute hosts.
 | 
			
		||||
 | 
			
		||||
  It is `recommended`__ to achieve this by configuring an ``rlimit`` at
 | 
			
		||||
  the ``/machine.slice`` top-level ``cgroup`` on the host, with all VMs
 | 
			
		||||
  placed inside that.  (For extreme detail, see `this discussion on the
 | 
			
		||||
  spec`__.)
 | 
			
		||||
 | 
			
		||||
  __ http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html#memory-reservation-solutions
 | 
			
		||||
  __ https://review.opendev.org/#/c/641994/2/specs/train/approved/amd-sev-libvirt-support.rst@167
 | 
			
		||||
 | 
			
		||||
  An alternative approach is to configure the
 | 
			
		||||
  :oslo.config:option:`reserved_host_memory_mb` option in the
 | 
			
		||||
  ``[DEFAULT]`` section of :file:`nova.conf`, based on the expected
 | 
			
		||||
  maximum number of SEV guests simultaneously running on the host, and
 | 
			
		||||
  the details provided in `an earlier version of the AMD SEV spec`__
 | 
			
		||||
  regarding memory region sizes, which cover how to calculate it
 | 
			
		||||
  correctly.
 | 
			
		||||
 | 
			
		||||
  __ https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/amd-sev-libvirt-support.html#proposed-change
 | 
			
		||||
 | 
			
		||||
  See `the Memory Locking and Accounting section of the AMD SEV spec`__
 | 
			
		||||
  and `previous discussion for further details`__.
 | 
			
		||||
 | 
			
		||||
  __ http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html#memory-locking-and-accounting
 | 
			
		||||
  __ https://review.opendev.org/#/c/641994/2/specs/train/approved/amd-sev-libvirt-support.rst@167
 | 
			
		||||
 | 
			
		||||
- A cloud administrator will need to define one or more SEV-enabled
 | 
			
		||||
  flavors :ref:`as described below <extra-specs-memory-encryption>`, unless it
 | 
			
		||||
  is sufficient for users to define SEV-enabled images.
 | 
			
		||||
 | 
			
		||||
Additionally the cloud operator should consider the following optional
 | 
			
		||||
steps:
 | 
			
		||||
 | 
			
		||||
.. _num_memory_encrypted_guests:
 | 
			
		||||
 | 
			
		||||
- Configure the :oslo.config:option:`libvirt.num_memory_encrypted_guests`
 | 
			
		||||
  option in :file:`nova.conf` to represent the number of guests an SEV
 | 
			
		||||
  compute node can host concurrently with memory encrypted at the
 | 
			
		||||
  hardware level.  For example:
 | 
			
		||||
 | 
			
		||||
  .. code-block:: ini
 | 
			
		||||
 | 
			
		||||
     [libvirt]
 | 
			
		||||
     num_memory_encrypted_guests = 15
 | 
			
		||||
 | 
			
		||||
  This option exists because on AMD SEV-capable hardware, the memory
 | 
			
		||||
  controller has a fixed number of slots for holding encryption keys,
 | 
			
		||||
  one per guest.  For example, at the time of writing, earlier
 | 
			
		||||
  generations of hardware only have 15 slots, thereby limiting the
 | 
			
		||||
  number of SEV guests which can be run concurrently to 15.  Nova
 | 
			
		||||
  needs to track how many slots are available and used in order to
 | 
			
		||||
  avoid attempting to exceed that limit in the hardware.
 | 
			
		||||
 | 
			
		||||
  At the time of writing (September 2019), work is in progress to
 | 
			
		||||
  allow QEMU and libvirt to expose the number of slots available on
 | 
			
		||||
  SEV hardware; however until this is finished and released, it will
 | 
			
		||||
  not be possible for Nova to programmatically detect the correct
 | 
			
		||||
  value.
 | 
			
		||||
 | 
			
		||||
  So this configuration option serves as a stop-gap, allowing the
 | 
			
		||||
  cloud operator the option of providing this value manually.  It may
 | 
			
		||||
  later be demoted to a fallback value for cases where the limit
 | 
			
		||||
  cannot be detected programmatically, or even removed altogether when
 | 
			
		||||
  Nova's minimum QEMU version guarantees that it can always be
 | 
			
		||||
  detected.
 | 
			
		||||
 | 
			
		||||
  .. note::
 | 
			
		||||
 | 
			
		||||
     When deciding whether to use the default of ``None`` or manually
 | 
			
		||||
     impose a limit, operators should carefully weigh the benefits
 | 
			
		||||
     vs. the risk.  The benefits of using the default are a) immediate
 | 
			
		||||
     convenience since nothing needs to be done now, and b) convenience
 | 
			
		||||
     later when upgrading compute hosts to future versions of Nova,
 | 
			
		||||
     since again nothing will need to be done for the correct limit to
 | 
			
		||||
     be automatically imposed.  However the risk is that until
 | 
			
		||||
     auto-detection is implemented, users may be able to attempt to
 | 
			
		||||
     launch guests with encrypted memory on hosts which have already
 | 
			
		||||
     reached the maximum number of guests simultaneously running with
 | 
			
		||||
     encrypted memory.  This risk may be mitigated by other limitations
 | 
			
		||||
     which operators can impose, for example if the smallest RAM
 | 
			
		||||
     footprint of any flavor imposes a maximum number of simultaneously
 | 
			
		||||
     running guests which is less than or equal to the SEV limit.
 | 
			
		||||
 | 
			
		||||
- Configure :oslo.config:option:`libvirt.hw_machine_type` on all
 | 
			
		||||
  SEV-capable compute hosts to include ``x86_64=q35``, so that all
 | 
			
		||||
  x86_64 images use the ``q35`` machine type by default.  (Currently
 | 
			
		||||
  Nova defaults to the ``pc`` machine type for the ``x86_64``
 | 
			
		||||
  architecture, although `it is expected that this will change in the
 | 
			
		||||
  future`__.)
 | 
			
		||||
 | 
			
		||||
  Changing the default from ``pc`` to ``q35`` makes the creation and
 | 
			
		||||
  configuration of images by users more convenient by removing the
 | 
			
		||||
  need for the ``hw_machine_type`` property to be set to ``q35`` on
 | 
			
		||||
  every image for which SEV booting is desired.
 | 
			
		||||
 | 
			
		||||
  .. caution::
 | 
			
		||||
 | 
			
		||||
     Consider carefully whether to set this option.  It is
 | 
			
		||||
     particularly important since a limitation of the implementation
 | 
			
		||||
     prevents the user from receiving an error message with a helpful
 | 
			
		||||
     explanation if they try to boot an SEV guest when neither this
 | 
			
		||||
     configuration option nor the image property are set to select
 | 
			
		||||
     a ``q35`` machine type.
 | 
			
		||||
 | 
			
		||||
     On the other hand, setting it to ``q35`` may have other
 | 
			
		||||
     undesirable side-effects on other images which were expecting to
 | 
			
		||||
     be booted with ``pc``, so it is suggested to set it on a single
 | 
			
		||||
     compute node or aggregate, and perform careful testing of typical
 | 
			
		||||
     images before rolling out the setting to all SEV-capable compute
 | 
			
		||||
     hosts.
 | 
			
		||||
 | 
			
		||||
  __ https://bugs.launchpad.net/nova/+bug/1780138
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
.. _extra-specs-memory-encryption:
 | 
			
		||||
 | 
			
		||||
Configuring a flavor or image
 | 
			
		||||
-----------------------------
 | 
			
		||||
 | 
			
		||||
Once an operator has covered the above steps, users can launch SEV
 | 
			
		||||
instances either by requesting a flavor for which the operator set the
 | 
			
		||||
:nova:extra-spec:`hw:mem_encryption` extra spec to ``True``, or by using an
 | 
			
		||||
image with the ``hw_mem_encryption`` property set to ``True``. For example, to
 | 
			
		||||
enable SEV for a flavor:
 | 
			
		||||
 | 
			
		||||
.. code-block:: console
 | 
			
		||||
 | 
			
		||||
   $ openstack flavor set FLAVOR-NAME \
 | 
			
		||||
       --property hw:mem_encryption=true
 | 
			
		||||
 | 
			
		||||
These do not inherently cause a preference for SEV-capable hardware,
 | 
			
		||||
but for now SEV is the only way of fulfilling the requirement for
 | 
			
		||||
memory encryption.  However in the future, support for other
 | 
			
		||||
hardware-level guest memory encryption technology such as Intel MKTME
 | 
			
		||||
may be added.  If a guest specifically needs to be booted using SEV
 | 
			
		||||
rather than any other memory encryption technology, it is possible to
 | 
			
		||||
ensure this by setting the :nova:extra-spec:`trait{group}:HW_CPU_X86_AMD_SEV`
 | 
			
		||||
extra spec or equivalent image metadata property to ``required``.
 | 
			
		||||
 | 
			
		||||
In all cases, SEV instances can only be booted from images which have
 | 
			
		||||
the ``hw_firmware_type`` property set to ``uefi``, and only when the
 | 
			
		||||
machine type is set to ``q35``.  This can be set per image by setting
 | 
			
		||||
the image property ``hw_machine_type=q35``, or per compute node by
 | 
			
		||||
the operator via :oslo.config:option:`libvirt.hw_machine_type` as
 | 
			
		||||
explained above.
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
Limitations
 | 
			
		||||
-----------
 | 
			
		||||
 | 
			
		||||
Impermanent limitations
 | 
			
		||||
~~~~~~~~~~~~~~~~~~~~~~~
 | 
			
		||||
 | 
			
		||||
The following limitations may be removed in the future as the
 | 
			
		||||
hardware, firmware, and various layers of software receive new
 | 
			
		||||
features:
 | 
			
		||||
 | 
			
		||||
- SEV-encrypted VMs cannot yet be live-migrated or suspended,
 | 
			
		||||
  therefore they will need to be fully shut down before migrating off
 | 
			
		||||
  an SEV host, e.g. if maintenance is required on the host.
 | 
			
		||||
 | 
			
		||||
- SEV-encrypted VMs cannot contain directly accessible host devices
 | 
			
		||||
  (PCI passthrough).  So for example mdev vGPU support will not
 | 
			
		||||
  currently work.  However technologies based on `vhost-user`__ should
 | 
			
		||||
  work fine.
 | 
			
		||||
 | 
			
		||||
  __ https://wiki.qemu.org/Features/VirtioVhostUser
 | 
			
		||||
 | 
			
		||||
- The boot disk of SEV-encrypted VMs can only be ``virtio``.
 | 
			
		||||
  (``virtio-blk`` is typically the default for libvirt disks on x86,
 | 
			
		||||
  but can also be explicitly set e.g. via the image property
 | 
			
		||||
  ``hw_disk_bus=virtio``). Valid alternatives for the disk
 | 
			
		||||
  include using ``hw_disk_bus=scsi`` with
 | 
			
		||||
  ``hw_scsi_model=virtio-scsi`` , or ``hw_disk_bus=sata``.
 | 
			
		||||
 | 
			
		||||
- QEMU and libvirt cannot yet expose the number of slots available for
 | 
			
		||||
  encrypted guests in the memory controller on SEV hardware.  Until
 | 
			
		||||
  this is implemented, it is not possible for Nova to programmatically
 | 
			
		||||
  detect the correct value.  As a short-term workaround, operators can
 | 
			
		||||
  optionally manually specify the upper limit of SEV guests for each
 | 
			
		||||
  compute host, via the new
 | 
			
		||||
  :oslo.config:option:`libvirt.num_memory_encrypted_guests`
 | 
			
		||||
  configuration option :ref:`described above
 | 
			
		||||
  <num_memory_encrypted_guests>`.
 | 
			
		||||
 | 
			
		||||
Permanent limitations
 | 
			
		||||
~~~~~~~~~~~~~~~~~~~~~
 | 
			
		||||
 | 
			
		||||
The following limitations are expected long-term:
 | 
			
		||||
 | 
			
		||||
- The number of SEV guests allowed to run concurrently will always be
 | 
			
		||||
  limited.  `On the first generation of EPYC machines it will be
 | 
			
		||||
  limited to 15 guests`__; however this limit becomes much higher with
 | 
			
		||||
  the second generation (Rome).
 | 
			
		||||
 | 
			
		||||
  __ https://www.redhat.com/archives/libvir-list/2019-January/msg00652.html
 | 
			
		||||
 | 
			
		||||
- The operating system running in an encrypted virtual machine must
 | 
			
		||||
  contain SEV support.
 | 
			
		||||
 | 
			
		||||
Non-limitations
 | 
			
		||||
~~~~~~~~~~~~~~~
 | 
			
		||||
 | 
			
		||||
For the sake of eliminating any doubt, the following actions are *not*
 | 
			
		||||
expected to be limited when SEV encryption is used:
 | 
			
		||||
 | 
			
		||||
- Cold migration or shelve, since they power off the VM before the
 | 
			
		||||
  operation at which point there is no encrypted memory (although this
 | 
			
		||||
  could change since there is work underway to add support for `PMEM
 | 
			
		||||
  <https://pmem.io/>`_)
 | 
			
		||||
 | 
			
		||||
- Snapshot, since it only snapshots the disk
 | 
			
		||||
 | 
			
		||||
- ``nova evacuate`` (despite the name, more akin to resurrection than
 | 
			
		||||
  evacuation), since this is only initiated when the VM is no longer
 | 
			
		||||
  running
 | 
			
		||||
 | 
			
		||||
- Attaching any volumes, as long as they do not require attaching via
 | 
			
		||||
  an IDE bus
 | 
			
		||||
 | 
			
		||||
- Use of spice / VNC / serial / RDP consoles
 | 
			
		||||
 | 
			
		||||
- `VM guest virtual NUMA (a.k.a. vNUMA)
 | 
			
		||||
  <https://www.suse.com/documentation/sles-12/singlehtml/article_vt_best_practices/article_vt_best_practices.html#sec.vt.best.perf.numa.vmguest>`_
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
References
 | 
			
		||||
----------
 | 
			
		||||
 | 
			
		||||
- `libvirt driver launching AMD SEV-encrypted instances (spec)`__
 | 
			
		||||
 | 
			
		||||
.. __: http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html
 | 
			
		||||
@@ -184,18 +184,6 @@ Performance Monitoring Unit (vPMU)
 | 
			
		||||
  required, such workloads should set ``hw:pmu=False``. For most workloads
 | 
			
		||||
  the default of unset or enabling the vPMU ``hw:pmu=True`` will be correct.
 | 
			
		||||
 | 
			
		||||
.. _extra-specs-memory-encryption:
 | 
			
		||||
 | 
			
		||||
Hardware encryption of guest memory
 | 
			
		||||
  If there are compute hosts which support encryption of guest memory
 | 
			
		||||
  at the hardware level, this functionality can be requested via the
 | 
			
		||||
  ``hw:mem_encryption`` extra spec parameter:
 | 
			
		||||
 | 
			
		||||
  .. code-block:: console
 | 
			
		||||
 | 
			
		||||
     $ openstack flavor set FLAVOR-NAME \
 | 
			
		||||
         --property hw:mem_encryption=True
 | 
			
		||||
 | 
			
		||||
.. _extra-specs-hiding-hypervisor-signature:
 | 
			
		||||
 | 
			
		||||
Hiding hypervisor signature
 | 
			
		||||
 
 | 
			
		||||
		Reference in New Issue
	
	Block a user