openstack/nova
Code Issues Proposed changes

docs: Fold in MDS security flaw doc

Browse Source 
There's no real need for this to exist as its own standalone document
now that we have a separate CPU models doc. Combine them.

Change-Id: I3a3e19b1f2660dd773fd3d47332abadc0c0e5c55
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This commit is contained in:
Stephen Finucane
2021-03-23 11:06:23 +00:00
parent be03ca7be7
commit 04b8693703
6 changed files with 130 additions and 131 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
doc/source/_extra/.htaccess
View File

@@ -77,3 +77,4 @@ redirectmatch 301 ^/nova/([^/]+)/admin/adv-config.html$ /nova/$1/admin/index.htm
redirectmatch 301 ^/nova/([^/]+)/admin/system-admin.html$ /nova/$1/admin/index.html redirectmatch 301 ^/nova/([^/]+)/admin/system-admin.html$ /nova/$1/admin/index.html
redirectmatch 301 ^/nova/([^/]+)/admin/port_with_resource_request.html$ /nova/$1/admin/ports-with-resource-requests.html redirectmatch 301 ^/nova/([^/]+)/admin/port_with_resource_request.html$ /nova/$1/admin/ports-with-resource-requests.html
redirectmatch 301 ^/nova/([^/]+)/admin/manage-users.html$ /nova/$1/admin/arch.html redirectmatch 301 ^/nova/([^/]+)/admin/manage-users.html$ /nova/$1/admin/arch.html
redirectmatch 301 ^/nova/([^/]+)/admin/mitigation-for-Intel-MDS-security-flaws.html /nova/$1/admin/cpu-models.html

127
doc/source/admin/cpu-models.rst
View File

@@ -191,3 +191,130 @@ example:
As ``Haswell`` is the first CPU model supporting both of these CPU features, As ``Haswell`` is the first CPU model supporting both of these CPU features,
the instance would be configured with this model. the instance would be configured with this model.
.. _mitigation-for-Intel-MDS-security-flaws:
Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws
----------------------------------------------------------------------
In May 2019, four new microprocessor flaws, known as `MDS`__ and also referred
to as `RIDL and Fallout`__ or `ZombieLoad`__, were discovered.
These flaws affect unpatched Nova compute nodes and instances running on Intel
x86_64 CPUs.
.. __: https://access.redhat.com/security/vulnerabilities/mds
.. __: https://mdsattacks.com/
.. __: https://zombieloadattack.com
Resolution
~~~~~~~~~~
To get mitigation for the said MDS security flaws, a new CPU flag,
``md-clear``, needs to be exposed to the Nova instances. This can be done as
follows.
#. Update the following components to the versions from your Linux
distribution that have fixes for the MDS flaws, on all compute nodes
with Intel x86_64 CPUs:
- ``microcode_ctl``
- ``kernel``
- ``qemu-system-x86``
- ``libvirt``
#. When using the libvirt driver, ensure that the CPU flag ``md-clear``
is exposed to the Nova instances. This can be done in one of three ways,
depending on your configured CPU mode:
#. :oslo.config:option:`libvirt.cpu_mode`\ =host-model
When using the ``host-model`` CPU mode, the ``md-clear`` CPU flag
will be passed through to the Nova guests automatically.
This mode is the default, when
:oslo.config:option:`libvirt.virt_type`\ =kvm|qemu is set in
``/etc/nova/nova-cpu.conf`` on compute nodes.
#. :oslo.config:option:`libvirt.cpu_mode`\ =host-passthrough
When using the ``host-passthrough`` CPU mode, the ``md-clear`` CPU
flag will be passed through to the Nova guests automatically.
#. :oslo.config:option:`libvirt.cpu_mode`\ =custom
When using the ``custom`` CPU mode, you must *explicitly* enable the
CPU flag ``md-clear`` to the Nova instances, in addition to the
flags required for previous vulnerabilities, using the
:oslo.config:option:`libvirt.cpu_model_extra_flags`. For example:
.. code-block:: ini
[libvirt]
cpu_mode = custom
cpu_models = IvyBridge
cpu_model_extra_flags = spec-ctrl,ssbd,md-clear
#. Reboot the compute node for the fixes to take effect.
To minimize workload downtime, you may wish to live migrate all guests to
another compute node first.
Once the above steps have been taken on every vulnerable compute node in the
deployment, each running guest in the cluster must be fully powered down, and
cold-booted (i.e. an explicit stop followed by a start), in order to activate
the new CPU models. This can be done by the guest administrators at a time of
their choosing.
Validation
~~~~~~~~~~
After applying relevant updates, administrators can check the kernel's
``sysfs`` interface to see what mitigation is in place, by running the
following command on the host:
.. code-block:: bash
# cat /sys/devices/system/cpu/vulnerabilities/mds
Mitigation: Clear CPU buffers; SMT vulnerable
To unpack the message "Mitigation: Clear CPU buffers; SMT vulnerable":
- ``Mitigation: Clear CPU buffers`` means you have the "CPU buffer clearing"
mitigation enabled, which is mechanism to invoke a flush of various
exploitable CPU buffers by invoking a CPU instruction called "VERW".
- ``SMT vulnerable`` means, depending on your workload, you may still be
vulnerable to SMT-related problems. You need to evaluate whether your
workloads need SMT (also called "Hyper-Threading") to be disabled or not.
Refer to the guidance from your Linux distribution and processor vendor.
To see the other possible values for
``/sys/devices/system/cpu/vulnerabilities/mds``, refer to the `MDS system
information`__ section in Linux kernel's documentation for MDS.
On the host, validate that KVM is capable of exposing the ``md-clear`` flag to
guests:
.. code-block:: bash
# virsh domcapabilities kvm | grep md-clear
<feature policy='require' name='md-clear'/>
More information can be found on the 'Diagnosis' tab of `this security notice
document`__.
.. __: https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html#mds-system-information
.. __: https://access.redhat.com/security/vulnerabilities/mds
Performance Impact
~~~~~~~~~~~~~~~~~~
Refer to this section titled "Performance Impact and Disabling MDS" from
`this security notice document`__, under the *Resolve* tab.
.. note::
Although the article referred to is from Red Hat, the findings and
recommendations about performance impact apply for other distributions also.
.. __: https://access.redhat.com/security/vulnerabilities/mds

1
doc/source/admin/index.rst
View File

@@ -151,6 +151,5 @@ Additional guides
ssh-configuration ssh-configuration
support-compute support-compute
secure-live-migration-with-qemu-native-tls secure-live-migration-with-qemu-native-tls
mitigation-for-Intel-MDS-security-flaws
vendordata vendordata
hw-machine-type hw-machine-type

128
doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst
View File

@@ -1,128 +0,0 @@
======================================================================
Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws
======================================================================
Issue
~~~~~
In May 2019, four new microprocessor flaws, known as `MDS
<https://access.redhat.com/security/vulnerabilities/mds>`_ , have been
discovered. These flaws affect unpatched Nova compute nodes and
instances running on Intel x86_64 CPUs. (The said MDS security flaws
are also referred to as `RIDL and Fallout <https://mdsattacks.com/>`_ or
`ZombieLoad <https://zombieloadattack.com>`_).
Resolution
~~~~~~~~~~
To get mitigation for the said MDS security flaws, a new CPU flag,
`md-clear`, needs to be exposed to the Nova instances. It can be done
as follows.
(1) Update the following components to the versions from your Linux
distribution that have fixes for the MDS flaws, on all compute nodes
with Intel x86_64 CPUs:
- microcode_ctl
- kernel
- qemu-system-x86
- libvirt
(2) When using the libvirt driver, ensure that the CPU flag ``md-clear``
is exposed to the Nova instances. It can be done so in one of the
three following ways, given that Nova supports three distinct CPU
modes:
a. :oslo.config:option:`libvirt.cpu_mode`\ =host-model
When using ``host-model`` CPU mode, the ``md-clear`` CPU flag
will be passed through to the Nova guests automatically.
This mode is the default, when
:oslo.config:option:`libvirt.virt_type`\ =kvm|qemu is set in
``/etc/nova/nova-cpu.conf`` on compute nodes.
b. :oslo.config:option:`libvirt.cpu_mode`\ =host-passthrough
When using ``host-passthrough`` CPU mode, the ``md-clear`` CPU
flag will be passed through to the Nova guests automatically.
c. Specific custom CPU models — this can be enabled using the
Nova config attributes :oslo.config:option:`libvirt.cpu_mode`\ =custom
plus particular named CPU models, e.g.
:oslo.config:option:`libvirt.cpu_models`\ =IvyBridge.
(The list of all valid named CPU models that are supported by
your host, QEMU, and libvirt can be found by running the
command ``virsh domcapabilities``.)
When using a custom CPU mode, you must *explicitly* enable the
CPU flag ``md-clear`` to the Nova instances, in addition to the
flags required for previous vulnerabilities, using the
:oslo.config:option:`libvirt.cpu_model_extra_flags`. E.g.::
[libvirt]
cpu_mode = custom
cpu_models = IvyBridge
cpu_model_extra_flags = spec-ctrl,ssbd,md-clear
(3) Reboot the compute node for the fixes to take effect. (To minimize
workload downtime, you may wish to live migrate all guests to
another compute node first.)
Once the above steps have been taken on every vulnerable compute
node in the deployment, each running guest in the cluster must be
fully powered down, and cold-booted (i.e. an explicit stop followed
by a start), in order to activate the new CPU models. This can be done
by the guest administrators at a time of their choosing.
Validate that the fixes are in effect
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
After applying relevant updates, administrators can check the kernel's
"sysfs" interface to see what mitigation is in place, by running the
following command (on the host)::
# cat /sys/devices/system/cpu/vulnerabilities/mds
Mitigation: Clear CPU buffers; SMT vulnerable
To unpack the message "Mitigation: Clear CPU buffers; SMT vulnerable":
- The ``Mitigation: Clear CPU buffers`` bit means, you have the "CPU
buffer clearing" mitigation enabled (which is mechanism to invoke a
flush of various exploitable CPU buffers by invoking a CPU
instruction called "VERW").
- The ``SMT vulnerable`` bit means, depending on your workload, you may
still be vulnerable to SMT-related problems. You need to evaluate
whether your workloads need SMT (also called "Hyper-Threading") to
be disabled or not. Refer to the guidance from your Linux
distribution and processor vendor.
To see the other possible values for the sysfs file,
``/sys/devices/system/cpu/vulnerabilities/mds``, refer to the `MDS
system information
<https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html#mds-system-information>`_
section in Linux kernel's documentation for MDS.
On the host, validate that KVM is capable of exposing the ``md-clear``
flag to guests::
# virsh domcapabilities kvm | grep md-clear
<feature policy='require' name='md-clear'/>
Also, refer to the 'Diagnosis' tab in this security notice document
`here <https://access.redhat.com/security/vulnerabilities/mds>`_
Performance Impact
~~~~~~~~~~~~~~~~~~
Refer to this section titled "Performance Impact and Disabling MDS" from
the security notice document `here
<https://access.redhat.com/security/vulnerabilities/mds>`_, under the
'Resolve' tab. (Note that although the article referred to is from Red
Hat, the findings and recommendations about performance impact apply
for other distributions as well.)

3
doc/source/admin/security.rst
View File

@@ -56,5 +56,4 @@ Mitigation for MDS (Microarchitectural Data Sampling) security flaws
It is strongly recommended to patch all compute nodes and nova instances It is strongly recommended to patch all compute nodes and nova instances
against the processor-related security flaws, such as MDS (and other against the processor-related security flaws, such as MDS (and other
previous vulnerabilities). For details on applying mitigation for the previous vulnerabilities). For details on applying mitigation for the
MDS flaws, refer to the :doc:`mitigation-for-Intel-MDS-security-flaws` MDS flaws, refer to :ref:`mitigation-for-Intel-MDS-security-flaws`.
document.

1
doc/test/redirect-tests.txt
View File

@@ -77,3 +77,4 @@
/nova/latest/admin/system-admin.html 301 /nova/latest/admin/index.html /nova/latest/admin/system-admin.html 301 /nova/latest/admin/index.html
/nova/latest/admin/port_with_resource_request.html 301 /nova/latest/admin/ports-with-resource-requests.html /nova/latest/admin/port_with_resource_request.html 301 /nova/latest/admin/ports-with-resource-requests.html
/nova/latest/admin/manage-users.html 301 /nova/latest/admin/arch.html /nova/latest/admin/manage-users.html 301 /nova/latest/admin/arch.html
/nova/latest/admin/mitigation-for-Intel-MDS-security-flaws.html 301 /nova/latest/admin/cpu-models.html