Merge "docs: Add a real-time guide"
This commit is contained in:
commit
ad6882b45b
|
@ -56,6 +56,8 @@ VCPU
|
||||||
Resource class representing a unit of CPU resources for a single guest
|
Resource class representing a unit of CPU resources for a single guest
|
||||||
approximating the processing power of a single physical processor.
|
approximating the processing power of a single physical processor.
|
||||||
|
|
||||||
|
.. _numa-topologies:
|
||||||
|
|
||||||
Customizing instance NUMA placement policies
|
Customizing instance NUMA placement policies
|
||||||
--------------------------------------------
|
--------------------------------------------
|
||||||
|
|
||||||
|
@ -113,12 +115,14 @@ filter must be enabled. Details on this filter are provided in
|
||||||
|
|
||||||
When used, NUMA awareness allows the operating system of the instance to
|
When used, NUMA awareness allows the operating system of the instance to
|
||||||
intelligently schedule the workloads that it runs and minimize cross-node
|
intelligently schedule the workloads that it runs and minimize cross-node
|
||||||
memory bandwidth. To restrict an instance's vCPUs to a single host NUMA node,
|
memory bandwidth. To configure guest NUMA nodes, you can use the
|
||||||
|
:nova:extra-spec:`hw:numa_nodes` flavor extra spec.
|
||||||
|
For example, to restrict an instance's vCPUs to a single host NUMA node,
|
||||||
run:
|
run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property hw:numa_nodes=1
|
$ openstack flavor set $FLAVOR --property hw:numa_nodes=1
|
||||||
|
|
||||||
Some workloads have very demanding requirements for memory access latency or
|
Some workloads have very demanding requirements for memory access latency or
|
||||||
bandwidth that exceed the memory bandwidth available from a single NUMA node.
|
bandwidth that exceed the memory bandwidth available from a single NUMA node.
|
||||||
|
@ -129,34 +133,56 @@ nodes, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property hw:numa_nodes=2
|
$ openstack flavor set $FLAVOR --property hw:numa_nodes=2
|
||||||
|
|
||||||
The allocation of instances vCPUs and memory from different host NUMA nodes can
|
The allocation of instance vCPUs and memory from different host NUMA nodes can
|
||||||
be configured. This allows for asymmetric allocation of vCPUs and memory, which
|
be configured. This allows for asymmetric allocation of vCPUs and memory, which
|
||||||
can be important for some workloads. To spread the 6 vCPUs and 6 GB of memory
|
can be important for some workloads. You can configure the allocation of
|
||||||
|
instance vCPUs and memory across each **guest** NUMA node using the
|
||||||
|
:nova:extra-spec:`hw:numa_cpus.{id}` and :nova:extra-spec:`hw:numa_mem.{id}`
|
||||||
|
extra specs respectively.
|
||||||
|
For example, to spread the 6 vCPUs and 6 GB of memory
|
||||||
of an instance across two NUMA nodes and create an asymmetric 1:2 vCPU and
|
of an instance across two NUMA nodes and create an asymmetric 1:2 vCPU and
|
||||||
memory mapping between the two nodes, run:
|
memory mapping between the two nodes, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property hw:numa_nodes=2
|
$ openstack flavor set $FLAVOR --property hw:numa_nodes=2
|
||||||
# configure guest node 0
|
# configure guest node 0
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:numa_cpus.0=0,1 \
|
--property hw:numa_cpus.0=0,1 \
|
||||||
--property hw:numa_mem.0=2048
|
--property hw:numa_mem.0=2048
|
||||||
# configure guest node 1
|
# configure guest node 1
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:numa_cpus.1=2,3,4,5 \
|
--property hw:numa_cpus.1=2,3,4,5 \
|
||||||
--property hw:numa_mem.1=4096
|
--property hw:numa_mem.1=4096
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
The ``{num}`` parameter is an index of *guest* NUMA nodes and may not
|
||||||
|
correspond to *host* NUMA nodes. For example, on a platform with two NUMA
|
||||||
|
nodes, the scheduler may opt to place guest NUMA node 0, as referenced in
|
||||||
|
``hw:numa_mem.0`` on host NUMA node 1 and vice versa. Similarly, the
|
||||||
|
CPUs bitmask specified in the value for ``hw:numa_cpus.{num}`` refer to
|
||||||
|
*guest* vCPUs and may not correspond to *host* CPUs. As such, this feature
|
||||||
|
cannot be used to constrain instances to specific host CPUs or NUMA nodes.
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
If the combined values of ``hw:numa_cpus.{num}`` or ``hw:numa_mem.{num}``
|
||||||
|
are greater than the available number of CPUs or memory respectively, an
|
||||||
|
exception will be raised.
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
Hyper-V does not support asymmetric NUMA topologies, and the Hyper-V
|
Hyper-V does not support asymmetric NUMA topologies, and the Hyper-V
|
||||||
driver will not spawn instances with such topologies.
|
driver will not spawn instances with such topologies.
|
||||||
|
|
||||||
For more information about the syntax for ``hw:numa_nodes``, ``hw:numa_cpus.N``
|
For more information about the syntax for ``hw:numa_nodes``, ``hw:numa_cpus.N``
|
||||||
and ``hw:num_mem.N``, refer to the :ref:`NUMA
|
and ``hw:num_mem.N``, refer to :doc:`/configuration/extra-specs`.
|
||||||
topology <extra-specs-numa-topology>` guide.
|
|
||||||
|
|
||||||
|
.. _cpu-pinning-policies:
|
||||||
|
|
||||||
Customizing instance CPU pinning policies
|
Customizing instance CPU pinning policies
|
||||||
-----------------------------------------
|
-----------------------------------------
|
||||||
|
@ -189,14 +215,16 @@ CPUs can use the CPUs of another pinned instance, thus preventing resource
|
||||||
contention between instances.
|
contention between instances.
|
||||||
|
|
||||||
CPU pinning policies can be used to determine whether an instance should be
|
CPU pinning policies can be used to determine whether an instance should be
|
||||||
pinned or not. There are three policies: ``dedicated``, ``mixed`` and
|
pinned or not. They can be configured using the
|
||||||
|
:nova:extra-spec:`hw:cpu_policy` extra spec and equivalent image metadata
|
||||||
|
property. There are three policies: ``dedicated``, ``mixed`` and
|
||||||
``shared`` (the default). The ``dedicated`` CPU policy is used to specify
|
``shared`` (the default). The ``dedicated`` CPU policy is used to specify
|
||||||
that all CPUs of an instance should use pinned CPUs. To configure a flavor to
|
that all CPUs of an instance should use pinned CPUs. To configure a flavor to
|
||||||
use the ``dedicated`` CPU policy, run:
|
use the ``dedicated`` CPU policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property hw:cpu_policy=dedicated
|
$ openstack flavor set $FLAVOR --property hw:cpu_policy=dedicated
|
||||||
|
|
||||||
This works by ensuring ``PCPU`` allocations are used instead of ``VCPU``
|
This works by ensuring ``PCPU`` allocations are used instead of ``VCPU``
|
||||||
allocations. As such, it is also possible to request this resource type
|
allocations. As such, it is also possible to request this resource type
|
||||||
|
@ -204,9 +232,9 @@ explicitly. To configure this, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property resources:PCPU=N
|
$ openstack flavor set $FLAVOR --property resources:PCPU=N
|
||||||
|
|
||||||
where ``N`` is the number of vCPUs defined in the flavor.
|
(where ``N`` is the number of vCPUs defined in the flavor).
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
|
@ -218,28 +246,28 @@ use pinned CPUs. To configure a flavor to use the ``shared`` CPU policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property hw:cpu_policy=shared
|
$ openstack flavor set $FLAVOR --property hw:cpu_policy=shared
|
||||||
|
|
||||||
The ``mixed`` CPU policy is used to specify that an instance use pinned CPUs
|
The ``mixed`` CPU policy is used to specify that an instance use pinned CPUs
|
||||||
along with unpinned CPUs. The instance pinned CPU could be specified in the
|
along with unpinned CPUs. The instance pinned CPU could be specified in the
|
||||||
``hw:cpu_dedicated_mask`` or, if real-time is enabled
|
:nova:extra-spec:`hw:cpu_dedicated_mask` or, if :doc:`real-time <real-time>` is
|
||||||
(``hw:cpu_realtime``\ = yes), in the ``hw:cpu_realtime_mask`` extra spec. For
|
enabled, in the :nova:extra-spec:`hw:cpu_realtime_mask` extra spec. For
|
||||||
example, to configure a flavor to use the ``mixed`` CPU policy with 4 vCPUs in
|
example, to configure a flavor to use the ``mixed`` CPU policy with 4 vCPUs in
|
||||||
total and the first 2 vCPUs as pinned CPUs, with the ``hw:cpu_realtime_mask``
|
total and the first 2 vCPUs as pinned CPUs, run:
|
||||||
extra spec, run:
|
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--vcpus=4 \
|
--vcpus=4 \
|
||||||
--property hw:cpu_policy=mixed \
|
--property hw:cpu_policy=mixed \
|
||||||
--property hw:cpu_dedicated_mask=0-1
|
--property hw:cpu_dedicated_mask=0-1
|
||||||
|
|
||||||
To create the mixed instance with the real-time extra specs, run:
|
To configure a flavor to use the ``mixed`` CPU policy with 4 vCPUs in total and
|
||||||
|
the first 2 vCPUs as pinned **real-time** CPUs, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--vcpus=4 \
|
--vcpus=4 \
|
||||||
--property hw:cpu_policy=mixed \
|
--property hw:cpu_policy=mixed \
|
||||||
--property hw:cpu_realtime=yes \
|
--property hw:cpu_realtime=yes \
|
||||||
|
@ -249,7 +277,12 @@ To create the mixed instance with the real-time extra specs, run:
|
||||||
|
|
||||||
For more information about the syntax for ``hw:cpu_policy``,
|
For more information about the syntax for ``hw:cpu_policy``,
|
||||||
``hw:cpu_dedicated_mask``, ``hw:realtime_cpu`` and ``hw:cpu_realtime_mask``,
|
``hw:cpu_dedicated_mask``, ``hw:realtime_cpu`` and ``hw:cpu_realtime_mask``,
|
||||||
refer to the :doc:`/user/flavors` guide.
|
refer to :doc:`/configuration/extra-specs`
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
For more information about real-time functionality, refer to the
|
||||||
|
:doc:`documentation <real-time>`.
|
||||||
|
|
||||||
It is also possible to configure the CPU policy via image metadata. This can
|
It is also possible to configure the CPU policy via image metadata. This can
|
||||||
be useful when packaging applications that require real-time or near real-time
|
be useful when packaging applications that require real-time or near real-time
|
||||||
|
@ -259,13 +292,13 @@ policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack image set [IMAGE_ID] --property hw_cpu_policy=dedicated
|
$ openstack image set $IMAGE --property hw_cpu_policy=dedicated
|
||||||
|
|
||||||
Likewise, to configure an image to use the ``shared`` CPU policy, run:
|
Likewise, to configure an image to use the ``shared`` CPU policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack image set [IMAGE_ID] --property hw_cpu_policy=shared
|
$ openstack image set $IMAGE --property hw_cpu_policy=shared
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
|
@ -312,7 +345,7 @@ performance, you can request hosts **with** SMT. To configure this, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:cpu_policy=dedicated \
|
--property hw:cpu_policy=dedicated \
|
||||||
--property hw:cpu_thread_policy=require
|
--property hw:cpu_thread_policy=require
|
||||||
|
|
||||||
|
@ -322,7 +355,7 @@ request this trait explicitly. To configure this, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property resources:PCPU=N \
|
--property resources:PCPU=N \
|
||||||
--property trait:HW_CPU_HYPERTHREADING=required
|
--property trait:HW_CPU_HYPERTHREADING=required
|
||||||
|
|
||||||
|
@ -331,7 +364,7 @@ you can request hosts **without** SMT. To configure this, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:cpu_policy=dedicated \
|
--property hw:cpu_policy=dedicated \
|
||||||
--property hw:cpu_thread_policy=isolate
|
--property hw:cpu_thread_policy=isolate
|
||||||
|
|
||||||
|
@ -341,7 +374,7 @@ possible to request this trait explicitly. To configure this, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property resources:PCPU=N \
|
--property resources:PCPU=N \
|
||||||
--property trait:HW_CPU_HYPERTHREADING=forbidden
|
--property trait:HW_CPU_HYPERTHREADING=forbidden
|
||||||
|
|
||||||
|
@ -351,7 +384,7 @@ is the default, but it can be set explicitly:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:cpu_policy=dedicated \
|
--property hw:cpu_policy=dedicated \
|
||||||
--property hw:cpu_thread_policy=prefer
|
--property hw:cpu_thread_policy=prefer
|
||||||
|
|
||||||
|
@ -360,7 +393,7 @@ This does not utilize traits and, as such, there is no trait-based equivalent.
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
For more information about the syntax for ``hw:cpu_thread_policy``, refer to
|
For more information about the syntax for ``hw:cpu_thread_policy``, refer to
|
||||||
the :doc:`/user/flavors` guide.
|
:doc:`/configuration/extra-specs`.
|
||||||
|
|
||||||
As with CPU policies, it also possible to configure the CPU thread policy via
|
As with CPU policies, it also possible to configure the CPU thread policy via
|
||||||
image metadata. This can be useful when packaging applications that require
|
image metadata. This can be useful when packaging applications that require
|
||||||
|
@ -370,7 +403,7 @@ image are always pinned regardless of flavor. To configure an image to use the
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack image set [IMAGE_ID] \
|
$ openstack image set $IMAGE \
|
||||||
--property hw_cpu_policy=dedicated \
|
--property hw_cpu_policy=dedicated \
|
||||||
--property hw_cpu_thread_policy=require
|
--property hw_cpu_thread_policy=require
|
||||||
|
|
||||||
|
@ -378,7 +411,7 @@ Likewise, to configure an image to use the ``isolate`` CPU thread policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack image set [IMAGE_ID] \
|
$ openstack image set $IMAGE \
|
||||||
--property hw_cpu_policy=dedicated \
|
--property hw_cpu_policy=dedicated \
|
||||||
--property hw_cpu_thread_policy=isolate
|
--property hw_cpu_thread_policy=isolate
|
||||||
|
|
||||||
|
@ -386,7 +419,7 @@ Finally, to configure an image to use the ``prefer`` CPU thread policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack image set [IMAGE_ID] \
|
$ openstack image set $IMAGE \
|
||||||
--property hw_cpu_policy=dedicated \
|
--property hw_cpu_policy=dedicated \
|
||||||
--property hw_cpu_thread_policy=prefer
|
--property hw_cpu_thread_policy=prefer
|
||||||
|
|
||||||
|
@ -400,6 +433,8 @@ an exception will be raised.
|
||||||
For more information about image metadata, refer to the `Image metadata`_
|
For more information about image metadata, refer to the `Image metadata`_
|
||||||
guide.
|
guide.
|
||||||
|
|
||||||
|
.. _emulator-thread-pinning-policies:
|
||||||
|
|
||||||
Customizing instance emulator thread pinning policies
|
Customizing instance emulator thread pinning policies
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
@ -429,25 +464,45 @@ the ``isolate`` emulator thread policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:cpu_policy=dedicated \
|
--property hw:cpu_policy=dedicated \
|
||||||
--property hw:emulator_threads_policy=isolate
|
--property hw:emulator_threads_policy=isolate
|
||||||
|
|
||||||
The ``share`` policy is used to specify that emulator threads from a given
|
The ``share`` policy is used to specify that emulator threads from a given
|
||||||
instance should be run on the pool of host cores listed in
|
instance should be run on the pool of host cores listed in
|
||||||
:oslo.config:option:`compute.cpu_shared_set`. To configure a flavor to use the
|
:oslo.config:option:`compute.cpu_shared_set` if configured, else across all
|
||||||
``share`` emulator thread policy, run:
|
pCPUs of the instance.
|
||||||
|
To configure a flavor to use the ``share`` emulator thread policy, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:cpu_policy=dedicated \
|
--property hw:cpu_policy=dedicated \
|
||||||
--property hw:emulator_threads_policy=share
|
--property hw:emulator_threads_policy=share
|
||||||
|
|
||||||
|
The above behavior can be summarized in this helpful table:
|
||||||
|
|
||||||
|
.. list-table::
|
||||||
|
:header-rows: 1
|
||||||
|
:stub-columns: 1
|
||||||
|
|
||||||
|
* -
|
||||||
|
- :oslo.config:option:`compute.cpu_shared_set` set
|
||||||
|
- :oslo.config:option:`compute.cpu_shared_set` unset
|
||||||
|
* - ``hw:emulator_treads_policy`` unset (default)
|
||||||
|
- Pinned to all of the instance's pCPUs
|
||||||
|
- Pinned to all of the instance's pCPUs
|
||||||
|
* - ``hw:emulator_threads_policy`` = ``share``
|
||||||
|
- Pinned to :oslo.config:option:`compute.cpu_shared_set`
|
||||||
|
- Pinned to all of the instance's pCPUs
|
||||||
|
* - ``hw:emulator_threads_policy`` = ``isolate``
|
||||||
|
- Pinned to a single pCPU distinct from the instance's pCPUs
|
||||||
|
- Pinned to a single pCPU distinct from the instance's pCPUs
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
For more information about the syntax for ``hw:emulator_threads_policy``,
|
For more information about the syntax for ``hw:emulator_threads_policy``,
|
||||||
refer to the :doc:`/user/flavors` guide.
|
refer to :nova:extra-spec:`the documentation <hw:emulator_threads_policy>`.
|
||||||
|
|
||||||
Customizing instance CPU topologies
|
Customizing instance CPU topologies
|
||||||
-----------------------------------
|
-----------------------------------
|
||||||
|
@ -476,17 +531,17 @@ sockets.
|
||||||
|
|
||||||
Some workloads benefit from a custom topology. For example, in some operating
|
Some workloads benefit from a custom topology. For example, in some operating
|
||||||
systems, a different license may be needed depending on the number of CPU
|
systems, a different license may be needed depending on the number of CPU
|
||||||
sockets. To configure a flavor to use a maximum of two sockets, run:
|
sockets. To configure a flavor to use two sockets, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property hw:cpu_sockets=2
|
$ openstack flavor set $FLAVOR --property hw:cpu_sockets=2
|
||||||
|
|
||||||
Similarly, to configure a flavor to use one core and one thread, run:
|
Similarly, to configure a flavor to use one core and one thread, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] \
|
$ openstack flavor set $FLAVOR \
|
||||||
--property hw:cpu_cores=1 \
|
--property hw:cpu_cores=1 \
|
||||||
--property hw:cpu_threads=1
|
--property hw:cpu_threads=1
|
||||||
|
|
||||||
|
@ -501,22 +556,25 @@ Similarly, to configure a flavor to use one core and one thread, run:
|
||||||
with ten cores fails.
|
with ten cores fails.
|
||||||
|
|
||||||
For more information about the syntax for ``hw:cpu_sockets``, ``hw:cpu_cores``
|
For more information about the syntax for ``hw:cpu_sockets``, ``hw:cpu_cores``
|
||||||
and ``hw:cpu_threads``, refer to the :doc:`/user/flavors` guide.
|
and ``hw:cpu_threads``, refer to :doc:`/configuration/extra-specs`.
|
||||||
|
|
||||||
It is also possible to set upper limits on the number of sockets, cores, and
|
It is also possible to set upper limits on the number of sockets, cores, and
|
||||||
threads used. Unlike the hard values above, it is not necessary for this exact
|
threads used. Unlike the hard values above, it is not necessary for this exact
|
||||||
number to used because it only provides a limit. This can be used to provide
|
number to used because it only provides a limit. This can be used to provide
|
||||||
some flexibility in scheduling, while ensuring certain limits are not
|
some flexibility in scheduling, while ensuring certain limits are not
|
||||||
exceeded. For example, to ensure no more than two sockets are defined in the
|
exceeded. For example, to ensure no more than two sockets, eight cores and one
|
||||||
instance topology, run:
|
thread are defined in the instance topology, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack flavor set [FLAVOR_ID] --property hw:cpu_max_sockets=2
|
$ openstack flavor set $FLAVOR \
|
||||||
|
--property hw:cpu_max_sockets=2 \
|
||||||
|
--property hw:cpu_max_cores=8 \
|
||||||
|
--property hw:cpu_max_threads=1
|
||||||
|
|
||||||
For more information about the syntax for ``hw:cpu_max_sockets``,
|
For more information about the syntax for ``hw:cpu_max_sockets``,
|
||||||
``hw:cpu_max_cores``, and ``hw:cpu_max_threads``, refer to the
|
``hw:cpu_max_cores``, and ``hw:cpu_max_threads``, refer to
|
||||||
:doc:`/user/flavors` guide.
|
:doc:`/configuration/extra-specs`.
|
||||||
|
|
||||||
Applications are frequently packaged as images. For applications that prefer
|
Applications are frequently packaged as images. For applications that prefer
|
||||||
certain CPU topologies, configure image metadata to hint that created instances
|
certain CPU topologies, configure image metadata to hint that created instances
|
||||||
|
@ -525,7 +583,7 @@ request a two-socket, four-core per socket topology, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack image set [IMAGE_ID] \
|
$ openstack image set $IMAGE \
|
||||||
--property hw_cpu_sockets=2 \
|
--property hw_cpu_sockets=2 \
|
||||||
--property hw_cpu_cores=4
|
--property hw_cpu_cores=4
|
||||||
|
|
||||||
|
@ -535,7 +593,7 @@ maximum of one thread, run:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
$ openstack image set [IMAGE_ID] \
|
$ openstack image set $IMAGE \
|
||||||
--property hw_cpu_max_sockets=2 \
|
--property hw_cpu_max_sockets=2 \
|
||||||
--property hw_cpu_max_threads=1
|
--property hw_cpu_max_threads=1
|
||||||
|
|
||||||
|
|
|
@ -56,6 +56,7 @@ Enabling huge pages on the host
|
||||||
-------------------------------
|
-------------------------------
|
||||||
|
|
||||||
.. important::
|
.. important::
|
||||||
|
|
||||||
Huge pages may not be used on a host configured for file-backed memory. See
|
Huge pages may not be used on a host configured for file-backed memory. See
|
||||||
:doc:`file-backed-memory` for details
|
:doc:`file-backed-memory` for details
|
||||||
|
|
||||||
|
@ -163,7 +164,7 @@ By default, an instance does not use huge pages for its underlying memory.
|
||||||
However, huge pages can bring important or required performance improvements
|
However, huge pages can bring important or required performance improvements
|
||||||
for some workloads. Huge pages must be requested explicitly through the use of
|
for some workloads. Huge pages must be requested explicitly through the use of
|
||||||
flavor extra specs or image metadata. To request an instance use huge pages,
|
flavor extra specs or image metadata. To request an instance use huge pages,
|
||||||
run:
|
you can use the :nova:extra-spec:`hw:mem_page_size` flavor extra spec:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
|
@ -205,7 +206,7 @@ run:
|
||||||
$ openstack flavor set m1.large --property hw:mem_page_size=any
|
$ openstack flavor set m1.large --property hw:mem_page_size=any
|
||||||
|
|
||||||
For more information about the syntax for ``hw:mem_page_size``, refer to
|
For more information about the syntax for ``hw:mem_page_size``, refer to
|
||||||
:doc:`flavors`.
|
:nova:extra-spec:`the documentation <hw:mem_page_size>`.
|
||||||
|
|
||||||
Applications are frequently packaged as images. For applications that require
|
Applications are frequently packaged as images. For applications that require
|
||||||
the IO performance improvements that huge pages provides, configure image
|
the IO performance improvements that huge pages provides, configure image
|
||||||
|
|
|
@ -107,6 +107,7 @@ instance for these kind of workloads.
|
||||||
|
|
||||||
pci-passthrough
|
pci-passthrough
|
||||||
cpu-topologies
|
cpu-topologies
|
||||||
|
real-time
|
||||||
huge-pages
|
huge-pages
|
||||||
virtual-gpu
|
virtual-gpu
|
||||||
file-backed-memory
|
file-backed-memory
|
||||||
|
|
|
@ -187,8 +187,9 @@ with ``provider:physical_network=foo`` must be scheduled on host cores from
|
||||||
NUMA nodes 0, while instances using one or more networks with
|
NUMA nodes 0, while instances using one or more networks with
|
||||||
``provider:physical_network=bar`` must be scheduled on host cores from both
|
``provider:physical_network=bar`` must be scheduled on host cores from both
|
||||||
NUMA nodes 2 and 3. For the latter case, it will be necessary to split the
|
NUMA nodes 2 and 3. For the latter case, it will be necessary to split the
|
||||||
guest across two or more host NUMA nodes using the ``hw:numa_nodes``
|
guest across two or more host NUMA nodes using the
|
||||||
:ref:`flavor extra spec <extra-specs-numa-topology>`.
|
:nova:extra-spec:`hw:numa_nodes` extra spec, as discussed :ref:`here
|
||||||
|
<numa-topologies>`.
|
||||||
|
|
||||||
Now, take an example for a deployment using L3 networks.
|
Now, take an example for a deployment using L3 networks.
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,153 @@
|
||||||
|
=========
|
||||||
|
Real Time
|
||||||
|
=========
|
||||||
|
|
||||||
|
.. versionadded:: 13.0.0 (Mitaka)
|
||||||
|
|
||||||
|
Nova supports configuring `real-time policies`__ for instances. This builds upon
|
||||||
|
the improved performance offered by :doc:`CPU pinning <cpu-topologies>` by
|
||||||
|
providing stronger guarantees for worst case scheduler latency for vCPUs.
|
||||||
|
|
||||||
|
.. __: https://en.wikipedia.org/wiki/Real-time_computing
|
||||||
|
|
||||||
|
|
||||||
|
Enabling Real-Time
|
||||||
|
------------------
|
||||||
|
|
||||||
|
Currently the creation of real-time instances is only supported when using the
|
||||||
|
libvirt compute driver with a :oslo.config:option:`libvirt.virt_type` of
|
||||||
|
``kvm`` or ``qemu``. It requires extensive configuration of the host and this
|
||||||
|
document provides but a rough overview of the changes required. Configuration
|
||||||
|
will vary depending on your hardware, BIOS configuration, host and guest OS'
|
||||||
|
and application.
|
||||||
|
|
||||||
|
BIOS configuration
|
||||||
|
~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Configure your host BIOS as recommended in the `rt-wiki`__ page.
|
||||||
|
The most important steps are:
|
||||||
|
|
||||||
|
- Disable power management, including CPU sleep states
|
||||||
|
- Disable SMT (hyper-threading) or any option related to logical processors
|
||||||
|
|
||||||
|
These are standard steps used in benchmarking as both sets of features can
|
||||||
|
result in non-deterministic behavior.
|
||||||
|
|
||||||
|
.. __: https://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application
|
||||||
|
|
||||||
|
OS configuration
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is inherently specific to the distro used, however, there are some common
|
||||||
|
steps:
|
||||||
|
|
||||||
|
- Install the real-time (preemptible) kernel (``PREEMPT_RT_FULL``) and
|
||||||
|
real-time KVM modules
|
||||||
|
- Configure hugepages
|
||||||
|
- Isolate host cores to be used for instances from the kernel
|
||||||
|
- Disable features like CPU frequency scaling (e.g. P-States on Intel
|
||||||
|
processors)
|
||||||
|
|
||||||
|
RHEL and RHEL-derived distros like CentOS provide packages in their
|
||||||
|
repositories to accomplish. The ``kernel-rt`` and ``kernel-rt-kvm``
|
||||||
|
packages will provide the real-time kernel and real-time KVM module,
|
||||||
|
respectively, while the ``tuned-profiles-realtime`` package will provide
|
||||||
|
`tuned`__ profiles to configure the host for real-time workloads. You should
|
||||||
|
refer to your distro documentation for more information.
|
||||||
|
|
||||||
|
.. __: https://tuned-project.org/
|
||||||
|
|
||||||
|
Validation
|
||||||
|
~~~~~~~~~~
|
||||||
|
|
||||||
|
Once your BIOS and the host OS have been configured, you can validate
|
||||||
|
"real-time readiness" using the ``hwlatdetect`` and ``rteval`` utilities. On
|
||||||
|
RHEL and RHEL-derived hosts, you can install these using the ``rt-tests``
|
||||||
|
package. More information about the ``rteval`` tool can be found `here`__.
|
||||||
|
|
||||||
|
.. __: https://git.kernel.org/pub/scm/utils/rteval/rteval.git/tree/README
|
||||||
|
|
||||||
|
|
||||||
|
Configuring a flavor or image
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 22.0.0 (Victoria)
|
||||||
|
|
||||||
|
Previously, it was necessary to specify
|
||||||
|
:nova:extra-spec:`hw:cpu_realtime_mask` when
|
||||||
|
:nova:extra-spec:`hw:cpu_realtime` was set to ``true``.
|
||||||
|
Starting in Victoria, it is possible
|
||||||
|
to omit this when an emulator thread policy is configured using the
|
||||||
|
:nova:extra-spec:`hw:emulator_threads_policy` extra spec, thus allowing all
|
||||||
|
guest cores to be be allocated as real-time cores.
|
||||||
|
|
||||||
|
.. versionchanged:: 22.0.0 (Victoria)
|
||||||
|
|
||||||
|
Previously, a leading caret was necessary when specifying the value for
|
||||||
|
:nova:extra-spec:`hw:cpu_realtime_mask` and omitting it would be equivalent
|
||||||
|
to not setting the mask, resulting in a failure to spawn the instance.
|
||||||
|
|
||||||
|
Compared to configuring the host, configuring the guest is relatively trivial
|
||||||
|
and merely requires a combination of flavor extra specs and image metadata
|
||||||
|
properties, along with a suitable real-time guest OS.
|
||||||
|
|
||||||
|
Enable real-time by setting the :nova:extra-spec:`hw:cpu_realtime` flavor extra
|
||||||
|
spec to ``true``. When this is configured, it is necessary to specify where
|
||||||
|
guest overhead processes should be scheduled to. This can be accomplished in
|
||||||
|
one of three ways. Firstly, the :nova:extra-spec:`hw:cpu_realtime_mask` extra
|
||||||
|
spec or equivalent image metadata property can be used to indicate which guest
|
||||||
|
cores should be scheduled as real-time cores, leaving the remainder to be
|
||||||
|
scheduled as non-real-time cores and to handle overhead processes. For example,
|
||||||
|
to allocate the first two cores of an 8 core instance as the non-real-time
|
||||||
|
cores:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ openstack flavor set $FLAVOR \
|
||||||
|
--property hw:cpu_realtime=yes \
|
||||||
|
--property hw:cpu_realtime_mask=2-7 # so 0,1 are non-real-time
|
||||||
|
|
||||||
|
In this configuration, any non-real-time cores configured will have an implicit
|
||||||
|
``dedicated`` :ref:`CPU pinning policy <cpu-pinning-policies>` applied. It is
|
||||||
|
possible to apply a ``shared`` policy for these non-real-time cores by
|
||||||
|
specifying the ``mixed`` :ref:`CPU pinning policy <cpu-pinning-policies>` via
|
||||||
|
the :nova:extra-spec:`hw:cpu_policy` extra spec. This can be useful to increase
|
||||||
|
resource utilization of the host. For example:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ openstack flavor set $FLAVOR \
|
||||||
|
--property hw:cpu_policy=mixed \
|
||||||
|
--property hw:cpu_realtime=yes \
|
||||||
|
--property hw:cpu_realtime_mask=2-7 # so 0,1 are non-real-time and unpinned
|
||||||
|
|
||||||
|
Finally, you can explicitly :ref:`offload guest overhead processes to another
|
||||||
|
host core <emulator-thread-pinning-policies>` using the
|
||||||
|
:nova:extra-spec:`hw:emulator_threads_policy` extra spec. For example:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ openstack flavor set $FLAVOR \
|
||||||
|
--property hw:cpu_realtime=yes \
|
||||||
|
--property hw:emulator_thread_policy=share
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Emulator thread pinning requires additional host configuration.
|
||||||
|
Refer to :ref:`the documentation <emulator-thread-pinning-policies>` for
|
||||||
|
more information.
|
||||||
|
|
||||||
|
In addition to configuring the instance CPUs, it is also likely that you will
|
||||||
|
need to configure guest huge pages. For information on how to configure these,
|
||||||
|
refer to :doc:`the documentation <huge-pages>`
|
||||||
|
|
||||||
|
References
|
||||||
|
----------
|
||||||
|
|
||||||
|
* `Libvirt real time instances (spec)`__
|
||||||
|
* `The Real Time Linux collaborative project`__
|
||||||
|
* `Deploying Real Time OpenStack`__
|
||||||
|
|
||||||
|
.. __: https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/libvirt-real-time.html
|
||||||
|
.. __: https://wiki.linuxfoundation.org/realtime/start
|
||||||
|
.. __: https://that.guru/blog/deploying-real-time-openstack/
|
|
@ -114,9 +114,8 @@ driver-specific.
|
||||||
~~~~~~~~~
|
~~~~~~~~~
|
||||||
|
|
||||||
The following extra specs are used to configure quotas for various
|
The following extra specs are used to configure quotas for various
|
||||||
paravirtualized devices.
|
paravirtualized devices. Different quotas are supported by different virt
|
||||||
|
drivers, as noted below.
|
||||||
They are only supported by the libvirt virt driver.
|
|
||||||
|
|
||||||
.. extra-specs:: quota
|
.. extra-specs:: quota
|
||||||
|
|
||||||
|
|
|
@ -184,103 +184,6 @@ Performance Monitoring Unit (vPMU)
|
||||||
required, such workloads should set ``hw:pmu=False``. For most workloads
|
required, such workloads should set ``hw:pmu=False``. For most workloads
|
||||||
the default of unset or enabling the vPMU ``hw:pmu=True`` will be correct.
|
the default of unset or enabling the vPMU ``hw:pmu=True`` will be correct.
|
||||||
|
|
||||||
.. _extra-specs-cpu-topology:
|
|
||||||
|
|
||||||
CPU topology
|
|
||||||
For the libvirt driver, you can define the topology of the processors in the
|
|
||||||
virtual machine using properties. The properties with ``max`` limit the
|
|
||||||
number that can be selected by the user with image properties.
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
$ openstack flavor set FLAVOR-NAME \
|
|
||||||
--property hw:cpu_sockets=FLAVOR-SOCKETS \
|
|
||||||
--property hw:cpu_cores=FLAVOR-CORES \
|
|
||||||
--property hw:cpu_threads=FLAVOR-THREADS \
|
|
||||||
--property hw:cpu_max_sockets=FLAVOR-SOCKETS \
|
|
||||||
--property hw:cpu_max_cores=FLAVOR-CORES \
|
|
||||||
--property hw:cpu_max_threads=FLAVOR-THREADS
|
|
||||||
|
|
||||||
Where:
|
|
||||||
|
|
||||||
- FLAVOR-SOCKETS: (integer) The number of sockets for the guest VM. By
|
|
||||||
default, this is set to the number of vCPUs requested.
|
|
||||||
- FLAVOR-CORES: (integer) The number of cores per socket for the guest VM. By
|
|
||||||
default, this is set to ``1``.
|
|
||||||
- FLAVOR-THREADS: (integer) The number of threads per core for the guest VM.
|
|
||||||
By default, this is set to ``1``.
|
|
||||||
|
|
||||||
.. _extra-specs-cpu-policy:
|
|
||||||
|
|
||||||
CPU pinning policy
|
|
||||||
For the libvirt driver, you can pin the virtual CPUs (vCPUs) of instances to
|
|
||||||
the host's physical CPU cores (pCPUs) using properties. You can further
|
|
||||||
refine this by stating how hardware CPU threads in a simultaneous
|
|
||||||
multithreading-based (SMT) architecture be used. These configurations will
|
|
||||||
result in improved per-instance determinism and performance.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
SMT-based architectures include Intel processors with Hyper-Threading
|
|
||||||
technology. In these architectures, processor cores share a number of
|
|
||||||
components with one or more other cores. Cores in such architectures are
|
|
||||||
commonly referred to as hardware threads, while the cores that a given
|
|
||||||
core share components with are known as thread siblings.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
Host aggregates should be used to separate these pinned instances from
|
|
||||||
unpinned instances as the latter will not respect the resourcing
|
|
||||||
requirements of the former.
|
|
||||||
|
|
||||||
.. code:: console
|
|
||||||
|
|
||||||
$ openstack flavor set FLAVOR-NAME \
|
|
||||||
--property hw:cpu_policy=CPU-POLICY \
|
|
||||||
--property hw:cpu_thread_policy=CPU-THREAD-POLICY
|
|
||||||
|
|
||||||
Valid CPU-POLICY values are:
|
|
||||||
|
|
||||||
- ``shared``: (default) The guest vCPUs will be allowed to freely float
|
|
||||||
across host pCPUs, albeit potentially constrained by NUMA policy.
|
|
||||||
- ``dedicated``: The guest vCPUs will be strictly pinned to a set of host
|
|
||||||
pCPUs. In the absence of an explicit vCPU topology request, the drivers
|
|
||||||
typically expose all vCPUs as sockets with one core and one thread. When
|
|
||||||
strict CPU pinning is in effect the guest CPU topology will be setup to
|
|
||||||
match the topology of the CPUs to which it is pinned. This option implies
|
|
||||||
an overcommit ratio of 1.0. For example, if a two vCPU guest is pinned to a
|
|
||||||
single host core with two threads, then the guest will get a topology of
|
|
||||||
one socket, one core, two threads.
|
|
||||||
- ``mixed``: This policy will create an instance combined with the ``shared``
|
|
||||||
policy vCPUs and ``dedicated`` policy vCPUs, as a result, some guest vCPUs
|
|
||||||
will be freely float across host pCPUs and the rest of guest vCPUs will be
|
|
||||||
pinned to host pCPUs. The pinned guest vCPUs are configured using the
|
|
||||||
``hw:cpu_dedicated_mask`` extra spec.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
The ``hw:cpu_dedicated_mask`` option is only valid if ``hw:cpu_policy``
|
|
||||||
is set to ``mixed`` and cannot be configured with ``hw:cpu_realtime_mask``
|
|
||||||
at the same time.
|
|
||||||
|
|
||||||
Valid CPU-THREAD-POLICY values are:
|
|
||||||
|
|
||||||
- ``prefer``: (default) The host may or may not have an SMT architecture.
|
|
||||||
Where an SMT architecture is present, thread siblings are preferred.
|
|
||||||
- ``isolate``: The host must not have an SMT architecture or must emulate a
|
|
||||||
non-SMT architecture. Hosts that support SMT (by reporting the
|
|
||||||
``HW_CPU_HYPERTHREADING`` trait) are excluded.
|
|
||||||
- ``require``: The host must have an SMT architecture and must report the
|
|
||||||
``HW_CPU_HYPERTHREADING`` trait. Each vCPU is allocated on thread siblings.
|
|
||||||
If the host does not have an SMT architecture, then it is not used. If the
|
|
||||||
host has an SMT architecture, but not enough cores with free thread
|
|
||||||
siblings are available, then scheduling fails.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
The ``hw:cpu_thread_policy`` option is valid if ``hw:cpu_policy`` is set
|
|
||||||
to ``dedicated`` or ``mixed``.
|
|
||||||
|
|
||||||
.. _pci_numa_affinity_policy:
|
.. _pci_numa_affinity_policy:
|
||||||
|
|
||||||
PCI NUMA Affinity Policy
|
PCI NUMA Affinity Policy
|
||||||
|
@ -336,60 +239,6 @@ PCI NUMA Affinity Policy
|
||||||
|
|
||||||
* There is no information about PCI-NUMA affinity available
|
* There is no information about PCI-NUMA affinity available
|
||||||
|
|
||||||
.. _extra-specs-numa-topology:
|
|
||||||
|
|
||||||
NUMA topology
|
|
||||||
For the libvirt driver, you can define the host NUMA placement for the
|
|
||||||
instance vCPU threads as well as the allocation of instance vCPUs and memory
|
|
||||||
from the host NUMA nodes. For flavors whose memory and vCPU allocations are
|
|
||||||
larger than the size of NUMA nodes in the compute hosts, the definition of a
|
|
||||||
NUMA topology allows hosts to better utilize NUMA and improve performance of
|
|
||||||
the instance OS.
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
$ openstack flavor set FLAVOR-NAME \
|
|
||||||
--property hw:numa_nodes=FLAVOR-NODES \
|
|
||||||
--property hw:numa_cpus.N=FLAVOR-CORES \
|
|
||||||
--property hw:numa_mem.N=FLAVOR-MEMORY
|
|
||||||
|
|
||||||
Where:
|
|
||||||
|
|
||||||
- FLAVOR-NODES: (integer) The number of host NUMA nodes to restrict execution
|
|
||||||
of instance vCPU threads to. If not specified, the vCPU threads can run on
|
|
||||||
any number of the host NUMA nodes available.
|
|
||||||
- N: (integer) The instance NUMA node to apply a given CPU or memory
|
|
||||||
configuration to, where N is in the range ``0`` to ``FLAVOR-NODES - 1``.
|
|
||||||
- FLAVOR-CORES: (comma-separated list of integers) A list of instance vCPUs
|
|
||||||
to map to instance NUMA node N. If not specified, vCPUs are evenly divided
|
|
||||||
among available NUMA nodes.
|
|
||||||
- FLAVOR-MEMORY: (integer) The number of MB of instance memory to map to
|
|
||||||
instance NUMA node N. If not specified, memory is evenly divided among
|
|
||||||
available NUMA nodes.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
``hw:numa_cpus.N`` and ``hw:numa_mem.N`` are only valid if
|
|
||||||
``hw:numa_nodes`` is set. Additionally, they are only required if the
|
|
||||||
instance's NUMA nodes have an asymmetrical allocation of CPUs and RAM
|
|
||||||
(important for some NFV workloads).
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
The ``N`` parameter is an index of *guest* NUMA nodes and may not
|
|
||||||
correspond to *host* NUMA nodes. For example, on a platform with two NUMA
|
|
||||||
nodes, the scheduler may opt to place guest NUMA node 0, as referenced in
|
|
||||||
``hw:numa_mem.0`` on host NUMA node 1 and vice versa. Similarly, the
|
|
||||||
integers used for ``FLAVOR-CORES`` are indexes of *guest* vCPUs and may
|
|
||||||
not correspond to *host* CPUs. As such, this feature cannot be used to
|
|
||||||
constrain instances to specific host CPUs or NUMA nodes.
|
|
||||||
|
|
||||||
.. warning::
|
|
||||||
|
|
||||||
If the combined values of ``hw:numa_cpus.N`` or ``hw:numa_mem.N`` are
|
|
||||||
greater than the available number of CPUs or memory respectively, an
|
|
||||||
exception is raised.
|
|
||||||
|
|
||||||
.. _extra-specs-memory-encryption:
|
.. _extra-specs-memory-encryption:
|
||||||
|
|
||||||
Hardware encryption of guest memory
|
Hardware encryption of guest memory
|
||||||
|
@ -402,146 +251,6 @@ Hardware encryption of guest memory
|
||||||
$ openstack flavor set FLAVOR-NAME \
|
$ openstack flavor set FLAVOR-NAME \
|
||||||
--property hw:mem_encryption=True
|
--property hw:mem_encryption=True
|
||||||
|
|
||||||
.. _extra-specs-realtime-policy:
|
|
||||||
|
|
||||||
CPU real-time policy
|
|
||||||
For the libvirt driver, you can state that one or more of your instance
|
|
||||||
virtual CPUs (vCPUs), though not all of them, run with a real-time policy.
|
|
||||||
When used on a correctly configured host, this provides stronger guarantees
|
|
||||||
for worst case scheduler latency for vCPUs and is a requirement for certain
|
|
||||||
applications.
|
|
||||||
|
|
||||||
.. todo::
|
|
||||||
|
|
||||||
Document the required steps to configure hosts and guests. There are a lot
|
|
||||||
of things necessary, from isolating hosts and configuring the
|
|
||||||
``[compute] cpu_dedicated_set`` nova configuration option on the host, to
|
|
||||||
choosing a correctly configured guest image.
|
|
||||||
|
|
||||||
.. important::
|
|
||||||
|
|
||||||
While most of your instance vCPUs can run with a real-time policy, you must
|
|
||||||
either mark at least one vCPU as non-real-time to be account for emulator
|
|
||||||
overhead (housekeeping) or explicitly configure an :ref:`emulator thread
|
|
||||||
policy <extra-specs-emulator-threads-policy>`.
|
|
||||||
|
|
||||||
.. important::
|
|
||||||
|
|
||||||
To use this extra spec, you must enable pinned CPUs. Refer to
|
|
||||||
:ref:`CPU policy <extra-specs-cpu-policy>` for more information.
|
|
||||||
|
|
||||||
.. code:: console
|
|
||||||
|
|
||||||
$ openstack flavor set FLAVOR-NAME \
|
|
||||||
--property hw:cpu_realtime=CPU-REALTIME-POLICY \
|
|
||||||
--property hw:cpu_realtime_mask=CPU-REALTIME-MASK
|
|
||||||
|
|
||||||
Where:
|
|
||||||
|
|
||||||
CPU-REALTIME-POLICY (enum):
|
|
||||||
One of:
|
|
||||||
|
|
||||||
- ``no``: (default) The guest vCPUs will not have a real-time policy
|
|
||||||
- ``yes``: The guest vCPUs will have a real-time policy
|
|
||||||
|
|
||||||
CPU-REALTIME-MASK (coremask):
|
|
||||||
A coremask indicating which vCPUs **will** or, if starting with a ``^``,
|
|
||||||
**will not** have a real-time policy. For example, a value of ``0-5``
|
|
||||||
indicates that vCPUs ``0`` to ``5`` will have a real-time policy.
|
|
||||||
Conversely, a value of ``^0-1`` indicates that all vCPUs *except* vCPUs
|
|
||||||
``0`` and ``1`` will have a real-time policy.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
The ``hw:cpu_realtime_mask`` option is only valid if ``hw:cpu_realtime``
|
|
||||||
is set to ``yes``.
|
|
||||||
|
|
||||||
.. versionchanged:: 22.0.0 (Victoria)
|
|
||||||
|
|
||||||
Previously, it was necessary to specify ``hw:cpu_realtime_mask`` when
|
|
||||||
``hw:cpu_realtime`` was set to yes. Starting in Victoria, it is possible
|
|
||||||
to omit this when an emulator thread policy is configured using the
|
|
||||||
``hw:emulator_threads_policy`` extra spec.
|
|
||||||
|
|
||||||
.. versionchanged:: 22.0.0 (Victoria)
|
|
||||||
|
|
||||||
Previously, the leading caret was necessary and omitting it would be
|
|
||||||
equivalent to not setting the mask, resulting in a failure to spawn
|
|
||||||
the instance.
|
|
||||||
|
|
||||||
.. _extra-specs-emulator-threads-policy:
|
|
||||||
|
|
||||||
Emulator threads policy
|
|
||||||
For the libvirt driver, you can assign a separate pCPU to an instance that
|
|
||||||
will be used for emulator threads, which are emulator processes not directly
|
|
||||||
related to the guest OS. This pCPU will used in addition to the pCPUs used
|
|
||||||
for the guest. This is generally required for use with a :ref:`real-time
|
|
||||||
workload <extra-specs-realtime-policy>`.
|
|
||||||
|
|
||||||
.. important::
|
|
||||||
|
|
||||||
To use this extra spec, you must enable pinned CPUs. Refer to :ref:`CPU
|
|
||||||
policy <extra-specs-cpu-policy>` for more information.
|
|
||||||
|
|
||||||
.. code:: console
|
|
||||||
|
|
||||||
$ openstack flavor set FLAVOR-NAME \
|
|
||||||
--property hw:emulator_threads_policy=THREAD-POLICY
|
|
||||||
|
|
||||||
The expected behavior of emulator threads depends on the value of the
|
|
||||||
``hw:emulator_threads_policy`` flavor extra spec and the value of
|
|
||||||
:oslo.config:option:`compute.cpu_shared_set`. It is presented in the
|
|
||||||
following table:
|
|
||||||
|
|
||||||
.. list-table::
|
|
||||||
:header-rows: 1
|
|
||||||
:stub-columns: 1
|
|
||||||
|
|
||||||
* -
|
|
||||||
- :oslo.config:option:`compute.cpu_shared_set` set
|
|
||||||
- :oslo.config:option:`compute.cpu_shared_set` unset
|
|
||||||
* - ``hw:emulator_treads_policy`` unset (default)
|
|
||||||
- Pinned to all of the instance's pCPUs
|
|
||||||
- Pinned to all of the instance's pCPUs
|
|
||||||
* - ``hw:emulator_threads_policy`` = ``share``
|
|
||||||
- Pinned to :oslo.config:option:`compute.cpu_shared_set`
|
|
||||||
- Pinned to all of the instance's pCPUs
|
|
||||||
* - ``hw:emulator_threads_policy`` = ``isolate``
|
|
||||||
- Pinned to a single pCPU distinct from the instance's pCPUs
|
|
||||||
- Pinned to a single pCPU distinct from the instance's pCPUs
|
|
||||||
|
|
||||||
.. _extra-specs-large-pages-allocation:
|
|
||||||
|
|
||||||
Large pages allocation
|
|
||||||
You can configure the size of large pages used to back the VMs.
|
|
||||||
|
|
||||||
.. code:: console
|
|
||||||
|
|
||||||
$ openstack flavor set FLAVOR-NAME \
|
|
||||||
--property hw:mem_page_size=PAGE_SIZE
|
|
||||||
|
|
||||||
Valid ``PAGE_SIZE`` values are:
|
|
||||||
|
|
||||||
- ``small``: (default) The smallest page size is used. Example: 4 KB on x86.
|
|
||||||
- ``large``: Only use larger page sizes for guest RAM. Example: either 2 MB
|
|
||||||
or 1 GB on x86.
|
|
||||||
- ``any``: It is left up to the compute driver to decide. In this case, the
|
|
||||||
libvirt driver might try to find large pages, but fall back to small pages.
|
|
||||||
Other drivers may choose alternate policies for ``any``.
|
|
||||||
- pagesize: (string) An explicit page size can be set if the workload has
|
|
||||||
specific requirements. This value can be an integer value for the page size
|
|
||||||
in KB, or can use any standard suffix. Example: ``4KB``, ``2MB``,
|
|
||||||
``2048``, ``1GB``.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
Large pages can be enabled for guest RAM without any regard to whether the
|
|
||||||
guest OS will use them or not. If the guest OS chooses not to use huge
|
|
||||||
pages, it will merely see small pages as before. Conversely, if a guest OS
|
|
||||||
does intend to use huge pages, it is very important that the guest RAM be
|
|
||||||
backed by huge pages. Otherwise, the guest OS will not be getting the
|
|
||||||
performance benefit it is expecting.
|
|
||||||
|
|
||||||
.. _extra-spec-pci-passthrough:
|
.. _extra-spec-pci-passthrough:
|
||||||
|
|
||||||
PCI passthrough
|
PCI passthrough
|
||||||
|
|
Loading…
Reference in New Issue