Document remote-managed port usage considerations
While most of the SR-IOV related documentation resides in the Neutron repository which is going to have a separate section on the topic of supporting remote-managed ports and off-path networking backends, there are still some things specific to Nova which are worth documenting in Nova docs. https://docs.openstack.org/neutron/latest/admin/config-sriov.html Implements: blueprint integration-with-off-path-network-backends Change-Id: I3c5fe8ec0539e10d07b1b4888e9833bc7ede1d04
This commit is contained in:
Dmitrii Shcherbakov
parent
db15cb9513
commit
419391e97d
@ -37,6 +37,42 @@ A full guide on configuring and using SR-IOV is provided in the
|
|||||||
Nova will ignore PCI devices reported by the hypervisor if the address is
|
Nova will ignore PCI devices reported by the hypervisor if the address is
|
||||||
outside of these ranges.
|
outside of these ranges.
|
||||||
|
|
||||||
|
.. versionadded:: 25.0.0
|
||||||
|
|
||||||
|
For information on creating servers with remotely-managed SR-IOV network
|
||||||
|
interfaces of SmartNIC DPUs, refer to the relevant section in
|
||||||
|
:neutron-doc:`Networking Guide <admin/ovn/smartnic_dpu>`.
|
||||||
|
|
||||||
|
**Limitations**
|
||||||
|
|
||||||
|
* Only VFs are supported and they must be tagged in the Nova Compute
|
||||||
|
configuration in the ``passthrough_whitelist`` option as
|
||||||
|
``remote_managed: "true"``. There is no auto-discovery of this based
|
||||||
|
on vendor and product IDs;
|
||||||
|
* Either VF or its respective PF must expose a PCI VPD capability with a
|
||||||
|
unique card serial number according to the PCI/PCIe specifications
|
||||||
|
(see `the Libvirt docs <https://libvirt.org/drvnodedev.html#VPDCap>`_ to
|
||||||
|
get an example of how VPD data is represented and what to expect). If
|
||||||
|
this is not the case, those devices will not appear in allocation pools;
|
||||||
|
* Only the Libvirt driver is capable of supporting this feature at the
|
||||||
|
time of writing;
|
||||||
|
* The support for VPD capability handling in Libvirt was added in release
|
||||||
|
`7.9.0 <https://libvirt.org/news.html#v7-9-0-2021-11-01>`_ - older
|
||||||
|
versions are not supported by this feature;
|
||||||
|
* All compute nodes must be upgraded to the Yoga release in order for
|
||||||
|
scheduling of nodes with ``VNIC_TYPE_REMOTE_MANAGED`` ports to succeed;
|
||||||
|
* The same limitations apply to operations like live migration as with
|
||||||
|
`legacy SR-IOV ports <https://docs.openstack.org/neutron/latest/admin/config-sriov.html#known-limitations>`_;
|
||||||
|
* Clearing a VLAN by programming VLAN 0 must not result in errors in the
|
||||||
|
VF kernel driver at the compute host. Before v8.1.0 Libvirt clears
|
||||||
|
a VLAN by programming VLAN 0 before passing a VF through to the guest
|
||||||
|
which may result in an error depending on your driver and kernel version
|
||||||
|
(see, for example, `this bug <https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1957753>`_
|
||||||
|
which discusses a case relevant to one driver). As of Libvirt v8.1.0,
|
||||||
|
EPERM errors encountered while programming VLAN 0 are ignored if
|
||||||
|
VLAN clearning is not explicitly requested in the device XML (i.e.
|
||||||
|
VLAN 0 is not specified explicitly).
|
||||||
|
|
||||||
NUMA Affinity
|
NUMA Affinity
|
||||||
-------------
|
-------------
|
||||||
|
|
||||||
|
|||||||
@ -156,6 +156,70 @@ Refer to :oslo.config:option:`pci.alias` for syntax information.
|
|||||||
|
|
||||||
Once configured, restart the :program:`nova-compute` service.
|
Once configured, restart the :program:`nova-compute` service.
|
||||||
|
|
||||||
|
Special Tags
|
||||||
|
^^^^^^^^^^^^
|
||||||
|
|
||||||
|
When specified in :oslo.config:option:`pci.passthrough_whitelist` some tags
|
||||||
|
have special meaning:
|
||||||
|
|
||||||
|
``physical_network``
|
||||||
|
Associates a device with a physical network label which corresponds to the
|
||||||
|
``physical_network`` attribute of a network segment object in Neutron. For
|
||||||
|
virtual networks such as overlays a value of ``null`` should be specified
|
||||||
|
as follows: ``"physical_network": null``. In the case of physical networks,
|
||||||
|
this tag is used to supply the metadata necessary for identifying a switched
|
||||||
|
fabric to which a PCI device belongs and associate the port with the correct
|
||||||
|
network segment in the networking backend. Besides typical SR-IOV scenarios,
|
||||||
|
this tag can be used for remote-managed devices in conjunction with the
|
||||||
|
``remote_managed`` tag.
|
||||||
|
|
||||||
|
``remote_managed``
|
||||||
|
Used to specify whether a PCI device is managed remotely or not. By default,
|
||||||
|
devices are implicitly tagged as ``"remote_managed": "false"`` but and they
|
||||||
|
must be tagged as ``"remote_managed": "true"`` if ports with
|
||||||
|
``VNIC_TYPE_REMOTE_MANAGED`` are intended to be used. Once that is done,
|
||||||
|
those PCI devices will not be available for allocation for regular
|
||||||
|
PCI passthrough use. Specifying ``"remote_managed": "true"`` is only valid
|
||||||
|
for SR-IOV VFs and specifying it for PFs is prohibited.
|
||||||
|
|
||||||
|
.. important::
|
||||||
|
It is recommended that PCI VFs that are meant to be remote-managed
|
||||||
|
(e.g. the ones provided by SmartNIC DPUs) are tagged as remote-managed in
|
||||||
|
order to prevent them from being allocated for regular PCI passthrough since
|
||||||
|
they have to be programmed accordingly at the host that has access to the
|
||||||
|
NIC switch control plane. If this is not done, instances requesting regular
|
||||||
|
SR-IOV ports may get a device that will not be configured correctly and
|
||||||
|
will not be usable for sending network traffic.
|
||||||
|
|
||||||
|
.. important::
|
||||||
|
For the Libvirt virt driver, clearing a VLAN by programming VLAN 0 must not
|
||||||
|
result in errors in the VF kernel driver at the compute host. Before v8.1.0
|
||||||
|
Libvirt clears a VLAN before passing a VF through to the guest which may
|
||||||
|
result in an error depending on your driver and kernel version (see, for
|
||||||
|
example, `this bug <https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1957753>`_
|
||||||
|
which discusses a case relevant to one driver). As of Libvirt v8.1.0, EPERM
|
||||||
|
errors encountered while programming a VLAN are ignored if VLAN clearning is
|
||||||
|
not explicitly requested in the device XML.
|
||||||
|
|
||||||
|
``trusted``
|
||||||
|
If a port is requested to be trusted by specifying an extra option during
|
||||||
|
port creation via ``--binding-profile trusted=true``, only devices tagged as
|
||||||
|
``trusted: "true"`` will be allocated to instances. Nova will then configure
|
||||||
|
those devices as trusted by the network controller through its PF device driver.
|
||||||
|
The specific set of features allowed by the trusted mode of a VF will differ
|
||||||
|
depending on the network controller itself, its firmware version and what a PF
|
||||||
|
device driver version allows to pass to the NIC. Common features to be affected
|
||||||
|
by this tag are changing the VF MAC address, enabling promiscuous mode or
|
||||||
|
multicast promiscuous mode.
|
||||||
|
|
||||||
|
.. important::
|
||||||
|
While the ``trusted tag`` does not directly conflict with the
|
||||||
|
``remote_managed`` tag, network controllers in SmartNIC DPUs may prohibit
|
||||||
|
setting the ``trusted`` mode on a VF via a PF device driver in the first
|
||||||
|
place. It is recommended to test specific devices, drivers and firmware
|
||||||
|
versions before assuming this feature can be used.
|
||||||
|
|
||||||
|
|
||||||
Configure ``nova-scheduler``
|
Configure ``nova-scheduler``
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user