Document remote-managed port usage considerations
While most of the SR-IOV related documentation resides in the Neutron repository which is going to have a separate section on the topic of supporting remote-managed ports and off-path networking backends, there are still some things specific to Nova which are worth documenting in Nova docs. https://docs.openstack.org/neutron/latest/admin/config-sriov.html Implements: blueprint integration-with-off-path-network-backends Change-Id: I3c5fe8ec0539e10d07b1b4888e9833bc7ede1d04
This commit is contained in:
Dmitrii Shcherbakov
parent
db15cb9513
commit
419391e97d
@ -37,6 +37,42 @@ A full guide on configuring and using SR-IOV is provided in the
|
||||
Nova will ignore PCI devices reported by the hypervisor if the address is
|
||||
outside of these ranges.
|
||||
|
||||
.. versionadded:: 25.0.0
|
||||
|
||||
For information on creating servers with remotely-managed SR-IOV network
|
||||
interfaces of SmartNIC DPUs, refer to the relevant section in
|
||||
:neutron-doc:`Networking Guide <admin/ovn/smartnic_dpu>`.
|
||||
|
||||
**Limitations**
|
||||
|
||||
* Only VFs are supported and they must be tagged in the Nova Compute
|
||||
configuration in the ``passthrough_whitelist`` option as
|
||||
``remote_managed: "true"``. There is no auto-discovery of this based
|
||||
on vendor and product IDs;
|
||||
* Either VF or its respective PF must expose a PCI VPD capability with a
|
||||
unique card serial number according to the PCI/PCIe specifications
|
||||
(see `the Libvirt docs <https://libvirt.org/drvnodedev.html#VPDCap>`_ to
|
||||
get an example of how VPD data is represented and what to expect). If
|
||||
this is not the case, those devices will not appear in allocation pools;
|
||||
* Only the Libvirt driver is capable of supporting this feature at the
|
||||
time of writing;
|
||||
* The support for VPD capability handling in Libvirt was added in release
|
||||
`7.9.0 <https://libvirt.org/news.html#v7-9-0-2021-11-01>`_ - older
|
||||
versions are not supported by this feature;
|
||||
* All compute nodes must be upgraded to the Yoga release in order for
|
||||
scheduling of nodes with ``VNIC_TYPE_REMOTE_MANAGED`` ports to succeed;
|
||||
* The same limitations apply to operations like live migration as with
|
||||
`legacy SR-IOV ports <https://docs.openstack.org/neutron/latest/admin/config-sriov.html#known-limitations>`_;
|
||||
* Clearing a VLAN by programming VLAN 0 must not result in errors in the
|
||||
VF kernel driver at the compute host. Before v8.1.0 Libvirt clears
|
||||
a VLAN by programming VLAN 0 before passing a VF through to the guest
|
||||
which may result in an error depending on your driver and kernel version
|
||||
(see, for example, `this bug <https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1957753>`_
|
||||
which discusses a case relevant to one driver). As of Libvirt v8.1.0,
|
||||
EPERM errors encountered while programming VLAN 0 are ignored if
|
||||
VLAN clearning is not explicitly requested in the device XML (i.e.
|
||||
VLAN 0 is not specified explicitly).
|
||||
|
||||
NUMA Affinity
|
||||
-------------
|
||||
|
||||
|
||||
@ -156,6 +156,70 @@ Refer to :oslo.config:option:`pci.alias` for syntax information.
|
||||
|
||||
Once configured, restart the :program:`nova-compute` service.
|
||||
|
||||
Special Tags
|
||||
^^^^^^^^^^^^
|
||||
|
||||
When specified in :oslo.config:option:`pci.passthrough_whitelist` some tags
|
||||
have special meaning:
|
||||
|
||||
``physical_network``
|
||||
Associates a device with a physical network label which corresponds to the
|
||||
``physical_network`` attribute of a network segment object in Neutron. For
|
||||
virtual networks such as overlays a value of ``null`` should be specified
|
||||
as follows: ``"physical_network": null``. In the case of physical networks,
|
||||
this tag is used to supply the metadata necessary for identifying a switched
|
||||
fabric to which a PCI device belongs and associate the port with the correct
|
||||
network segment in the networking backend. Besides typical SR-IOV scenarios,
|
||||
this tag can be used for remote-managed devices in conjunction with the
|
||||
``remote_managed`` tag.
|
||||
|
||||
``remote_managed``
|
||||
Used to specify whether a PCI device is managed remotely or not. By default,
|
||||
devices are implicitly tagged as ``"remote_managed": "false"`` but and they
|
||||
must be tagged as ``"remote_managed": "true"`` if ports with
|
||||
``VNIC_TYPE_REMOTE_MANAGED`` are intended to be used. Once that is done,
|
||||
those PCI devices will not be available for allocation for regular
|
||||
PCI passthrough use. Specifying ``"remote_managed": "true"`` is only valid
|
||||
for SR-IOV VFs and specifying it for PFs is prohibited.
|
||||
|
||||
.. important::
|
||||
It is recommended that PCI VFs that are meant to be remote-managed
|
||||
(e.g. the ones provided by SmartNIC DPUs) are tagged as remote-managed in
|
||||
order to prevent them from being allocated for regular PCI passthrough since
|
||||
they have to be programmed accordingly at the host that has access to the
|
||||
NIC switch control plane. If this is not done, instances requesting regular
|
||||
SR-IOV ports may get a device that will not be configured correctly and
|
||||
will not be usable for sending network traffic.
|
||||
|
||||
.. important::
|
||||
For the Libvirt virt driver, clearing a VLAN by programming VLAN 0 must not
|
||||
result in errors in the VF kernel driver at the compute host. Before v8.1.0
|
||||
Libvirt clears a VLAN before passing a VF through to the guest which may
|
||||
result in an error depending on your driver and kernel version (see, for
|
||||
example, `this bug <https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1957753>`_
|
||||
which discusses a case relevant to one driver). As of Libvirt v8.1.0, EPERM
|
||||
errors encountered while programming a VLAN are ignored if VLAN clearning is
|
||||
not explicitly requested in the device XML.
|
||||
|
||||
``trusted``
|
||||
If a port is requested to be trusted by specifying an extra option during
|
||||
port creation via ``--binding-profile trusted=true``, only devices tagged as
|
||||
``trusted: "true"`` will be allocated to instances. Nova will then configure
|
||||
those devices as trusted by the network controller through its PF device driver.
|
||||
The specific set of features allowed by the trusted mode of a VF will differ
|
||||
depending on the network controller itself, its firmware version and what a PF
|
||||
device driver version allows to pass to the NIC. Common features to be affected
|
||||
by this tag are changing the VF MAC address, enabling promiscuous mode or
|
||||
multicast promiscuous mode.
|
||||
|
||||
.. important::
|
||||
While the ``trusted tag`` does not directly conflict with the
|
||||
``remote_managed`` tag, network controllers in SmartNIC DPUs may prohibit
|
||||
setting the ``trusted`` mode on a VF via a PF device driver in the first
|
||||
place. It is recommended to test specific devices, drivers and firmware
|
||||
versions before assuming this feature can be used.
|
||||
|
||||
|
||||
Configure ``nova-scheduler``
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user