starlingx
/
docs
Code Issues Proposed changes

CI-PT configuration when SR-IOV is not available (stx 7.0, stx8, ds7) 

There is a known limitation [1] and NICs that do not support SR-IOV
require a different procedure [2] when configuring PCI-PT.

This change adds a note on checking SR-IOV support for the target NIC,
when configuring PCI-Passthrough for it, and adds the necessary
steps for the configuration to work properly whit this type of NIC.

For completeness, it also duplicates the PCI-PT example for when
configuring PCI SRIOV Ethernet Interfaces, with the necessary
changes to the procedure.

Fix merge conflict.

[1] https://bugs.launchpad.net/starlingx/+bug/1836682
[2] https://wiki.openstack.org/wiki/StarlingX/Networking#Useful_Networking_Commands

Partial-bug: 1836682

Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: I7258ab34cb7ce69a2f4b82c682f72d9467d95c70
(cherry picked from commit bc0870eade)
This commit is contained in:
Thales Elero Cervi 2022-08-16 15:59:13 -03:00 committed by Elisamara Aoki Goncalves
parent 2011610bba
commit c4aaec1c73
5 changed files with 272 additions and 79 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
doc/source/node_management/figures/ptj1538163621290.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 103 KiB

167
doc/source/node_management/openstack/configuring-pci-passthrough-ethernet-interfaces.rst
View File

@ -13,66 +13,105 @@ considerations.
.. rubric:: |context|
You can specify interfaces when you launch an instance.
Configure a |PCI| Passthrough Ethernet Interface on a host and request it for an
instance at boot/create time.
.. rubric:: |prereq|
.. note::
To use |PCI| passthrough or |SRIOV| devices, you must have Intel VT-x and
- To use |PCI| passthrough or |SRIOV| devices, you must have Intel VT-x and
Intel VT-d features enabled in the BIOS.
The exercise assumes that the underlying data network **group0-data0** exists
already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
**project1**.
- The exercise assumes that the underlying data network **group0-data0**
exists already, and that |VLAN| ID 10 is a valid segmentation ID assigned
to **project1**.
.. rubric:: |proc|
#. Log in as the **admin** user to the |os-prod-hor| interface.
#. Log in as the **admin** user to the |prod-p| |prod-hor-long|.
#. Lock the compute node you want to configure.
#. Configure the Ethernet interface to be used as a PCI passthrough interface.
#. Configure the Ethernet interface to be used as a |PCI| passthrough
interface. You can do this using Horizon or the CLI.
- Using Horison:
#. Select **Admin** \> **Platform** \> **Host Inventory** from the
left-hand pane.
#. Select the **Hosts** tab.
#. Click the name of the compute host.
#. Select the **Interfaces** tab.
#. Click the **Edit Interface** button associated with the interface
you want to configure.
#. Select **Admin** \> **Platform** \> **Host Inventory** from the left-hand pane.
The Edit Interface dialog appears.
#. Select the **Hosts** tab.
.. image:: /node_management/figures/ptj1538163621289.png
#. Click the name of the compute host.
#. Select **pci-passthrough**, from the **Interface Class** drop-down,
and then select the data network to attach the interface.
#. Select the **Interfaces** tab.
#. (Optional) You may also need to change the |MTU|.
#. Click the **Edit Interface** button associated with the interface you
want to configure.
- Using the CLI:
Assign the ``pci-sriov`` class to the interface.
.. code-block:: none
~(keystone_admin)$ system host-if-modify -c pci-passthrough compute-0 enp0s3
~(keystone_admin)$ system interface-datanetwork-assign compute-0 <enp0s3_interface_uuid> <group0_data0_data_network_uuid>
The Edit Interface dialog appears.
.. image:: /node_management/figures/ptj1538163621289.png
#. Check if the Ethernet interface supports |SRIOV|.
#. Check the host port associated with the configured |PCI|-passthrough interface.
Select **pci-passthrough**, from the **Interface Class** drop-down, and
then select the data network to attach the interface.
.. code-block:: none
You may also need to change the |MTU|.
~(keystone_admin)$ system host-if-list <host-name> | grep pci-passthrough
The interface can also be configured from the |CLI| as illustrated below:
#. Review the value of ``sriov_totalvfs`` on the target port.
.. code-block:: none
If the value is ``None``, the Ethernet interface does not support
|SRIOV|. Otherwise, it does.
~(keystone_admin)$ system host-if-modify -c pci-passthrough compute-0 enp0s3
~(keystone_admin)$ system interface-datanetwork-assign compute-0 <enp0s3_interface_uuid> <group0_data0_data_network_uuid>
.. code-block:: none
#. Create the **net0** project network
Select **Admin** \> **Network** \> **Networks**, select the Networks tab, and then click **Create Network**. Fill in the Create Network dialog box as illustrated below. You must ensure that:
~(keystone_admin)$ system host-port-show <host-name> <port-name> | grep sriov_totalvfs
- **project1** has access to the project network, either assigning it as
the owner, as in the illustration \(using **Project**\), or by enabling
the shared flag.
.. note::
For Ethernet interfaces without |SRIOV| support, there is a known limitation
reported `here <https://bugs.launchpad.net/starlingx/+bug/1836682>`__.
This limitation is overcome with a specific step later on this procedure.
.. _create-the-net0-project-network:
#. Create the ``net0`` project network for Ethernet interfaces that support
|SRIOV|.
.. warning::
If the Ethernet interface does not support |SRIOV|, **skip** this step.
Log in as the **admin** user to the |os-prod-hor-long|.
Select **Admin** \> **Network** \> **Networks**, select the Networks tab,
and then click **Create Network**. Fill in the Create Network dialog box as
illustrated below. You must ensure that:
- **project1** has access to the project network. Either by assigning it
as the owner, as in the illustration \(using **Project**\), or by
enabling the shared flag.
- The segmentation ID is set to 10.
@ -80,18 +119,19 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
.. image:: /node_management/figures/bek1516655307871.png
Click the **Next** button to proceed to the **Subnet** tab.
Click the **Next** button to proceed to the Subnet tab.
Click the **Next** button to proceed to the Subnet Details tab.
Click the **Next** button to proceed to the **Subnet Details** tab.
#. Configure the access switch. Refer to the OEM documentation to configure
the access switch.
Log in as the **admin** user to the |prod-p| |prod-hor-long|.
Configure the physical port on the access switch used to connect to
Ethernet interface **enp0s3** as an access port with default |VLAN| ID of 10.
Traffic across the connection is therefore untagged, and effectively
integrated into the targeted project network.
Ethernet interface ``enp0s3`` to be an access port with the default |VLAN|
ID of 10. Traffic across the connection is therefore untagged, and
effectively integrated into the targeted project network.
You can also use a trunk port on the access switch so that it handles
tagged packets as well. However, this opens the possibility for guest
@ -103,10 +143,14 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
#. Unlock the compute node.
#. Create a neutron port with a |VNIC| type, direct-physical.
#. Create a neutron port with a |VNIC| of type ``direct-physical`` for
Ethernet interfaces that support |SRIOV|.
The neutron port can also be created from the |CLI|, using the following
command. First, you must set up the environment and determine the correct
.. warning::
If the Ethernet interface does not support |SRIOV|, **skip** this step.
First, you must set up the environment and determine the correct
network |UUID| to use with the port.
.. code-block:: none
@ -119,16 +163,49 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
You have now created a port to be used when launching the server in the
next step.
#. Launch the virtual machine, specifying the port uuid created in *Step 7*.
#. Complete the following Nova configuration, for Ethernet interfaces that do
not support |SRIOV|.
.. warning::
If the Ethernet interface supports |SRIOV|, **skip** this step.
#. Get the Ethernet interface ``vendor_id`` and ``product_id``:
.. code-block:: none
~(keystone_admin)$ source /etc/platform/openrc
~(keystone_admin)$ system host-port-show <host-name> <port-name> | grep -E '(pvendor |pdevice )'
#. Use the retrieved IDs to create a |PCI| alias with
``"device_type":"type-PCI"``, as peer :ref:`Configure a PCI Alias in
Nova <configuring-a-pci-alias-in-nova>`.
#. Configure a flavor with the extra spec key ``pci_passthrough:alias``
pointing to the previously created |PCI| alias, as peer :ref:`Configure
a Flavor to Use a Generic PCI Device
<configuring-a-flavor-to-use-a-generic-pci-device>`
#. Launch the virtual machine
.. note::
You will need to source to the same project selected in the Create
Network 'net0' in *step 4*.
You will need to source to the same project selected in the :ref:`Create
Network net0 <create-the-net0-project-network>` step.
.. code-block:: none
- For Ethernet interfaces with |SRIOV| support: specify the port uuid
created.
~(keystone_admin)$ openstack server create --flavor <flavor_name> --image <image_name> --nic port-id=<port_uuid> <name>
.. code-block:: none
~(keystone_admin)$ openstack server create --flavor <flavor_name> --image <image_name> --nic port-id=<port_uuid> <name>
- For Ethernet interfaces without |SRIOV| support: specify the created
flavor to use the |PCI| device.
.. code-block:: none
~(keystone_admin)$ openstack server create --flavor <pci_flavor_name> --image <image_name>
For more information, see the Neutron documentation at:
`https://docs.openstack.org/neutron/train/admin/config-sriov.html

8
doc/source/node_management/openstack/index-node-mgmt-os-ccb47338adbc.rst
View File

@ -22,12 +22,12 @@ PCI Device Access for VMs
.. toctree::
:maxdepth: 1
sr-iov-encryption-acceleration
configuring-pci-passthrough-ethernet-interfaces
pci-passthrough-ethernet-interface-devices
configuring-a-flavor-to-use-a-generic-pci-device
configuring-pci-passthrough-ethernet-interfaces
generic-pci-passthrough
pci-device-access-for-vms
pci-sr-iov-ethernet-interface-devices
sr-iov-encryption-acceleration
pci-device-access-for-vms
configuring-a-flavor-to-use-a-generic-pci-device
exposing-a-generic-pci-device-for-use-by-vms
exposing-a-generic-pci-device-using-the-cli

175
doc/source/node_management/openstack/pci-sr-iov-ethernet-interface-devices.rst
View File

@ -2,30 +2,28 @@
.. vic1596720744539
.. _pci-sr-iov-ethernet-interface-devices:
=====================================
PCI SR-IOV Ethernet Interface Devices
=====================================
===============================================
Configure PCI SR-IOV Ethernet Interface Devices
===============================================
A |SRIOV| ethernet interface is a physical |PCI| ethernet |NIC| that implements
hardware-based virtualization mechanisms to expose multiple virtual network
interfaces that can be used by one or more virtual machines simultaneously.
An |SRIOV| Ethernet interface is a physical |PCI| Ethernet |NIC| that
implements hardware-based virtualization mechanisms to expose multiple virtual
network interfaces that can be used by one or more virtual machines
simultaneously.
The |PCI|-SIG Single Root I/O Virtualization and Sharing \(|SRIOV|\) specification
defines a standardized mechanism to create individual virtual ethernet devices
from a single physical ethernet interface. For each exposed virtual ethernet
device, formally referred to as a Virtual Function \(VF\), the |SRIOV| interface
provides separate management memory space, work queues, interrupts resources,
and |DMA| streams, while utilizing common resources behind the host interface.
Each VF therefore has direct access to the hardware and can be considered to be
an independent ethernet interface.
The |PCI|-SIG Single Root I/O Virtualization and Sharing \(|SRIOV|\)
specification defines a standardized mechanism to create individual virtual
Ethernet devices from a single physical Ethernet interface. For each exposed
virtual Ethernet device, formally referred to as a |VF|, the
|SRIOV| interface provides separate management memory space, work queues,
interrupts resources, and |DMA| streams, while utilizing common resources
behind the host interface. Each |VF| therefore has direct access to the hardware
and can be considered to be an independent Ethernet interface.
When compared with a |PCI| Passthrough ethernet interface, a |SRIOV| ethernet
When compared with a |PCI| Passthrough Ethernet interface, a |SRIOV| Ethernet
interface:
.. _pci-sr-iov-ethernet-interface-devices-ul-tyq-ymg-rr:
- Provides benefits similar to those of a |PCI| Passthrough ethernet interface,
- Provides benefits similar to those of a |PCI| Passthrough Ethernet interface,
including lower latency packet processing.
- Scales up more easily in a virtualized environment by providing multiple
@ -40,22 +38,139 @@ interface:
- Provides a similar configuration workflow when used on |prod-os|.
The configuration of a |PCI| |SRIOV| ethernet interface is identical to
:ref:`Configure PCI Passthrough ethernet Interfaces
<configure-pci-passthrough-ethernet-interfaces>` except that
The configuration of a |PCI| |SRIOV| Ethernet interface is almost identical to
:ref:`Configure PCI Passthrough Ethernet Interfaces
<configure-pci-passthrough-ethernet-interfaces>` and will be detailed bellow.
.. rubric:: |context|
.. _pci-sr-iov-ethernet-interface-devices-ul-ikt-nvz-qmb:
Configure a |PCI| |SRIOV| on a host and request it for an
instance at boot/create time.
- you use **pci-sriov** instead of **pci-passthrough** when defining the
network type of an interface
.. rubric:: |prereq|
- the segmentation ID of the project network\(s\) used is more significant
here since this identifies the particular |VF| of the |SRIOV| interface
- To use |PCI| passthrough or |SRIOV| devices, you must have Intel VT-x and
Intel VT-d features enabled in the BIOS.
- when creating the neutron port, you must use ``--vnic-typedirect``
- The exercise assumes that the underlying data network **group0-data0**
exists already, and that |VLAN| ID 10 is a valid segmentation ID assigned
to **project1**.
- when creating a neutron port backed by an |SRIOV| |VF|, you must use
``--vnic-type direct``
.. rubric:: |proc|
#. Log in as the **admin** user to the |prod-p| |prod-hor-long|.
#. Lock the compute node you want to configure.
#. Configure the Ethernet interface to be used as a |PCI| passthrough
interface. You can do this using Horizon or the CLI.
- Using Horison:
#. Select **Admin** \> **Platform** \> **Host Inventory** from the
left-hand pane.
#. Select the **Hosts** tab.
#. Click the name of the compute host.
#. Select the **Interfaces** tab.
#. Click the **Edit Interface** button associated with the interface
you want to configure.
The Edit Interface dialog appears.
.. image:: /node_management/figures/ptj1538163621290.png
#. Select **pci-sriov**, from the **Interface Class** drop-down, and
then select the data network to attach the interface.
#. (Optional) You may also need to change the |MTU|.
- Using the CLI:
Assign the ``pci-sriov`` class to the interface.
.. code-block:: none
~(keystone_admin)$ system host-if-modify -c pci-sriov compute-0 enp0s3
~(keystone_admin)$ system interface-datanetwork-assign compute-0 <enp0s3_interface_uuid> <group0_data0_data_network_uuid>
#. Create the ``net0`` project network.
Log in as the **admin** user to the |os-prod-hor-long|.
Select **Admin** \> **Network** \> **Networks**, select the **Networks**
tab, and then click **Create Network**. Fill in the **Create Network**
dialog box as illustrated below. You must ensure that:
- **project1** has access to the project network, either assigning it as
the owner, as in the illustration \(using **Project**\), or by enabling
the shared flag.
- The segmentation ID is set to 10.
.. image:: /node_management/figures/bek1516655307871.png
The segmentation ID of the project network\(s\) used is more significant
here since this identifies the particular |VF| of the |SRIOV| interface.
Click the **Next** button to proceed to the **Subnet** tab.
Click the **Next** button to proceed to the **Subnet Details** tab.
#. Configure the access switch. Refer to your |OEM| documentation for more
details.
Log in as the **admin** user to the |prod-p| |prod-hor-long|.
Configure the physical port on the access switch used to connect to
Ethernet interface ``enp0s3`` as an access port with default |VLAN| ID of 10.
Traffic across the connection is therefore untagged, and effectively
integrated into the targeted project network.
You can also use a trunk port on the access switch so that it handles
tagged packets as well. However, this opens the possibility for guest
applications to join other project networks using tagged packets with
different |VLAN| IDs, which might compromise the security of the system.
See |os-intro-doc|: :ref:`L2 Access Switches
<network-planning-l2-access-switches>` for other details regarding the
configuration of the access switch.
#. Unlock the compute node.
#. Create a neutron port with a |VNIC| of type ``direct-physical``.
Set up the environment and determine the correct network |UUID| to use with
the port.
.. code-block:: none
~(keystone_admin)$ source /etc/platform/openrc
~(keystone_admin)$ OS_AUTH_URL=http://keystone.openstack.svc.cluster.local/v3
~(keystone_admin)$ openstack network list | grep net0
~(keystone_admin)$ openstack port create --network <uuid_of_net0> --vnic-type direct <port_name>
You have now created a port to be used when launching the server in the
next step.
#. Launch the virtual machine specifying the |UUID| of the port previously
created.
.. note::
You will need to source to the same project selected in the
:ref:`Create Network net0 <create-the-net0-project-network>` step.
Specify the port uuid created.
.. code-block:: none
~(keystone_admin)$ openstack server create --flavor <flavor_name> --image <image_name> --nic port-id=<port_uuid> <name>
For more information, see the Neutron documentation at:
`https://docs.openstack.org/neutron/train/admin/config-sriov.html
<https://docs.openstack.org/neutron/train/admin/config-sriov.html>`__.

1
doc/source/shared/abbrevs.txt
View File

@ -89,6 +89,7 @@
.. |NVMe| replace:: :abbr:`NVMe (Non-Volatile Memory express)`
.. |OAM| replace:: :abbr:`OAM (Operations, administration and management)`
.. |OC| replace:: :abbr:`OC (Ordinary Clock)`
.. |OEM| replace:: :abbr:`OEM (Original Equipment Manufacturer)`
.. |OIDC| replace:: :abbr:`OIDC (OpenID Connect)`
.. |ONAP| replace:: :abbr:`ONAP (Open Network Automation Program)`
.. |OVS| replace:: :abbr:`OVS (Open Virtual Switch)`