From d8ff7d58f2e84177d74ace87d5ce9bdb36c1f7fc Mon Sep 17 00:00:00 2001 From: Lucas Alvares Gomes Date: Fri, 24 Apr 2020 14:08:25 +0100 Subject: [PATCH] [OVN] Documentation: SR-IOV guide for OVN This patch is adding some documentation about OVN + SR-IOV. Change-Id: I0353c24d1ff795ad09a9cdad28b6a839e916644c Signed-off-by: Lucas Alvares Gomes --- doc/source/admin/ovn/index.rst | 1 + doc/source/admin/ovn/sriov.rst | 171 +++++++++++++++++++++++++++++++++ 2 files changed, 172 insertions(+) create mode 100644 doc/source/admin/ovn/sriov.rst diff --git a/doc/source/admin/ovn/index.rst b/doc/source/admin/ovn/index.rst index ec7ffb9eaf6..6ef372d75ff 100644 --- a/doc/source/admin/ovn/index.rst +++ b/doc/source/admin/ovn/index.rst @@ -13,3 +13,4 @@ OVN Driver Administration Guide refarch/refarch dpdk troubleshooting + sriov diff --git a/doc/source/admin/ovn/sriov.rst b/doc/source/admin/ovn/sriov.rst new file mode 100644 index 00000000000..63dbb4ec5c5 --- /dev/null +++ b/doc/source/admin/ovn/sriov.rst @@ -0,0 +1,171 @@ +.. _ovn_sriov: + +==================== +SR-IOV guide for OVN +==================== + +The purpose of this page is to describe how SR-IOV works with OVN. Prior +to reading this document, it is recommended to first read :ref:`the +basic guide for SR-IOV`. + +External ports +~~~~~~~~~~~~~~ + +In order for SR-IOV to work with the +Neutron driver we are leveraging the `external ports +`_ +feature from the OVN project. When virtual machines are booted +on hypervisors supporting SR-IOV nics, the local ovn-controllers are +unable to reply to the VM's DHCP, internal DNS, IPv6 router solicitation +requests, etc... since the hypervisor is bypassed in the SR-IOV case. OVN +then introduced the idea of having ``external`` ports which are able to +reply on behalf of those VM ports external to the hypervisor that they +are running on. + +The OVN Neutron driver will create a port of the type ``external`` +for ports with the following VNICs set: + +* direct +* direct-physical +* macvtap + +Also, ports of the type ``external`` will be scheduled on the gateway +nodes (controller or networker nodes) in HA mode by the OVN Neutron +driver. Check the `OVN Database information`_ section for more +information. + +Environment setup for OVN SR-IOV +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are a very few differences between setting up an environment for +SR-IOV for the OVS and OVN Neutron drivers. As mentioned at the beginning +of this document, the instructions from the :ref:`the basic guide for +SR-IOV` are required for getting SR-IOV working with the +OVN driver. + +The only differences required for an OVN deployment are: + +* When configuring the ``mechanism_drivers`` in the *ml2_conf.ini* file + we should specify ``ovn`` driver instead of the ``openvswitch`` driver +* Disabling the Neutron DHCP agent +* Deploying the OVN Metadata agent on the gateway nodes (controller + or networker nodes) + + +OVN Database information +~~~~~~~~~~~~~~~~~~~~~~~~ + +Before getting into the ports information, the previous sections +talks about **gateway nodes**, the OVN Neutron driver identifies +a gateway node by the ``ovn-cms-options=enable-chassis-as-gw`` and +``ovn-bridge-mappings`` options in the external_ids column from the +``Chassis`` table in the OVN Southbound database: + +.. code-block:: bash + + $ ovn-sbctl list Chassis + _uuid : 12b13aff-a821-4cde-a4ac-d9cf8e2c91bc + external_ids : {ovn-cms-options=enable-chassis-as-gw, ovn-bridge-mappings="public:br-ex", ...} + hostname : controller-0 + name : "1a462946-ccfd-46a6-8abf-9dca9eb558fb" + ... + +.. end + +For more information about both of these options, please +take a look at the `ovn-controller documentation +`_. + +These options can be set by running the following command locally on each +gateway node (note, the ``ovn-bridge-mappings`` will need to be adapted +to your environment): + +.. code-block:: bash + + $ ovs-vsctl set Open_vSwitch . external-ids:ovn-cms-options=\"enable-chassis-as-gw\" external-ids:ovn-bridge-mappings=\"public:br-ex\" + +.. end + +As mentioned in the `External ports`_ section, every time a Neutron port +with a certain VNIC is created the OVN driver will create a port of the +type ``external`` in the OVN Northbound database. These ports can be +found by issuing the following command: + +.. code-block:: bash + + $ ovn-nbctl find Logical_Switch_Port type=external + _uuid : 105e83ae-252d-401b-a1a7-8d28ec28a359 + ha_chassis_group : [43047e7b-4c78-4984-9788-6263fcc69885] + type : external + ... + +.. end + +The ``ha_chassis_group`` column indicates which HA Chassis Group that +port belongs to, to find that group do: + +.. code-block:: bash + + # The UUID is the one from the ha_chassis_group column from + # the Logical_Switch_Port table + $ ovn-nbctl list HA_Chassis_Group 43047e7b-4c78-4984-9788-6263fcc69885 + _uuid : 43047e7b-4c78-4984-9788-6263fcc69885 + external_ids : {} + ha_chassis : [3005bf84-fc95-4361-866d-bfa1c980adc8, 72c7671e-dd48-4100-9741-c47221672961] + name : default_ha_chassis_group + +.. end + +.. note:: + + For now, the OVN driver only has one HA Chassis Group created called + ``default_ha_chassis_group``. All ``external`` ports in the system + will belong to this group. + +The chassis that are members of the ``default_ha_chassis_group`` HA +Chassis Group are listed in the ``ha_chassis`` column. Those are the +gateway nodes (controller or networker nodes) in the deployment and +it's where the ``external`` ports will be scheduled. In order to +find which gateway node the external ports are scheduled on use the +following command: + +.. code-block:: bash + + # The UUIDs are the UUID members of the HA Chassis Group + # (ha_chassis column from the HA_Chassis_Group table) + $ ovn-nbctl list HA_Chassis 3005bf84-fc95-4361-866d-bfa1c980adc8 72c7671e-dd48-4100-9741-c47221672961 + _uuid : 3005bf84-fc95-4361-866d-bfa1c980adc8 + chassis_name : "1a462946-ccfd-46a6-8abf-9dca9eb558fb" + external_ids : {} + priority : 32767 + + _uuid : 72c7671e-dd48-4100-9741-c47221672961 + chassis_name : "a0cb9d55-a6da-4f84-857f-d4b674088c8c" + external_ids : {} + priority : 32766 + +.. end + +Note the ``priority`` column from the previous command, the chassis with +the highest ``priority`` from that list is the chassis that will have +the external ports scheduled on it. In our example above, the chassis +with the UUID ``1a462946-ccfd-46a6-8abf-9dca9eb558fb`` is the one. + +Whenever the chassis with the highest priority goes down, the ports will +be automatically scheduled on the next chassis with the highest priority +which is alive. So, the external ports are HA out of the box. + +Known limitations +~~~~~~~~~~~~~~~~~ + +The current SR-IOV implementation for the OVN Neutron driver has a few +known limitations that should be addressed in the future: + +#. At the moment, **all** external ports will be scheduled on a single + gateway node since there's only one HA Chassis Group for all of those + ports. + +#. Routing on VLAN tenant network will not work with SR-IOV. This + is because the external ports are not being co-located with + the logical router's gateway ports, for more information take a look at + `bug #1875852 `_.