8.1 KiB

Raw Blame History

Strict minimum bandwidth support for tunnelled networks

Launchpad bug: https://bugs.launchpad.net/neutron/+bug/1998608

The aim of the RFE is to create a tool to perform generic tasks in the compute nodes when using ML2/OVN.

Problem Description

ML2/OVN is a mechanism driver that doesn't have backend agents. This mechanism driver exposes two agent types: the OVN controller agent and the metadata agent. The OVN controller agent is a representation of the status of the ovn-controller service on the node; the metadata agent is the service that provides the metadata to the virtual machines.

However the ML2/OVN mechanism driver does not have an agent to perform the local Open vSwitch service configuration as in ML2/OVS, with extension drivers (like QoS or Trunk). The ovn-controller reads the OVN Southbound database to configure the local OVS database.

The lack of an agent running on the local node, owned by Neutron, prevents us from implementing some functionalities currently not supported by OVN nor the drivers. For example, an this will be the first feature that the OVN monitor will implement, hardware offloaded ports cannot apply the OVS QoS rules due to limitations in the drivers.

Proposed Change

This RFE proposes to create a generic agent running on the compute node. This agent will be called "OVN monitor". The execution of this monitor is discretionary and will be needed only if the specific features implemented on it are requested in a particular compute node. In other words, initially this service will be needed if a compute node has hardware offloaded ports and QoS is needed (new features could be implemented in the future).

Unless other mechanism driver agents (like in ML2/OVS or ML2/SRIOV), this agent does not implement an RPC client. The information required to work will be retrieved from the local OVS database, the OVN Northbound database and the OVN Southbound database.

QoS for hardware offloaded ports

The first feature that will be implemented within this agent is the QoS extension for hardware offloaded ports. Because of the current drivers limitations, these ports cannot be properly configured with the current QoS rules.

This RFE proposes to use the corresponding "ip-link"¹ commands (or their "pyroute2" equivalents) to configure the port representor virtual function rates (maximum and minimum). This is similar to what is done in ML2/SRIOV for a virtual function (VF).

A port representor (or a devlink port²) is an API to expose the device information; it is not a physical port but a representative of a hardware port. The devlink port is connected to the local OVS switch; using the "devlink" tool (or the "pyroute2" equivalent), it is possible to retrieve the port information, including the physical function (PF) PCI address. E.g.:

$ devlink port show enp4s0f1_5 -jp
 "port": {
     "pci/0000:04:00.1/65542": {
         "type": "eth",
         "netdev": "enp4s0f1_5",
         "flavour": "pcivf",
         "pfnum": 1,
         "vfnum": 5,
         "splittable": false,
         "function": {
             "hw_addr": "fa:16:3e:f8:7b:10"
         }
     }
 }

With the PF PCI address, it is possible to retrieve the PF name, that will be stored in:

$ cat /sys/bus/pci/devices/$pf_pci_address/net

Using the ML2/SRIOV class PciDeviceIPWrapper, it is possible to set the maximum and minimum rates of a VF, knowing the VF index and the PF name.

OVN QoS information location

The OVN QoS information is stored in two different places (due to how QoS is currently implemented in ML2/OVN and core OVN):

The maximum bandwidth and maximum burst limits are stored in the "Qos:rate" dictionary:

$ ovn-nbctl list Qos
 _uuid               : 376303c4-5290-4c4e-a489-d35894315199
 action              : {}
 bandwidth           : {rate=1000, burst=800}
 direction           : to-lport
 external_ids        : {"neutron:port_id"="89a81cc0-7b3e-473d-8c01-2539cf2a9a6a"}
 match               : "outport == \"89a81cc0-7b3e-473d-8c01-2539cf2a9a6a\""
 priority            : 2002

The minimum bandwidth rate is stored in the "Logical_Switch_Port:options" dictionary:

$ ovn-nbctl list Logical_Switch_Port
 _uuid               : 502b3852-4a46-4fcb-9b49-03063cfc0b34
 addresses           : ["fa:16:3e:c4:b8:29 10.0.0.8"]
 dhcpv4_options      : a7e4cfb1-ef22-490a-9ffe-fea04de3fe1c
 dhcpv6_options      : []
 dynamic_addresses   : []
 enabled             : true
 external_ids        : {...(skipped)}
 ha_chassis_group    : []
 name                : "e6808371-c9ac-4015-94a3-7f16ac3fbb2d"
 options             : {mcast_flood_reports="true", qos_min_rate="1000",
                        requested-chassis=u20ovn}
 parent_name         : []
 port_security       : ["fa:16:3e:c4:b8:29 10.0.0.8"]
 tag                 : []
 tag_request         : []
 type                : ""
 up                  : true

OVN QoS events

The hardware offloaded QoS extension will configure the QoS setting on a port reacting to the following events (please check the working POC³ for more information):

The local OVS interface creation: with this event, the OVN monitor will store what ports are bound to the local instance. It will store the Neutron port ID (stored in the "Interface.external_ids:iface-id" string) and the OVS port name.
The local OVS interface deletion: this event will trigger the QoS reset and the interface local cache deletion.
The OVN Southbound "Port_Binding" creation event: this event is received after a port is created in a local OVS. If this event is received, that will trigger the QoS update of the local port. It's worth mentioning that this event happens always after the local OVS interface creation; that means the OVN monitor has already registered that this port is bound locally.
The OVN Northbound "Logical_Switch_Port" register change: if minimum bandwidth of a locally bound LSP changes, this event triggers the QoS update.
The OVN Northbound "QoS" register change: similar to the previous one but affecting the maximum bandwidth limit.

REST API Impact

This RFE does not introduce any API change.

Data Model Impact

This RFE does not introduce any model change.

Security Impact

None.

Performance Impact

Each monitor will have a connection to the local OVS database and the remotes OVN Northbound and Southbound databases. The connection to the remove OVN databases can have a severe impact on the load of the OVN database node (that are usually the OpenStack controllers). This initial implementation will subscribe to the following tables:

Northbound: QoS, Logical_Switch_Port and Logical_Switch
Southbound: Chassis, Encap, Port_Binding, Datapath_Binding and SB_Global

Other Impact

None.

Implementation

Assignee(s)

Primary assignees:: Rodolfo Alonso Hernandez <ralonsoh@redhat.com> (IRC: ralonsoh)

Work Items

OVN monitor implementation and the hardware offloaded QoS extension.
Tests and CI related changes.

Testing

Unit/functional tests.

Note

The hardware offloaded QoS extension requires specific hardware to test this feature. Currently is not possible to implement any tempest test on the CI.

Documentation Impact

User Documentation

Information about how to configure and spawn the OVN monitor.
Information about the hardware offloaded QoS extension.

8.1 KiB Raw Blame History