openstack/openstack-ansible-os_neutron
Code Issues Proposed changes

Add Support for DPDK Bonding

Browse Source 
This patch will allow you to setup DPDK bonds within a given OVS
bridge using DPDK-accelerated interfaces. A new provider network
key, network_bond_interfaces, and related keys, have been
introduced.

Co-Authored-By: James Denton <james.denton@rackspace.com>

Change-Id: I3fc2846a0c2b6579e4cdb54c3a7c36620700cd44
This commit is contained in:
Satish Patel 2021-10-20 16:11:52 -04:00 committed by Dmitriy Rabotyagov
parent 782771f197
commit 0122ca1d65
4 changed files with 165 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

89
doc/source/app-openvswitch-dpdk.rst
View File

@ -27,7 +27,7 @@ We recommend that you read the following documents before proceeding:
* Open vSwitch with DPDK datapath:
`<https://docs.openstack.org/neutron/latest/admin/config-ovs-dpdk.html>`_
* Getting the best performance from DPDK:
`<https://doc.dpdk.org/guides-16.04/linux_gsg/nic_perf_intel_platform.html>`_
`<https://doc.dpdk.org/guides/linux_gsg/nic_perf_intel_platform.html>`_
* OpenStack documentation on hugepages:
`<https://docs.openstack.org/nova/latest/admin/huge-pages.html>`_
@ -156,11 +156,13 @@ The remaining cores can be reserved for virtual machine instances.
In this example, the breakdown would resemble the following:
```
| Reserved Cores | Purpose | node0 | node1 |
| ---------------------- | --------------------- | --------- | ----- |
| 0,8,16,24 | Host Operating System | 0,16 | 8,24 |
| 1,9,17,25 | DPDK PMDs | 1,17 | 9,25 |
| 2-7,18-23 | Virtual Machines | 2-7,18-23 | N/A |
```
The variables are overrides used to define this configuration are discussed
in the following sections.
@ -280,9 +282,11 @@ to a hexidecimal mask and defined using the ``ovs_dpdk_lcore_mask`` override.
To convert to a hex mask you must first establish the binary mask of chosen
cores using the following table:
```
| 31 | 30 | . | 24 | 23 | . | 17 | 16 | 15 | . | 9 | 8 | 7 | . | 1 | 0 |
| -- | -- | - | -- | -- | - | -- | -- | -- | - | -- | -- | -- | - | -- | -- |
| 0 | 0 | . | 1 | 0 | . | 0 | 1 | 0 | . | 0 | 1 | 0 | . | 0 | 1 |
```
The ellipses represent cores not shown. The binary mask for cores 0,8,16,24
can be determined in the following way:
@ -302,9 +306,11 @@ file or ``openstack_user_config.yml``:
The mask for cores 1,9,17,25 reserved for DPDK PMDs can be determined in
a similar fashion. The table would resemble the following:
```
| 31 | 30 | . | 25 | 24 | . | 17 | 16 | 15 | . | 9 | 8 | 7 | . | 1 | 0 |
| -- | -- | - | -- | -- | - | -- | -- | -- | - | -- | -- | -- | - | -- | -- |
| 0 | 0 | . | 1 | 0 | . | 1 | 0 | 0 | . | 1 | 0 | 0 | . | 1 | 0 |
```
The ellipses represent cores not shown. The binary mask for cores 1,9,17,254
can be determined in the following way:
@ -401,6 +407,12 @@ Specify provider network definitions in your
``/etc/openstack_deploy/openstack_user_config.yml`` that define one or more
Neutron provider bridges and related configuration:
.. note::
Bridges specified here will be created automatically. If *network_interface*
is defined, the interface will be placed into the bridge automatically
as a *DPDK-accelerated* interface.
.. code-block:: yaml
- network:
@ -409,13 +421,55 @@ Neutron provider bridges and related configuration:
type: "vlan"
range: "101:200,301:400"
net_name: "physnet1"
network_interface: "eno49"
group_binds:
- neutron_openvswitch_agent
.. note::
A *DPDK-accelerated* **bond** interface can be created by specifying a list
of member interfaces using `network_bond_interfaces`. The bond port will
be created automatically and added to the respective bridge in OVS:
A single DPDK interface can be connected to an OVS provider bridge, and
must be done using the ``ovs-vsctl`` command as a post-installation step.
.. code-block:: yaml
- network:
container_bridge: "br-provider"
container_type: "veth"
type: "vlan"
range: "101:200,301:400"
net_name: "physnet1"
network_bond_interfaces:
- "0000:04:00.0"
- "0000:04:00.1"
group_binds:
- neutron_openvswitch_agent
Additional OVS bond parameters can be specified using the following keys:
* bond_mode (Default: active-backup)
* lacp (Default: off)
* bond_downdelay (Default: 100)
* bond_updelay (Default: 100)
.. code-block:: yaml
- network:
container_bridge: "br-provider"
container_type: "veth"
type: "vlan"
range: "101:200,301:400"
net_name: "physnet1"
network_bond_interfaces:
- "0000:04:00.0"
- "0000:04:00.1"
bond_mode: balance-tcp
lacp: active
bond_downdelay: 200
bond_updelay: 200
group_binds:
- neutron_openvswitch_agent
For more information on possible values, visit:
`<https://docs.ansible.com/ansible/latest/collections/openvswitch/openvswitch/openvswitch_bond_module.html>`_
Set the following user variables in your
``/etc/openstack_deploy/user_variables.yml`` to enable the Open vSwitch driver
@ -430,7 +484,9 @@ and DPDK support:
ovs_dpdk_support: True
# Add these overrides or set on per-host basis in openstack_user_config.yml
ovs_dpdk_pci_addresses: "0000:03:00.0"
ovs_dpdk_pci_addresses:
- "0000:04:00.0"
- "0000:04:00.1"
ovs_dpdk_lcore_mask: 1010101
ovs_dpdk_pmd_cpu_mask: 2020202
ovs_dpdk_socket_mem: "1024,1024"
@ -442,20 +498,29 @@ and DPDK support:
Post-installation
~~~~~~~~~~~~~~~~~
Once the playbooks have been run and OVS/DPDK has been configured, it will be
Once the playbooks have been run and OVS/DPDK has been configured, it may be
necessary to add a physical interface to the provider bridge before networking
can be fully established.
can be fully established *if* `network_interface` or `network_bond_interfaces`
have not been defined.
On compute nodes, the following command can be used to attach a NIC port
``0000:03:00.0`` to the provider bridge ``br-provider``:
``0000:04:00.0`` to the provider bridge ``br-provider``:
.. code-block:: console
ovs-vsctl add-port br-provider 0000:03:00.0 -- set Interface 0000:03:00.0 type=dpdk options:dpdk-devargs=0000:03:00.0
ovs-vsctl add-port br-provider 0000:04:00.0 -- set interface 0000:04:00.0 type=dpdk options:dpdk-devargs=0000:04:00.0
The command can be adjusted according to your configuration.
Additionally, it may be necessary to make post-installation adjustments to
interface queues or other parameters to avoid errors within Open vSwitch:
.. code-block:: console
ovs-vsctl set interface 0000:04:00.0 options:n_txq=5
ovs-vsctl set interface 0000:04:00.0 options:n_rxq=5
The command(s) can be adjusted according to your configuration.
.. warning::
Adding multiple ports to the bridge may result in bridging loops unless
bonding is configured. DPDK bonding is outside the scope of this guide.
Adding multiple ports to a bridge may result in bridging loops unless
bonding is configured.

57
library/provider_networks
View File

@ -23,7 +23,7 @@ from ansible.module_utils.basic import AnsibleModule
DOCUMENTATION = """
---
module: provider_networks
version_added: "1.8.6"
version_added: "1.8.7"
short_description:
- Parse a list of networks and return data that Ansible can use
description:
@ -116,6 +116,22 @@ EXAMPLES = """
# group_binds:
# - neutron_openvswitch_agent
# - network:
# container_bridge: "br-provider"
# container_type: "veth"
# container_interface: "eth11"
# network_bond_interfaces:
# - "0000:02:00.0"
# - "0000:02:00.1"
# bond_mode: balance-tcp
# bond_updelay: 100
# bond_downdelay: 100
# lacp: active
# type: "vlan"
# range: "1:1, 101:101"
# net_name: "physnet1"
# group_binds:
# - neutron_openvswitch_agent
# - network:
# container_bridge: "br-storage"
# container_type: "veth"
# container_interface: "eth2"
@ -157,6 +173,14 @@ EXAMPLES = """
# "flat:brx-eth12",
# "vlan:brx-eth11"
# ],
# "network_bond_interfaces_mappings": [{
# 'bridge': 'br-provider',
# 'interfaces': ['0000:02:00.0', '0000:02:00.1'],
# 'bond_mode': 'balance-tcp',
# 'lacp': 'active',
# 'bond_updelay': 100,
# 'bond_downdelay': 100
# }],
# "network_sriov_mappings": "physnet1:p1p1,physnet1:p1p2",
# "network_sriov_mappings_list": [
# "physnet1:p1p1"
@ -202,6 +226,7 @@ class ProviderNetworksParsing(object):
self.network_types = list()
self.network_sriov_mappings = list()
self.network_interface_mappings = list()
self.network_bond_interfaces_mappings = list()
def load_networks(self, provider_networks, is_metal=False,
bind_prefix=None, group_names=None):
@ -299,6 +324,29 @@ class ProviderNetworksParsing(object):
)
)
# Builds a list of provider bridge to physical
# interface (bond member) mappings and is used
# when constructing OVS bonds
if 'network_bond_interfaces' in net['network']:
self.network_bond_interfaces_mappings.append({
'bridge': net['network'][
'container_bridge'
],
'interfaces': net['network'][
'network_bond_interfaces'
],
'bond_mode': net['network'].get(
'bond_mode', 'active-backup'
),
'lacp': net['network'].get('lacp', 'off'),
'bond_updelay': net['network'].get(
'bond_updelay', 100
),
'bond_downdelay': net['network'].get(
'bond_downdelay', 100
)
})
# SR-IOV interface mappings
if 'sriov_host_interfaces' in net['network']:
host_interfaces = \
@ -367,7 +415,12 @@ def main():
'network_interface_mappings': ','.join(
pnp.network_interface_mappings
),
'network_interface_mappings_list': pnp.network_interface_mappings
'network_interface_mappings_list': pnp.network_interface_mappings,
'network_bond_interfaces_mappings': ','.join(
map(str, pnp.network_bond_interfaces_mappings)
),
'network_bond_interfaces_mappings_list':
pnp.network_bond_interfaces_mappings
}
module.exit_json(changed=True, **resp)

10
releasenotes/notes/neutron-dpdk-bonds-4dd98fc0b341ebfb.yaml Normal file
View File

@ -0,0 +1,10 @@
---
features:
- |
The ``provider_networks`` library has been updated to support the
definition of bond member interfaces that can automatically be added as
bond ports to OVS provider bridges setup during a deployment. This
feature is currently limited to DPDK-based deployments. To activate this
feature, add the ``network_bond_interfaces`` key to the respective provider
network definition in ``openstack_user_config.yml``. For more information,
refer to the latest Open vSwitch w/ DPDK deployment guide.

24
tasks/providers/setup_ovs_dpdk.yml
View File

@ -28,6 +28,7 @@
dest: "/etc/dpdk/interfaces"
owner: "root"
group: "root"
mode: "0640"
when:
- neutron_services['neutron-openvswitch-agent']['group'] in group_names
- '"nova_compute" in group_names'
@ -38,6 +39,7 @@
dest: "/etc/dpdk/dpdk.conf"
owner: "root"
group: "root"
mode: "0640"
when:
- neutron_services['neutron-openvswitch-agent']['group'] in group_names
- '"nova_compute" in group_names'
@ -111,7 +113,7 @@
- neutron_plugin_type in ['ml2.ovs', 'ml2.ovs.dvr']
- neutron_provider_networks.network_mappings is defined
# (todo) Loop thru ints or build a bond with ints. TBD.
# Adds a single host interface to an OVS bridge
- name: Add ports to Network Provider Bridges
openvswitch_port:
bridge: "{{ interface_mapping.split(':')[0] }}"
@ -125,3 +127,23 @@
- neutron_services['neutron-openvswitch-agent']['group'] in group_names
- neutron_plugin_type in ['ml2.ovs', 'ml2.ovs.dvr']
- neutron_provider_networks.network_interface_mappings is defined and (neutron_provider_networks.network_interface_mappings|length > 0)
# Adds a DPDK-accelerated bond interface to an OVS bridge
- name: Add Bonds to Network Provider Bridges
openvswitch.openvswitch.openvswitch_bond:
bridge: "{{ bond_interfaces_mapping.bridge }}"
port: "{{ bond_interfaces_mapping.bridge }}-dpdkbond"
interfaces: "{{ bond_interfaces_mapping.interfaces }}"
bond_mode: "{{ bond_interfaces_mapping.bond_mode | default('active-backup') }}"
lacp: "{{ bond_interfaces_mapping.lacp | default('off') }}"
bond_updelay: "{{ bond_interfaces_mapping.bond_updelay | default(100) }}"
bond_downdelay: "{{ bond_interfaces_mapping.bond_downdelay | default(100) }}"
set: "{% for interface in bond_interfaces_mapping.interfaces %}interface {{ interface }} type=dpdk options:dpdk-devargs='{{ interface }}'{% if not loop.last %},{% endif %}{% endfor %}"
state: present
with_items: "{{ neutron_provider_networks.network_bond_interfaces_mappings }}"
loop_control:
loop_var: bond_interfaces_mapping
when:
- neutron_services['neutron-openvswitch-agent']['group'] in group_names
- neutron_plugin_type in ['ml2.ovs', 'ml2.ovs.dvr']
- neutron_provider_networks.network_bond_interfaces_mappings is defined and (neutron_provider_networks.network_bond_interfaces_mappings|length > 0)