Merge "Adding deployment guide for Openstack Helm with OVS-DPDK"
This commit is contained in:
Zuul
Gerrit Code Review
commit
b226026504
|
|
@ -0,0 +1,189 @@
|
|||
===============
|
||||
Deploy OVS-DPDK
|
||||
===============
|
||||
|
||||
Requirements
|
||||
============
|
||||
|
||||
A correct DPDK configuration depends heavily on the specific hardware resources
|
||||
and its configuration. Before deploying Openvswitch with DPDK, check the amount
|
||||
and type of available hugepages on the host OS.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cat /proc/meminfo | grep Huge
|
||||
AnonHugePages: 0 kB
|
||||
ShmemHugePages: 0 kB
|
||||
HugePages_Total: 8
|
||||
HugePages_Free: 6
|
||||
HugePages_Rsvd: 0
|
||||
HugePages_Surp: 0
|
||||
Hugepagesize: 1048576 kB
|
||||
|
||||
In this example, 8 hugepages of 1G size have been allocated. 2 of those are
|
||||
being used and 6 are still available.
|
||||
|
||||
More information on how to allocate and configure hugepages on the host OS can
|
||||
be found in the `Openvswitch documentation
|
||||
<http://docs.openvswitch.org/en/latest/intro/install/dpdk/>`_.
|
||||
|
||||
In order to allow OVS inside a pod to make use of hugepages, the corresponding
|
||||
type and amount of hugepages must be specified in the resource section of the
|
||||
OVS chart's values.yaml:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
resources:
|
||||
enabled: true
|
||||
ovs:
|
||||
db:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "1024Mi"
|
||||
cpu: "2000m"
|
||||
vswitchd:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "1024Mi"
|
||||
cpu: "2000m"
|
||||
# set resources to enabled and specify one of the following when using dpdk
|
||||
hugepages-1Gi: "1Gi"
|
||||
# hugepages-2Mi: "512Mi"
|
||||
|
||||
Additionally, the default configuration of the neutron chart must be adapted according
|
||||
to the underlying hardware. The corresponding configuration parameter is labeled with
|
||||
"CHANGE-ME" in the script "values_overrides/dpdk.yaml". Specifically, the "ovs_dpdk"
|
||||
configuration section should list all NICs which should be bound to DPDK with
|
||||
their corresponding PCI-IDs. Moreover, the name of each NIC needs to be unique,
|
||||
e.g., dpdk0, dpdk1, etc.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
network:
|
||||
interface:
|
||||
tunnel: br-phy
|
||||
conf:
|
||||
ovs_dpdk:
|
||||
enabled: true
|
||||
driver: uio_pci_generic
|
||||
nics:
|
||||
- name: dpdk0
|
||||
# CHANGE-ME: modify pci_id according to hardware
|
||||
pci_id: '0000:05:00.0'
|
||||
bridge: br-phy
|
||||
migrate_ip: true
|
||||
bridges:
|
||||
- name: br-phy
|
||||
bonds: []
|
||||
|
||||
In the example above, bonding isn't used and hence an empty list is passed in the "bonds"
|
||||
section.
|
||||
|
||||
Deployment
|
||||
==========
|
||||
|
||||
Once the above requirements are met, start deploying Openstack Helm using the deployment
|
||||
scripts under the dpdk directory in an increasing order
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./tools/deployment/developer/dpdk/<script-name>
|
||||
|
||||
One can also specify the name of Openstack release and container OS distribution as
|
||||
overrides before running the deployment scripts, for instance,
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export OPENSTACK_RELEASE=rocky
|
||||
export CONTAINER_DISTRO_NAME=ubuntu
|
||||
export CONTAINER_DISTRO_VERSION=bionic
|
||||
|
||||
Note that OVS-DPDK deployment has been tested with Openstack Rocky release and Ubuntu
|
||||
Bionic container distributions. If the above variables aren't set, the defaults (currently
|
||||
Openstack Ocata and Ubuntu Xenial) will be used.
|
||||
|
||||
Troubleshooting
|
||||
===============
|
||||
|
||||
OVS startup failure
|
||||
-------------------
|
||||
|
||||
If OVS fails to start up because of no hugepages are available, check the
|
||||
configuration of the OVS daemonset. Older versions of helm-toolkit were not
|
||||
able to render hugepage configuration into the Kubernetes manifest and just
|
||||
removed the hugepage attributes. If no hugepage configuration is defined for
|
||||
the OVS daemonset, consider using a newer version of helm-toolkit.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
kubectl get daemonset openvswitch-vswitchd -n openstack -o yaml
|
||||
[...]
|
||||
resources:
|
||||
limits:
|
||||
cpu: "2"
|
||||
hugepages-1Gi: 1Gi
|
||||
memory: 1Gi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
[...]
|
||||
|
||||
Adding a DPDK port to Openvswitch fails
|
||||
---------------------------------------
|
||||
|
||||
When adding a DPDK port (a NIC bound to DPDK) to OVS fails, one source of error
|
||||
is related to an incorrect configuration with regards to the NUMA topology of
|
||||
the underlying hardware. Every NIC is connected to one specific NUMA socket. In
|
||||
order to use a NIC as DPDK port in OVS, the OVS configuration regarding
|
||||
hugepages and PMD threas needs to match the NUMA topology.
|
||||
|
||||
The NUMA socket a given NIC is connected to can be found in the ovs-vswitchd log:
|
||||
|
||||
.. code-block::
|
||||
|
||||
kubectl logs -n openstack openvswitch-vswitchd-6h928
|
||||
[...]
|
||||
2019-07-02T13:42:06Z|00016|dpdk|INFO|EAL: PCI device 0000:00:04.0 on NUMA socket 1
|
||||
2019-07-02T13:42:06Z|00018|dpdk|INFO|EAL: probe driver: 1af4:1000 net_virtio
|
||||
[...]
|
||||
|
||||
In this example, the NIC with PCI-ID 0000:00:04.0 is connected to NUMA socket
|
||||
1. As a result, this NIC can only be used by OVS if
|
||||
|
||||
1. hugepages have been allocated on NUMA socket 1 by OVS, and
|
||||
2. PMD threads have been assigned to NUMA socket 1.
|
||||
|
||||
To allocate hugepages to NUMA sockets in OVS, ensure that the
|
||||
``socket_memory`` attribute in values.yaml specifies a value for the
|
||||
corresponding NUMA socket. In the following example, OVS will use one 1G
|
||||
hugepage for NUMA socket 0 and socket 1.
|
||||
|
||||
.. code-block::
|
||||
|
||||
socket_memory: 1024,1024
|
||||
|
||||
|
||||
To allocate PMD threads to NUMA sockets in OVS, ensure that the ``pmd_cpu_mask``
|
||||
attribute in values.yaml includes CPU sockets on the corresponding NUMA socket.
|
||||
In the example below, the mask of 0xf covers the first 4 CPU cores which are
|
||||
dsitributed across NUMA sockets 0 and 1.
|
||||
|
||||
.. code-block::
|
||||
|
||||
pmd_cpu_mask: 0xf
|
||||
|
||||
The mapping of CPU cores to NUMA sockets can be determined by means of ``lspci``, for instance:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
lspci | grep NUMA
|
||||
NUMA node(s): 2
|
||||
NUMA node0 CPU(s): 0,2,4,6,8,10,12,14
|
||||
NUMA node1 CPU(s): 1,3,5,7,9,11,13,15
|
||||
|
||||
More information can be found in the `Openvswitch documentation
|
||||
<http://docs.openvswitch.org/en/latest/intro/install/dpdk/>`_.
|
||||
|
|
@ -10,5 +10,6 @@ Contents:
|
|||
kubernetes-and-common-setup
|
||||
deploy-with-nfs
|
||||
deploy-with-ceph
|
||||
deploy-ovs-dpdk.rst
|
||||
exercise-the-cloud
|
||||
cleaning-deployment
|
||||
|
|
|
|||
Loading…
Reference in New Issue