tripleo_ovs_dpdk: add support for pmd load based sleeping

PMD threads constantly poll Rx queues which are assigned to them. In
order to reduce the CPU cycles they use, they can sleep for small
periods of time when there is no load or very-low load on all the Rx
queues they poll.

This can be enabled by setting the max requested sleep time (in
microseconds) for any PMD thread:

  $ ovs-vsctl set open_vswitch . other_config:pmd-maxsleep=50

With a non-zero max value a PMD may request to sleep by an incrementing
amount of time up to the maximum time. If at any point the threshold of
at least half a batch of packets (i.e. 16) is received from an Rx queue
that the PMD is polling is met, the requested sleep time will be reset
to 0. At that point no sleeps will occur until the no/low load
conditions return.

Sleeping in a PMD thread will mean there is a period of time when the
PMD thread will not process packets. Sleep times requested are not
guaranteed and can differ significantly depending on system
configuration. The actual time not processing packets will be determined
by the sleep and processor wake-up times and should be tested with each
system configuration.

Link: https://github.com/openvswitch/ovs/blob/v3.1.2/Documentation/topics/dpdk/pmd.rst#pmd-load-based-sleeping-experimental
Cc: Christophe Fontaine <cfontain@redhat.com>
Change-Id: Ifa0def0eb6fb21a7bb9313a88ff5f4de2e811c27
Signed-off-by: Robin Jarry <rjarry@redhat.com>

doc/source/roles/role-tripleo-ovs-dpdk.rst

@@ -123,6 +123,17 @@ Role Defaults
   - **Default**: ``''``
   - **Examples**: ``'5'``
 
+- ``tripleo_ovs_dpdk_pmd_sleep_max``
+
+  - **Description**: (*Optional*) PMD maximum sleep time, in range 0 to 10,000.
+    Its a string with a number, specifies the maximum sleep time that will be
+    requested in microseconds per iteration for a PMD thread which has received
+    zero or a small amount of packets from the Rx queues it is polling. The
+    actual sleep time requested is based on the load of the Rx queues that the
+    PMD polls and may be less than the maximum value.
+  - **Default**: ``''``
+  - **Examples**: ``'50'``
+
 Modules
 -------
 

tripleo_ansible/playbooks/cli-overcloud-openvswitch-dpdk.yaml

@@ -33,6 +33,7 @@
     pmd_load_threshold: ''
     pmd_improvement_threshold: ''
     pmd_rebal_interval: ''
+    pmd_sleep_max: ''
     nova_postcopy: true
     tuned_profile: 'cpu-partitioning'
 
@@ -76,3 +77,4 @@
             tripleo_ovs_dpdk_pmd_load_threshold: "{{ pmd_load_threshold }}"
             tripleo_ovs_dpdk_pmd_improvement_threshold: "{{ pmd_improvement_threshold }}"
             tripleo_ovs_dpdk_pmd_rebal_interval: "{{ pmd_rebal_interval }}"
+            tripleo_ovs_dpdk_pmd_sleep_max: "{{ pmd_sleep_max }}"

tripleo_ansible/roles/tripleo_ovs_dpdk/defaults/main.yml

@@ -32,5 +32,6 @@ tripleo_ovs_dpdk_pmd_auto_lb: false
 tripleo_ovs_dpdk_pmd_load_threshold: ""
 tripleo_ovs_dpdk_pmd_improvement_threshold: ""
 tripleo_ovs_dpdk_pmd_rebal_interval: ""
+tripleo_ovs_dpdk_pmd_sleep_max: ""
 tripleo_ovs_dpdk_vhost_postcopy_support: false
 tripleo_ovs_dpdk_vhost_postcopy_ovs_options: "--mlockall=no"

tripleo_ansible/roles/tripleo_ovs_dpdk/molecule/default/tests/test_default.py

@@ -90,3 +90,8 @@ def test_positive_pmd_improvement_threshold(host):
 def test_positive_pmd_rebal_interval(host):
     other_config = get_config(host)
     assert 'pmd-auto-lb-rebal-interval' not in other_config
+
+
+def test_positive_pmd_sleep_max(host):
+    other_config = get_config(host)
+    assert 'pmd-maxsleep' not in other_config

tripleo_ansible/roles/tripleo_ovs_dpdk/molecule/positive/converge.yml

@@ -34,3 +34,4 @@
         tripleo_ovs_dpdk_pmd_load_threshold: 50
         tripleo_ovs_dpdk_pmd_improvement_threshold: 10
         tripleo_ovs_dpdk_pmd_rebal_interval: 5
+        tripleo_ovs_dpdk_pmd_sleep_max: 50

tripleo_ansible/roles/tripleo_ovs_dpdk/molecule/positive/tests/test_positive.py

@@ -100,3 +100,9 @@ def test_positive_pmd_rebal_interval(host):
     other_config = get_config(host)
     pmd_rebal_interval = other_config['pmd-auto-lb-rebal-interval'].replace('"', '')
     assert pmd_rebal_interval == "5"
+
+
+def test_positive_pmd_sleep_max(host):
+    other_config = get_config(host)
+    pmd_sleep_max = other_config['pmd-maxsleep'].replace('"', '')
+    assert pmd_sleep_max == "50"

tripleo_ansible/roles/tripleo_ovs_dpdk/tasks/config.yml

@@ -267,3 +267,21 @@
     col: other_config
     key: pmd-auto-lb-rebal-interval
   when: not tripleo_ovs_dpdk_pmd_rebal_interval|string
+
+- name: Set PMD maximum sleep time
+  openvswitch_db:
+    table: open_vswitch
+    record: .
+    col: other_config
+    key: pmd-maxsleep
+    value: "{{ tripleo_ovs_dpdk_pmd_sleep_max }}"
+  when: tripleo_ovs_dpdk_pmd_sleep_max|string
+
+- name: Remove PMD maximum sleep time
+  openvswitch_db:
+    state: absent
+    table: open_vswitch
+    record: .
+    col: other_config
+    key: pmd-maxsleep
+  when: not tripleo_ovs_dpdk_pmd_sleep_max|string