Enhance Iperf executor to accept "unmapped" args

Issue: The current iperf executor is setup nicely to handle the
majority, let's say 90%, of the test cases a user might want to run.
However, it still leaves a bit to be desired for the remaining 10%. For
example we use a custom qcow2 image for our tests  and the verison of
iperf in this image (and the latest iperf which shaker doesn't have) you
can specify --sctp as an argument to run this type of traffic. This is
not possible with the current executor, but most users may not want or
care about this ability.

This commit proposes to extend the current executor to accept any
argument as part of an "advanced" mode, rather than mapping every
argument that may or may not used or exists only in certain versions.
This allows different version of iperf (e.g. --sctp capable) of
executing a supported argument without having to directly add support
for this new argument into the executor

I know that any argument could be passed to iperf using the script
executor, but this has the downside of not providing the nice graphs.
I believe this commit is a good compromise between general
useability (pre-mapping arguments) and granular control (using any
agrument that is not mapped). This will also be useful for other
arguments that may not be widely used, or are newly introduced, such
as --zerocopy or --omit along with other sctp related argument like
--xbind or --nstreams.

This would be for advanced users familiar with iperf and would not
impact current tests or general use. The only "safety net" provided is
not allowing mutually exclusive arguments with time to be specified with
the time argument. Since time is currently added by default, even if we
were to create as mapping for say --blockcount(-k), we would have to
handle removing the time argument for a correct run. Arguments that
don't mix will lead to an iperf error, but this error is correctly
logged and handled. Overriding time if a mutually exclusive argument is
used seems like a fair way to handle the situation while allowing other
possible argument combinations.

Change-Id: I072278d0ab3912aac375605dc36987a7f8c20461

doc/source/catalog.rst

@@ -402,6 +402,19 @@ overridden by command-line parameter, e.g. ``--matrix "{host: 172.10.1.2}"``.
 To use this scenario specify parameter ``--scenario spot/udp``.
 Scenario source is available at: https://github.com/openstack/shaker/blob/master/shaker/scenarios/spot/udp.yaml
 
+.. _scenario_sample_tcp_test_with_advanced_iperf_arguments:
+
+Sample TCP Test with Advanced Iperf Arguments
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+This test definition demonstrates the use of advanced arguments with iperf. In
+this scenario Shaker launches pairs of instances in the same tenant network.
+Every instance is hosted on a separate compute node, 1 compute node is
+utilized. The traffic goes within the tenant network (L2 domain) and uses
+arguments not directly mapped by the iperf executor.
+
+To use this scenario specify parameter ``--scenario test/sample_with_advanced_iperf``.
+Scenario source is available at: https://github.com/openstack/shaker/blob/master/shaker/scenarios/test/sample_with_advanced_iperf.yaml
+
 .. _scenario_sample_tcp_test_with_environment_file:
 
 Sample TCP Test with Environment File

shaker/engine/executors/iperf.py

@@ -17,9 +17,12 @@ import csv
 import json
 import yaml
 
+from oslo_log import log as logging
 from shaker.engine.executors import base
 from shaker.engine import utils
 
+LOG = logging.getLogger(__name__)
+
 
 def add_common_iperf_params(cmd, executor):
     cmd.add('--client', executor.test_definition.get('host') or
@@ -35,7 +38,23 @@ def add_common_iperf_params(cmd, executor):
             cmd.add('--len', executor.test_definition.get('datagram_size'))
     if executor.test_definition.get('bandwidth') is not None:
         cmd.add('--bandwidth', executor.test_definition.get('bandwidth'))
-    cmd.add('--time', executor.get_expected_duration())
+
+    if executor.test_definition.get('args'):
+        args = executor.test_definition.get('args')
+        for arg in args.split(' '):
+            cmd.add(arg)
+
+    # if any of these arguments are present user is using advanced mode so
+    # these should overwrite the default addition of the time argument
+    mutually_exclusive_with_time = {"-k", "--blockcount", "-n", "--num", "-l",
+                                    "--length"}
+    if mutually_exclusive_with_time.isdisjoint(cmd.tokens):
+        cmd.add('--time', executor.get_expected_duration())
+    else:
+        LOG.info('Time argument was not added since it is '
+                 'mutually exclusive with %s',
+                 ",".join(mutually_exclusive_with_time))
+
     cmd.add('--parallel', executor.test_definition.get('threads') or 1)
     if executor.test_definition.get('interval'):
         cmd.add('--interval', executor.test_definition.get('interval'))

shaker/scenarios/test/sample_with_advanced_iperf.yaml

@@ -0,0 +1,25 @@
+title: Sample TCP Test with Advanced Iperf Arguments
+
+description:
+  This test definition demonstrates the use of advanced arguments with iperf.
+
+  In this scenario Shaker launches pairs of instances in the same tenant
+  network. Every instance is hosted on a separate compute node, 1 compute node
+  is utilized. The traffic goes within the tenant network (L2 domain) and uses
+  arguments not directly mapped by the iperf executor.
+
+deployment:
+  template: l2_with_env.hot
+  env_file: env/sample.env
+
+  accommodation: [pair, compute_nodes: 1]
+
+execution:
+  tests:
+  -
+    title: tcp
+    class: iperf3
+    buffer_size: 9000
+    args: "--blockcount 1M --zerocopy --omit 1"
+    sla:
+    - "[type == 'agent'] >> (stats.bandwidth.avg > 5000)"

shaker/tests/test_iperf_executor.py

@@ -121,6 +121,56 @@ class TestIperfGraphExecutor(testtools.TestCase):
             base.ExecutorException, executor.process_reply, message)
 
 
+class IperfExecutor(testtools.TestCase):
+
+    def test_add_common_iperf_params(self):
+        executor = iperf.IperfExecutor(
+            {'bandwidth': '100M', 'time': 30}, AGENT)
+
+        cmd = base.CommandLine('iperf')
+        iperf.add_common_iperf_params(cmd, executor)
+
+        expected = {'data': ('iperf --client %s --format m --bandwidth 100M'
+                             ' --time 30 --parallel 1 --nodelay') % IP,
+                    'type': 'program'}
+
+        self.assertEqual(expected, executor.get_command())
+
+    def test_add_common_iperf_params_with_args(self):
+        executor = iperf.IperfExecutor(
+            {'bandwidth': '100M', 'time': 30, 'args': '--zerocopy --omit 1', },
+            AGENT)
+
+        cmd = base.CommandLine('iperf')
+        iperf.add_common_iperf_params(cmd, executor)
+
+        expected = {'data': ('iperf --client %s --format m --bandwidth 100M'
+                             ' --zerocopy --omit 1 --time 30 --parallel 1 '
+                             '--nodelay') % IP,
+                    'type': 'program'}
+
+        self.assertEqual(expected, executor.get_command())
+
+    def test_add_common_iperf_params_args_with_exclusion(self):
+        executor = iperf.IperfExecutor(
+            # add time on purpose to make sure it is excluded
+            {'bandwidth': '100M', 'time': 30,
+             'args': '--blockcount 1M --zerocopy --omit 1', },
+            AGENT)
+
+        cmd = base.CommandLine('iperf')
+        iperf.add_common_iperf_params(cmd, executor)
+
+        # time should not be present since it is mutually exclusive with
+        # -k/blockcount
+        expected = {'data': ('iperf --client %s --format m --bandwidth 100M'
+                             ' --blockcount 1M --zerocopy --omit 1 '
+                             '--parallel 1 --nodelay') % IP,
+                    'type': 'program'}
+
+        self.assertEqual(expected, executor.get_command())
+
+
 class TestIperf3Executor(testtools.TestCase):
 
     def test_get_command(self):