openstack/rally
Code Issues Proposed changes

Port all samples to task v2 format

Browse Source 
Change-Id: I893f2655311e2312341cc93637046bb08f8fb7ef
Signed-off-by: Andriy Kurilin <andr.kurilin@gmail.com>
This commit is contained in:
Andriy Kurilin committed 2025-08-10 12:28:11 +02:00
1 parent cd921eafc4
commit 6565717213
36 files changed
+639 -261
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CHANGELOG.rst
+1
View File
@@ -33,6 +33,7 @@ Changed
for plugins discovery and loading.
* Implements pep-517 (pyproject.toml) and replaces pbr dependency
with setuptools-scm
* Ported all samples to Task format v2
Removed
~~~~~~~
doc/source/quick_start/tutorial/step_2_input_task_format.rst
+167 -13
View File
@@ -28,7 +28,35 @@ Rally comes with a really great collection of
:ref:`plugins <tutorial_step_8_discovering_more_plugins>` and in most
real-world cases you will use multiple plugins to test your OpenStack cloud.
Rally makes it very easy to run **different test cases defined in a single
task**. To do so, use the following syntax:
task**.
**Current format (v2)**: Rally currently uses task format version 2, which
provides better organization, metadata support, and flexibility:
.. code-block:: json
{
"version": 2,
"title": "Task title",
"description": "Task description",
"tags": ["tag1", "tag2"],
"subtasks": [
{
"title": "Subtask title",
"scenario": {
"ScenarioName": { <scenario-specific arguments> }
},
"runner": {
"runner_type": { <runner parameters> }
},
"contexts": { <contexts needed for this scenario> },
"sla": { <different SLA configs> }
}
]
}
**Legacy format (v1) - DEPRECATED**: The old format is still supported but
deprecated:
.. code-block:: json
@@ -37,7 +65,7 @@ task**. To do so, use the following syntax:
"<ScenarioName2>": [<config>, ...]
}
where *<config>*, as before, is a dictionary:
where *<config>*, in the legacy format, is a dictionary:
.. code-block:: json
@@ -48,6 +76,11 @@ where *<config>*, as before, is a dictionary:
"sla": { <different SLA configs> }
}
.. note::
The legacy v1 format is deprecated. New tasks should use v2 format for better
organization and metadata support.
Rally automatically converts v1 to v2 internally for compatibility.
Multiple subtasks in a single task
----------------------------------
@@ -55,11 +88,64 @@ As an example, let's edit our configuration file from
:ref:`step 1 <tutorial_step_1_setting_up_env_and_running_benchmark_from_samples>`
so that it prescribes Rally to launch not only the
**NovaServers.boot_and_delete_server** scenario, but also the
**KeystoneBasic.create_delete_user** scenario. All we have to do is to append
the configuration of the second scenario as yet another top-level key of our
JSON file:
**KeystoneBasic.create_delete_user** scenario.
*multiple-scenarios.json*
**Using v2 format (recommended):**
*multiple-scenarios-v2.json*
.. code-block:: json
{
"version": 2,
"title": "Multiple scenarios example",
"description": "Example demonstrating multiple scenarios in a single task",
"tags": ["nova", "keystone", "example"],
"subtasks": [
{
"title": "Nova server lifecycle test",
"scenario": {
"NovaServers.boot_and_delete_server": {
"flavor": {
"name": "m1.tiny"
},
"image": {
"name": "^cirros.*-disk$"
},
"force_delete": false
}
},
"runner": {
"constant": {
"times": 10,
"concurrency": 2
}
},
"contexts": {
"users": {
"tenants": 3,
"users_per_tenant": 2
}
}
},
{
"title": "Keystone user management test",
"scenario": {
"KeystoneBasic.create_delete_user": {}
},
"runner": {
"constant": {
"times": 10,
"concurrency": 3
}
}
}
]
}
**Legacy v1 format (deprecated but still supported):**
*multiple-scenarios-v1.json*
.. code-block:: json
@@ -100,11 +186,11 @@ JSON file:
]
}
Now you can start this task as usually:
Now you can start this task as usually (using v2 format):
.. code-block:: console
$ rally task start multiple-scenarios.json
$ rally task start multiple-scenarios-v2.json
...
+--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
| action | min (sec) | avg (sec) | max (sec) | 90 percentile | 95 percentile | success | count |
@@ -148,13 +234,81 @@ Multiple configurations of the same scenario
--------------------------------------------
Yet another thing you can do in Rally is to launch **the same scenario multiple
times with different configurations**. That's why our configuration file stores
a list for the key *"NovaServers.boot_and_delete_server"*: you can just append
a different configuration of this scenario to this list to get it. Let's say,
times with different configurations**. In v2 format, this is done by creating
multiple subtasks with the same scenario but different parameters. Let's say,
you want to run the **boot_and_delete_server** scenario twice: first using the
*"m1.tiny"* flavor and then using the *"m1.small"* flavor:
*multiple-configurations.json*
**Using v2 format (recommended):**
*multiple-configurations-v2.json*
.. code-block:: json
{
"version": 2,
"title": "Multiple configurations example",
"description": "Running the same scenario with different configurations",
"tags": ["nova", "flavors", "configurations"],
"subtasks": [
{
"title": "Boot server with m1.tiny flavor",
"scenario": {
"NovaServers.boot_and_delete_server": {
"flavor": {
"name": "m1.tiny"
},
"image": {
"name": "^cirros.*-disk$"
},
"force_delete": false
}
},
"runner": {
"constant": {
"times": 10,
"concurrency": 2
}
},
"contexts": {
"users": {
"tenants": 3,
"users_per_tenant": 2
}
}
},
{
"title": "Boot server with m1.small flavor",
"scenario": {
"NovaServers.boot_and_delete_server": {
"flavor": {
"name": "m1.small"
},
"image": {
"name": "^cirros.*-disk$"
},
"force_delete": false
}
},
"runner": {
"constant": {
"times": 10,
"concurrency": 2
}
},
"contexts": {
"users": {
"tenants": 3,
"users_per_tenant": 2
}
}
}
]
}
**Legacy v1 format (deprecated):**
*multiple-configurations-v1.json*
.. code-block:: json
@@ -193,7 +347,7 @@ That's it! You will get again the results for each configuration separately:
.. code-block:: console
$ rally task start --task=multiple-configurations.json
$ rally task start --task=multiple-configurations-v2.json
...
+--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
| action | min (sec) | avg (sec) | max (sec) | 90 percentile | 95 percentile | success | count |
samples/tasks/contexts/dummy-context.json
+15 -7
View File
@@ -1,15 +1,23 @@
{
"Dummy.dummy": [
"version": 2,
"title": "Context usage sample",
"description": "Sample task demonstrating context usage with dummy_context",
"tags": ["context", "sample", "dummy"],
"subtasks": [
{
"args": {
"sleep": 0.1
"title": "Dummy scenario with context",
"scenario": {
"Dummy.dummy": {
"sleep": 0.1
}
},
"runner": {
"type": "constant",
"times": 4,
"concurrency": 2
"constant": {
"times": 4,
"concurrency": 2
}
},
"context": {
"contexts": {
"dummy_context": {
"fail_setup": false,
"fail_cleanup": false
samples/tasks/contexts/dummy-context.yaml
+14 -9
View File
@@ -1,13 +1,18 @@
---
Dummy.dummy:
-
args:
version: 2
title: "Context usage sample"
description: "Sample task demonstrating context usage with dummy_context"
tags: ["context", "sample", "dummy"]
subtasks:
- title: "Dummy scenario with context"
scenario:
Dummy.dummy:
sleep: 0.1
runner:
type: "constant"
runner:
constant:
times: 4
concurrency: 2
context:
dummy_context:
fail_setup: false
fail_cleanup: false
contexts:
dummy_context:
fail_setup: false
fail_cleanup: false
samples/tasks/runners/constant/constant-for-duration.json
+14 -6
View File
@@ -1,13 +1,21 @@
{
"Dummy.dummy": [
"version": 2,
"title": "Constant for duration runner sample",
"description": "Sample task demonstrating constant_for_duration runner usage",
"tags": ["runner", "constant_for_duration", "sample"],
"subtasks": [
{
"args": {
"sleep": 5
"title": "Dummy scenario with constant_for_duration runner",
"scenario": {
"Dummy.dummy": {
"sleep": 5
}
},
"runner": {
"type": "constant_for_duration",
"concurrency": 5,
"duration": 30
"constant_for_duration": {
"concurrency": 5,
"duration": 30
}
}
}
]
samples/tasks/runners/constant/constant-for-duration.yaml
+10 -5
View File
@@ -1,9 +1,14 @@
---
Dummy.dummy:
-
args:
version: 2
title: "Constant for duration runner sample"
description: "Sample task demonstrating constant_for_duration runner usage"
tags: ["runner", "constant_for_duration", "sample"]
subtasks:
- title: "Dummy scenario with constant_for_duration runner"
scenario:
Dummy.dummy:
sleep: 5
runner:
type: "constant_for_duration"
runner:
constant_for_duration:
concurrency: 5
duration: 30
samples/tasks/runners/constant/constant-timeout.json
+15 -7
View File
@@ -1,14 +1,22 @@
{
"Dummy.dummy": [
"version": 2,
"title": "Constant runner with timeout sample",
"description": "Sample task demonstrating constant runner with timeout configuration",
"tags": ["runner", "constant", "timeout", "sample"],
"subtasks": [
{
"args": {
"sleep": 10
"title": "Dummy scenario with constant runner and timeout",
"scenario": {
"Dummy.dummy": {
"sleep": 10
}
},
"runner": {
"type": "constant",
"times": 20,
"concurrency": 5,
"timeout": 5
"constant": {
"times": 20,
"concurrency": 5,
"timeout": 5
}
}
}
]
samples/tasks/runners/constant/constant-timeout.yaml
+10 -5
View File
@@ -1,10 +1,15 @@
---
Dummy.dummy:
-
args:
version: 2
title: "Constant runner with timeout sample"
description: "Sample task demonstrating constant runner with timeout configuration"
tags: ["runner", "constant", "timeout", "sample"]
subtasks:
- title: "Dummy scenario with constant runner and timeout"
scenario:
Dummy.dummy:
sleep: 10
runner:
type: "constant"
runner:
constant:
times: 20
concurrency: 5
timeout: 5
samples/tasks/runners/rps/rps.json
+15 -7
View File
@@ -1,14 +1,22 @@
{
"Dummy.dummy": [
"version": 2,
"title": "RPS runner sample",
"description": "Sample task demonstrating RPS (Requests Per Second) runner usage",
"tags": ["runner", "rps", "sample"],
"subtasks": [
{
"args": {
"sleep": 5
"title": "Dummy scenario with RPS runner",
"scenario": {
"Dummy.dummy": {
"sleep": 5
}
},
"runner": {
"type": "rps",
"times": 20,
"rps": 3,
"timeout": 6
"rps": {
"times": 20,
"rps": 3,
"timeout": 6
}
}
}
]
samples/tasks/runners/rps/rps.yaml
+10 -5
View File
@@ -1,10 +1,15 @@
---
Dummy.dummy:
-
args:
version: 2
title: "RPS runner sample"
description: "Sample task demonstrating RPS (Requests Per Second) runner usage"
tags: ["runner", "rps", "sample"]
subtasks:
- title: "Dummy scenario with RPS runner"
scenario:
Dummy.dummy:
sleep: 5
runner:
type: "rps"
runner:
rps:
times: 20
rps: 3
timeout: 6
samples/tasks/runners/serial/serial.json
+13 -5
View File
@@ -1,12 +1,20 @@
{
"Dummy.dummy": [
"version": 2,
"title": "Serial runner sample",
"description": "Sample task demonstrating serial runner usage for sequential execution",
"tags": ["runner", "serial", "sample"],
"subtasks": [
{
"args": {
"sleep": 5
"title": "Dummy scenario with serial runner",
"scenario": {
"Dummy.dummy": {
"sleep": 5
}
},
"runner": {
"type": "serial",
"times": 20
"serial": {
"times": 20
}
}
}
]
samples/tasks/runners/serial/serial.yaml
+10 -5
View File
@@ -1,8 +1,13 @@
---
Dummy.dummy:
-
args:
version: 2
title: "Serial runner sample"
description: "Sample task demonstrating serial runner usage for sequential execution"
tags: ["runner", "serial", "sample"]
subtasks:
- title: "Dummy scenario with serial runner"
scenario:
Dummy.dummy:
sleep: 5
runner:
type: "serial"
runner:
serial:
times: 20
samples/tasks/scenarios/dummy/dummy-exception-probability.json
+14 -6
View File
@@ -1,13 +1,21 @@
{
"Dummy.dummy_exception_probability": [
"version": 2,
"title": "Dummy scenario with probabilistic exceptions",
"description": "Sample task demonstrating dummy scenario with exception probability functionality",
"tags": ["dummy", "exception", "probability", "sample"],
"subtasks": [
{
"args": {
"exception_probability": 0.5
"title": "Dummy scenario with 50% exception probability",
"scenario": {
"Dummy.dummy_exception_probability": {
"exception_probability": 0.5
}
},
"runner": {
"type": "constant",
"times": 1000,
"concurrency": 1
"constant": {
"times": 1000,
"concurrency": 1
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/dummy/dummy-exception-probability.yaml
+13 -8
View File
@@ -1,12 +1,17 @@
---
Dummy.dummy_exception_probability:
-
args:
version: 2
title: "Dummy scenario with probabilistic exceptions"
description: "Sample task demonstrating dummy scenario with exception probability functionality"
tags: ["dummy", "exception", "probability", "sample"]
subtasks:
- title: "Dummy scenario with 50% exception probability"
scenario:
Dummy.dummy_exception_probability:
exception_probability: 0.5
runner:
type: "constant"
runner:
constant:
times: 1000
concurrency: 1
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/dummy/dummy-exception.json
+14 -6
View File
@@ -1,13 +1,21 @@
{
"Dummy.dummy_exception": [
"version": 2,
"title": "Dummy exception scenario sample",
"description": "Sample task demonstrating Dummy.dummy_exception scenario usage",
"tags": ["dummy", "exception", "sample"],
"subtasks": [
{
"args": {
"size_of_message": 5
"title": "Dummy scenario that raises exception",
"scenario": {
"Dummy.dummy_exception": {
"size_of_message": 5
}
},
"runner": {
"type": "constant",
"times": 20,
"concurrency": 5
"constant": {
"times": 20,
"concurrency": 5
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/dummy/dummy-exception.yaml
+13 -8
View File
@@ -1,12 +1,17 @@
---
Dummy.dummy_exception:
-
args:
version: 2
title: "Dummy exception scenario sample"
description: "Sample task demonstrating Dummy.dummy_exception scenario usage"
tags: ["dummy", "exception", "sample"]
subtasks:
- title: "Dummy scenario that raises exception"
scenario:
Dummy.dummy_exception:
size_of_message: 5
runner:
type: "constant"
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/dummy/dummy-failure.json
+17 -9
View File
@@ -1,16 +1,24 @@
{
"Dummy.failure": [
"version": 2,
"title": "Dummy failure scenario sample",
"description": "Sample task demonstrating Dummy.failure scenario with iteration-based failures",
"tags": ["dummy", "failure", "sample"],
"subtasks": [
{
"args": {
"sleep": 0.2,
"from_iteration": 5,
"to_iteration": 10,
"each": 2
"title": "Dummy scenario with controlled failures",
"scenario": {
"Dummy.failure": {
"sleep": 0.2,
"from_iteration": 5,
"to_iteration": 10,
"each": 2
}
},
"runner": {
"type": "constant",
"times": 20,
"concurrency": 5
"constant": {
"times": 20,
"concurrency": 5
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/dummy/dummy-failure.yaml
+13 -8
View File
@@ -1,15 +1,20 @@
---
Dummy.failure:
-
args:
version: 2
title: "Dummy failure scenario sample"
description: "Sample task demonstrating Dummy.failure scenario with iteration-based failures"
tags: ["dummy", "failure", "sample"]
subtasks:
- title: "Dummy scenario with controlled failures"
scenario:
Dummy.failure:
sleep: 0.2
from_iteration: 5
to_iteration: 10
each: 2
runner:
type: "constant"
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/dummy/dummy-output.json
+14 -6
View File
@@ -1,13 +1,21 @@
{
"Dummy.dummy_output": [
"version": 2,
"title": "Dummy scenario demonstrating output collection",
"description": "Sample task demonstrating dummy scenario output collection functionality",
"tags": ["dummy", "output", "sample"],
"subtasks": [
{
"args": {
"random_range": 25
"title": "Dummy scenario with random output collection",
"scenario": {
"Dummy.dummy_output": {
"random_range": 25
}
},
"runner": {
"type": "constant",
"times": 50,
"concurrency": 5
"constant": {
"times": 50,
"concurrency": 5
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/dummy/dummy-output.yaml
+13 -8
View File
@@ -1,12 +1,17 @@
---
Dummy.dummy_output:
-
args:
version: 2
title: "Dummy scenario demonstrating output collection"
description: "Sample task demonstrating dummy scenario output collection functionality"
tags: ["dummy", "output", "sample"]
subtasks:
- title: "Dummy scenario with random output collection"
scenario:
Dummy.dummy_output:
random_range: 25
runner:
type: "constant"
runner:
constant:
times: 50
concurrency: 5
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/dummy/dummy-random-action.json
+16 -8
View File
@@ -1,15 +1,23 @@
{
"Dummy.dummy_random_action": [
"version": 2,
"title": "Dummy scenario with random action selection",
"description": "Sample task demonstrating dummy scenario with random action selection functionality",
"tags": ["dummy", "random", "action", "sample"],
"subtasks": [
{
"args": {
"actions_num": 5,
"sleep_min": 0,
"sleep_max": 2
"title": "Dummy scenario with random action selection and timing",
"scenario": {
"Dummy.dummy_random_action": {
"actions_num": 5,
"sleep_min": 0,
"sleep_max": 2
}
},
"runner": {
"type": "constant",
"times": 5,
"concurrency": 5
"constant": {
"times": 5,
"concurrency": 5
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/dummy/dummy-random-action.yaml
+13 -8
View File
@@ -1,14 +1,19 @@
---
Dummy.dummy_random_action:
-
args:
version: 2
title: "Dummy scenario with random action selection"
description: "Sample task demonstrating dummy scenario with random action selection functionality"
tags: ["dummy", "random", "action", "sample"]
subtasks:
- title: "Dummy scenario with random action selection and timing"
scenario:
Dummy.dummy_random_action:
actions_num: 5
sleep_min: 0
sleep_max: 2
runner:
type: "constant"
runner:
constant:
times: 5
concurrency: 5
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/dummy/dummy-random-fail-in-atomic.json
+14 -6
View File
@@ -1,13 +1,21 @@
{
"Dummy.dummy_random_fail_in_atomic": [
"version": 2,
"title": "Dummy scenario with random atomic action failures",
"description": "Sample task demonstrating dummy scenario with random atomic action failures functionality",
"tags": ["dummy", "random", "atomic", "failure", "sample"],
"subtasks": [
{
"args": {
"exception_probability": 0.5
"title": "Dummy scenario with 50% atomic action failure probability",
"scenario": {
"Dummy.dummy_random_fail_in_atomic": {
"exception_probability": 0.5
}
},
"runner": {
"type": "constant",
"times": 100,
"concurrency": 10
"constant": {
"times": 100,
"concurrency": 10
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/dummy/dummy-random-fail-in-atomic.yaml
+14 -8
View File
@@ -1,11 +1,17 @@
Dummy.dummy_random_fail_in_atomic:
-
args:
---
version: 2
title: "Dummy scenario with random atomic action failures"
description: "Sample task demonstrating dummy scenario with random atomic action failures functionality"
tags: ["dummy", "random", "atomic", "failure", "sample"]
subtasks:
- title: "Dummy scenario with 50% atomic action failure probability"
scenario:
Dummy.dummy_random_fail_in_atomic:
exception_probability: 0.5
runner:
type: "constant"
runner:
constant:
times: 100
concurrency: 10
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/dummy/dummy-timed-atomic-actions.json
+15 -7
View File
@@ -1,14 +1,22 @@
{
"Dummy.dummy_timed_atomic_actions": [
"version": 2,
"title": "Dummy scenario with timed atomic actions",
"description": "Sample task demonstrating dummy scenario with timed atomic actions functionality",
"tags": ["dummy", "timed", "atomic", "actions", "sample"],
"subtasks": [
{
"args": {
"number_of_actions": 1,
"sleep_factor": 1
"title": "Dummy scenario with timed atomic actions and SLA monitoring",
"scenario": {
"Dummy.dummy_timed_atomic_actions": {
"number_of_actions": 1,
"sleep_factor": 1
}
},
"runner": {
"type": "constant",
"times": 5,
"concurrency": 5
"constant": {
"times": 5,
"concurrency": 5
}
},
"sla": {
"max_avg_duration_per_atomic": {
samples/tasks/scenarios/dummy/dummy-timed-atomic-actions.yaml
+13 -8
View File
@@ -1,13 +1,18 @@
---
Dummy.dummy_timed_atomic_actions:
-
args:
version: 2
title: "Dummy scenario with timed atomic actions"
description: "Sample task demonstrating dummy scenario with timed atomic actions functionality"
tags: ["dummy", "timed", "atomic", "actions", "sample"]
subtasks:
- title: "Dummy scenario with timed atomic actions and SLA monitoring"
scenario:
Dummy.dummy_timed_atomic_actions:
number_of_actions: 1
sleep_factor: 1
runner:
type: "constant"
runner:
constant:
times: 5
concurrency: 5
sla:
max_avg_duration_per_atomic:
action_0: 1.0
sla:
max_avg_duration_per_atomic:
action_0: 1.0
samples/tasks/scenarios/dummy/dummy.json
+14 -6
View File
@@ -1,13 +1,21 @@
{
"Dummy.dummy": [
"version": 2,
"title": "Dummy scenario sample",
"description": "Sample task demonstrating basic Dummy.dummy scenario usage with constant runner",
"tags": ["dummy", "sample", "basic"],
"subtasks": [
{
"args": {
"sleep": 5
"title": "Dummy.dummy basic scenario",
"scenario": {
"Dummy.dummy": {
"sleep": 5
}
},
"runner": {
"type": "constant",
"times": 20,
"concurrency": 5
"constant": {
"times": 20,
"concurrency": 5
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/dummy/dummy.yaml
+13 -8
View File
@@ -1,12 +1,17 @@
---
Dummy.dummy:
-
args:
version: 2
title: "Dummy scenario sample"
description: "Sample task demonstrating basic Dummy.dummy scenario usage with constant runner"
tags: ["dummy", "sample", "basic"]
subtasks:
- title: "Dummy.dummy basic scenario"
scenario:
Dummy.dummy:
sleep: 5
runner:
type: "constant"
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/requests/check-random-request.json
+17 -9
View File
@@ -1,16 +1,24 @@
{
"HttpRequests.check_random_request": [
"version": 2,
"title": "HTTP random request sample",
"description": "Sample task demonstrating random HTTP request selection scenario",
"tags": ["http", "requests", "random", "sample"],
"subtasks": [
{
"args": {
"requests": [{"url": "http://www.example.com", "method": "GET",
"status_code": 200},
{"url": "http://www.openstack.org", "method": "GET"}],
"status_code": 200
"title": "Random HTTP request validation",
"scenario": {
"HttpRequests.check_random_request": {
"requests": [{"url": "http://www.example.com", "method": "GET",
"status_code": 200},
{"url": "http://www.openstack.org", "method": "GET"}],
"status_code": 200
}
},
"runner": {
"type": "constant",
"times": 20,
"concurrency": 5
"constant": {
"times": 20,
"concurrency": 5
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/requests/check-random-request.yaml
+13 -8
View File
@@ -1,7 +1,12 @@
---
HttpRequests.check_random_request:
-
args:
version: 2
title: "HTTP random request sample"
description: "Sample task demonstrating random HTTP request selection scenario"
tags: ["http", "requests", "random", "sample"]
subtasks:
- title: "Random HTTP request validation"
scenario:
HttpRequests.check_random_request:
requests:
-
url: "http://www.example.com"
@@ -11,10 +16,10 @@
url: "http://www.openstack.org"
method: "GET"
status_code: 200
runner:
type: "constant"
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/scenarios/requests/check-request.json
+17 -9
View File
@@ -1,16 +1,24 @@
{
"HttpRequests.check_request": [
"version": 2,
"title": "HTTP request check sample",
"description": "Sample task demonstrating HTTP request validation scenario",
"tags": ["http", "requests", "sample"],
"subtasks": [
{
"args": {
"url": "http://www.example.com",
"method": "GET",
"status_code": 200,
"allow_redirects": false
"title": "HTTP request validation",
"scenario": {
"HttpRequests.check_request": {
"url": "http://www.example.com",
"method": "GET",
"status_code": 200,
"allow_redirects": false
}
},
"runner": {
"type": "constant",
"times": 20,
"concurrency": 5
"constant": {
"times": 20,
"concurrency": 5
}
},
"sla": {
"failure_rate": {
samples/tasks/scenarios/requests/check-request.yaml
+13 -8
View File
@@ -1,15 +1,20 @@
---
HttpRequests.check_request:
-
args:
version: 2
title: "HTTP request check sample"
description: "Sample task demonstrating HTTP request validation scenario"
tags: ["http", "requests", "sample"]
subtasks:
- title: "HTTP request validation"
scenario:
HttpRequests.check_request:
url: "http://www.example.com"
method: "GET"
status_code: 200
allow_redirects: False
runner:
type: "constant"
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
max: 0
sla:
failure_rate:
max: 0
samples/tasks/sla/dummy.json
+14 -6
View File
@@ -1,13 +1,21 @@
{
"Dummy.dummy": [
"version": 2,
"title": "SLA criteria sample",
"description": "Sample task demonstrating various SLA criteria usage",
"tags": ["sla", "sample", "dummy"],
"subtasks": [
{
"args": {
"sleep": 5
"title": "Dummy scenario with multiple SLA criteria",
"scenario": {
"Dummy.dummy": {
"sleep": 5
}
},
"runner": {
"type": "constant",
"times": 20,
"concurrency": 5
"constant": {
"times": 20,
"concurrency": 5
}
},
"sla": {
"max_seconds_per_iteration": 4.0,
samples/tasks/sla/dummy.yaml
+19 -14
View File
@@ -1,18 +1,23 @@
---
Dummy.dummy:
-
args:
version: 2
title: "SLA criteria sample"
description: "Sample task demonstrating various SLA criteria usage"
tags: ["sla", "sample", "dummy"]
subtasks:
- title: "Dummy scenario with multiple SLA criteria"
scenario:
Dummy.dummy:
sleep: 5
runner:
type: "constant"
runner:
constant:
times: 20
concurrency: 5
sla:
max_seconds_per_iteration: 4.0
failure_rate:
max: 1
max_avg_duration: 3.0
outliers:
max: 1
min_iterations: 10
sigmas: 10
sla:
max_seconds_per_iteration: 4.0
failure_rate:
max: 1
max_avg_duration: 3.0
outliers:
max: 1
min_iterations: 10
sigmas: 10
tests/samples/test_task_samples.py
+24 -13
View File
@@ -14,7 +14,6 @@
# under the License.
import inspect
import itertools
import os
import traceback
from unittest import mock
@@ -59,12 +58,17 @@ class TaskSampleTestCase(test.TestCase):
if "tasks/scenarios" not in path:
continue
with open(path) as task_file:
task_config = yaml.safe_load(
self.rapi.task.render_template(
task_template=task_file.read()))
for workload in itertools.chain(*task_config.values()):
if not workload.get("sla", {}):
failures.append(path)
rendered_config = self.rapi.task.render_template(
task_template=task_file.read())
raw_config = yaml.safe_load(rendered_config)
# Use TaskConfig to get normalized format
task_config_obj = task_cfg.TaskConfig(raw_config)
# Extract workloads from normalized v2 format
for subtask in task_config_obj.subtasks:
for workload in subtask["workloads"]:
if not workload.get("sla", {}):
failures.append(path)
if failures:
self.fail("One or several workloads from the list of samples below"
" doesn't have SLA section: \n %s" %
@@ -77,20 +81,27 @@ class TaskSampleTestCase(test.TestCase):
with open(path) as task_file:
try:
try:
task_config = yaml.safe_load(
self.rapi.task.render_template(
task_template=task_file.read()))
rendered_config = self.rapi.task.render_template(
task_template=task_file.read())
raw_config = yaml.safe_load(rendered_config)
except Exception:
print(traceback.format_exc())
self.fail("Invalid JSON file: %s" % path)
eng = engine.TaskEngine(task_cfg.TaskConfig(task_config),
# Use TaskConfig to get normalized format and validate
task_config_obj = task_cfg.TaskConfig(raw_config)
eng = engine.TaskEngine(task_config_obj,
mock.MagicMock(), mock.Mock())
eng.validate(only_syntax=True)
# Extract scenario names from normalized format
for subtask in task_config_obj.subtasks:
for workload in subtask["workloads"]:
scenarios.add(workload["name"])
except Exception:
print(traceback.format_exc())
self.fail("Invalid task file: %s" % path)
else:
scenarios.update(task_config.keys())
missing = set(s.get_name() for s in scenario.Scenario.get_all())
missing -= scenarios
tests/unit/doc/test_format.py
+5 -2
View File
@@ -16,6 +16,10 @@ import os
import re
import unittest
import rally
ROOT_DIR = os.path.dirname(os.path.dirname(rally.__file__))
class TestFormat(unittest.TestCase):
def _check_lines_wrapping(self, doc_file, raw):
@@ -65,8 +69,7 @@ class TestFormat(unittest.TestCase):
def test_lines(self):
files = []
docs_dir = os.path.join(os.path.dirname(__file__), os.pardir,
os.pardir, os.pardir, "doc")
docs_dir = os.path.join(ROOT_DIR, "doc")
for root, dirnames, filenames in os.walk(docs_dir):
for filename in fnmatch.filter(filenames, "*.rst"):
files.append(os.path.join(root, filename))