rally-openstack/rally-jobs/self-rally.yaml
Andrey Kurilin cb653e5e23 Port hooks to the new format
Originally, we did not find a good terms for hook entities, so term "hook"
had several meanings: the plugin of action and the combination of
trigger+action configs.

While we are working on task format v2, we can make the things easier.

* rename plugin base Hook to HookAction;
* unification section "hooks" of task config format v2. Now, it
  looks similar to other sections.

Change-Id: Ic800fd4c9cb1ca19b7f6896e2994ca2e30f5138a
2017-10-05 23:01:06 +00:00

479 lines
12 KiB
YAML

---
version: 2
title: Task for rally-tox-self job
description: >
This task contains various scenarios for testing rally features
subtasks:
-
title: Test SLA plugins
workloads:
-
description: "Check SLA"
scenario:
Dummy.dummy:
sleep: 0.25
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
max: 0
max_seconds_per_iteration: 1.0
max_avg_duration: 0.5
outliers:
max: 1
min_iterations: 10
sigmas: 10
performance_degradation:
max_degradation: 50
-
description: Check failure_rate SLA plugin
scenario:
Dummy.failure:
sleep: 0.2
from_iteration: 5
to_iteration: 15
each: 2
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
min: 25
max: 25
-
description: Check max_avg_duration_per_atomic SLA plugin
scenario:
Dummy.dummy_timed_atomic_actions:
number_of_actions: 5
sleep_factor: 1
runner:
constant:
times: 3
concurrency: 3
sla:
max_avg_duration_per_atomic:
action_0: 1.0
action_1: 2.0
action_2: 3.0
action_3: 4.0
action_4: 5.0
-
title: Test constant runner
workloads:
-
description: "Check 'constant' runner."
scenario:
Dummy.dummy:
sleep: 0.25
runner:
constant:
times: 8
concurrency: 4
max_cpu_count: 2
sla:
failure_rate:
max: 0
-
scenario:
Dummy.dummy:
sleep: 0
runner:
constant:
times: 4500
concurrency: 20
sla:
failure_rate:
max: 0
-
description: >
Check the ability of constant runner to terminate scenario by timeout.
scenario:
Dummy.dummy:
sleep: 30
runner:
constant:
times: 2
concurrency: 2
timeout: 1
sla:
failure_rate:
min: 100
-
title: Test constant_for_duration runner
workloads:
-
description: "Check 'constant_for_duration' runner."
scenario:
Dummy.dummy:
sleep: 0.1
runner:
constant_for_duration:
duration: 5
concurrency: 5
sla:
failure_rate:
max: 0
-
title: Test rps runner
workloads:
-
description: "Check 'rps' runner."
scenario:
Dummy.dummy:
sleep: 0.001
runner:
rps:
times: 2000
rps: 200
sla:
failure_rate:
max: 0
-
description: >
Check 'rps' runner with float value of requests per second.
scenario:
Dummy.dummy:
sleep: 0.1
runner:
rps:
times: 5
rps: 0.5
sla:
failure_rate:
max: 0
-
description: >
Check 'rps' runner with float value of requests per second.
scenario:
Dummy.dummy:
sleep: 0.1
runner:
rps:
times: 5
rps: 0.2
sla:
failure_rate:
max: 0
-
description: >
Check 'max_concurrency' and 'max_cpu_count' properties of 'rps' runner.
scenario:
Dummy.dummy:
sleep: 0.001
runner:
rps:
times: 200
rps: 20
max_concurrency: 10
max_cpu_count: 3
sla:
failure_rate:
max: 0
-
description: "Check 'rps' with start, end, step arguments"
scenario:
Dummy.dummy:
sleep: 0.25
runner:
rps:
times: 55
rps:
start: 1
end: 10
step: 1
max_concurrency: 10
max_cpu_count: 3
sla:
failure_rate:
max: 0
-
description: "Check 'rps' with start, end, step arguments"
scenario:
Dummy.dummy:
sleep: 0.5
runner:
rps:
times: 55
rps:
start: 1
end: 10
step: 1
duration: 2
max_concurrency: 10
max_cpu_count: 3
sla:
failure_rate:
max: 0
-
description: >
Check the ability of rps runner to terminate scenario by timeout.
scenario:
Dummy.dummy:
sleep: 30
runner:
rps:
times: 1
rps: 1
timeout: 1
sla:
failure_rate:
min: 100
-
title: Test serial runner
workloads:
-
description: "Check 'serial' runner."
scenario:
Dummy.dummy:
sleep: 0.1
runner:
serial:
times: 20
sla:
failure_rate:
max: 0
-
title: Test Hook and Trigger plugins
workloads:
-
description: "Check sys_call hook."
scenario:
Dummy.dummy:
sleep: 0.75
runner:
constant:
times: 20
concurrency: 2
hooks:
- description: Run script
action:
sys_call: sh rally-jobs/extra/hook_example_script.sh
trigger:
event:
unit: iteration
at: [2, 5, 8, 13, 17]
- description: Show time
action:
sys_call: date +%Y-%m-%dT%H:%M:%S
trigger:
event:
unit: time
at: [0, 2, 5, 6, 9]
- description: Show system name
action:
sys_call: uname -a
trigger:
event:
unit: iteration
at: [2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 17, 18]
sla:
failure_rate:
max: 0
-
description: "Check periodic trigger with iteration unit."
scenario:
Dummy.dummy:
sleep: 0.25
runner:
constant:
times: 10
concurrency: 2
hooks:
- description: test hook
action:
sys_call: /bin/true
trigger:
periodic:
unit: iteration
step: 2
start: 4
end: 8
sla:
failure_rate:
max: 0
-
description: "Check event trigger args."
scenario:
Dummy.dummy:
sleep: 1
runner:
serial:
times: 10
hooks:
- description: Show system name
action:
sys_call: uname -a
trigger:
event:
unit: time
at: [0, 2, 4, 6, 8, 10]
sla:
failure_rate:
max: 0
-
description: "Check periodic trigger with time unit."
scenario:
Dummy.dummy:
sleep: 1
runner:
serial:
times: 10
hooks:
- description: test hook
action:
sys_call: /bin/true
trigger:
periodic:
unit: time
step: 2
start: 0
end: 6
-
title: Test Dummy scenarios
workloads:
-
scenario:
Dummy.dummy_exception:
size_of_message: 5
sla:
failure_rate:
max: 100
runner:
constant:
times: 20
concurrency: 5
-
scenario:
Dummy.dummy_exception_probability:
exception_probability: 0.5
runner:
serial:
times: 2042
sla:
failure_rate:
min: 5
max: 95
-
scenario:
Dummy.dummy_output: {}
runner:
constant:
times: 20
concurrency: 10
sla:
failure_rate:
max: 0
-
scenario:
Dummy.dummy_random_fail_in_atomic:
exception_probability: 0.5
sla:
failure_rate:
min: 5
max: 95
runner:
constant:
times: 50
concurrency: 10
-
scenario:
Dummy.dummy_random_action: {}
runner:
constant:
times: 10
concurrency: 5
-
title: Test a subtask with only one workload
scenario:
FakePlugin.testplugin: {}
runner:
constant:
times: 4
concurrency: 4
sla:
failure_rate:
max: 0
-
title: Profile generate_random_name method
workloads:
-
scenario:
RallyProfile.generate_names_in_atomic:
number_of_names: 100
runner:
constant:
times: 1000
concurrency: 10
sla:
max_avg_duration_per_atomic:
generate_100_names: 0.015
failure_rate:
max: 0
-
scenario:
RallyProfile.generate_names_in_atomic:
number_of_names: 1000
runner:
constant:
times: 500
concurrency: 10
sla:
max_avg_duration_per_atomic:
generate_1000_names: 0.1
failure_rate:
max: 0
-
scenario:
RallyProfile.generate_names_in_atomic:
number_of_names: 10000
runner:
constant:
times: 200
concurrency: 10
sla:
max_avg_duration_per_atomic:
generate_10000_names: 1
failure_rate:
max: 0
-
title: Profile atomic actions
workloads:
-
scenario:
RallyProfile.calculate_atomic:
number_of_atomics: 100
runner:
constant:
times: 300
concurrency: 10
sla:
max_avg_duration_per_atomic:
calculate_100_atomics: 0.04
failure_rate:
max: 0
-
scenario:
RallyProfile.calculate_atomic:
number_of_atomics: 500
runner:
constant:
times: 100
concurrency: 10
sla:
max_avg_duration_per_atomic:
calculate_500_atomics: 0.5
failure_rate:
max: 0