openstack/rally
Code Issues Proposed changes
rally/rally-jobs/self-rally.yaml
Boris Pavlovic 5f73c84595 Make self-rally more stable 
Even if we have failure probability 0.5 it may occure that failure rate
will be any particual number (hello to the math) so we should extend
min and max failure rates from 20 to 80 to 5 to 95 to reduce the possibility
of such event

Change-Id: Id5c79408bbe6a0a1850c386bc3d4710d03a0512c
2017-09-06 20:16:10 +00:00

485 lines
12 KiB
YAML
Raw Blame History

---
version: 2
title: Task for rally-tox-self job
description: >
This task contains various scenarios for testing rally features
subtasks:
-
title: Test SLA plugins
workloads:
-
description: "Check SLA"
scenario:
Dummy.dummy:
sleep: 0.25
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
max: 0
max_seconds_per_iteration: 1.0
max_avg_duration: 0.5
outliers:
max: 1
min_iterations: 10
sigmas: 10
performance_degradation:
max_degradation: 50
-
description: Check failure_rate SLA plugin
scenario:
Dummy.failure:
sleep: 0.2
from_iteration: 5
to_iteration: 15
each: 2
runner:
constant:
times: 20
concurrency: 5
sla:
failure_rate:
min: 25
max: 25
-
description: Check max_avg_duration_per_atomic SLA plugin
scenario:
Dummy.dummy_timed_atomic_actions:
number_of_actions: 5
sleep_factor: 1
runner:
constant:
times: 3
concurrency: 3
sla:
max_avg_duration_per_atomic:
action_0: 1.0
action_1: 2.0
action_2: 3.0
action_3: 4.0
action_4: 5.0
-
title: Test constant runner
workloads:
-
description: "Check 'constant' runner."
scenario:
Dummy.dummy:
sleep: 0.25
runner:
constant:
times: 8
concurrency: 4
max_cpu_count: 2
sla:
failure_rate:
max: 0
-
scenario:
Dummy.dummy:
sleep: 0
runner:
constant:
times: 4500
concurrency: 20
sla:
failure_rate:
max: 0
-
description: >
Check the ability of constant runner to terminate scenario by timeout.
scenario:
Dummy.dummy:
sleep: 30
runner:
constant:
times: 2
concurrency: 2
timeout: 1
sla:
failure_rate:
min: 100
-
title: Test constant_for_duration runner
workloads:
-
description: "Check 'constant_for_duration' runner."
scenario:
Dummy.dummy:
sleep: 0.1
runner:
constant_for_duration:
duration: 5
concurrency: 5
sla:
failure_rate:
max: 0
-
title: Test rps runner
workloads:
-
description: "Check 'rps' runner."
scenario:
Dummy.dummy:
sleep: 0.001
runner:
rps:
times: 2000
rps: 200
sla:
failure_rate:
max: 0
-
description: >
Check 'rps' runner with float value of requests per second.
scenario:
Dummy.dummy:
sleep: 0.1
runner:
rps:
times: 5
rps: 0.5
sla:
failure_rate:
max: 0
-
description: >
Check 'rps' runner with float value of requests per second.
scenario:
Dummy.dummy:
sleep: 0.1
runner:
rps:
times: 5
rps: 0.2
sla:
failure_rate:
max: 0
-
description: >
Check 'max_concurrency' and 'max_cpu_count' properties of 'rps' runner.
scenario:
Dummy.dummy:
sleep: 0.001
runner:
rps:
times: 200
rps: 20
max_concurrency: 10
max_cpu_count: 3
sla:
failure_rate:
max: 0
-
description: "Check 'rps' with start, end, step arguments"
scenario:
Dummy.dummy:
sleep: 0.25
runner:
rps:
times: 55
rps:
start: 1
end: 10
step: 1
max_concurrency: 10
max_cpu_count: 3
sla:
failure_rate:
max: 0
-
description: "Check 'rps' with start, end, step arguments"
scenario:
Dummy.dummy:
sleep: 0.5
runner:
rps:
times: 55
rps:
start: 1
end: 10
step: 1
duration: 2
max_concurrency: 10
max_cpu_count: 3
sla:
failure_rate:
max: 0
-
description: >
Check the ability of rps runner to terminate scenario by timeout.
scenario:
Dummy.dummy:
sleep: 30
runner:
rps:
times: 1
rps: 1
timeout: 1
sla:
failure_rate:
min: 100
-
title: Test serial runner
workloads:
-
description: "Check 'serial' runner."
scenario:
Dummy.dummy:
sleep: 0.1
runner:
serial:
times: 20
sla:
failure_rate:
max: 0
-
title: Test Hook and Trigger plugins
workloads:
-
description: "Check sys_call hook."
scenario:
Dummy.dummy:
sleep: 0.75
runner:
constant:
times: 20
concurrency: 2
hooks:
- name: sys_call
description: Run script
args: sh ~/.rally/extra/hook_example_script.sh
trigger:
name: event
args:
unit: iteration
at: [2, 5, 8, 13, 17]
- name: sys_call
description: Show time
args: date +%Y-%m-%dT%H:%M:%S
trigger:
name: event
args:
unit: time
at: [0, 2, 5, 6, 9]
- name: sys_call
description: Show system name
args: uname -a
trigger:
name: event
args:
unit: iteration
at: [2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 17, 18]
sla:
failure_rate:
max: 0
-
description: "Check periodic trigger with iteration unit."
scenario:
Dummy.dummy:
sleep: 0.25
runner:
constant:
times: 10
concurrency: 2
hooks:
- name: sys_call
description: test hook
args: /bin/true
trigger:
name: periodic
args:
unit: iteration
step: 2
start: 4
end: 8
sla:
failure_rate:
max: 0
-
description: "Check event trigger args."
scenario:
Dummy.dummy:
sleep: 1
runner:
serial:
times: 10
hooks:
- name: sys_call
description: Get system name
args: uname -a
trigger:
name: event
args:
unit: time
at: [0, 2, 4, 6, 8, 10]
sla:
failure_rate:
max: 0
-
description: "Check periodic trigger with time unit."
scenario:
Dummy.dummy:
sleep: 1
runner:
serial:
times: 10
hooks:
- name: sys_call
description: test hook
args: /bin/true
trigger:
name: periodic
args:
unit: time
step: 2
start: 0
end: 6
-
title: Test Dummy scenarios
workloads:
-
scenario:
Dummy.dummy_exception:
size_of_message: 5
sla:
failure_rate:
max: 100
runner:
constant:
times: 20
concurrency: 5
-
scenario:
Dummy.dummy_exception_probability:
exception_probability: 0.5
runner:
serial:
times: 2042
sla:
failure_rate:
min: 5
max: 95
-
scenario:
Dummy.dummy_output: {}
runner:
constant:
times: 20
concurrency: 10
sla:
failure_rate:
max: 0
-
scenario:
Dummy.dummy_random_fail_in_atomic:
exception_probability: 0.5
sla:
failure_rate:
min: 5
max: 95
runner:
constant:
times: 50
concurrency: 10
-
scenario:
Dummy.dummy_random_action: {}
runner:
constant:
times: 10
concurrency: 5
-
title: Test a subtask with only one workload
scenario:
FakePlugin.testplugin: {}
runner:
constant:
times: 4
concurrency: 4
sla:
failure_rate:
max: 0
-
title: Profile generate_random_name method
workloads:
-
scenario:
RallyProfile.generate_names_in_atomic:
number_of_names: 100
runner:
constant:
times: 1000
concurrency: 10
sla:
max_avg_duration_per_atomic:
generate_100_names: 0.015
failure_rate:
max: 0
-
scenario:
RallyProfile.generate_names_in_atomic:
number_of_names: 1000
runner:
constant:
times: 500
concurrency: 10
sla:
max_avg_duration_per_atomic:
generate_1000_names: 0.1
failure_rate:
max: 0
-
scenario:
RallyProfile.generate_names_in_atomic:
number_of_names: 10000
runner:
constant:
times: 200
concurrency: 10
sla:
max_avg_duration_per_atomic:
generate_10000_names: 1
failure_rate:
max: 0
-
title: Profile atomic actions
workloads:
-
scenario:
RallyProfile.calculate_atomic:
number_of_atomics: 100
runner:
constant:
times: 300
concurrency: 10
sla:
max_avg_duration_per_atomic:
calculate_100_atomics: 0.04
failure_rate:
max: 0
-
scenario:
RallyProfile.calculate_atomic:
number_of_atomics: 500
runner:
constant:
times: 100
concurrency: 10
sla:
max_avg_duration_per_atomic:
calculate_500_atomics: 0.5
failure_rate:
max: 0
View Git Blame Copy Permalink