49c5056968
In the common.streaming_algorithms module, we add the base for the streaming
algorithms that take values from the input stream and compute certain
quantities needed for the benchmark results processing. These algorithms
should use constant memory and be able to scale.
Two streaming algorithms for computing mean and standard deviation are implemented.
There is also a new "max_outliers" SLA that checks for the maximum number of
outliers based on the mean and standard deviation of the durations, computed
using these new streaming algorithms. By default, the outliers are detected
only when the total number of iterations reaches 10 (can be configured).
Example:
3.1 4.2 3.6 4.5 2.8 3.3 4.1 3.8 4.3 2.9 10.2 11.2 3.4
has 2 outliers (10.2 and 11.2), so:
{"outliers": {"max": 2}} -> PASS
{"outliers": {"max": 1}} -> FAIL
Bonus:
* Add gate testing of different SLAs with the Dummy scenario
* Add samples for all SLAs
* Fix detailed message for max_avg_duration
Change-Id: I7c4f77c418c7b61f71b43216110fa4c7aaccc2f5
|
||
|---|---|---|
| .. | ||
| create-and-delete-user.json | ||
| create-and-delete-user.yaml | ||
| README.rst | ||
SLA Configuration Samples
This directory contains SLA configuration samples.
SLA (Service-level agreement) is set of details for determining compliance with contracted values such as maximum error rate or minimum response time.
Currently supported criteria:
failure_rate
A maximum and/or a minimum failure rate expressed as max or min sub-keys.
max_seconds_per_iteration
Maximum time in seconds per one iteration.