[doc] Add documentation for hook and trigger plugins
update and fix json errors in hook_plugins.rst Change-Id: I8b9d7b40048fdcae2d997a0a757f226d3487d297
This commit is contained in:
Anton Studenov
parent
4c53056382
commit
2e5bd14ec5
BIN
doc/source/images/Hook-Aggregated-Report.png
Normal file
BIN
doc/source/images/Hook-Aggregated-Report.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 63 KiB |
BIN
doc/source/images/Hook-Per-Hook-Report.png
Normal file
BIN
doc/source/images/Hook-Per-Hook-Report.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 86 KiB |
BIN
doc/source/images/Hook-Results.png
Normal file
BIN
doc/source/images/Hook-Results.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
402
doc/source/plugins/implementation/hook_and_trigger_plugins.rst
Normal file
402
doc/source/plugins/implementation/hook_and_trigger_plugins.rst
Normal file
@ -0,0 +1,402 @@
|
||||
..
|
||||
Copyright 2016 Mirantis Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _hook_and_trigger_plugins:
|
||||
|
||||
|
||||
Hooks. Hook trigger plugins
|
||||
===========================
|
||||
|
||||
|
||||
Why Hooks?
|
||||
----------
|
||||
|
||||
All Rally workloads repeat their actions as many times as it is configured by
|
||||
runner. Once run, there is no way to interupt the runner to evaluate any change
|
||||
or restart event on the stability of the cloud under test.
|
||||
For example we would like to test how configuration change or cloud component
|
||||
restart would affect performance and stability.
|
||||
|
||||
Task hooks were added to fill this gap and allow to use Rally for reliability
|
||||
and high availability testing. Generally, hooks allow to perform any
|
||||
actions on specified iteration or specified time since the workload has been
|
||||
started.
|
||||
|
||||
Also, task html-report provides results of hook execution. They can contain
|
||||
graphical or textual information with timing and statistics.
|
||||
|
||||
|
||||
Hooks & Triggers Overview
|
||||
-------------------------
|
||||
|
||||
|
||||
Architecture
|
||||
^^^^^^^^^^^^
|
||||
|
||||
Rally uses runners to specify how many times the workload should be executed.
|
||||
Hooks do not use runners, instead they rely on trigger plugins to specify when
|
||||
and how many times hook should be called. Therefore hooks are isolated from
|
||||
workload runners and do not affect them because each hook is executed in
|
||||
separate thread.
|
||||
|
||||
|
||||
Sample of usage
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
Hooks can be added to the task configuration. Lets take a look at hook
|
||||
configuration:
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"name": "sys_call",
|
||||
"args": "/bin/echo 123",
|
||||
"trigger": {
|
||||
"name": "event",
|
||||
"args": {
|
||||
"unit": "iteration",
|
||||
"at": [5, 50, 200, 1000]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
It specifies hook plugin with name "sys_call". "args" field contains string
|
||||
that will be used by sys_call plugin, but in case of any other hook plugin it
|
||||
can contain any other Python object, that is assumed to be passed to the hook.
|
||||
"trigger" field specifies which trigger plugin should be used to run this hook.
|
||||
"trigger" contains similar fields "name" and "args" which represent trigger
|
||||
plugin name and arguments for trigger plugin. In this example "event" trigger
|
||||
is specified and configured to run the hook at 5th, 50th, 200th and 1000th
|
||||
iterations.
|
||||
|
||||
Here is a full task config that contains previous hook configuraiton:
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"Dummy.dummy": [
|
||||
{
|
||||
"args": {
|
||||
"sleep": 0.01
|
||||
},
|
||||
"runner": {
|
||||
"type": "constant",
|
||||
"times": 1500,
|
||||
"concurrency": 1
|
||||
},
|
||||
"hooks": [
|
||||
{
|
||||
"name": "sys_call",
|
||||
"args": "/bin/echo 123",
|
||||
"trigger": {
|
||||
"name": "event",
|
||||
"args": {
|
||||
"unit": "iteration",
|
||||
"at": [5, 50, 200, 1000]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
.. note::
|
||||
In this example, runner is configured to run workload 1500 times. So there
|
||||
is a limit for iterations and hook will be triggered only if certain
|
||||
iteration is started by runner. In other words, if trigger specifies
|
||||
iteration out of runner iterations scope then such trigger will not be
|
||||
called.
|
||||
|
||||
Task report for this example will contain minimal information about hook
|
||||
execution: duration of each hook call and its status(success of failure).
|
||||
|
||||
Let's take a look at more complicated config that can produce graphical
|
||||
and textual information.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
Dummy.dummy:
|
||||
-
|
||||
args:
|
||||
sleep: 0.75
|
||||
runner:
|
||||
type: "constant"
|
||||
times: 20
|
||||
concurrency: 2
|
||||
hooks:
|
||||
- name: sys_call
|
||||
description: Run script
|
||||
args: sh rally/rally-jobs/extra/hook_example_script.sh
|
||||
trigger:
|
||||
name: event
|
||||
args:
|
||||
unit: iteration
|
||||
at: [2, 5, 8, 13, 17]
|
||||
- name: sys_call
|
||||
description: Show time
|
||||
args: date +%Y-%m-%dT%H:%M:%S
|
||||
trigger:
|
||||
name: event
|
||||
args:
|
||||
unit: time
|
||||
at: [0, 2, 5, 6, 9]
|
||||
- name: sys_call
|
||||
description: Show system name
|
||||
args: uname -a
|
||||
trigger:
|
||||
name: event
|
||||
args:
|
||||
unit: iteration
|
||||
at: [2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 17, 18]
|
||||
sla:
|
||||
failure_rate:
|
||||
max: 0
|
||||
|
||||
|
||||
hook_example_script.sh generates dummy output in JSON format. Grafical
|
||||
information format is the same as for workloads and the same types of
|
||||
charts are supported for the hooks.
|
||||
|
||||
Here is a report that shows aggregated table and chart with hook results:
|
||||
|
||||
.. image:: ../../images/Hook-Aggregated-Report.png
|
||||
|
||||
Here is report that shows lines chart and pie chart for first hook on
|
||||
the second iteration:
|
||||
|
||||
.. image:: ../../images/Hook-Per-Hook-Report.png
|
||||
|
||||
Browse existing Hooks_ and Triggers_.
|
||||
|
||||
|
||||
Writing your own Hook plugin
|
||||
----------------------------
|
||||
|
||||
Problem description
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Hook plugin should implement custom action that can be done one or multiple
|
||||
times during the workload. Examples of such actions might be the following:
|
||||
|
||||
- Destructive action inside cloud (`Fault Injection`_)
|
||||
- Getting information about current state of cloud (load/health)
|
||||
- Upgrading/downgrading a component of cloud
|
||||
- Changing configuration of cloud
|
||||
- etc.
|
||||
|
||||
|
||||
Plugin code
|
||||
^^^^^^^^^^^
|
||||
|
||||
The following example shows simple hook code that performs system call.
|
||||
It is inherited from the base *Hook* class and contains implemented ``run()``
|
||||
method:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import shlex
|
||||
import subprocess
|
||||
|
||||
from rally import consts
|
||||
from rally.task import hook
|
||||
|
||||
|
||||
@hook.configure(name="simple_sys_call")
|
||||
class SimpleSysCallHook(hook.Hook):
|
||||
"""Performs system call."""
|
||||
|
||||
CONFIG_SCHEMA = {
|
||||
"$schema": consts.JSON_SCHEMA,
|
||||
"type": "string",
|
||||
}
|
||||
|
||||
def run(self):
|
||||
proc = subprocess.Popen(shlex.split(self.config),
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT)
|
||||
proc.wait()
|
||||
if proc.returncode:
|
||||
self.set_error(
|
||||
exception_name="n/a", # no exception class
|
||||
description="Subprocess returned {}".format(proc.returncode),
|
||||
details=proc.stdout.read(),
|
||||
)
|
||||
|
||||
Any exceptions risen during execution of ``run`` method will be caught by Hook
|
||||
base class and saved as a result. Although hook should manually call
|
||||
``Hook.set_error()`` to indicate logical error in case if there is no exception
|
||||
raised.
|
||||
|
||||
Also there is a method for saving charts data: ``Hook.add_output()``.
|
||||
|
||||
|
||||
Plugin Placement
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
There are two folders for hook plugins:
|
||||
|
||||
- `OpenStack Hooks`_
|
||||
- `Common Hooks`_
|
||||
|
||||
|
||||
Sample of task that uses Hook
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"Dummy.dummy": [
|
||||
{
|
||||
"args": {
|
||||
"sleep": 0.01
|
||||
},
|
||||
"runner": {
|
||||
"type": "constant",
|
||||
"times": 10,
|
||||
"concurrency": 1
|
||||
},
|
||||
"hooks": [
|
||||
{
|
||||
"name": "simple_sys_call",
|
||||
"args": "/bin/echo 123",
|
||||
"trigger": {
|
||||
"name": "event",
|
||||
"args": {
|
||||
"unit": "iteration",
|
||||
"at": [3, 6]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
Results of task execution
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Result of previous task example:
|
||||
|
||||
.. image:: ../../images/Hook-Results.png
|
||||
|
||||
|
||||
Writing your own Trigger plugin
|
||||
-------------------------------
|
||||
|
||||
|
||||
Problem description
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Trigger plugin should implement an event processor that decides whether to
|
||||
start hook or not. Rally has two basic triggers that should cover most cases:
|
||||
|
||||
- `Event Trigger`_
|
||||
- `Periodic Trigger`_
|
||||
|
||||
|
||||
Plugin code
|
||||
^^^^^^^^^^^
|
||||
|
||||
This example shows the code of the existing Event trigger:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from rally import consts
|
||||
from rally.task import trigger
|
||||
|
||||
|
||||
@trigger.configure(name="event")
|
||||
class EventTrigger(trigger.Trigger):
|
||||
"""Triggers hook on specified event and list of values."""
|
||||
|
||||
CONFIG_SCHEMA = {
|
||||
"type": "object",
|
||||
"$schema": consts.JSON_SCHEMA,
|
||||
"oneOf": [
|
||||
{
|
||||
"properties": {
|
||||
"unit": {"enum": ["time"]},
|
||||
"at": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": True,
|
||||
"items": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
}
|
||||
},
|
||||
},
|
||||
"required": ["unit", "at"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
{
|
||||
"properties": {
|
||||
"unit": {"enum": ["iteration"]},
|
||||
"at": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": True,
|
||||
"items": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
}
|
||||
},
|
||||
},
|
||||
"required": ["unit", "at"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
def get_listening_event(self):
|
||||
return self.config["unit"]
|
||||
|
||||
def on_event(self, event_type, value=None):
|
||||
if not (event_type == self.get_listening_event()
|
||||
and value in self.config["at"]):
|
||||
# do nothing
|
||||
return
|
||||
super(EventTrigger, self).on_event(event_type, value)
|
||||
|
||||
|
||||
Trigger plugins must override two methods:
|
||||
|
||||
- ``get_listening_event`` - this method should return currently configured
|
||||
event name. (So far Rally supports only "time" and "iteration")
|
||||
- ``on_event`` - this method is called each time certain events occur.
|
||||
It calls base method when the hook is triggered on specified event.
|
||||
|
||||
|
||||
Plugin Placement
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
All trigger plugins should be placed in `Trigger folder`_.
|
||||
|
||||
|
||||
.. references:
|
||||
|
||||
.. _Hooks: ../plugin_reference.html#task-hooks
|
||||
.. _Triggers: ../plugin_reference.html#task-hook-triggers
|
||||
.. _Fault Injection: ../plugin_reference.html#fault-injection-hook
|
||||
.. _Event Trigger: ../plugin_reference.html#event-hook-trigger
|
||||
.. _Periodic Trigger: ../plugin_reference.html#periodic-hook-trigger
|
||||
.. _Common Hooks: https://github.com/openstack/rally/tree/master/rally/plugins/common/hook
|
||||
.. _OpenStack Hooks: https://github.com/openstack/rally/tree/master/rally/plugins/openstack/hook
|
||||
.. _Trigger folder: https://github.com/openstack/rally/tree/master/rally/plugins/common/trigger
|
||||
@ -41,7 +41,7 @@ of hook plugins that should be executed on this iteration.
|
||||
"args": {
|
||||
"cmd": "bash enable_factor_1"
|
||||
},
|
||||
"trigger: {
|
||||
"trigger": {
|
||||
"name": "event",
|
||||
"args": {
|
||||
"unit": "time",
|
||||
@ -54,7 +54,7 @@ of hook plugins that should be executed on this iteration.
|
||||
"args": {
|
||||
"cmd": "bash enable_factor_2"
|
||||
},
|
||||
"trigger: {
|
||||
"trigger": {
|
||||
"name": "event",
|
||||
"args": {
|
||||
"unit": "iteration",
|
||||
@ -67,7 +67,7 @@ of hook plugins that should be executed on this iteration.
|
||||
"args": {
|
||||
"cmd": "bash enable_factor_3"
|
||||
},
|
||||
"trigger: {
|
||||
"trigger": {
|
||||
"name": "periodic",
|
||||
"args": {
|
||||
"unit": "iteration",
|
||||
@ -82,7 +82,7 @@ of hook plugins that should be executed on this iteration.
|
||||
"args": {
|
||||
"cmd": "bash enable_factor_4"
|
||||
},
|
||||
"trigger: {
|
||||
"trigger": {
|
||||
"name": "periodic",
|
||||
"args": {
|
||||
"unit": "time",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user