rally/doc/source/plugins/implementation/hook_and_trigger_plugins.rst

Hooks. Hook trigger plugins

Why Hooks?

All Rally workloads repeat their actions as many times as it is configured by runner. Once run, there is no way to interrupt the runner to evaluate any change or restart event on the stability of the cloud under test. For example we would like to test how configuration change or cloud component restart would affect performance and stability.

Task hooks were added to fill this gap and allow to use Rally for reliability and high availability testing. Generally, hooks allow to perform any actions on specified iteration or specified time since the workload has been started.

Also, task html-report provides results of hook execution. They can contain graphical or textual information with timing and statistics.

Hooks & Triggers Overview

Architecture

Rally uses runners to specify how many times the workload should be executed. Hooks do not use runners, instead they rely on trigger plugins to specify when and how many times hook should be called. Therefore hooks are isolated from workload runners and do not affect them because each hook is executed in separate thread.

Sample of usage

Hooks can be added to the task configuration. Lets take a look at hook configuration:

{
    "name": "sys_call",
    "args": "/bin/echo 123",
    "trigger": {
        "name": "event",
        "args": {
            "unit": "iteration",
            "at": [5, 50, 200, 1000]
        }
    }
}

It specifies hook plugin with name "sys_call". "args" field contains string that will be used by sys_call plugin, but in case of any other hook plugin it can contain any other Python object, that is assumed to be passed to the hook. "trigger" field specifies which trigger plugin should be used to run this hook. "trigger" contains similar fields "name" and "args" which represent trigger plugin name and arguments for trigger plugin. In this example "event" trigger is specified and configured to run the hook at 5th, 50th, 200th and 1000th iterations.

Here is a full task config that contains previous hook configuration:

{
    "Dummy.dummy": [
        {
            "args": {
                "sleep": 0.01
            },
            "runner": {
                "type": "constant",
                "times": 1500,
                "concurrency": 1
            },
            "hooks": [
                {
                    "name": "sys_call",
                    "args": "/bin/echo 123",
                    "trigger": {
                        "name": "event",
                        "args": {
                            "unit": "iteration",
                            "at": [5, 50, 200, 1000]
                        }
                    }
                }
            ]
        }
    ]
}

Note

In this example, runner is configured to run workload 1500 times. So there is a limit for iterations and hook will be triggered only if certain iteration is started by runner. In other words, if trigger specifies iteration out of runner iterations scope then such trigger will not be called.

Task report for this example will contain minimal information about hook execution: duration of each hook call and its status(success of failure).

Let's take a look at more complicated config that can produce graphical and textual information.

---
  Dummy.dummy:
    -
      args:
        sleep: 0.75
      runner:
        type: "constant"
        times: 20
        concurrency: 2
      hooks:
        - name: sys_call
          description: Run script
          args: sh rally/rally-jobs/extra/hook_example_script.sh
          trigger:
            name: event
            args:
              unit: iteration
              at: [2, 5, 8, 13, 17]
        - name: sys_call
          description: Show time
          args: date +%Y-%m-%dT%H:%M:%S
          trigger:
            name: event
            args:
              unit: time
              at: [0, 2, 5, 6, 9]
        - name: sys_call
          description: Show system name
          args: uname -a
          trigger:
            name: event
            args:
              unit: iteration
              at: [2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 17, 18]
      sla:
        failure_rate:
          max: 0

hook_example_script.sh generates dummy output in JSON format. Graphical information format is the same as for workloads and the same types of charts are supported for the hooks.

Here is a report that shows aggregated table and chart with hook results:

Here is report that shows lines chart and pie chart for first hook on the second iteration:

Browse existing Hooks and Triggers.

Writing your own Hook plugin

Problem description

Hook plugin should implement custom action that can be done one or multiple times during the workload. Examples of such actions might be the following:

• Destructive action inside cloud (Fault Injection)
• Getting information about current state of cloud (load/health)
• Upgrading/downgrading a component of cloud
• Changing configuration of cloud
• etc.

Plugin code

The following example shows simple hook code that performs system call. It is inherited from the base Hook class and contains implemented run() method:

import shlex
import subprocess

from rally import consts
from rally.task import hook


@hook.configure(name="simple_sys_call")
class SimpleSysCallHook(hook.Hook):
    """Performs system call."""

    CONFIG_SCHEMA = {
        "$schema": consts.JSON_SCHEMA,
        "type": "string",
    }

    def run(self):
        proc = subprocess.Popen(shlex.split(self.config),
                                stdout=subprocess.PIPE,
                                stderr=subprocess.STDOUT)
        proc.wait()
        if proc.returncode:
            self.set_error(
                exception_name="n/a",  # no exception class
                description="Subprocess returned {}".format(proc.returncode),
                details=proc.stdout.read(),
            )

Any exceptions risen during execution of run method will be caught by Hook base class and saved as a result. Although hook should manually call Hook.set_error() to indicate logical error in case if there is no exception raised.

Also there is a method for saving charts data: Hook.add_output().

Plugin Placement

There are two folders for hook plugins:

• OpenStack Hooks
• Common Hooks

Sample of task that uses Hook

{
    "Dummy.dummy": [
        {
            "args": {
                "sleep": 0.01
            },
            "runner": {
                "type": "constant",
                "times": 10,
                "concurrency": 1
            },
            "hooks": [
                {
                    "name": "simple_sys_call",
                    "args": "/bin/echo 123",
                    "trigger": {
                        "name": "event",
                        "args": {
                            "unit": "iteration",
                            "at": [3, 6]
                        }
                    }
                }
            ]
        }
    ]
}

Results of task execution

Result of previous task example:

Writing your own Trigger plugin

Problem description

Trigger plugin should implement an event processor that decides whether to start hook or not. Rally has two basic triggers that should cover most cases:

• Event Trigger
• Periodic Trigger

Plugin code

This example shows the code of the existing Event trigger:

from rally import consts
from rally.task import trigger


@trigger.configure(name="event")
class EventTrigger(trigger.Trigger):
    """Triggers hook on specified event and list of values."""

    CONFIG_SCHEMA = {
        "type": "object",
        "$schema": consts.JSON_SCHEMA,
        "oneOf": [
            {
                "properties": {
                    "unit": {"enum": ["time"]},
                    "at": {
                        "type": "array",
                        "minItems": 1,
                        "uniqueItems": True,
                        "items": {
                            "type": "integer",
                            "minimum": 0,
                        }
                    },
                },
                "required": ["unit", "at"],
                "additionalProperties": False,
            },
            {
                "properties": {
                    "unit": {"enum": ["iteration"]},
                    "at": {
                        "type": "array",
                        "minItems": 1,
                        "uniqueItems": True,
                        "items": {
                            "type": "integer",
                            "minimum": 1,
                        }
                    },
                },
                "required": ["unit", "at"],
                "additionalProperties": False,
            },
        ]
    }

    def get_listening_event(self):
        return self.config["unit"]

    def on_event(self, event_type, value=None):
        if not (event_type == self.get_listening_event()
                and value in self.config["at"]):
            # do nothing
            return
        super(EventTrigger, self).on_event(event_type, value)

Trigger plugins must override two methods:

• get_listening_event - this method should return currently configured event name. (So far Rally supports only "time" and "iteration")
• on_event - this method is called each time certain events occur. It calls base method when the hook is triggered on specified event.

Plugin Placement

All trigger plugins should be placed in Trigger folder.