openstack/openstack-ansible-ops
Code Issues Proposed changes
3aa5fcd8fc
openstack-ansible-ops/bowling_ball
History
Jean-Philippe Evrard 4df9332649 Use clouds.yaml 
Instead of passing all things to the client, and forgetting some,
we should directly load the config, and use the cloud
configuration for creating connections.

Change-Id: I8e66ff033b6d49537b161710e983750f40d5e93d
2018-02-13 11:02:27 +00:00
..
README.rst Implement CLI test picker 2017-05-19 11:47:49 +00:00
rolling_restart.py Unify output formatting, add logging 2017-03-15 16:36:32 -04:00
rolling_tests.py Use clouds.yaml 2018-02-13 11:02:27 +00:00

README.rst

Bowling Ball - OpenStack-Ansible Rolling Downtime Simulator

date

2017-03-09

tags

rackspace, openstack, ansible

category

*openstack, *nix

About

This project aims to test for issues with rolling downtime on OpenStack-Ansible deployments. It's comprised of two main components:

  • The rolling_restart.py script
  • The rolling_test.py script

The rolling_restart.py script will stop containers from a specified group in a rolling fashion - node 1 will stop, then start, then node 2, then node 3 and so on. This script runs from the deployment host.

The tests directory contains scripts to generate traffic against the target services. system that will be restarted by rolling_restart.py in order to measure the effects. These scripts run from a utility container.

The rolling_test.py script contains tests to generate traffic against the target services. These vary per service, but attempt to apply usage to a system that will be restarted by rolling_restart.py in order to measure the effects. This script runs from a utility container.

Usage

  1. Start your test from a utility container. ./rolling_test.py keystone runs the Keystone test. ./rolling_test.py list will list tests and their descriptions

  2. From the deployment node, run rolling_restart.py in the playbooks directory (necessary to find the inventory script). Specify the service you're targeting with the -s parameter.

    rolling_restart.py -s keystone_container

    You can specify a wait time in seconds between stopping and starting individual nodes.

    rolling_restart.py -s keystone_container -w 60

Assumptions

These tools are currently coupled to OSA, and they assume paths to files as specified by the multi-node-aio scripts.

Container stopping and starting is done with an ansible command, and the physical host to target is derived from the current inventory.

rolling_restart.py must currently be run from the playbooks directory. This will be fixed later.

You must source openrc before running rolling_test.py.

Creating New Tests

Tests should subclass from the ServiceTest class in the same file and implement the following properties and methods:

  1. run - The actual test to run should be placed in this method. Timings

    will be gathered based on when this function starts and stops.

  2. pre_test - Any pre-test setup that needs to happen, like creating a

    file for Glance, Cinder, or Swift upload.

  3. post_test - Any post-test teardown that might be needed.
  4. service_name - The primary service that is being tested.
  5. description - Brief description of what the test does.

Finally, add the test to the available_tests dictionary with the invocation name as the key and the class as the value.

Why the name?

It sets 'em up and knocks em down.