Files

Boris Pavlovic 3d9b1e43a6 Rename certification/ to tasks/

Every day Rally step by step closer to the point where Rally is going to be
generic framework and it would be possible to split Rally & Rally OpenStack Plugins

One of the blockers for doing this is certification & samples directories and
their contents. Current apporach creates a lot of confusion:

- Rally Users are running samples against production
- Rally samples are actually not that usefull, because creating from them
  tasks takes a lot of time
- Rally Users don't know what certification task is and how to use it
- Samples are hard to ship/find in packaged way (when rally is installed)
- Samples are going to be impossible to ship after the split
- We have to keep hunderds of yaml/json files in actual state
  in our repo (extra work for developers)

Proposed approach is to:

- Create mechansim CLI command that generates samples using plugin names
- Rename certification -> tasks
- Put in tasks directory pre created and tested tasks for different use cases:
  key performance metrics, functional testing, load testing of key functionality,
  ha Testing
- Remove samples as they are not needed anymore

Change-Id: I4c45224c3af637d17dab0edea038c69a2bae8b1a

2017-09-19 18:01:25 -07:00

hooks

Rename certification/ to tasks/

2017-09-19 18:01:25 -07:00

__init__.py

Reorganize test module structure

2014-10-07 13:50:40 +00:00

cover.sh

[ci]merge 3 patchs to unblock our ci

2017-06-30 12:38:07 -07:00

osresources.py

Simplify deployment config format

2017-08-22 16:26:17 +03:00

pytest_launcher.py

[ci] Rework launching cli job

2017-08-03 01:17:58 +03:00

rally_app.py

[Test] Add test for simple app based on rally api

2017-01-19 13:23:01 +03:00

rally_gate_functions.sh

Simplify deployment config format

2017-08-22 16:26:17 +03:00

rally_self_job.sh

[CLI] Add support for emtpy deployment

2017-09-09 09:15:43 +00:00

rally_verify.py

mofify gate reports to use static html because of some unreachable links.

2017-08-11 06:44:44 +00:00

rally-gate.sh

Fix identity v3 detection

2017-05-02 15:19:54 +03:00

README.rst

Adjusting introduction of gate

2017-03-22 01:01:20 +08:00

render.py

[Reports] Get rid of Mako in reports, finally

2016-10-19 15:29:03 +03:00

sync_requirements.py

Handle log message interpolation by the logger in common/

2017-08-18 04:57:12 +00:00

test_install.sh

Make sure we don't have a stale package cache

2016-04-05 09:28:54 -04:00

wip-rally-gate.py

[CLI] group commands in help message

2016-11-28 22:58:14 +03:00

README.rst

Rally Gate Jobs

For each patch submitted for review on Gerrit, there is a set of tests called gate jobs to be run against it. These tests check whether the Rally code works correctly after applying the patch and provide additional guarantees that it won't break the software when it gets merged. Rally gate jobs contain tests checking the codestyle (via pep8), unit tests suites, functional tests and a set of Rally benchmark tasks that are executed against a real devstack deployment.

rally-gate.sh

This script runs a set of real Rally benchmark tasks and fetches their results in textual / visualized form (available via a special html page by clicking the corresponding job title in Gerrit). It checks that scenarios don't fail while being executed against a devstack deployment and also tests SLA criteria to ensure that benchmark tasks have completed successfully. Jenkins uses this script by runnint the 'gate-rally-dsvm-rally' job, 'gate-rally-dsvm-manila-multibackend' job and 'gate-rally-dsvm-neutron-rally' job.

rally_verify.py

This script runs various "rally verify" commands. This set of commands allow us to perform Tempest tests of OpenStack live cloud and display verification results. The verification results obtained by running various "rally verify <cmd>" commands including "start", "show", "list" are compared using the "rally verify results" command, which are then saved in csv, html and json formats in the "rally-verify" directory. Jenkins uses this script by running the 'gate-rally-dsvm-verify' job.

test_install.sh

This script tests the correct working of the install_rally.sh, used for the installation of Rally. Jenkins tests this script by running it against Centos7 and Ubuntu 16.04 in the corresponding jobs 'gate-rally-install-centos-7' and 'gate-rally-install-ubuntu-xenial'.

Jenkins

Jenkins is a Continuous Integration system which works as the scheduler. It receives events related to proposed changes, triggers tests based on those events, and reports back. For each patch that is uploaded for review on Gerrit, Jenkins runs it against the various rally gate jobs listed below along with their functions and local equivalents:

gate-rally-pep8 : code style check

(equal to tox -epep8)
gate-rally-docs : documention generation

(equal to tox -edocs)
gate-rally-python27 : unit tests against python27

(equal to tox -epy27)
gate-rally-python35 : unit tests against python35

(equal to tox -epy35)
rally-coverage : generates unit test

coverage (equal to tox -cover)
gate-rally-install-centos-7 : testing of test_install.sh

(described above) against Centos7
gate-rally-install-ubuntu-xenial : testing of test_install.sh

(described above) against Ubuntu 16.04
gate-rally-dsvm-rally : runs rally-gate.sh

(described above) against OpenStack deployed by devstack with nova-network (It is standard dsvm job)
gate-rally-dsvm-manila-multibackend(non-voting) : runs rally-gate.sh against

manila
gate-rally-dsvm-neutron-rally : runs rally-gate.sh against

OpenStack deployed by devastack with neutron
gate-rally-dsvm-cli : runs tests/functional/*

(equal to tox -ecli)
gate-rally-dsvm-verify(non-voting) : runs rally_verify.py and

tests Rally and Tempest integration in all possible ways
gate-rally-tox-self(non-voting) : not yet used

and a success in these tests(except non-voting) would mean that the patch is approved by Jenkins.