27 Commits

Author SHA1 Message Date
Mikhail Dubov
e16d5c3cec Fix bug in atomic actions
Some code cleanup for the recently merged https://review.openstack.org/111977

We took the atomic actions list by mistake always from the first iteration;
we should continue taking this list from the first non-failed iteration.
Add functional testing for this case.

We also fix the missing "%" sign in the output CLI table in case there were
no successful atomic action iterations.

Co-authored-by: Oleg Anufriev <oanufriev@mirantis.com>
Change-Id: Id4095122be8dd8bffeeaeabe407ff76c0f294686
Closes-Bug: 1372739
Closes-Bug: 1372998
Closes-Bug: 1373488
Closes-Bug: 1373576
2014-09-24 23:22:01 +04:00
Oleh Anufriiev
94dd8648b8 Rally cli functional testing
Added test for:
    'rally deployment'
    'rally show'
    'rally info'
    'rally task'

  test_cli was parted

  blueprint tests-integrated

Change-Id: If9febf155d8178b5c941317d2a7b837bbce128f1
2014-09-18 15:45:42 +03:00
Boris Pavlovic
5ed84f0cdd Fix one more time rally perf job
We don't have actually access to create plugin directory

Change-Id: I23fbc71bea7e8bd509925f3b2b48aebb4a58b948
2014-09-09 19:57:00 +04:00
Boris Pavlovic
1deb53c9f1 Fix rally job in projects that don't have rally-scenario/plugins dir
*) Fix job
*) Improve a bit structure of index.html

Change-Id: I2996caf80f14ab64474bc4719ecf6ac0f89bde8e
2014-09-09 17:02:56 +04:00
Boris Pavlovic
83040abc84 Make rally perf job work in other projects
I made small mistake in previos patch. So we are copy index.html from wrong place

Change-Id: Ie02f13b3714ca893c3c967f6797a008ab5be9826
2014-09-09 00:25:11 +04:00
Boris Pavlovic
58e17ab929 Improve Rally gate job
Make a nice page when you open rally job results.


Change-Id: I8db18548f0e007102bf5bd357c69b3eac59de08b
2014-09-08 14:58:48 +00:00
Sergey Skripnick
cb5295c3a6 Fix rally-gate.sh exit status
Exit status should be set to non-zero if rally sla_check was not
successful. Bash option pipefail save any non-zero exit status
from piped commands.

Changed some default scenarios runners agrument

Remove SLA from 2 non working benchmark scenarios.
Related-Bug: #1354683

Change-Id: I327f23e460737ffe3a4fec42afcfbb387ae81bfb
2014-08-09 16:55:38 +04:00
Kumar Rishabh
a92d7eaeb1 Adding sla_check to gate jobs
Adds proper sla checks to each benchmark configuration to be run
in the gates and adds sla check to the script to check if non-zero
status is returned

Change-Id: Ibba485aa1b070fc5eff0b4cfece0f249e9fae166
2014-08-02 22:07:34 +00:00
Oleh Anufriiev
65d882c2b3 Periodic runner refactoring
'preiodic' runner renamed to rps (runs per second).
  Extensible pool of multiprocessing.Process was implemented instead
  of using constant length multiprocessing.Pool. That should decrease memory
  usage on large iterations and related errors.

Change-Id: I6e367ea30d89d5d0e7ba212ed22e8658421ac9bd
2014-07-24 18:58:59 +03:00
Oleh Anufriiev
7f18130563 Updated rally gate scenarios
Change-Id: I021e4be45cf3e9d92c9e099cec63ae844831c995
2014-07-23 18:21:38 +03:00
Sergey Skripnick
98d7946e3c Add basic tools for integration tests
Add class for represent task config
Add integration tests for rally SLA
Generate html test results

Change-Id: Ifc77d1efad1456df0325fcb0d0d9dca0d9b46866
2014-07-15 13:57:42 +03:00
Boris Pavlovic
f76ac9738e Add rally task detailed info to rally-plot dir
I am already tired rolling to the end of console.log to see the
aggregated infromation, so it makes sense to put this info to separated
txt file in rally-plot dir

Change-Id: Ic492f0acd293fd0a90595658c4da0dea720758ed
2014-07-11 21:38:09 +04:00
Duncan Thomas
56fb2ba32a Add json results to the logs bundle, as well as html
The json file of the raw results is useful if you want to pull
the results into an external system, e.g. to plot the performance
of the gate over time.

Change-Id: I03d6b6f6ab5c5aee3a0f5de0c87eaf393f041d14
2014-07-08 18:29:35 +01:00
Jenkins
6afc443af2 Merge "Fix hacking 0.9.x issues" 2014-06-18 14:31:41 +00:00
Olga Kopylova
c800370cfd Fix hacking 0.9.x issues
Fixed issues:
H202 assertRaises Exception too broad
H305 imports not grouped correctly
H307 like imports should be grouped together
H405 multi line docstring summary not separated with an empty line
H904 Wrap long lines in parentheses instead of a backslash
E122 continuation line missing indentation or outdented
E128 continuation line under-indented for visual indent
E131 continuation line unaligned for hanging indent
E251 unexpected spaces around keyword / parameter equals
E265 block comment should start with '# '

Change-Id: Ia2b8994c58686e3570f303f0725a2850d6b36e47
2014-06-18 14:59:18 +03:00
Bruno Semperlotti
fcfb66d468 Update Rally gates to support VM scenarios
Add "extra" directory for rally scenarios in gates. All files
from this directory will be copy pasted so we will be able to use
them in gates, because the path will be determined.

Also, print information about test environnement like available
services, flavors, images, networks, security groups and keypairs.

Change-Id: I19dd861beb11fb7ffdcf353f69759d841e7e8a01
2014-06-18 14:25:37 +04:00
Jenkins
b6bb8fbcb9 Merge "Add command-line integrated tests" 2014-06-04 16:58:53 +00:00
Sergey Skripnick
24ef12bc78 Use plugins from project in rally-gate
Plugins must be stored in $PROJECT/rally-scenarios/plugins/

Change-Id: I751216e69c11e8743d5256c4e8e5b1bfa193091e
2014-05-27 14:20:31 +03:00
Sergey Skripnick
b194c2e2fe Add command-line integrated tests
Change-Id: I75b4efa104cb1991a439a891c2135af286d2283f
Related: blueprint rally-gate-jobs
2014-05-14 13:52:21 +03:00
Joe Gordon
36afe95c2a Don't run rally in debug mode in ci
Running rally with debug mode can result in extremely large console
output.

Change-Id: I1fd8c226cc2bcbc25f8acbc97b4eb8e767e7ad89
2014-04-24 17:14:52 -07:00
Sergey Skripnick
b0a2a14e47 Add scenarios directory for ci
This directoy contains scenarios to be run by jenkins.

Add script which runs specified scenario and plot graph.

Add abbility to specify cloud by --name in "rally use deployment"

Removed rally/exercises

Change-Id: I7803a6950ec5fce3be11277ae0e4d830f0cf6bad
Related: blueprint rally-gate-jobs
2014-04-23 20:45:52 +03:00
Sergey Skripnick
a822308969 Skip large-ops benchmarks
This benchmarks should be tested in another job.
Also small fix in devstack-integration script

Change-Id: I31944026dfbb4ffb3dc6cf791208cfaffb1a0364
Related: blueprint rally-gate-jobs
2014-04-07 23:16:06 +03:00
Kun Huang
2acebe6a71 move install script to project root dir
This is helpful for new guys who can't or wouldn't find this file.

Change-Id: I9239a388ba7831a4f6c9b70e938a805903238796
2014-04-06 11:59:15 +08:00
Kun Huang
e81904a46d add +x to pre_test_hook.sh
Change-Id: I199c208db47ec35c5f367908265b807040884a14
2014-04-03 15:09:28 +08:00
Sergey Skripnick
3eee106094 Add tests_ci/pre_test_hook.sh
This script is executed inside pre_test_hook function in desvstack gate.
It installs rally devstack integration and exports necessary environment
variables.

Change-Id: Idf6d09ca33337c3b4a32912048653c13f53e2cf3
Related: blueprint rally-gate-jobs
2014-04-02 15:56:57 +03:00
Sergey Skripnick
b2988f06f6 Fix dependencies for rpm based systems
Add gmp-devel to rhel dependencies in install_rally.sh
Also add +x permissions to test_install.sh

Related: blueprint rally-gate-jobs
Change-Id: I9effe4d70e6f7d43b52640cea96bf129071064bb
2014-04-01 21:39:56 +03:00
Sergey Skripnick
276a235b0f Add shell script for integrated tests
This script does rally installation check

Change-Id: I85c37416ef4cb2daaddedde49fd0d6c820298fc8
2014-03-25 16:58:46 +02:00