Bash completion script is installed in /usr/local/etc, so
install_rally.sh should make a symlink in /etc/bash_completion.d/
Also move `data_files` section from setup.py to setup.cfg
Closes-Bug: 1463038
Change-Id: Ifd5f69e41aa8121129d6314130385f48e1690843
*.pyc files break cover-script for situations like:
some/module.py -> some.py
So we should remove all *.pyc files before launching `testr --coverage`
Change-Id: Ica22a3cd1a26d27c01f314a48a7cd49a529552eb
* Install wget/curl in install_required_sw(), where all other required
software is installed.
* Fix and standardize exit. install_rally.sh used three different ways
to stop execution: abort(); die(); and err() + exit. die() has a
strange usage pattern, and there are several places that use a
two-argument die() which happily waits for stdin forever. In other
cases we had unique and exciting hacks to get around die()'s
strangeness. This fixes all of those, and makes everything either
use abort() or die() (correctly).
* Fix inconsistent say()/echo usage. This eliminates the rarely-used
say() function entirely.
* Quoted things that need quoted. Some other minor changes
recommended by shellcheck.
Change-Id: I1a9d0b2ff5daba02998245d5c7e624f46bd2242a
This is an almost complete rewrite of the installation script, mainly
directed to allow installation of Rally also from unprivileged users,
but also adding a few more options and checks.
More in details, the script now:
* can run as unprivileged user.
* support different database types
* allow to specify custom python binary
* always asks confirmation before doing potentially dangerous actions
(removing directories or installing software)
* automatically install needed software if run as root
* this script is self-sufficient: it automatically downloads Rally
from the git if needed
* if interrupted, cleans up the virtualenv and/or the downloaded repository
Co-Authored-By: Antonio Messina <antonio.s.messina@gmail.com>
Co-Authored-By: Sergey Skripnick <sskripnick@mirantis.com>
Change-Id: I574d4fd7cc7c8e71720a6a5ff8db22ee4fd4fc81
Closes-bug: #1393887
Script rally-gate.py is designed to work if RALLY_SCENARIO
does contain scenario filename with extension.
It should work if there is no extension in RALLY_SCENARIO
as done in OpenStack infra scripts.
Change-Id: I95c77ae7d9109c6f9cf6a12b1a56bb73357c7d70
Add --rally-debug flag to the task execution to show
rally logs in scenarios that are tested on gate.
Change-Id: I877cf605ead7e2e7172ef6e3d88c57942edb9b0e
New version of pbr 0.11 was released and it checks
that version in setup.cfg is bigger then latest tag
To avoid future issues we will let pbr calulate version
instead of hardcode them in setup.cfg
Temporary disable coverage job to merge this patch
Partial-bug: #1450731
Change-Id: I0c00c231cea1a0506107133803406eb56694f690
Allow only 4 lines non covered by unit tets
Currently we are testing patch against master, instead of HEAD^
which produce issue when checked commit is not after current rebase.
master
|
patch1
|
patch2
|
old_master <- your_patch
Proper way to check coverage is to do diff between
your_patch and old_master instead of master.
Change-Id: Ib0a177eafa9c7dcce799bde1c6b45d541fedde05
* Make decision based on missing lines not difference
between percentage of coverage.
* Fix issue in case when stash has elements but
there is no changes to stash. We pop stash only
if coverage job created it
* Changes in UX:
* Show diff of coverage reports if proposed patch
coverage is less than in master
* Better UI messages
Change-Id: I61bb5fff812628a163025eef0a6e2ed29cdd4353
We are not able to use relative imports for plugins that are
outside of rally package.
To fix this we should add to sys.path all directories with Rally
plugins.
This is quite easy fix. But in any case it includes functional test
that checks that this actually works in real life.
+ Fix of Rally gates to copy whole tree not just *.py files from
rally-jobs/plugins/
Change-Id: Ie53dc2ba5c32a5e83d84d0ea5703e2e04b10ff33
Closes-bug: #1446923
Instead of asking people to write unit tests let's just make
coverage job voting.
This can be implemented in very simple maner:
1) checkout to master
2) run coverage and get base line
3) checkout back
4) run coverage to get current coverage
5) If coverage was reduced more then 0.01% mark job as failed
Change-Id: Ib6bc4dbb11c69b107e6fcd6464a4041337ea1a66
Changes:
- Replaced 'use deployment' with 'deployment use' in various files
as the former command has been deprecated.
Change-Id: I5f5779d6b6f36fa4f21cfe257ba88c2d9fce0d57
Closes-Bug: #1433683
This script will run scenario specified by $RALLY_SCENARIO
environment variable.
Also tags now supported. Tags are stand for modifying
deployment or environment somehow.
Currently implemented only "v3" tag. This tag is changing
deployment endpoints to use keystone v3 protocol.
For example we want to run two scenarios: one for keysone v2
and one for v3. Then we need to create two files:
my-rally.yaml # will be run vs keystone v2 (v2 is used by default)
my-rally_v3.yaml # will be run vs keystone v3
And export $RALLY_SCENARIO=my-rally.yaml
Change-Id: I44d1788e3892be59afbf8eaecb5d3e3b995fc3a8
Add alias "rally task use" for "rally use task",
"rally deployment use" for "rally use deployment"
and "rally verify use" for "rally use verification"
Change-Id: I6e63f4b60e1faf794f050f98be127744fe8011a4
Implement task templates based on jinja2.
This allow us to pass as a task jinja2 template
and it's argument via arguments --task-args and
--task-args-file that should be dict in JSON or YAML
presentations.
So now command looks like:
rally task start <file> --task-args <template-args-json-or-yaml> \
--task-args-file <file-with-args-in-json-yaml>
If both --task-args and --task-args-file then file dict is updated
by task args file.
Extend rally CI performance job. Now we can set template args
via file with name: ${TASK}_args.yaml
Bonus:
* Better message on InvalidTask format
* Remove redudant catch of "keyboardinterrupt"
it should be implement in different way.
* Replace ' -> " in rally.cmd.commands.task
and tests.unit.cmd.commands.task
* Imporve a bit CLI messages on rally task start
* Remove old plot2html command (it's enough deprecated)
* Improve test coverage of rally/cmd/commands/task
* Fix rally/cmd/commands/validate return 1 if bad format
* Write errors to stderr (in whole cmd/commands/task.py)
Change-Id: I7dadf2986bb10407865bc73bb2fb8c96a5162d9a
Recently, we added new job "gate-rally-dsvm-verify" for Rally[1].
This job implied functional testing integration Rally with Tempest[2].
This patch adds tests/ci/rally-verify.sh script, which is used by new job.
rally-verify.sh does:
- tempest installation
- run "rally verify start" twice and print results
- compare results of two verifications
- list verifications
- generate html page based on results
To implement gate-rally-dsvm-verify some changes were requered to
existing Rally code:
- Added ability for rally/ui/utils.py to accept arguments to render html-pages
- Fixed logging debug-messages in tempest verifier
- Fixed check "is debug mode turned on or not"(also, added hacking rule for it)
TODO for future patches:
- add launch of rally task for Tempest
- add launch of random test set
- add check for successful tests
[1] https://review.openstack.org/#/c/137232
[2] https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler
Closes-Bug: #1400465
Closes-Bug: #1400518
Change-Id: I8e1fbab22c2da109bbc442f040fe259e5d22a62a
The overview page shows all scenarios and their summarized results
in single table, so it is easier to understand the whole result
and compare specific scenarios.
Also, there are some minor features and fixes in html report.
Changes:
* Benchmark overview page
* Task input file is available now
* Scenario full duration is added to overview
* Scenario duration is renamed to `Load duration'
* Overview is sortable
* Fix: control from browser history: back/forward buttons now work
* Fix: durations are reset for iterations with errors
* AngularJS version is updated to release 1.3.3
* task sla_check output: column `success' with options True/False
is renamed to `status' with options PASS/FAIL
Change-Id: I0eb7af01432c9c10e4ec55720bb53417478a5789
This patch renames misleading rally-scenarios to rally-jobs.
rally-jobs describes much better purpose of this directory
Add readme files that describes structure and meanings of files
and directories
Change-Id: I8f5a2816186a2cafd251981ba7ad50e631e548f5
Closes-bug: #1395502
This patch adds UI templates directory and utils,
which serve and unify HTML generation.
Also, there are some fixes and improvements for HTML task report.
In this patch:
* Base mako templates (package rally.ui)
* Rework tests/ci/rally-gate templates
in order to use base template with generic header and styles
* Show scenario errors (if any) in task report tab
* Show scenario output (if any) in task report tab
* Show SLA data in Overview tab
* Show total scenario duration value (after the scenario name, above tabs)
* If got some iteration error, save exception class name in the database
instead of its repr()
* Prevent layout from breaking and show proper message if JS libs
can not be loaded for some reason
* Fix bug 1387661 - the cause of the bug is wrong input json data,
generated by plot.py. This happens when some atomic actions data
missed (which is a result of scenario errors) - and we have different
atomic actions sets between iterations. The fix saves atomic actions
integrity by adding missed atomic actions (with 0 value).
* Fix: if unexistend task uuid is specified in `task report' command,
then proper exception is raised
Closes-Bug: 1387661
Change-Id: I4bcbf86e6fad844e6752306eb6c1ccfefa6c6909
Provide plugins loading for runners and contexts, improve samples.
Plugins now can be loaded from ~/.rally/plugins and /opt/rally/plugins
and any it's subdirectories.
Closes-Bug: #1320942
Change-Id: I213086e7eae17e297c398c301f5ed8d2331d951e
CLI command `task plot2html' is renamed to `task report',
however old name is still available for compatibility.
HTML output of benchmark results is reworked:
* scenario select dropdown is changed to aside menu,
with scenarios methods grouped by classes
* initially select scenario from URL hash
* scenario results are divided into tabs
* new layout and styling, from bootstrap framework
* jQuery is removed
* AngularJS is used for rendering all the data
Change-Id: I00db7016900952f397681045ff99d07a0714fafe
