The OSprofiler is a distributed trace toolkit library.
It helps to trace internal calls of Openstack services including RPC, DB
and WSGI.
This patch integrates OSprofiler in Rally. Rally can trigger the
profiling on a per-iteration basis. To do so a secret key
(profiler_hmac_key) is stored alongside the credentials and used to
initialize the profiler in the constructor of the scenarios. A
configuration parameter (enable_profiler) can disabled the profiling.
Note that in this patch we don't embed the full osprofiler report but
only a trace id. This trace id can be used to retrieve the full trace
from the osprofiler tool later.
Change-Id: I7602856d094e073fde80d287b4d92b5750aacc3c
Co-Authored-By: rcherrueau <Ronan-Alexandre.Cherrueau@inria.fr>
Implements: spec osprofiler
This command could be used to import json results
of a test into rally database.
example:
rally task import --file <json_file> [--deployment <uuid>] [--tag <tag>]
Change-Id: Ie2db6ef19d73ca49e0bf0a4e888f65c07cec3a6c
Default image in devstack was changed recently from uec to qcow2.
All rally jobs except rally-mos now need to be updated accordingly.
devstack changes:
* Id65ebae73b28da7185cb349b714b659af51ef77f
* I5c154ec25555e768954538fc22b4f5d5975b2deb
Change-Id: Ia4d4afb661d8bcc7c38b0a16b508294f789eef14
* Added --filename option to deployment recreate that
will update configuration from provided file.
Change-Id: Icefca03fd86ab8977d0910bf192187c713cb355f
We have a lot of information about verifiers and main horizontal table from
``rally verify list-verifiers`` command could not include all of it.
Co-Authored-By: Yaroslav Lobankov <ylobankov@mirantis.com>
Change-Id: If425e9c373483fa092ebbcefdaf9a46173275c8a
This patch allows us to add some tags to verifications when starting them.
When we have a lot of verifications, it is not a simple task to find the
desired verification from the list. But now we can filter verifications by
tags and it should help us.
Change-Id: I7c6c6a54f31100100b67a750d0381d88fa98dc96
1. Small bugs were fixed.
2. Missing doc strings were added.
3. Now verifier cannot have 'configuring' and 'configured' statuses.
It was done to support work with a few deployments.
Change-Id: I064e863f7597c015e8a8f7c64a4d3049d6baac95
The following changes were done:
recreate -> reconfigure
replace-by -> override
add-options -> extend
Also, some printed messages in CLI interface were improved.
Change-Id: I47bfe88fabc22992b2bcde4c7ea03961c34bc1b6
Two or more plugins can have the same name if they have different plugin
base classes. In the table we did not mention relations to base classes, so
it is hard to identify right plugins.
Change-Id: I31e5633af9a34d59216c281113e32c1839a6172f
Some time ago we added new entity "exporters". For task we created a command
"rally task export" which allows to export task results to external systems.
Exporters can "easily" extende by plugins.
In case of verification component, proper command wasn't created yet. While I
thing idea of exporters is good enough, implementation can be improved.
To simplify usage, entity "exporters" was renamed to "reporters" and
integrated with "rally verify report" command. Generation of regular rally
reports (like html, json) is done in the same way as in plugabble reporters.
Proposed interface:
rally verify report --uuid <uuid-1> [<uuid-2>...<uuid-n>] \
--type <reporter-name> --to <destination>
Examples of usage:
rally verify report --uuid some-uuid --type html --to /path/to/save
# such exporter is not implemented yet, but we expect it soon
rally verify report --uuids some-uuid --type elasticsearch \
--to https://username@passwd:example.com
Change-Id: I4fb38984a73f92503bf2988e509477b10b308cac
There is a possible case when something went wrong and verifier stuck in
configuring state. In such situation there is not way to do un-stuck it,
just full recreation of verifier.
New flag --force should help in such situations.
Change-Id: Icf41f53ec3f6a6a8952a2f5922e1f396ca705ffc
* Split task results into chunks with size
configured by raw_result_chunk_size parameter
* Removed load_finished event as not used anywhere
Change-Id: I09841d02c60ab66eebd851bdf1c0a6d5f9e5a7be
We are planning to rewrite the whole Verification component. It will be done
from scratch. To reduce diff in following patches, let's remove all redundant
"outdated" code now.
Co-Authored-By: Yaroslav Lobankov <ylobankov@mirantis.com>
Change-Id: I64dd3851e381324a8d5274029690b5cfd2333911
In subcommand(for example in verification) we have several groups of
methods(managements, launchers, results). It would be nice to split
these groups in help message.
Before:
compare Deprecated. Use `rally verify results' instead.
detailed Display results table of a verification with detailed errors.
discover Show a list of discovered tests.
genconfig Generate Tempest configuration file.
import Import Tempest tests results into the Rally database.
install Install Tempest.
installplugin Install Tempest plugin.
list List verification runs.
listplugins List all installed Tempest plugins.
reinstall Uninstall Tempest and install again.
results Display results of verifications.
show Display results table of a verification.
showconfig Show Tempest configuration file.
start Start verification (run Tempest tests).
uninstall Remove the deployment's local Tempest installation.
uninstallplugin Uninstall Tempest plugin.
use Set active verification.
After:
genconfig Generate Tempest configuration file.
install Install Tempest.
installplugin Install Tempest plugin.
listplugins List all installed Tempest plugins.
reinstall Uninstall Tempest and install again.
showconfig Show Tempest configuration file.
uninstall Remove the deployment's local Tempest installation.
uninstallplugin Uninstall Tempest plugin.
discover Show a list of discovered tests.
start Start verification (run Tempest tests).
compare Deprecated. Use `rally verify results' instead.
detailed Display results table of a verification with detailed errors.
import-results Import Tempest tests results into the Rally database.
list List verification runs.
results Display results of verifications.
show Display results table of a verification.
use Set active verification.
Also this change transforms all _ to - in cli methods names.
Change-Id: I292e71d159ee35e933119f7fb57209f071aa37d4
* ability to show several verifications results (this
deprecates command `rally verify compare')
* command `rally verify compare' is removed - new report
compares results in better way
* new, unified, AngularJS/Jinja2-based template
* new verifications results processing
* new module rally.ui.report which cares about
report generation (this is a place where code from
rally.task.processing.plot should be also moved)
Examples:
rally verify results\
--uuid <uuid> --html > single_verification_result.html
rally verify results\
--uuid <uuid1> <uuid2> <uuid3> --html > compare_3.html
Co-Authored-By: Oleksandr Savatieiev <osavatieiev@mirantis.com>
Co-Authored-By: Alexander Maretskiy <amaretskiy@mirantis.com>
Change-Id: I942e0d9bf2094f3254dbeccbaa76dbbc3a3ca40e
Magnum renamed baymodels and bays to cluster_templates and clusters
respectively [1].
[1] https://blueprints.launchpad.net/magnum/+spec/rename-bay-to-cluster
Change-Id: I4390406d45a705cf93f8eb6522ebe4a65e468d2c
Partially-Implements: blueprint benchmark-scenarios-for-magnum
Some config options have changed their help text like 'sqlite_db',
so run 'tox -e genconfig' to generate latest config sample file.
Change-Id: Ifad41b2e1203308c02392ad63e3e90853dbbca68
Measure time to create a bay
In this scenario, a single bay is created. This can be used to establish
the baseline in a test environment.
The job yaml file describes the attribute of the bay to be created in
the scenario, such as node count. The baymodel should already be
created by the context: Kubernetes, Swarm, Mesos.
Change-Id: I182abf34212d693d2bb246d195a5daba61e0efe4
Co-Authored-By: Spyros Trigazis <strigazi@gmail.com>
Co-Authored-By: Ton Ngo <ton@us.ibm.com>
Partially-Implements: blueprint benchmark-scenarios-for-magnum
This patch set adds audit template context with two scenarios:
list_audit_templates
create_audit_and_delete
Change-Id: I2d3adef7893fbef1b90e69f56818fbe4d3ce4366
Tempest supports a mechanism that allows us to plug in various tests as
a Tempest plugin. Each plugin needs to be configured in the proper way.
Rally cannot configure all plugins on the planet. We should add some
option that will allow us to add extra configuration for plugins to
Tempest config file.
For example, if we want to run Ceilometer tests, the workflow will be
as follows:
$ rally verify install
$ rally verify installplugin --source https://github.com/openstack/ceilometer.git
$ rally verify genconfig --add-options /path/to/config/file/for/ceilometer/tests
$ rally verify start --regex ceilometer
where config file for Ceilometer tests is something like this:
...
[service_available]
ceilometer = True
event_enabled = True
...
Change-Id: Ida02a36e2757a742ca91919cc756846f137a23d3
This patch deprecates the following args for `rally verify start` cmd:
Old New
tests-file -> load-list
xfails-file -> xfail-list
So now we have consistent args that allow us to manage the workflow for
a certain list of tests:
1. Arg 'load-list' - Run tests listed in the named file
2. Arg 'skip-list' - Skip run of tests listed in the named file
3. Arg 'xfail-list' - Consider tests listed in the named file as
expected failures
Change-Id: Ida1252172c7edf7e7dd5350123a52c085867ba0f
This command allows us to uninstall a Tempest plugin.
E.g.
$ rally verify uninstallplugin --repo-name hello-world-tempest-plugin
Also, the patch has the following changes:
1. Now Tempest is installed in the local env (without deps) when providing
the --sistem-wide argument. It is needed to list plugins installed in the
local env, not in Tempest virtual env.
2. Now plugins are installed in the 'plugins/system-wide' directory when
providing the --system-wide argument. It is needed to separate plugins
installed in the local env and in Tempest virtual env.
Change-Id: Ic30a2705edb321a8d5af70e9d2bf5d5d5d9e8781
This command allows us to list all Tempest plugins.
E.g.
$ rally verify listplugins
+----------------------------+--------------------------------------------+
| Name | EntryPoint |
+----------------------------+--------------------------------------------+
| hello-world-tempest-plugin | hello_world_tempest_plugin.plugin:MyPlugin |
+----------------------------+--------------------------------------------+
Change-Id: Ifd53afd749ac47893b0fa11168b30d2536bf89fa
Sometimes we need to run the full test suite without some tests.
Now the new '--skip-list' argument allows us to do it. In order to skip
the desired tests we just need to list the tests in some file and provide
the '--skip-list' argument with the path to the file:
rally verify start --skip-list /path/to/the/file
Also, missing messages about incompatibility of arguments were added.
Change-Id: I882bfacd2e39bed2b54af4092752f642c964975a
This command allows us to install a Tempest plugin from a repository.
E.g.
$ rally verify installplugin --source https://github.com/MBonell/hello-world-tempest-plugin
$ rally verify installplugin --source /home/ubuntu/my-plugin
Implements Blueprint: install-tempest-plugins
Change-Id: I03ac6063b6b9d7f860379b99f3b524adedba6df9
Moved the old image and role options to the [tempest] section and added a
few flavor related config options. With these new options it is possible
to configure the RAM size of the two flavors used when running tempest
and the RAM size of the orchestration instance type flavor.
These were previously hardcoded to 64M, 128M and 64M respectively which
may cause the test image to fail under certain conditions, due to lack of
memory (e.g. AArch64 VMs seem to fall into this category).
After applying this fix, the three ram sizes and all other tempest related
options can be configured in the [tempest] section of /etc/rally/rally.conf.
Closes-Bug: #1580695
Change-Id: I33b8682310a42e66b408802a0d955fc886f6c8ed
Signed-off-by: Ciprian Barbu <ciprian.barbu@gmail.com>
New HTML report for statistics trends for given tasks
which is generated by cli command "rally task trends".
Blueprint: trends-report
Change-Id: I0805058e8bd225796b02516fad094d73037d5495
For now Rally always installs Tempest from the master branch. It is not
very convenient because Tempest may be broken in the master branch and
it would be very useful to use some commit or tag for Tempest.
So this commit adds a possibility to checkout the specified commit or
tag before Tempest installation.
Change-Id: Icfbb4221005466139c14d3f4aae34bbb3e013982
When we request a list of images to choose one of them for tests,
we should make sure all images are active, they have names and they
are PUBLIC. If images are not public, we will have failures of Tempest
tests as described in the bug.
Closes-Bug: #1564431
Change-Id: Ia2cc1586956dcf1fff4c7859faf39fdd98ba35e2
This is the first task exporter plugin. It used to export rally task
results to the file in json format.
Change-Id: I25117933ac0881453a001d96600d4c95e6064fff
