Here we create a special rally.benchmark.processing.utils module for
computing a couple of useful functions on the benchmark data.
Change-Id: I9f3e6e50b9ba37946fbb56d7546b87b30beb0233
There is a new support library named 'oslotest'
extracted from oslo incubator. So, we add it to
test-requirements.txt and remove deprecated modules
from rally.openstack.common
Change-Id: I1d9efaf2f8a5a4948c454ea3f2e75262479133e8
This directoy contains scenarios to be run by jenkins.
Add script which runs specified scenario and plot graph.
Add abbility to specify cloud by --name in "rally use deployment"
Removed rally/exercises
Change-Id: I7803a6950ec5fce3be11277ae0e4d830f0cf6bad
Related: blueprint rally-gate-jobs
The createfromenv deployment function uses the environment variable
OS_REGION_NAME, but the tests did not correctly mock this variable.
Consequently setting this env var before running tests would cause the
tests to fail. This patch corrects the mock.
Change-Id: I15fe174cd9700ff34791c6980fa52ff975055e02
Closes-Bug: #1308029
Class TempestConf is added that generates tempest configuration file
for specific OpenStack deployment.
Co-Authored-By: Alexander Maretskiy <amaretskiy@mirantis.com>
blueprint tempest-improve-autoconfig
Change-Id: Id0a33998f99820f8a108305eea14b9bb9918c102
When there are multiple regions shared with one keystone, we should be
able to specify the region for the endpoint.
Change-Id: I18bb2c7630d0b1927969f6e06f30998644902020
Implements: blueprint region-support
Command `rally verify list` displays list of verifications
Command `rally verify results` prints raw tests results
Command `rally verify show` prints verification results.
Command `rally verify detailed` prints verification results
with traceback of failed tests.
Blueprint tempest-verification
Change-Id: I19991219c2f184c5158fecc0d234c7a40031e092
* Scenario methods now use @with_default_task_id which initialize
the task_id parameter: it sets task_id to the 'RALLY_TASK' env var
available in the global file ~/.rally/globals
* Added @with_default_task_id global to use @default_from_global
decorator with task_id, so we can use directly @with_default_task_id
and don't have to use @default_from_global that requires parameters.
Implements blueprint: cli-use-task-subcommand
Change-Id: I3fdd80b0fa89354ba24aa78b2b7db84d8a6d58c1
* Since envutils.get_global doesn't raise anymore 'InvalidArgumentsException'
(due to this fix https://launchpad.net/rally/+bug/1291298) we are encountering
issues when get_global simply returns None. That's why the bool parameter
doRaise was added to get_global, so when needed the method will raise
'InvalidArgumentsException' and in the case of #1291298 get_global won't raise
any exception and will simply return None.
* Fixed unit test of Tempest Install command, deploy_id was always None in the
unit test and once the above modification applied, an InvalidArgumentsException
was raised which caused the failing of the test.
Change-Id: I18e2ac99e6412c3ae67769390e390d4210243381
Closes-Bug: #1293454
* The decorator is now more generic and is able to manage any default id.
* Refactored _deployment_default_id in the generic function _get_global
The function was simplified by avoiding the use of 'except KeyError' and
it makes it more readable.
Change-Id: I4b822f954059f604cf5248550df1e20ef94a8d64
*) Change task config format
. Split "context" & "runner" stuff
*) Refactor Verification
. Move validation to context.base, runner.base and scenario.base
. Validate whole config fully before starting any of tasks
. Optimize scenario args validation (create only one time clients)
. Optimize order of validation:
1) Validate names of benchmarks
2) Validate all static parameters, e.g. configuration of runner
and context
3) If everything is ok in all benchmarks, then start validation
of scenario args.
. Store validation result (exception) in task["verification_log"]
. Remove verification logic from BenchmarkEngine.__exit__
. Remove scenario args verification results from task["results"]
*) Fix & Swtich to new format doc/samples/tasks
. Switch to new fromat
. Add missing task configratuion
. Better formatting
. json & yaml samples
*) Refactored unit tests
. tests.rally.benchmark.test_engine
. tests.rally.benchmark.context.base
. tests.orcestrator.test_api.start_task
cover validation step as well and new change format
*) Refactor orchestrator api start task
. Remove benchmark engine context
. Call verify explicity
. Do not raise any excpetion in case of validation error
. Catch in start task any unexcepted Exceptions a set deployment
in incosistance state
*) Refactor CLI
. Properly handle new behaviour of verification
. Replace table on task start to just message
. Add HINTs to task detailed command
*) Add unit test for checking doc samples
*) Improve benchmark engine LOGing
blueprint benchmark-new-task-config
Change-Id: I23d3f6b3439fdb44946a7c2491d5a9b3559dc671
Currently envutuils.default_deployment_id raises an exception if there
is no active deployment (one that has been recorded previously with use
ore create). This patch changes the method to return None instead.
Change-Id: I57052e5d4862d80b95a45eb582702ce009a4f22f
Closes-Bug: #1291298
* Lazy load client handles instead of eager instantiation.
* Update all client handle references to use osclients.Clients.
Change-Id: Iffd17eb37886ca07190d1e2fbb6c02ff6081f69a
When running "rally use deployment", use try...except to avoid exposing
exceptions to end users.
Change-Id: I5b005e140784374f9a2d68da9b32fa57b2ca86a8
Closes-Bug: #1289789
When running 'rally deployment list' curent active deployment is being
higlighted by a "*" on a new column called "active". Active deployment id is
comming from envutils._default_deployment_id() so this function is now public.
Change-Id: Ib911f10ee320d1ae6536f69587361fe1e35d8208
Trello: https://trello.com/c/d4wMvMsl
Add unit tests for the methods of the ShowCommands class that it has
been committed without tests.
Change-Id: I94e3f4bc8c166a612c653065c6f032eb5d71a231
This would add support to tag a task with a human friendly name.
e.g.
rally task start --task doc/samples/tasks/authenticate/keystone.json \
--tag "keystone test"
Change-Id: I93aa43b12a6a6e2488c3db223e65337d2556b388
As per trello request: https://trello.com/c/tqjav3dY remove task.plot function
and related rally/processing.py and tests/test_processing.py files.
Change-Id: I255e13db294928c76d42541cea24e001b2f41071
Added a new subcommand in use command: 'rally use task <task_id>'
which sets a RALLY_TASK global var in ~/.rally/use_command
Hence the user does not need to specify a task UUID in commands
requiring this parameter.
Partially implements blueprint: use-task-subcommand
Change-Id: I51fffa017c56321493d1a5eba866d7163b9fc813
* Renamed 'deployment' filename in 'globals' in order to
use this file with other use subcommands such as 'task'
* Refactored by moving code into new generic functions that
* check if the 'use_command' file exists:
_ensure_rally_configuration_dir_exists(self)
* update env var in the 'use_comand' file:
_update_attribute_in_use_file(self, attribute, value)
Change-Id: I7734d34f99b468001bd431f98d97e4dade2236cd
Use tempest (https://github.com/openstack/tempest) inside rally to
organize base for verification functionality & for benchmarks scenario
based on tempest.
This patch adds ability to install Tempest tests (no parameters needs)
and to verify the specified cloud (takes a deploy_id as parameter, this
parameter is now optional). For now, there are smoke tests.
Usage:
rally-manage tempest install - install tempest tests
rally verify start --deploy-id=<deploy_id> - start cloud verification
Change-Id: I70bb37476dd2d12370ea77156adab3ceaddb93cb
Related: blueprint tempest-verification
Instead of storing the endpoints as a single admin credentials dictionary
(containing the username, password etc.) we should be able to store several
endpoints which can be possibly returned from the deploy object, as well as
the information whether these endpoints have the admin permission or not.
If not, then the deployment is going to be used with the predefined set of
users (that will be implemented next). If there is an admin endpoint, then
the users for benchmarking will be generated (the "admin" role).
This information now gets stored in a dictionary of form:
"endpoints": [
{"username": ..., "password": ...,
"tenant_name": ..., "auth_url": ..., "permission": "admin|user"},
{"username": ..., "password": ...,
"tenant_name": ..., "auth_url": ...,
"permission": "admin|user"},
...
]
Blueprint benchmarking-with-predefined-users
Change-Id: If36e33e89daa036bd4b0d7db8e2d3b5b5b0f4aca
Move all the currently supported rally commands (i.e. deployment, task,
use) and their equivalent tests that are described as classes in
the 'main.py' module (and 'test_main.py' module), to separate modules in
a new directory. This is done for organizing the code of the rally command
line utility, before it become too big.
Implements: blueprint rally-show-command
Change-Id: I5982f854dfe64604ae112d93fb7ea68f72657b98
I simply moved the logic of this method which was called at the
beginning of each method in a decorator
Implements blueprint: rally-use-command
Change-Id: I0fd41399933c0c12dbe7d58fe06dcba2452daf6d
Currently 'task detailed' shows status, config and results, but not whether
the task failed.
Change-Id: I9f58ebff0ef6dc289a317c482272403afe73c797
Trello: https://trello.com/c/Qjq5iQqW
This completes the behaviour of the 'rally use' command.
when the command is called, it creates a file called openrc-{deployid}
in ~/.rally/ which contains instructions to export deployment specific
environment variables
It also creates a symlink ~/.rally/openrc that refers to the latest
deployment called with 'rally use'
It becomes simple to just do 'source ~/.rally/openrc' and then
'nova list'
Implements blueprint: rally-use-command
Change-Id: I51ddfdbf7b8ab41b5b26685dd7b671468d306fe1
Call use deployment when a deployment is created so that the newly
created deployment is default for future operations.
Change-Id: I3d223336d221f4794b3912fea8a381a0c4934aa5
This saves users from creating dummy config json files when they already
have an openrc for use with the other openstack clients.
Change-Id: I487af60e0c6a5eadfc12f04fbba9c4a4186740e0
Trello: https://trello.com/c/5ZkMb9zL
This adds a new group of commands : 'rally use'
the first implemented is 'rally use deployment <deploy_id>'
As it is not possible in python to export globaly an env var,it saves the
variable in a file at ~/.rally/deployment like :
RALLY_DEPLOYMENT=1234abcd
For each command that takes a deploy_id as parameter, this
parameter is now optional.
The algorithm to find the deploy_id is as follows :
- If the --deploy-id param is set, use it
- If the environ contains the key 'RALLY_DEPLOYMENT', use it
- load the content of ~/.rally/deployment and if it contains the key
RALLY_DEPLOYMENT, use it
- Else, raise an InvalidArgumentException
Note that I have not been able to use the default arg in @cliutils.args
because it is called before the effective method call. Setting
deploy_id=envutils.default_deployment_id() has the same problem
Partially implements blueprint: rally-use-command
Change-Id: I1e62e26d259bd9e81f1c008058ee9ce91cc3290c
Summary line is printed when task has been created but not started,
detailed information is printed once the task has completed.
Change-Id: I8a3cbb81d7fccffe158517d3023ad47d28d7ebc1
Trello: https://trello.com/c/wJbH8DqG
Unit test for ‘results’ method in TaskCommands class was missing
followed advices from Ilya Kharin
Change-Id: I5a4f110933d855994172716b3607b0ef5e24599f
Change the output of the detailed command when this one is called
with a non-existing id. Instead of having an error stacktrace, we now
have a message saying "The task %s can not be found"
Change-Id: I889831b0efeba451df429680007bae52987e8271
Closes-Bug: #1270153
