This patch contains:
- scenario NeutronNetworks.create_and_list_subnets is added
- method base.Scenario._generate_random_name is added
to provide a unified generation of random names.
- a piece of refactoring for scenario
NeutronNetworks.create_and_list_networks
blueprint benchmark-scenarios-for-neutron
Change-Id: I4ba67eb3babcf48e1842904c9e39b22a01351a8b
Several new scenarios added in tempest benchmark:
* scenario for launching all tests
* scenario for launching all tests from given set(special set_name
validator was added for this scenario)
* scenario for launching all tests from given list of test names
(Also, test coverage for validator `tempest_tests_exists` was increased
for situations like in this scenario)
* scenario for launching all tests which match given regex expression
bp benchmark-scenarios-based-on-tempest
Change-Id: I3e715cb360dec3d5d8683a9001c4f2221b49d1ac
Add 'use_public_urls' option to the deployment config
This option will be used later for switching access method to the cloud.
Because in some use-cases cloud may be accessible only by public endpoints.
Add 'admin_port' option to the deployment config
This option defines administrative keystone port on the public network.
Closes-Bug: #1306448
Change-Id: Ifb838c5fe9eab5e81353d697aa747bd8adf77572
1) Revert direct usage of subunit
Test repository should be used as launcher for tempest tests, since direct
usage of `subunit` broke execution of `compute` set as well as `full` set.
Tests discovery and execution can't be split at the moment.
2) Cleanup of tempest folder afterfaulty installation
Also, execution of tempest sets cannot be correct without confidence in
correct installation of tempest repository. Cleanup of tempest folder after
faulty installation should be added to fix check of tempest installation.
3) Length of database field for verification results
MutableJSONEncodedDict limited by 64k. This length is not sufficient for
storage results of those verifications, which have a huge number of failures
and tracebacks.
4) Incorrect usage of mock in tests
Class `mock.MagicMock` doesn't have method `assert_called_once`.
Instead of this one, method `assert_called_once_with` should be used.
Based on the fact that this is critical error in testing, this patch
affects not only tempest verification.
Change-Id: I68660e38f2cae3f60a01e99cdbbb57bdb8a9bb04
Closes-Bug: #1313744
Closes-Bug: #1313742
Closes-Bug: #1305991
test_verify fails when there is no internet connection. Fixed this issue
by mocking the urlopen which downloaded the image from internet.Also included
a try except block to detect the error and raise exception when downloading the
image when no internet is detected.
Closes-bug: #1308466
Change-Id: I29095eed8a4ff05720567223f2cb1f834b1d8f26
* using stopall() to stop patches avoids race conditions
that might arise when mock state is not cleaned up
between test cases
* use of stopall() from test case cleanup is recommended
when applying patches as part setUp()
Change-Id: I7918cc8c7c96a25ae874ea8bede93719d6baa634
Load plugins for scenarios from directories
/etc/rally/plugins/scenarios and ~/.rally/plugins/scenarios
Change-Id: Id3bbd7068b5dbe01d5107691d235be2109ae6a2a
Moved context/cleaner to its own package, called cleanup, under
benchmark.context and renamed it to cleanup.py. To improve cohesion,
splitted benchmark/utils.py into two, and all cleanup related functions
were moved to benchmark.context.cleanup.utils. Removed cleanup function
in cleanup.py and references to it, since it's obsolete. Finally updated
all references and unit tests accordingly to reflect the changes.
bp benchmark-context-cleanup-refactor
Change-Id: Iaffc4e35e21b734a7b4cb77ffaf810112054ebae
There is a new support library named 'oslotest'
extracted from oslo incubator. So, we add it to
test-requirements.txt and remove deprecated modules
from rally.openstack.common
Change-Id: I1d9efaf2f8a5a4948c454ea3f2e75262479133e8
Here we create a special rally.benchmark.processing.utils module for
computing a couple of useful functions on the benchmark data.
Change-Id: I9f3e6e50b9ba37946fbb56d7546b87b30beb0233
Neutron client accepts region_name parameter, so it should be passed to
neutron as it is to other clients.
Change-Id: I29cba149c8aa7d832a4f2f6b68878af82dbe5fd0
This patch adds new consts class for runner type and also fixs some
missing place of old runner type
Change-Id: I3dc75b31bae1a053ed705049c4c40d98c5e79679
This directoy contains scenarios to be run by jenkins.
Add script which runs specified scenario and plot graph.
Add abbility to specify cloud by --name in "rally use deployment"
Removed rally/exercises
Change-Id: I7803a6950ec5fce3be11277ae0e4d830f0cf6bad
Related: blueprint rally-gate-jobs
For testing heat needs to have users with special role. This patch introduce
simple implementation of new role context. It gives ability to add specific
roles to all users.
Implements: blueprint roles-context
Change-Id: I1908acf5ec970acd075077e8007bd9624e2c58a9
If you're in a private network, you need to setup a proxy
to access the public network. It would be a difficult thing
to set-up proxy for git:// protocol, while much more easy to
use https:// protocol instead.
Change-Id: Iec8447cc28760a98e1641162c50c152c89100775
This patch provides bases for tempest benchmarking:
1) Modified rally verification for launching tempest scenarios from benchmark
- moved to usage subunit as a test launcher;
- changed access to several methods;
- changed logging.
2) Added TempestContext, which checks tempest installation, initialization
of testrepository and existence of configuration file for tempest.
Also provides launcher for cleanup cloud after benchmarking.
3) Validator for tempest scenario names.
4) TempestScenario with launcher for single tempest test.
bp benchmark-scenarios-based-on-tempest
Change-Id: If67970a49674750b88d27e77c06d89b4e54d5337
There would be two kind of "users" appears in the task
configuration file when we're using runner "constant".
One in "context" means the number of openstack users,
while the other in "runner" simulate the activities
of multiple "human users", which actually controls the
number of concurrent scenarios.
Rally user might easily be misleaded.
Change-Id: I4333885d877364fbd19279cf338b758abbb804f7
1. Change "time" to "duration".
Because "time" and "times" are too close, and have
different meaning in rally. While, "duration" is much cleaner,
has only one meaning that means actually what we have in results.
2. Change "atomic_actions_time" to "atomic_actions"
Because not only "time" include in this column, and we already
have "duration" inside it.
3. Similar changes for functions and class members
Change-Id: Id3fa44451184603c7f044296ec6f1a9b75f236fc
