* Change the api of the SLA classes:
- add_iteration() - processes a single iteration result from the queue
- result() - return the SLAResult based on the data processed so far.
It now also returns a successful result in case there was no iterations
data.
* Add a new SLAChecker class:
- add_iteration() - processes a single iteration result with different SLAs
- results() - return cumulative SLA results for all SLAs
* Change the benchmark engine so that it supports scenario runner aborts
on SLA failure
* Support the "abort on SLA failure" feature in the CLI:
rally task start ... --abort-on-sla-failure
* Modify the SLA detailed messages (cleaner text and less decimal places)
* Update unit and functional tests correspondingly. Also remove the usage of the
deprecated "max_failure_percent" SLA from the functional tests.
Change-Id: I91894a81649815428fd1ac7afcfce9cf47160fc9
In case an input scenario had no atomic actions, the "--iterations-data"
table contained "n/a" values. Here we fix this and enhance the corresponding
functional tests.
Change-Id: Ic4d3809f64bcfcbce804023de7ce0c2d1f6e965f
Closes-Bug: 1420703
Keypairs and secgroup data are now stored separately
for each user. Secgroup data is now a dict not Nova object.
Change-Id: Ic4bacad2ff41c4ddcb010e8203559c0dd3945a63
Use the new API classes in CLI and other parts of Rally code
(still leaving old API methods for compability).
ToDo in next patches:
* Remove old API methods
* add get(), list() methods to API.
Change-Id: I1eca518bca8b383e3e6ce05e99251c2d954f1b4e
Add alias "rally task use" for "rally use task",
"rally deployment use" for "rally use deployment"
and "rally verify use" for "rally use verification"
Change-Id: I6e63f4b60e1faf794f050f98be127744fe8011a4
This patch makes output of `tox -epy34` to finish with following message:
> py34: commands succeeded
> congratulations :)
Issues:
* module "__builtin__" was renamed to "builtins" in Python 3
Related modules:
- rally.api
- tests.unit.benchmark.scenarios.vm.test_utils
* function "map"/"filter" returns "builtins.map"/"builtins.filter" object
in Python 3 instead of list in Python 2. "builtins.map" and
"builtins.filter" object is not subscriptable and has no len(), so list
comprehension is preferable to use and py2/py3 compatible way.
Related modules:
- rally.benchmark.context.sahara.sahara_edp
- rally.benchmark.processing.plot
- rally.benchmark.sla.base
- rally.benchmark.types
- rally.cmd.commands.task
- rally.cmd.commands.verify
- tests.unit.benchmark.scenarios.test_base
- tests.unit.benchmark.wrappers.test_keystone
- tests.unit.cmd.commands.test_task
- tests.unit.cmd.commands.test_verify
* dict.keys()/dict.values() returns "dict_keys"/"dict_values" object in
Python 3 instead of list in Python 2. so list(dict) and
list(dict.values()) should be used instead.
Related modules:
- rally.benchmark.scenarios.utils
- rally.benchmark.scenarios.vm.vmtasks
- tests.unit.cmd.commands.test_show
- tests.unit.common.test_broker
- tests.unit.deploy.engines.test_fuel
- tests.unit.fakes
* Some changes was made in Python 3 related to data model, so we should
change our inspect code. See code changes for more details
Related modules:
- rally.cmd.cliutils
- rally.common.utils
* ConfigParser is more strict for duplicate items in Python 3, so
duplicates are removed
Related files:
- rally/verification/tempest/config.ini
* Exception object doesn't have "message" attribute in Python 3, so
if we want to get it, the most proper way is using "getattr"
Related modules:
- rally.verification.tempest.config
* "mock.MagicMock" is not sortable in Python 3, so we should add required
attributes to fix that.
Related modules:
- tests.unit.benchmark.context.test_base
* assertSequenceEqual assertation method was added in tests.unit.test to
compare sequence objects
Related modules:
- tests.unit.benchmark.context.cleanup.test_resources
- tests.unit.benchmark.scenarios.nova.test_utils
* function "range" returns "range" object in Python 3 instead of list
in Python 2.
Related modules:
- tests.unit.benchmark.processing.test_utils
* keyword arguments should be transmitted to self.assertRaises as kwargs,
not like a dict
Related modules:
- tests.unit.benchmark.scenarios.dummy.test_dummy
Additional changes:
* Python 2.6 was added to setup.cfg, since Rally supports it.
* py33, py34 environments were added to tox.ini
* wrong ignore path was removed from tox.ini
* made items of bash complition sorted
Several tests are skipped in Python 3 env. For more details see notes in code:
- tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_main_time
- tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_atomic_time
- tests.unit.common.test_utils.MethodClassTestCase.test_method_class_for_class_level_method
During porting Rally to Python3, several issues found and fixed in
TempestContext and its unit tests:
- If process of cleanup is failed, exception is handled and cleanup is
marked as successfull. This issue was fixed and CleanUpException was
added to rally.exception module
- Cleanup was called with wrong path.
Change-Id: If04e873790dcb4c9c882d4be4bf40479deedd36d
According to related blueprint, installing oslo
libraries into a same namespace seems to cause
several type of errors. Every occurrence
of oslo.foo supported is renamed to oslo_foo.
Change-Id: I786b0b824be31c08e6795201bb95553434d8d634
Implements: blueprint 'oslo-incubator/drop-namespace-packages'
Closes-Bug: #1409213
RALLY_UNITTEST_DB_URL may be used to change test database.
Sample usage:
export RALLY_UNITTEST_DB_URL="mysql://user:secret@localhost/rally"
tox -epy27
Fix postgress related bugs in unit tests.
* All enums should have name
* Fix invalid unit tests
Closes-bug: #1413155
Partial-bug: #1405884
Co-Authored-By: Sergey Skripnick <sskripnick@mirantis.com>
Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>
Change-Id: Ic741cb79eadf952c3dea99182c1e7d52992c9904
Implement task templates based on jinja2.
This allow us to pass as a task jinja2 template
and it's argument via arguments --task-args and
--task-args-file that should be dict in JSON or YAML
presentations.
So now command looks like:
rally task start <file> --task-args <template-args-json-or-yaml> \
--task-args-file <file-with-args-in-json-yaml>
If both --task-args and --task-args-file then file dict is updated
by task args file.
Extend rally CI performance job. Now we can set template args
via file with name: ${TASK}_args.yaml
Bonus:
* Better message on InvalidTask format
* Remove redudant catch of "keyboardinterrupt"
it should be implement in different way.
* Replace ' -> " in rally.cmd.commands.task
and tests.unit.cmd.commands.task
* Imporve a bit CLI messages on rally task start
* Remove old plot2html command (it's enough deprecated)
* Improve test coverage of rally/cmd/commands/task
* Fix rally/cmd/commands/validate return 1 if bad format
* Write errors to stderr (in whole cmd/commands/task.py)
Change-Id: I7dadf2986bb10407865bc73bb2fb8c96a5162d9a
Python 3 dict.items() doesn't support + operand, so in order
to keep compatibility, it's supposed to concatenate using
dict.update(). This patch adds validation to avoid the use
of + operand.
Co-Authored-By: Martin Pavlásek <mpavlase@redhat.com>
Change-Id: Ie23f93f3ab2ac1cd53da87d0b71f5bd71e0226df
Current Rally provides "create_and_list_volume" scenario, but doesn't
provide "list_volumes". User can specify number of volumes which will
be created in context.
Change-Id: I700282388b839d735448850e0629a0a37ab781f1
Added ability to build one report from
results of multiple tasks. Tasks may be passed as UUID or as previously
saved json results. Also it may be mixed list of UUIDs and file pathes.
Closes bug 1406585
Change-Id: I11f2ca3ee6b868b79f91a4fa95e6a9ea918a79b0
1) Module urllib2 is missed in Python 3, so it should be replaced by other
http lib. "requests" is good library for such stuff, which supports both
Python 2.* and Python 3.*
2) Function 'rally.common.utils.parse_docstring' uses built-in function
'filter'. This function returns different types in Python 2.* and
Python 3.*. Since results are used as a list, 'filter' should be replaced
by list comprehension.
3) Extend H330 hacking rule to chech 'dict.iterkeys()', 'dict.itervalues()'
and 'dict.iterlist()'. Also, fixed all places, which are failed in terms
of this rule.
4) Function object does not have 'func_code' attribute in Python 3, but
'__code__' attribute exist in both Python 2 and Python 3
Also, 3 tests are broken due to new release(1.6.0) of oslo.config, so we
need to skip them to unblock our gates until new version of oslo.config will
be released with appropriate fix.
Closes-Bug: #1405919
Change-Id: Icc42e220ac3f15ae6c838a4698b3c9578177513c
Python 3 introduced a next() function to replace the next() method on
iterator objects. Rather than calling the method on the iterator, the next()
function is called with the iterable object as it's sole parameter, which
calls the underlying __next__() method.
The next() function was backported to Python 2.6 which allows us to use either
the 2.x or 3.x way simultaneously.
Change-Id: I945225b1b0c6c39842a77f4a136e9431b798229f
Closes-Bug: #1403074
"string.lowercase" and "string.uppercase" were removed in Python 3,
we can use only "string.ascii_lowercase" and "string.ascii_uppercase".
Change-Id: I97ddac01af7070129ad8baf5960c841482c346a2
Closes-Bug: #1405918
In Python 3, module "xrange" doesn't exist any more.
To support both Python 2 and Python 3, "six.moves.range"
should be used instead.
Change-Id: I9dcf79011a4b2672e4ba55298aa22cab5e2292ff
Closes-Bug: #1403447
In Python 3, module "itertools" doesn't contain "imap" object.
To support both Python 2 and Python 3, "six.moves.map"
should be used instead.
Change-Id: I5b82020d0721a91aa6d6090a42073a3285b80ff9
Closes-Bug: #1403434
In Python 3, urlparse module is gone.
To support both Python 2 and Python 3, "six.moves.urllib.parse"
should be used instead.
Change-Id: I0adddbdea5a80606907a1b78ca1a04d15f61452b
Closes-Bug: #1403433
* Remove semantic validation mechanism for context. We will use same
approach as for current scenarios (via decorators)
* Pass "deployment" instead of "tasks" to validators. This allow us to
remove access to DB from validators + we can call validation without
creating tasks
* Add hacking rule to force using construction "from rally import objects"
* Fix unit tests
Change-Id: Ibf08183b28479cc49ab6adfd12298612f6b4397f
In Python 3, StringIO module is gone.
Class StringIO.StringIO is not available from io module.
To support both Python 2 and Python 3, "six.moves.StringIO" should be used instead.
Change-Id: I17e79d8625c75d6903f024c648d3ef62e21d0c51
Closes-bug:#1403076
There is no global variable "basestring" in Python 3.
"six.string_types" should be used to support both Python 2 and Python 3
Change-Id: I556fef0e98d00ab819e3dea961666b403b927b06
Closes-bug:#1403077