- Adds new median metric both on CLI and HTML reports.
- Adds supports for table header that consolidates
units from each metric, and gives a title.
- Re-organizes metrics based on order statistics
(min, percentiles, max) then its followed by
aggregate statistics metrics.
- Includes unit tests.
Change-Id: Icd154c9f00b5e9d6df11797685fe8f92c4fdbb1b
This patch makes output of `tox -epy34` to finish with following message:
> py34: commands succeeded
> congratulations :)
Issues:
* module "__builtin__" was renamed to "builtins" in Python 3
Related modules:
- rally.api
- tests.unit.benchmark.scenarios.vm.test_utils
* function "map"/"filter" returns "builtins.map"/"builtins.filter" object
in Python 3 instead of list in Python 2. "builtins.map" and
"builtins.filter" object is not subscriptable and has no len(), so list
comprehension is preferable to use and py2/py3 compatible way.
Related modules:
- rally.benchmark.context.sahara.sahara_edp
- rally.benchmark.processing.plot
- rally.benchmark.sla.base
- rally.benchmark.types
- rally.cmd.commands.task
- rally.cmd.commands.verify
- tests.unit.benchmark.scenarios.test_base
- tests.unit.benchmark.wrappers.test_keystone
- tests.unit.cmd.commands.test_task
- tests.unit.cmd.commands.test_verify
* dict.keys()/dict.values() returns "dict_keys"/"dict_values" object in
Python 3 instead of list in Python 2. so list(dict) and
list(dict.values()) should be used instead.
Related modules:
- rally.benchmark.scenarios.utils
- rally.benchmark.scenarios.vm.vmtasks
- tests.unit.cmd.commands.test_show
- tests.unit.common.test_broker
- tests.unit.deploy.engines.test_fuel
- tests.unit.fakes
* Some changes was made in Python 3 related to data model, so we should
change our inspect code. See code changes for more details
Related modules:
- rally.cmd.cliutils
- rally.common.utils
* ConfigParser is more strict for duplicate items in Python 3, so
duplicates are removed
Related files:
- rally/verification/tempest/config.ini
* Exception object doesn't have "message" attribute in Python 3, so
if we want to get it, the most proper way is using "getattr"
Related modules:
- rally.verification.tempest.config
* "mock.MagicMock" is not sortable in Python 3, so we should add required
attributes to fix that.
Related modules:
- tests.unit.benchmark.context.test_base
* assertSequenceEqual assertation method was added in tests.unit.test to
compare sequence objects
Related modules:
- tests.unit.benchmark.context.cleanup.test_resources
- tests.unit.benchmark.scenarios.nova.test_utils
* function "range" returns "range" object in Python 3 instead of list
in Python 2.
Related modules:
- tests.unit.benchmark.processing.test_utils
* keyword arguments should be transmitted to self.assertRaises as kwargs,
not like a dict
Related modules:
- tests.unit.benchmark.scenarios.dummy.test_dummy
Additional changes:
* Python 2.6 was added to setup.cfg, since Rally supports it.
* py33, py34 environments were added to tox.ini
* wrong ignore path was removed from tox.ini
* made items of bash complition sorted
Several tests are skipped in Python 3 env. For more details see notes in code:
- tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_main_time
- tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_atomic_time
- tests.unit.common.test_utils.MethodClassTestCase.test_method_class_for_class_level_method
During porting Rally to Python3, several issues found and fixed in
TempestContext and its unit tests:
- If process of cleanup is failed, exception is handled and cleanup is
marked as successfull. This issue was fixed and CleanUpException was
added to rally.exception module
- Cleanup was called with wrong path.
Change-Id: If04e873790dcb4c9c882d4be4bf40479deedd36d
In python 3, the raise syntax no longer accepts comma-separated
arguments, instead working with exception instances.
Also, python 3 is more strict, tuple must be created explicit.
Change-Id: Ib56b6bdf9266c0a2ed410729dc1f33ca08921005
Closes-bug:#1405145
The overview page shows all scenarios and their summarized results
in single table, so it is easier to understand the whole result
and compare specific scenarios.
Also, there are some minor features and fixes in html report.
Changes:
* Benchmark overview page
* Task input file is available now
* Scenario full duration is added to overview
* Scenario duration is renamed to `Load duration'
* Overview is sortable
* Fix: control from browser history: back/forward buttons now work
* Fix: durations are reset for iterations with errors
* AngularJS version is updated to release 1.3.3
* task sla_check output: column `success' with options True/False
is renamed to `status' with options PASS/FAIL
Change-Id: I0eb7af01432c9c10e4ec55720bb53417478a5789
This patch adds UI templates directory and utils,
which serve and unify HTML generation.
Also, there are some fixes and improvements for HTML task report.
In this patch:
* Base mako templates (package rally.ui)
* Rework tests/ci/rally-gate templates
in order to use base template with generic header and styles
* Show scenario errors (if any) in task report tab
* Show scenario output (if any) in task report tab
* Show SLA data in Overview tab
* Show total scenario duration value (after the scenario name, above tabs)
* If got some iteration error, save exception class name in the database
instead of its repr()
* Prevent layout from breaking and show proper message if JS libs
can not be loaded for some reason
* Fix bug 1387661 - the cause of the bug is wrong input json data,
generated by plot.py. This happens when some atomic actions data
missed (which is a result of scenario errors) - and we have different
atomic actions sets between iterations. The fix saves atomic actions
integrity by adding missed atomic actions (with 0 value).
* Fix: if unexistend task uuid is specified in `task report' command,
then proper exception is raised
Closes-Bug: 1387661
Change-Id: I4bcbf86e6fad844e6752306eb6c1ccfefa6c6909
In this patch:
* responsive behavior of task report is changed:
now we have minimal width with scroll bars
and scenarios list is always on left side
* `bootstrap' framework css is removed - now its responsive grid
is useless, so other styles were reimplemented
* AngularJS, D3, NVD3 updated to recent versions
* Current tab is now a part of URL hash
* Expanding arrows are added on scenarios list
* Table row with `total' values is highlighted and always last
* Chart scope ends correctly on large count of iterations - e.g. if last
value is 1200 and limit is 1000, then scope ends with 1200, not 1000
* Fix: more accurate histograms calculation
Change-Id: Ie9ed98443a7f2163bc5fbf0d416b9bda8c5b3b6f
CLI command `task plot2html' is renamed to `task report',
however old name is still available for compatibility.
HTML output of benchmark results is reworked:
* scenario select dropdown is changed to aside menu,
with scenarios methods grouped by classes
* initially select scenario from URL hash
* scenario results are divided into tabs
* new layout and styling, from bootstrap framework
* jQuery is removed
* AngularJS is used for rendering all the data
Change-Id: I00db7016900952f397681045ff99d07a0714fafe
