973 Commits

Author SHA1 Message Date
Yaroslav Lobankov
b54d143695 [Verify] Make total time like H:M:S (<time in sec> s) in HTML report
Change-Id: I23ed5e7ca91c15ccd2d5bf88fb7e89c915bafa07
2015-11-28 18:19:23 +03:00
Bo Chi
9d89acf8fc Add region_name when generate tempest conf
Change-Id: I4aa3152dd192a38b4ebb43d57c3088913c2d752f
Closes-Bug: #1517237
2015-11-27 12:49:28 -05:00
Jenkins
059dae4238 Merge "[Verify] Reworking unit tests for config.py [part 1]" 2015-11-26 23:04:51 +00:00
Chris St. Pierre
eb8f9dc976 Remove old random name generator
Kill the old random name generator in favor of the fancy new one.

Implements blueprint: consistent-resource-names
Change-Id: Ia612ebb3ed3af27c6aaa1e35f459b7980b7a24b9
2015-11-26 15:43:50 +00:00
Yaroslav Lobankov
d60b195e08 [Verify] Reworking unit tests for config.py [part 1]
1. The setUp() method was reworked
   because no one could understand how it worked.
2. Now ddt module is used where it is possible.
3. Some tests were refactored.

In some tests you will see that

    self.assertIn(sorted(expected)[0], sorted(result))

was replaced by

    for item in expected:
        self.assertIn(item, result)

because sorted(expected)[0] is only the first item from expected tuple.
So we always checked only one item from config section. The wrong behavior
was introduced by the following commit: I487806fe9a8ccd8528922cfaa2700df30961ef9b.
Now it is to be fixed.

Change-Id: Iaabc1a9fbd77e73bd9d28a30b29f9827a22e0bbd
2015-11-26 16:49:11 +03:00
Jenkins
a56448948c Merge "Remove old generate_random_name() use from sahara context" 2015-11-26 05:56:55 +00:00
Jenkins
a2a597951d Merge "Add orchestration section to tempest.conf" 2015-11-26 04:20:05 +00:00
Chris St. Pierre
3b931007dc Remove old generate_random_name() use from sahara context
Change-Id: Ic83e0b24a41fe718bb4f1e93acb954ac583c5eee
2015-11-25 19:49:47 +00:00
Jenkins
555ef74f55 Merge "Implement new random name generator for wrapper plugins" 2015-11-25 16:42:53 +00:00
Jenkins
2b120d3022 Merge "Rename "endpoint" to "credential" (part 1)" 2015-11-25 09:19:02 +00:00
Arata Notsu
ff12c115a0 Add orchestration section to tempest.conf
For heat, stack_owner_role and stack_user_role should be set.

Change-Id: I4fda6560b851dc03335d92318800b395fdaa4b4f
Closes-Bug: #1519672
2015-11-25 16:42:33 +09:00
Alexander Maretskiy
695dffec7f [Reports] Add an option for embedding JS/CSS libs into report
This change allows running command `report' with `--html-static'
option so all required JS/CSS will be embedded to HTML file.
Having all required CSS/JS loaded directly from HTML makes
report working without access to the Internet.

Added third-party libraries files:
    rally/ui/templates/libs/nv.d3.1.1.15-beta.min.css
    rally/ui/templates/libs/angular.1.3.3.min.js
    rally/ui/templates/libs/nv.d3.1.1.15-beta.min.js
    rally/ui/templates/libs/d3.3.4.13.min.js

Notes about licenses compatibility:
    https://angularjs.org/
    https://github.com/angular/angular.js/blob/master/LICENSE
    The MIT License
    compatible with Apache License Version 2.0
    according to http://www.apache.org/legal/resolved.html

    http://d3js.org/
    https://github.com/mbostock/d3/blob/master/LICENSE
    BSD 3-clause License
    compatible with Apache License Version 2.0
    according to http://www.apache.org/legal/resolved.html

    http://nvd3.org/
    https://github.com/novus/nvd3/blob/master/LICENSE.md
    Apache License Version 2.0

Change-Id: I8913ecc585ee17affb1d8a0b29bb475c1e07427f
Closes-Bug: #1505533
2015-11-24 11:33:25 +02:00
Bo Chi
ee73295e93 Rename "endpoint" to "credential" (part 1)
Copied rally/common/objects/endpoint.py to credential.py
Marked objects.Endpoint as deprecated
Modified use of objects.Endpoint to objects.Credential
Variable and key names are not changed in this patch

Partial-Bug: #1447682
Change-Id: Ia62881473d1bf63f952528b66e7420eea6323f08
2015-11-24 01:18:13 -05:00
Chris St. Pierre
edce6deb73 Implement new random name generator for wrapper plugins
This switches all wrapper plugins to use the new consistent random
name generator.

Implements blueprint: consistent-resource-names
Change-Id: I6b27fe2f41d84c1961db3b13cf8ddeeae27a6591
2015-11-23 08:53:34 -06:00
Jenkins
8b14eea91d Merge "[report] Fix the dummy mismatch in report.mako" 2015-11-23 14:07:58 +00:00
Yaroslav Lobankov
9d276cd0b5 [Verify] Fixing issue with SSL deployments
Now options 'disable_ssl_certificate_validation' and 'ca_certificates_file'
are configured in the Tempest config file and values for these options are
taken from credentials that user specified.

Config options https_insecure and https_cacert were removed since they were
marked as "deprecated_for_removal".

Change-Id: I74083bc9af21969dd836f852712b81abd3ec74dc
2015-11-23 12:54:11 +03:00
Kun Huang
fa89cdd28e [report] Fix the dummy mismatch in report.mako
Here is an example [0]. The location(sc.ref) in url is dummy-2, but the
real scenario is "dummy [10]"

This patch corrects the sort of task results.

[0]
http://logs.openstack.org/70/181270/52/check/gate-rally-dsvm-rally/431d2c5/rally-plot/results.html.gz#/Dummy.dummy-2/overview

Change-Id: I749c6e9290d67d60b230df1520f9da2415699179
2015-11-23 17:51:29 +08:00
Jenkins
0ca784d9f4 Merge "[Sahara] Move sahara data to sahara context" 2015-11-19 17:02:32 +00:00
Jenkins
4fa19e1544 Merge "Fix broken logic in some neutron scenarios" 2015-11-19 07:12:41 +00:00
Jenkins
93d2cfa907 Merge "[Sahara] Add Swift to output data sources" 2015-11-18 20:47:10 +00:00
Sergey Skripnick
b908f84cbd Fix broken logic in some neutron scenarios
_get_or_create_network: create networks only if there is no networks
in context.

_get_or_create_subnets: create subnets only if there is no subnets
in context.

Unit tests are passing because of bug in ddt [0]
This issue was actually caught by rally-ci (rally-pg-py27-unit) but was
ignored by revievers.

Running tests without testr may be used as workaround:

    .tox/py27/bin/python -m unittest discover tests/unit

[0] https://github.com/txels/ddt/issues/35

Change-Id: Ib77bd4f7eeaec2ed3a712e1d56a20d1f87325264
2015-11-18 21:28:04 +02:00
Evgeny Sikachev
dc4e5e96c9 [Sahara] Move sahara data to sahara context
Create sahara context for sahara tests

Change-Id: I38abdde9c8209c6e2b038e27e944d0a17c7071d8
2015-11-18 17:52:14 +03:00
Evgeny Sikachev
2195f8de9c [Sahara] Add Swift to output data sources
Added support of Swift in Sahara Data Sources. The context does not
verify the output of the job. The cleanup deletes the output container.

Change-Id: I7b4cdc14f6c8a6af227a953be298f266e5851c92
2015-11-17 13:37:04 +03:00
Jenkins
457fb7187e Merge "[Verify] Refactoring HtmlOutput" 2015-11-16 17:39:16 +00:00
Jenkins
1029e4cf24 Merge "Extend list benchmarks for ceilometer" 2015-11-16 16:38:12 +00:00
Jenkins
dda2a81116 Merge "[Verify] Fixing tokens expiration issue when deleting Tempest resources" 2015-11-16 14:18:38 +00:00
Andrey Kurilin
e5ed7697c4 [Verify] Refactoring HtmlOutput
There is no reason to use OOP for one simple method. This patch replaces
class HtmlOutput with simple function "generate_report".

Also, it adds an improvement for skip reason: now ID of a bug is replaced
by the launchpad link to it in HTML report.

Change-Id: Ia75578a67a1fe626b80b877bbda659b5e6070ef0
2015-11-16 16:25:34 +03:00
Jenkins
9faaabe8fc Merge "[Verify] Adding creation of network resources to Tempest context" 2015-11-16 12:57:59 +00:00
Jenkins
033ca4ad0c Merge "[Sahara] Add Swift to input data sources" 2015-11-16 12:56:16 +00:00
Yaroslav Lobankov
75f7ed9f93 [Verify] Fixing tokens expiration issue when deleting Tempest resources
The current cleanup mechanism for deleting Tempest resources after
the tests finished doesn't work properly. If Tempest tests take more
than 1 hour, we will not be able to delete a resource because of token
expiration. So this mechanism was slightly reworked to fix the issue.

Change-Id: I9807109e6cf7e5652c5fe73c433b7a9a8f3f489d
2015-11-14 16:56:35 +03:00
Yaroslav Lobankov
2b844e78c8 [Verify] Adding creation of network resources to Tempest context
Tempest creates private network resources for all tests by default.
If a cloud has some shared networks, many Tempest tests will fail with
the following error:

    "Multiple possible networks found."

To avoid this issue we have to configure the fixed_network_name option
in the Tempest config file. So this patch proposes the following solution:

    if the cloud doesn't have any shared networks, the default behavior
    defined in Tempest will be used and Tempest itself will manage network
    resources;

    if the cloud has some shared networks, we will create our own shared
    network and specify its name in the Tempest config file;

    if user configures the fixed_network_name option in the tempest.conf
    file manually, then we don't need to do anything.

Unit tests for new methods and missing unit tests were added.

Change-Id: I487806fe9a8ccd8528922cfaa2700df30961ef9b
2015-11-14 13:12:18 +00:00
Jenkins
afcbab6f71 Merge "Use network context for Neutron scenarios" 2015-11-13 23:59:31 +00:00
Ilya Tyaptin
76b8df58d2 Extend list benchmarks for ceilometer
Existing benchmarks which process a list of resources and meters
are updated via new context and adding new configured query
parameters.

It allows to find a weak places in requesting ceilometer
data and extend count of scenarios which use a ceilometer.

Change-Id: Ifbb77915267f7e339ea7adbd9dae34ca59acb78d
2015-11-13 18:22:25 +03:00
Jenkins
6670658b5d Merge "Rename BenchmarkEngine to TaskEngine" 2015-11-12 23:25:11 +00:00
Jenkins
b56920fbbf Merge "Implement subunit_parser" 2015-11-12 13:03:49 +00:00
Jenkins
768fcbc3df Merge "Improve rally.common.utils.RAMInt code" 2015-11-12 12:05:08 +00:00
Illia Khudoshyn
d60bdaff80 Rename BenchmarkEngine to TaskEngine
The reame is performed as a part of transition
to the new task format.

Change-Id: Ieaef03138298ef04f52a8f0da2a87b3664130ec8
2015-11-12 11:43:13 +00:00
Jenkins
ff56404389 Merge "[Swift] Add additional Swift scenarios" 2015-11-12 06:23:31 +00:00
Boris Pavlovic
fdd366570d Improve rally.common.utils.RAMInt code
*) There is no need to create extra lock object. There is already
   instance of lock object in multiprocessing.Value

*) Improve the tests. Unit tests should test that method properly
   work and not mock everything to check what is in each python line
   (which is anti pattern in unit testing)

Change-Id: I7cc102aae0e083e2fccf98cbc21f8f43cef8d454
2015-11-11 18:00:52 -08:00
Andrey Kurilin
c91b7e24ff Implement subunit_parser
subunit2json module was implemented based on subunit2html module, which
cause several issues(see attached bugs for more details).

The main approach of new parser: it designed to be used from Python code,
not as live-filter for a stream.

Since such parser can be used not only for verification, the implementation
is moved to common dir.

Also, this patch:
 - removes fake log, which was left from xml parser;
 - removes all db calls(except list call) from cli.verify module.

Closes-Bug: #1456810
Closes-Bug: #1471400
Change-Id: Ifcfce827a0df1e75f58e198ee899d4bb85ca9376
2015-11-12 02:18:29 +02:00
Jenkins
97a81bb02c Merge "Add wrapper to support different API versions" 2015-11-11 13:40:21 +00:00
Jenkins
8347790441 Merge "[Sahara] Split Data Sources context" 2015-11-11 12:45:54 +00:00
John Wu
48a4866e14 [Swift] Add additional Swift scenarios
This patch adds additional swift scenarios to use what was created
during swift context.

Swift Scenarios:
- list_and_download_objects_in_containers
- list_objects_in_containers

Blueprint benchmark-scenarios-for-swift

Change-Id: I6d615889a1acfcd4686eb061f1041f5b22fd09f0
2015-11-11 00:05:44 -08:00
Chris St. Pierre
cee6df2f80 Use network context for Neutron scenarios
This permits users to create a network with the network context, or
use the new exiting_network context, to provide networks to Neutron
scenarios instead of always creating a new network. This makes it
possible to run Rally against Neutron Provider Network deployments
with "dumb" routers. It will also eventually permit us to deduplicate
some code by moving network creation into the network context.

Neutron scenarios that are benchmarking network creation continue to
create networks.

This deprecates network creation inside the other scenarios. Warning
messages are produced where appropriate.

Change-Id: I404eee0f14f40e348b2916854ed193c59364c31b
2015-11-10 15:27:30 -06:00
Jenkins
4d0979d2c0 Merge "Add boot-server-from-volume-and-resize for nova benchmark" 2015-11-08 11:52:59 +00:00
Jenkins
b72e71f763 Merge "Fix test_get_template_* tests" 2015-11-07 05:09:36 +00:00
Jenkins
597d1d7ae7 Merge "Add timestamps and metadata into creating samples" 2015-11-07 00:31:59 +00:00
Jenkins
695abe2f63 Merge "Fix 'merge lists'" 2015-11-06 18:18:24 +00:00
Jenkins
df5b57f2e3 Merge "Make name_matches_object accept list of objects" 2015-11-06 16:48:00 +00:00
Sergey Skripnick
03e417aceb Fix test_get_template_* tests
These tests was calling wrong function

Change-Id: I6da522c606bd4d79e9d57ee010624bca45a35cae
2015-11-06 18:43:02 +02:00