Commit Graph

100 Commits

Author SHA1 Message Date
Jenkins
32c02ab658 Merge "Add aliases "rally task|deployment|verify use"" 2015-02-19 13:37:42 +00:00
Jenkins
9d6f497f79 Merge "Fix rally task detailed --iterations-data" 2015-02-16 11:06:30 +00:00
Jenkins
2d5b452da9 Merge "Abort scenario execution on SLA failure" 2015-02-13 17:57:44 +00:00
Mikhail Dubov
e188a81a5f Abort scenario execution on SLA failure
* Change the api of the SLA classes:
  - add_iteration() - processes a single iteration result from the queue
  - result() - return the SLAResult based on the data processed so far.
    It now also returns a successful result in case there was no iterations
    data.

* Add a new SLAChecker class:
  - add_iteration() - processes a single iteration result with different SLAs
  - results() - return cumulative SLA results for all SLAs

* Change the benchmark engine so that it supports scenario runner aborts
  on SLA failure

* Support the "abort on SLA failure" feature in the CLI:
    rally task start ... --abort-on-sla-failure

* Modify the SLA detailed messages (cleaner text and less decimal places)

* Update unit and functional tests correspondingly. Also remove the usage of the
  deprecated "max_failure_percent" SLA from the functional tests.

Change-Id: I91894a81649815428fd1ac7afcfce9cf47160fc9
2015-02-13 14:29:32 +03:00
Mikhail Dubov
dc59e3e981 Fix rally task detailed --iterations-data
In case an input scenario had no atomic actions, the "--iterations-data"
table contained "n/a" values. Here we fix this and enhance the corresponding
functional tests.

Change-Id: Ic4d3809f64bcfcbce804023de7ce0c2d1f6e965f
Closes-Bug: 1420703
2015-02-12 03:01:26 +03:00
Pavel Boldin
df082d31d8 Store keypair and secgroup data on per-user basis
Keypairs and secgroup data are now stored separately
for each user. Secgroup data is now a dict not Nova object.

Change-Id: Ic4bacad2ff41c4ddcb010e8203559c0dd3945a63
2015-02-11 18:36:01 +02:00
Jenkins
7cabf5ca10 Merge "Publishing functional tests reports" 2015-02-10 13:27:06 +00:00
Oleh Anufriiev
89bd4aa865 Publishing functional tests reports
Change-Id: I4b7543e7244ecc776e6d0955c0428423cca3ad3c
2015-02-10 06:49:46 +02:00
Anastasia Kuznetsova
2348fd1b4a Add Mistral benchmark
- added Mistral client initialization
- added Mistral scenarios
- added unit tests

Change-Id: Ie60a36504a970e6acb369adc570a556e549c20e1
2015-02-06 16:56:53 +04:00
jacobliberman
a33b5b4d39 Replace ' with " in rally/benchmark and tests/unit
Partial bug: 1405884

Change-Id: I43cb7440b7b2a284ff08538860b1f85c1da0925d
2015-02-05 15:15:56 -06:00
Jenkins
70f9b2f6d6 Merge "Replace ' with " in tests/unit/[aas,cmd]" 2015-02-04 20:05:02 +00:00
Jenkins
7b14e43903 Merge "Add option to keep db settings in functional tests" 2015-02-04 19:10:50 +00:00
jacobliberman
1298d86398 Replace ' with " in tests/unit/[aas,cmd]
Partial bug: 1405884

Change-Id: I820e5c60b14f323a7d71f175eefe4560552c6f46
2015-02-04 10:41:00 -06:00
Jenkins
17d2702458 Merge "Switch to the new Rally API" 2015-02-04 11:06:28 +00:00
Sergey Skripnick
487cc5538b Add option to keep db settings in functional tests
Also use six.configparser to fix py3 compatibility

Change-Id: If75b5c5439f679e5b01078f18faecede0d104916
2015-02-03 23:49:56 +02:00
jacobliberman
9d3a7a27ef Replace ' with " in tests/unit/[common,deploy,doc,fixtures]
Partial bug: 1405884

Change-Id: I0100649c32bb911dab345c40492aa148ed1832c6
2015-02-03 10:48:15 -06:00
Mikhail Dubov
a37341667e Switch to the new Rally API
Use the new API classes in CLI and other parts of Rally code
(still leaving old API methods for compability).

ToDo in next patches:
 * Remove old API methods
 * add get(), list() methods to API.

Change-Id: I1eca518bca8b383e3e6ce05e99251c2d954f1b4e
2015-02-03 15:34:54 +03:00
Olga Kopylova
92ef11be17 Add aliases "rally task|deployment|verify use"
Add alias "rally task use" for "rally use task",
"rally deployment use" for "rally use deployment"
and "rally verify use" for "rally use verification"

Change-Id: I6e63f4b60e1faf794f050f98be127744fe8011a4
2015-02-02 12:20:47 +02:00
Jenkins
d249772721 Merge "Fix all py3 related issues" 2015-01-27 23:23:34 +00:00
Andrey Kurilin
4bffb73331 Fix all py3 related issues
This patch makes output of `tox -epy34` to finish with following message:
>  py34: commands succeeded
>  congratulations :)

Issues:
 * module "__builtin__" was renamed to "builtins" in Python 3

   Related modules:
    - rally.api
    - tests.unit.benchmark.scenarios.vm.test_utils

 * function "map"/"filter" returns "builtins.map"/"builtins.filter" object
   in Python 3 instead of list in Python 2. "builtins.map" and
   "builtins.filter" object is not subscriptable and has no len(), so list
   comprehension is preferable to use and py2/py3 compatible way.

   Related modules:
    - rally.benchmark.context.sahara.sahara_edp
    - rally.benchmark.processing.plot
    - rally.benchmark.sla.base
    - rally.benchmark.types
    - rally.cmd.commands.task
    - rally.cmd.commands.verify
    - tests.unit.benchmark.scenarios.test_base
    - tests.unit.benchmark.wrappers.test_keystone
    - tests.unit.cmd.commands.test_task
    - tests.unit.cmd.commands.test_verify

 * dict.keys()/dict.values() returns "dict_keys"/"dict_values" object in
   Python 3 instead of list in Python 2. so list(dict) and
   list(dict.values()) should be used instead.

   Related modules:
    - rally.benchmark.scenarios.utils
    - rally.benchmark.scenarios.vm.vmtasks
    - tests.unit.cmd.commands.test_show
    - tests.unit.common.test_broker
    - tests.unit.deploy.engines.test_fuel
    - tests.unit.fakes

 * Some changes was made in Python 3 related to data model, so we should
   change our inspect code. See code changes for more details

   Related modules:
    - rally.cmd.cliutils
    - rally.common.utils

 * ConfigParser is more strict for duplicate items in Python 3, so
   duplicates are removed

   Related files:
    - rally/verification/tempest/config.ini

 * Exception object doesn't have "message" attribute in Python 3, so
   if we want to get it, the most proper way is using "getattr"

   Related modules:
    - rally.verification.tempest.config

 * "mock.MagicMock" is not sortable in Python 3, so we should add required
   attributes to fix that.

   Related modules:
    - tests.unit.benchmark.context.test_base

 * assertSequenceEqual assertation method was added in tests.unit.test to
   compare sequence objects

   Related modules:
    - tests.unit.benchmark.context.cleanup.test_resources
    - tests.unit.benchmark.scenarios.nova.test_utils

 * function "range" returns "range" object in Python 3 instead of list
   in Python 2.

   Related modules:
    - tests.unit.benchmark.processing.test_utils

 * keyword arguments should be transmitted to self.assertRaises as kwargs,
   not like a dict

   Related modules:
    - tests.unit.benchmark.scenarios.dummy.test_dummy

Additional changes:
 * Python 2.6 was added to setup.cfg, since Rally supports it.
 * py33, py34 environments were added to tox.ini
 * wrong ignore path was removed from tox.ini
 * made items of bash complition sorted

Several tests are skipped in Python 3 env. For more details see notes in code:
 - tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_main_time
 - tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_atomic_time
 - tests.unit.common.test_utils.MethodClassTestCase.test_method_class_for_class_level_method

During porting Rally to Python3, several issues found and fixed in
TempestContext and its unit tests:

 - If process of cleanup is failed, exception is handled and cleanup is
   marked as successfull. This issue was fixed and CleanUpException was
   added to rally.exception module
 - Cleanup was called with wrong path.

Change-Id: If04e873790dcb4c9c882d4be4bf40479deedd36d
2015-01-26 15:18:36 +00:00
Filip Hubik
d82d02583d Refactor oslo.foo as oslo_foo
According to related blueprint, installing oslo
libraries into a same namespace seems to cause
several type of errors. Every occurrence
of oslo.foo supported is renamed to oslo_foo.

Change-Id: I786b0b824be31c08e6795201bb95553434d8d634
Implements: blueprint 'oslo-incubator/drop-namespace-packages'
Closes-Bug: #1409213
2015-01-26 11:50:39 +01:00
Sergey Skripnick
5533dea487 Allow setting DB for unittests via env & add psql support
RALLY_UNITTEST_DB_URL may be used to change test database.

Sample usage:

export RALLY_UNITTEST_DB_URL="mysql://user:secret@localhost/rally"
tox -epy27

Fix postgress related bugs in unit tests.

* All enums should have name
* Fix invalid unit tests

Closes-bug: #1413155
Partial-bug: #1405884

Co-Authored-By: Sergey Skripnick <sskripnick@mirantis.com>
Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>

Change-Id: Ic741cb79eadf952c3dea99182c1e7d52992c9904
2015-01-24 03:08:12 +03:00
Shahifali Agrawal
77b5110ff6 Moving rally/doc/samples to rally/samples
Moving samples to root forlder for easy access.

Change-Id: I050c8d7580282b898be77156f65cdbce75a4dbb4
Closes-Bug: 1409836
2015-01-16 05:54:09 -08:00
Jenkins
b4bca9856b Merge "Pulls up _test_atomic_action_timer to TestCase" 2015-01-15 18:11:32 +00:00
Jenkins
6ec6f3bf58 Merge "Input task templates and task cmd cleanup" 2015-01-15 15:31:06 +00:00
Jenkins
790d786de6 Merge "Changes Timestamp to hold float instead of datetime" 2015-01-15 14:47:43 +00:00
Jenkins
1ef6fcd472 Merge "Adding dict.items() concatenate validation" 2015-01-15 13:44:42 +00:00
Boris Pavlovic
254c0d68e8 Input task templates and task cmd cleanup
Implement task templates based on jinja2.

This allow us to pass as a task jinja2 template
and it's argument via arguments --task-args and
--task-args-file that should be dict in JSON or YAML
presentations.

So now command looks like:

rally task start <file> --task-args <template-args-json-or-yaml> \
  --task-args-file <file-with-args-in-json-yaml>

If both --task-args and --task-args-file then file dict is updated
by task args file.

Extend rally CI performance job. Now we can set template args
via file with name: ${TASK}_args.yaml

Bonus:
* Better message on InvalidTask format
* Remove redudant catch of "keyboardinterrupt"
  it should be implement in different way.
* Replace ' -> " in rally.cmd.commands.task
  and tests.unit.cmd.commands.task
* Imporve a bit CLI messages on rally task start
* Remove old plot2html command (it's enough deprecated)
* Improve test coverage of rally/cmd/commands/task
* Fix rally/cmd/commands/validate return 1 if bad format
* Write errors to stderr (in whole cmd/commands/task.py)

Change-Id: I7dadf2986bb10407865bc73bb2fb8c96a5162d9a
2015-01-15 16:29:34 +03:00
Arx Cruz
e59a81ece7 Adding dict.items() concatenate validation
Python 3 dict.items() doesn't support + operand, so in order
to keep compatibility, it's supposed to concatenate using
dict.update(). This patch adds validation to avoid the use
of + operand.

Co-Authored-By: Martin Pavlásek <mpavlase@redhat.com>

Change-Id: Ie23f93f3ab2ac1cd53da87d0b71f5bd71e0226df
2015-01-15 12:34:55 +01:00
Yair Fried
bc2c07e782 Pulls up _test_atomic_action_timer to TestCase
This method is being duplicated multiple times

Change-Id: I696d35f88fc94eccc3c351fc36db0cfb8add773c
2015-01-15 10:30:19 +02:00
Jenkins
73c17ffb85 Merge "Adding documentation" 2015-01-15 07:46:50 +00:00
Prabhjyot Singh Sodhi
8097553bcc Adding documentation
Documentation about rally-verify.sh and test_install.sh is added

Change-Id: If5f2911385b0df893867fc8711212d2d62351dfd
2015-01-14 00:18:06 -08:00
Yair Fried
9388d52449 Changes Timestamp to hold float instead of datetime
Processing is simpler for floats instead of objects/strings

Change-Id: Ib0ae26654736e70b3ff3064f59a4ad96c03e8ea5
2015-01-13 12:27:32 +02:00
Jenkins
f8b82e77f3 Merge "Adds timestamps to scenarios iterations" 2015-01-13 01:55:56 +00:00
Yair Fried
111d3f5a54 Adds timestamps to scenarios iterations
In preparation for parallel scenario runners

Change-Id: I27fbaa2a546f47b2e00dc8e54348ec45410865a6
2015-01-12 14:57:27 +00:00
Jenkins
0f55aeee32 Merge "Move files to common lib(Part 4)" 2015-01-12 12:47:47 +00:00
liyingjun
e3021b5485 Move files to common lib(Part 4)
Move log to common directory.

Change-Id: Ibcd891caf9a7fe53023010d359871d2a021f64d2
2015-01-12 14:17:27 +08:00
Wataru Takase
11623b10c9 Add list_volumes scenario for Cinder test
Current Rally provides "create_and_list_volume" scenario, but doesn't
provide "list_volumes". User can specify number of volumes which will
be created in context.

Change-Id: I700282388b839d735448850e0629a0a37ab781f1
2015-01-11 17:20:33 +01:00
liyingjun
109d261e19 Move JSON_SCHEMA to rally.consts
Change-Id: I918999306bbc6577b2b86761ea9da5e61dfa5a58
2015-01-09 09:09:15 +08:00
Oleh Anufriiev
0d3a3524b1 Support generation of report from N task
Added ability to build one report from
  results of multiple tasks. Tasks may be passed as UUID or as previously
  saved json results. Also it may be mixed list of UUIDs and file pathes.

  Closes bug 1406585

Change-Id: I11f2ca3ee6b868b79f91a4fa95e6a9ea918a79b0
2015-01-08 16:17:31 +02:00
Andrey Kurilin
4f4c5791b1 Fix py3k issues
1) Module urllib2 is missed in Python 3, so it should be replaced by other
   http lib. "requests" is good library for such stuff, which supports both
   Python 2.* and Python 3.*

2) Function 'rally.common.utils.parse_docstring' uses built-in function
   'filter'. This function returns different types in Python 2.* and
   Python 3.*. Since results are used as a list, 'filter' should be replaced
   by list comprehension.

3) Extend H330 hacking rule to chech 'dict.iterkeys()', 'dict.itervalues()'
   and 'dict.iterlist()'. Also, fixed all places, which are failed in terms
   of this rule.

4) Function object does not have 'func_code' attribute in Python 3, but
   '__code__' attribute exist in both Python 2 and Python 3

Also, 3 tests are broken due to new release(1.6.0) of oslo.config, so we
need to skip them to unblock our gates until new version of oslo.config will
be released with appropriate fix.

Closes-Bug: #1405919

Change-Id: Icc42e220ac3f15ae6c838a4698b3c9578177513c
2015-01-06 13:01:30 +02:00
li,chen
44b5f310c5 use function next() instead of next() method on iterator objects
Python 3 introduced a next() function to replace the next() method on
iterator objects. Rather than calling the method on the iterator, the next()
function is called with the iterable object as it's sole parameter, which
calls the underlying __next__() method.

The next() function was backported to Python 2.6 which allows us to use either
the 2.x or 3.x way simultaneously.

Change-Id: I945225b1b0c6c39842a77f4a136e9431b798229f
Closes-Bug: #1403074
2015-01-02 15:41:25 +02:00
liyingjun
bc4e1b657e Move files to common lib(Part 3)
Move utils to common directory.

Change-Id: Ifaf71ea4a2a1acc85ceb2af6ed978d3e08666b41
2014-12-31 09:21:14 +08:00
li,chen
4e6ebcf933 Use string.ascii_lowercase and string.ascii_uppercase only
"string.lowercase" and "string.uppercase" were removed in Python 3,
we can use only "string.ascii_lowercase" and "string.ascii_uppercase".

Change-Id: I97ddac01af7070129ad8baf5960c841482c346a2
Closes-Bug: #1405918
2014-12-30 08:28:43 +08:00
li,chen
080cff77fc Use six.moves.range instead of xrange
In Python 3, module "xrange" doesn't exist any more.
To support both Python 2 and Python 3, "six.moves.range"
should be used instead.

Change-Id: I9dcf79011a4b2672e4ba55298aa22cab5e2292ff
Closes-Bug: #1403447
2014-12-27 09:38:38 +08:00
li,chen
7576192914 Use six.moves.map instead of itertools.imap
In Python 3, module "itertools" doesn't contain "imap" object.
To support both Python 2 and Python 3, "six.moves.map"
should be used instead.

Change-Id: I5b82020d0721a91aa6d6090a42073a3285b80ff9
Closes-Bug: #1403434
2014-12-26 21:28:09 +08:00
li,chen
ced7b863e9 Use from six.moves.urllib.parse instead of urlparse
In Python 3, urlparse module is gone.
To support both Python 2 and Python 3, "six.moves.urllib.parse"
should be used instead.

Change-Id: I0adddbdea5a80606907a1b78ca1a04d15f61452b
Closes-Bug: #1403433
2014-12-26 08:27:56 +08:00
Boris Pavlovic
ac9cd9fedb Refactor validation (part 1)
* Remove semantic validation mechanism for context. We will use same
  approach as for current scenarios (via decorators)

* Pass "deployment" instead of "tasks" to validators. This allow us to
  remove access to DB from validators + we can call validation without
  creating tasks

* Add hacking rule to force using construction "from rally import objects"

* Fix unit tests

Change-Id: Ibf08183b28479cc49ab6adfd12298612f6b4397f
2014-12-24 18:27:17 +04:00
li,chen
7b4f35546c Use six.moves.StringIO instead of StringIO.StringIO
In Python 3, StringIO module is gone.
Class StringIO.StringIO is not available from io module.
To support both Python 2 and Python 3, "six.moves.StringIO" should be used instead.

Change-Id: I17e79d8625c75d6903f024c648d3ef62e21d0c51
Closes-bug:#1403076
2014-12-24 18:34:08 +08:00
li,chen
352df6d6af Use six.string_types instead of basestring
There is no global variable "basestring" in Python 3.
"six.string_types" should be used to support both Python 2 and Python 3

Change-Id: I556fef0e98d00ab819e3dea961666b403b927b06
Closes-bug:#1403077
2014-12-24 15:54:16 +08:00