Our cleanup manager is designed to accept subclass of
`RandomNameGeneratorMixin` to match names of resources to particular
task.
The new method `make_name_matcher` creates a subclass of
`RandomNameGeneratorMixin` with overriden check method to check exact
names instead of matching to specific patterns.
Also, while checking all context to port to new matcher, proper
required_platform validators were added to all contexts.
Change-Id: I42b9af878630c0e8080f50c7b489346121704255
Because of BackupHelper.__del__ that uses os.path.exists,
there is a probability that __del__ will happen during running
of test_generate_random_path and it will grab a value
from mock.side_effect that as a result will lead to an error.
Solution:
Replace side_effect tuple with lambda-function that will
react to a specific value instead of hard coded return values.
Change-Id: I965041049079520a0aa354ddef32aeba047195ba
This helper class provides mechanism for saving data at temporary folders and
process rollback if needed.
Co-Authored-By: Yaroslav Lobankov <ylobankov@mirantis.com>
Change-Id: If3555171b429039b54661f3af9d44ee0fc679181
Triggers determine when hooks should be executed.
Added event trigger.
Added utils required for implementation of hook_section spec.
implements spec: hook_section
Change-Id: I33d558468f221a7c731e4a3fd42deb4f76be671a
For now we have output of small floats in scientific notation:
3.79085540771e-05
Fix that to 0.000037. Also floats in fixed place will output as:
0.0000001 -> 0.0
0.000000 -> 0.0
37 -> 37.0
1.0000001 -> 1.0
1.0000011 -> 1.000001
1.0000019 -> 1.000002
This can be made in different ways like:
"%f" % float(decimal.Decimal(num).normalize())
or in more weird way but I choose that one (see in the patch) like
the most flat, simple and explicit.
Change-Id: I4b9cdbb01f3f8e2eeaedc9d33e1203247ea79f89
Decided to put thread instance to timeout_queue because
thread.ident in python is recycled. So this is the way to
check whether the thread alive or not.
Closes-bug: #1570798
Change-Id: I745030ec9942c37b47c6d53864ae5fc7ecae7ccd
Class LockedDict represents dict which is locked for updates
by default. It will be used in cases when we want to prevent
dict modification, like context configuration.
Usage of this class will be added in further patches.
Change-Id: I782e99bbe6d9ba4463103304ba152196eddd5227
This will support places where we create resources with names that
start with a given name pattern, but include some additional
identifier afterwards. For instance, when bulk creating instances,
Nova appends a UUID to each instance name.
Implements: blueprint cleanup-refactoring
Change-Id: Idaa8a8eb2f4edbc48cae324240623922537106a5
This will be used to ensure that we are only deleting resources for a
particular Rally task.
Implements: blueprint cleanup-refactoring
Change-Id: I487a685987b7a5e1bae7e4695c470141ff8f6824
There are some classes in the code that didn't inherited from
nothing and this is an old-style classes. A "New Class" is the
recommended way to create a class in modern Python.A "New Class"
should always inherit from `object` or another new-style class.
Hacking rule added as well.
Change-Id: I2c72f20695e35e05435d2526d4a804196c2ab2d3
TempestResourcesContext calls NeutronWrapper.create_network, and it will
be called to generate random name, so it needs to implements
utils.RandomNameGeneratorMixin
Changed generate_random_name to use self.task["uuid"] or
self.verification["uuid"] as task_id
This patch also refactor test__create_network_resources to test the
logic correctly
Change-Id: I8c145073386cb7f668461e050fe8de75f4675def
Closes-Bug: #1530770
Timeout for scenario runners doesn't work correctly.
To reproduce it just put in Dummy scenario args sleep bigger
then timeout and it won't fail, that will verify this bug.
Tasks are running in threads, so we need a way to kill threads.
New function `terminate_thread` added for this purpose.
Also added function that is run in separate thread and check
execution time of the threads. If they run longer then timeout
they would be terminated by terminate_thread.
Thread can not be killed during system call, so this mechanism
isn't 100% effective. Due to this interruptable_sleep method
was added in order to be used by Dummy scenarios.
Added timeout argument to constant and rps runners. Also added
two Dummy scenarios to test this.
New exception ThreadTimeoutException was added.
Closes-Bug: #1385365
Co-Authored-By: Roman Vasilets <rvasilets@mirantis.com>
Co-Authored-By: Sergey Scripnick <sskripnick@mirantis.com>
Change-Id: I2096ba6adaf81a18c80b2ae6a759ccc9e247c45a
Kill the old random name generator in favor of the fancy new one.
Implements blueprint: consistent-resource-names
Change-Id: Ia612ebb3ed3af27c6aaa1e35f459b7980b7a24b9
*) There is no need to create extra lock object. There is already
instance of lock object in multiprocessing.Value
*) Improve the tests. Unit tests should test that method properly
work and not mock everything to check what is in each python line
(which is anti pattern in unit testing)
Change-Id: I7cc102aae0e083e2fccf98cbc21f8f43cef8d454
rally.common.utils.merge now produces lists
of given length rather than ones with length greater or equal to.
Change-Id: I95f0eae869281b9ab65ea8e6a5b82918ed9faac6
This makes it easier and more efficient to check randomly-generated
names against multiple objects to see if they could have been
generated by any of the given objects.
Implements blueprint: consistent-resource-names
Change-Id: I0c7b7bc9bd65738984bf5b156a6c249756aaf4fe
The function added in the patch is to be used as an auxiliary
tool in a forthcoming implementation of distribluted load generation
functionality
Change-Id: I482d0e81a92eb9e783b73120478f98149fbcba96
* Moved log functions from rally/common/utils.py #L137-L234 to rally/common/log.y
* Moved test tunctions form test/unit/common/test_utils.py to test/unit/common/test_log.py
Change-Id: Icac3e87d507a3cf90e27071cb26bc871dac78514
If the task UUID passed to generate_random_name() was shorter than the
first set of Xs in the pattern, the UUID would be used which would
result in a random name that was too short and thus didn't match the
pattern. This fixes that.
Implements blueprint: consistent-resource-names
Closes-Bug: 1509402
Change-Id: I4f8a4415d565dbf737588a4db56fc2a6dfe893cc
It was already trying to use DDT, it just wasn't enabled for the
class.
Implements blueprint: consistent-resource-names
Change-Id: Ie9c4f59951e4e2e0d82c5aa9b493668f38ea30e6
It is unclear what is deprecated in current message.
For example, methods `wait_for`, `wait_is_ready` and `wait_for_delete` were
deprecated in https://review.openstack.org/#/c/232438 . Deprecation
messages are equal, so logs are equal too:
Deprecated, use wait_for_status instead. (deprecated in Rally v0.1.2)
This patch adds deprecated method's name, so it is easier to determine what
method produces deprecation log message.
Also, this patch adds ability to print warning message once, which can be
helpful for deprecated methods used in scenarios.
Change-Id: Id86e984986ecb61e7b92cc61882382179fc8a74b
This adds a new random name generator that will make it easier to do
out-of-band cleanup of Rally-created resources. It eliminates the
ability for scenarios to specify a custom random name length and
prefix; adds a task-specific portion to the random name; and improves
configurability of random name patterns.
Subsequent commits will implement the new generator.
Implements: blueprint consistent-resource-names
Change-Id: I189474fbb185ab402e3b248421b4188b7ccde7f4
This patch does a few things
- Introduce rally.common.plugin.info.InfoMixin
and adds it to plugin base. It adds method get_info()
to all plugins. This method returns information about plugin
such like title, description, arguments, plugin module
- Introduce rally plugin show and list commands that returns
information about all plugins
- Moves info related methods from rally.common.utils to
rally.common.plugin.info module
Related-Bug: #1470895
Change-Id: I8db12e3017d241a2b8bc549a817dad817f922437
Add following unit test for common.utils.EnumMixin
- Check if the name of variable start underscore (like _a), don't return value in iterate
Change-Id: Ief17f1012a0fd6b7917d5a031e03f45e5d963123
Closes-Bug: #1473331
Move all related to plugins from utils to discover module
and fix the code and tests to make it work
Change-Id: Ifaed1fadfa0760f121cb1584cb4d8ae3fee8768d
The patch introduces CLI option --plugin-paths to specify the list of
folders with plugins.
Co-Authored-By: Roman Vasilets <rvasilets@mirantis.com>
Co-Authored-By: Ilya Shakhat <ishakhat@mirantis.com>
Change-Id: Iee39217bfe3e5c77e6925910f3998942541387a3
Add decorator `log_deprecated_args' that issues a message whenever
the decorated function is called with deprecated keyword arguments
supplied. Message is issued only once or per each call depending
on the configuration.
Change-Id: Icedf72c40d9255f8d82233ebf640165d4827d53b
Changes:
- list() declarations replaced with []
- dict() declarations replaced with {}
- Added N351 to hacking for literal rule along with checks, test hacking, and to readme
- Minor fix to incorrect path in hacking guidelines pointing to test_hacking.py
Change-Id: Ie7b9201d61b6f2024217b6f4117605eb52fa916a
Closes-Bug: #1437515
* Remove tests/unit/aas and test/unit/fixture
fixtures are not used so just remove them
aas - didn't have __init__.py files so these
tests were not executed at all. By the way
code related to this tests was removed earlier
in this patch:
https://review.openstack.org/#/c/152847/
* Remove all old py3 related hacking rules
Now we have unit & functional jobs that checks them
* Add new hacking rule that protect us from usage of '
* Unified signature of all hacking methods
(otherwise we are not able to make decorator)
* Add hacking rules decorator for skiping 3 types of lines:
empty, commented, and with # noqa comment
Change-Id: I6bf2cc123325a4980edeb066a6c31aa685858f5b
* Move API methods to classes, make them @classmethod
* s/deploy/deployment
Also modify unit tests.
We leave the old API methods for compability, but they are going to be
deprecated in Rally v0.0.2.
Change-Id: I44155575621c715af1109690ed2788663a4c36c3
This patch makes output of `tox -epy34` to finish with following message:
> py34: commands succeeded
> congratulations :)
Issues:
* module "__builtin__" was renamed to "builtins" in Python 3
Related modules:
- rally.api
- tests.unit.benchmark.scenarios.vm.test_utils
* function "map"/"filter" returns "builtins.map"/"builtins.filter" object
in Python 3 instead of list in Python 2. "builtins.map" and
"builtins.filter" object is not subscriptable and has no len(), so list
comprehension is preferable to use and py2/py3 compatible way.
Related modules:
- rally.benchmark.context.sahara.sahara_edp
- rally.benchmark.processing.plot
- rally.benchmark.sla.base
- rally.benchmark.types
- rally.cmd.commands.task
- rally.cmd.commands.verify
- tests.unit.benchmark.scenarios.test_base
- tests.unit.benchmark.wrappers.test_keystone
- tests.unit.cmd.commands.test_task
- tests.unit.cmd.commands.test_verify
* dict.keys()/dict.values() returns "dict_keys"/"dict_values" object in
Python 3 instead of list in Python 2. so list(dict) and
list(dict.values()) should be used instead.
Related modules:
- rally.benchmark.scenarios.utils
- rally.benchmark.scenarios.vm.vmtasks
- tests.unit.cmd.commands.test_show
- tests.unit.common.test_broker
- tests.unit.deploy.engines.test_fuel
- tests.unit.fakes
* Some changes was made in Python 3 related to data model, so we should
change our inspect code. See code changes for more details
Related modules:
- rally.cmd.cliutils
- rally.common.utils
* ConfigParser is more strict for duplicate items in Python 3, so
duplicates are removed
Related files:
- rally/verification/tempest/config.ini
* Exception object doesn't have "message" attribute in Python 3, so
if we want to get it, the most proper way is using "getattr"
Related modules:
- rally.verification.tempest.config
* "mock.MagicMock" is not sortable in Python 3, so we should add required
attributes to fix that.
Related modules:
- tests.unit.benchmark.context.test_base
* assertSequenceEqual assertation method was added in tests.unit.test to
compare sequence objects
Related modules:
- tests.unit.benchmark.context.cleanup.test_resources
- tests.unit.benchmark.scenarios.nova.test_utils
* function "range" returns "range" object in Python 3 instead of list
in Python 2.
Related modules:
- tests.unit.benchmark.processing.test_utils
* keyword arguments should be transmitted to self.assertRaises as kwargs,
not like a dict
Related modules:
- tests.unit.benchmark.scenarios.dummy.test_dummy
Additional changes:
* Python 2.6 was added to setup.cfg, since Rally supports it.
* py33, py34 environments were added to tox.ini
* wrong ignore path was removed from tox.ini
* made items of bash complition sorted
Several tests are skipped in Python 3 env. For more details see notes in code:
- tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_main_time
- tests.unit.benchmark.processing.test_plot.PlotTestCase.test__process_atomic_time
- tests.unit.common.test_utils.MethodClassTestCase.test_method_class_for_class_level_method
During porting Rally to Python3, several issues found and fixed in
TempestContext and its unit tests:
- If process of cleanup is failed, exception is handled and cleanup is
marked as successfull. This issue was fixed and CleanUpException was
added to rally.exception module
- Cleanup was called with wrong path.
Change-Id: If04e873790dcb4c9c882d4be4bf40479deedd36d
1) Module urllib2 is missed in Python 3, so it should be replaced by other
http lib. "requests" is good library for such stuff, which supports both
Python 2.* and Python 3.*
2) Function 'rally.common.utils.parse_docstring' uses built-in function
'filter'. This function returns different types in Python 2.* and
Python 3.*. Since results are used as a list, 'filter' should be replaced
by list comprehension.
3) Extend H330 hacking rule to chech 'dict.iterkeys()', 'dict.itervalues()'
and 'dict.iterlist()'. Also, fixed all places, which are failed in terms
of this rule.
4) Function object does not have 'func_code' attribute in Python 3, but
'__code__' attribute exist in both Python 2 and Python 3
Also, 3 tests are broken due to new release(1.6.0) of oslo.config, so we
need to skip them to unblock our gates until new version of oslo.config will
be released with appropriate fix.
Closes-Bug: #1405919
Change-Id: Icc42e220ac3f15ae6c838a4698b3c9578177513c
Python 3 introduced a next() function to replace the next() method on
iterator objects. Rather than calling the method on the iterator, the next()
function is called with the iterable object as it's sole parameter, which
calls the underlying __next__() method.
The next() function was backported to Python 2.6 which allows us to use either
the 2.x or 3.x way simultaneously.
Change-Id: I945225b1b0c6c39842a77f4a136e9431b798229f
Closes-Bug: #1403074
