Looping over the .keys() method of a dictionary is an anti-pattern. In
fact, the method is actually creating a new list which is redundant and
unnecessary. Looping over a dictionary implicitly loops over its keys.
Change-Id: I937d3f060bf95bb86e50fcb5dec8def524f6208e
This is based on the devstack plugin detection code, modified
to check setup.cfg for a tempest_plugins string. It is to be
executed from a periodic proposal job, generating a list of
plugins in the same format.
Change-Id: I53d082319b6af6fb47083ea42c3ff6b109882e72
Now that tempest_lib code is copied back into tempest, stop using
tempest_lib in tempest, and start using the copied code.
Remove the dependency to tempest_lib from requirements, and drop
the script to use tempest_lib in tempest. Add os-testr to the
test-requirements.
Partially implements bp tempest-lib-reintegration
Change-Id: I21ab5fe6349f72c98ac9f960a29bf62e813f8b1b
This commit removes the old virtualenv management scripts from oslo
incubator. They were just wrappers around virtualenv and don't really
work well in a world with global requirements and constraints. This
commit just removes them since there isn't any reason to keep them
around.
Change-Id: Ib7dd6b34533880e48b367732ae3520026a92500e
exit() is used for interactive shell, it relies on the site module.
sys.exit() is considered good to use in programs.
Trivial-fix
Change-Id: I9e6e7a53581abefbb6cc3ba6ee6b853cd430d456
to keep Python 3.x compatibility, use six.StringIO/BytesIO to
replace StringIO.StringIO
StringIO works for unicode
BytesIO works for bytes
Change-Id: I063d62a4f005084f4088da403f348ad97656ad45
Partial-Bug: #1280100
The idempotent id decorator and the check-uuid script were migrated to
tempest-lib as part of tempest-lib 0.10.0. However, tempest was never
updated to use the library version of idempotent id or check-uuid.
This commit updates tempest to remove it's local copies of the
idempotent_id decorator and the check-uuid tool and use them from
tempest-lib instead.
Also, instead of updating the import of every single test module in
tempest an alias is just added to tempest.test to let all the old
imports work seamlessly. Updating the imports and removing the alias
may or may not be worth it in the future, but that will be a follow
on patch or discussion.
Change-Id: Iae3f491ada8f7b55c1fd817e033d968c2102d11e
As many files have been migrated to tempest-lib and
we need to remove those from tempest and make use of lib.
That is easy work but doing it manually takes times.
Automating that process same way it was done for migrating files to lib.
This commit adds new script use_tempest_lib for using the files
from tempest-lib.
In short this script does below steps-
- Get the latest tempest commit-id of file.
- Clone the tempest-lib and fetch the migration commit information
- Get the tempest commit-id of file which was migrated to lib.
- Check Diff of both (latest commit-id and migrated commit-id).
- If both are same then delete those files from tempest and edit import of
those file using tempest-lib file.
Sample - Id28c615534235c345e31b895a77a53ceac4af1a4
Change-Id: I8177f47113b6ab9b23af8ff86b87bb946ad549ad
This commit switches the run_tempest.sh script to use subunit-trace
instead of the colorizer output filter. Since the colorizer isn't used
anymore it is removed.
The colorizer output has several issues. Firstly the color is actually
meaningless. It's based on arbitrary run times for the tests but don't
actually indicate anything useful. Additionally, it lacks a lot of the
debugging information that subunit-trace provides. Things like skip
messages, the worker breakdown, etc. are missing with the colorizor
output. The last aspect is that switching to use subunit-trace will
provide a consistent output format for running tempest whether you
use tox or run_tempest.sh.
Change-Id: I26e9c45f8e674bdb889b6bc65ad30c4abf6e7655
There is a lot of H404/405 violations in Tempest now, and that leads
difficult to migrate the code to tempest-lib or the other projects'
repos. This patch fixes these violations for tools for enabling
H404/405 rules on Tempest.
Change-Id: I4e1d78532d8f96edd5fd28a7bd2d62181fe9144b
Init command was not working properly as it could not locate the
configuration sample file.
Steps taken:
- Move config-generator.tempest.conf to etc so it will be installed
with pbr.
- Update all files with new path to config-generator-tempest.conf as
needed.
- Refactor init command so if it detects we are not in a virtual
environment, try to find the global config directory /etc/tempest.
If that fails fallback to [sys.prefix]/etc/tempest.
Closes-Bug: #1491058
Closes-Bug: #1490670
Change-Id: I960bc711ff78ac2b0441ef63dff8ec4fb268fd3a
This removes the check on in tree sample generation to ensure the file
matches the config options. However, to counter the loss of the in-tree
the tempest init command will now automatically generate a new sample
config in the local etc dir when run. This will include all of the oslo
options that were previously included in the in-tree sample file.
Additionally, the docs now contain the complete text of the generated
sample output. These also get published on docs.openstack.org on each
commit to tempest. Between this 2 alternatives there shouldn't be a need
to keep the file in tree anymore, especially given the weekly headaches
with new oslo releases changing the config option help breaking the job.
Change-Id: Ie3bc368329d240ca9604cc2f9b2cac544b3ee715
Depends-On: I4064a041a965ed2419b68efc8dc31fce37b24cfd
Currently install_venv.py and with_venv.sh create virtual
environment in .venv directory and there is no way to specify
the virtual environment path explicitly.
This has been fixed in oslo-incubator.
This patch sync those virtual environment scripts from oslo-incubator.
Also make run_test.sh and run_tempest.sh scripts to use 'VENV' env variable.
Change-Id: I55311cff4dea1f6c5adaa2a6d51eeee2f6cc71ea
Closes-Bug: 1370086
Previously 'tox -epep8' just says run 'tox -v -euuidgen' but without
manually removing the duplicate uuid it fails to fix things and says run
'tox -v -euuidgen' again.
Change-Id: I1948c2d038d89af8dcfe69d01dcc49d024ae3210
Some decorators (like dtt) do have a func attribute
but no "attr" and "value" attribute defined.
This leads to errors during pep8 gate check (see [1]).
[1]: https://review.openstack.org/#/c/153234/4
Closes-Bug: #1431267
Change-Id: Id16bce40e30ecb6bc5bff9b55650acc538ece3a9
Use graduated libraries where available and update the incubated modules
that remain.
Update imports for libraries that were being used already to avoid the
'oslo' namespace package.
Update config file for creating example configuration files and provide
a new sample file based on the options defined in the libraries.
Note: because of the number of libraries involved there is no real way
to break this change up into smaller pieces. Fortunately, most of the
changes are just to import statements.
Change-Id: Ia0de957b681cb924a57af98d99a9389ee234ed5b
Adds uuid4 as a decorator of the form:
@test.idempotent_id('12345678-1234-1234-1234-123456789abc')
to every test in the Tempest tree. Includes a gate check to
ensure the existence and uniqueness of the ids.
Modify check tool to ignore Tempest unit tests.
Change-Id: I19e3c7dd555a3ea09d585fb9091c357a300e6559
Co-Authored-By: Sergey Slipushenko <sslypushenko@mirantis.com>
Implements: bp test-uuid
This patch adds a new tool to check for existence and uniqueness
of UUIDs across Tempest or any other test repository based on
Tempest. The tool also includes an option to automatically tag
a test repository with UUIDs if they don't exist. The tool
will be used in the gate to ensure UUID existence.
Change-Id: I25aa83c7836f5a607af2aaa4bf862fa72766f799
Co-Authored-By: Sergey Slipushenko <sslypushenko@mirantis.com>
Partially-Implements: bp test-uuid
This commit fixes the output from pretty_tox.sh so that the testr
slowest output isn't gobbled up by the pipe into subunit-trace.
Change-Id: I11e8fe80c35a1d3a90a4d7c8bf2b36c4114b5cc6
We've recently changed default behaviour to forbid any dirty logs for
services outside the list instead of explicitly enabling log checks for
specific services. The comment was not updated to reflect the change
though.
Change-Id: I5d5ba64d156a9b5318cfc2740b2a69d75de3c127
This commit removes the tempest copy of subunit-trace and uses the
entry-point from tempest-lib instead.
Partially implements bp tempest-library
Change-Id: Ic1aeddc2d0b217c41b6b9914ac3f269b80c4ec8e
Also fixes tempest reported bug with old generator in oslo-incubator,
when identical options are registered in different groups.
Change-Id: I274f48503c1b61ca09cabc8d8c50245ba1059943
Closes-bug: 1282887
This commit adds a check to subunit-trace to fail if the test run
doesn't actually run anything. Despite technically passing without
running anything, if you're passing a regex to testr that runs 0 tests
that is probably an error, so treat it as such. This also uncovered a
bug in the test_wrappers unit test where the test was using a bad test
regex but still passsing. Since we start enforcing it here the unit
test is also updated to use the proper regex.
Change-Id: I109a8cd6eb731e5b8daf5cc2d6a76498ddd48c6d
The check logs script was not properly detecting grenade and would fail
grenade jobs if the logs were not clean. We don't want to do this
because the upgrade process is not log clean yet. Correct this by
setting grenade flag to true only if the DEVSTACK_GATE_GRENADE variable
has content.
Change-Id: Id8a339d78d981376b9af3a80aae8e4e18a68a85f
This commit changes the output of check_logs.py to be a bit more
concise and clear. Instead of printing out every instance of errors
found in the log files it will only print out which individual files
had errors. If a file is marked as not allowing ERRORS or TRACES it
also prints that along with the filename. This should hopefully avoid
confusion around the operation of this script running in the gate.
Change-Id: Ib2a9a9736470cd3b445fb51a95a1a6c6696bb2cc
After bumping the hacking version to the 0.9.x series ignores were
added for several rules. This commit fixes the violations for a subset
of these rules and re-enables the checks.
E123 is ignored because this rule is ignored in the default pep8 by
default, due to a lack of consensus around it.
Change-Id: I8f14cd2ca6afc38d3fe8ee758272071111022896
This commit removes the generate_sample wrapper script and adds an
rc file for the config generator to specify the alternate path.
This makes console messages from the generator tooling point
consistent with what is actually being used. Before because a wrapper
was used error messages would recommend running the incorrect tool.
Change-Id: I0ac792454777af6e3f1d718836031f9a14850229
This commit cleans up how tempest failure logs are displayed. We'll
no longer dump the failure information to the console as a test fails.
Instead all the failure logs will be printed after the tests are run.
Change-Id: I7ecdc349d913b43f4fb0505d5c17c66f811774b4
This script isn't being used by anyone to my knowledge nor has it
really been maintained since it was first added. We will be working
on a plan for automating tempest configuration soon and this
doesn't really fit the initial set requirements of requirements. It
also does not have any testing, so I expect that it has bit rotted
and probably doesn't work anymore.
Change-Id: Id896d9877312dfb6b12b54ea1c598a1940cdd9ce
a summary report that includes counts of the test results so far
as well as a worker report to let us understand how our workers
are balanced at the moment, as well as if we are missing worker
0 in our data (which seems to happen quite often).
Change-Id: Ide0b63b86e87f8a18c2f0e45264692ac3797c791