Commit Graph

208 Commits

Author SHA1 Message Date
Joe H. Rahme
a72f2c6fb7 Removes explicit looping over dict .keys() method
Looping over the .keys() method of a dictionary is an anti-pattern. In
fact, the method is actually creating a new list which is redundant and
unnecessary. Looping over a dictionary implicitly loops over its keys.

Change-Id: I937d3f060bf95bb86e50fcb5dec8def524f6208e
2016-07-11 16:45:22 +02:00
Yatin Kumbhare
2e2c83a527 Keep py3.X compatibility for urllib
Change-Id: Iba10637688ada66f2e3003cd87bbba7d4db4abc7
Closes-Bug: #1280105
2016-05-30 22:50:15 +05:30
Ihar Hrachyshka
e43567667b Removed q-vpn from the list of dirty logs
The last gate that was producing the log was kilo. Since it's EOL now,
remove the reference.

Change-Id: I40cf34f53dcd9c3169922dc3b84756dfef09a381
2016-04-29 13:02:12 -07:00
Jenkins
dba61befee Merge "Remove virtualenv management scripts from oslo-incubator" 2016-04-13 02:18:33 +00:00
Jenkins
438084d0cf Merge "Prepare for dynamic generation of tempest plugin registry" 2016-04-08 01:43:38 +00:00
Clint Adams
a413652c25 Prepare for dynamic generation of tempest plugin registry
This is based on the devstack plugin detection code, modified
to check setup.cfg for a tempest_plugins string.  It is to be
executed from a periodic proposal job, generating a list of
plugins in the same format.

Change-Id: I53d082319b6af6fb47083ea42c3ff6b109882e72
2016-03-28 20:11:23 -04:00
zhang.lei
a4b1cef845 Optimize "open" method with context manager
Use opening context manager to open a file.
Change-Id: Ice9be6350fd01cbf5eec686ac68d7fc8b5a4d5bd
2016-03-10 11:21:58 +08:00
Andrea Frittoli (andreaf)
db9672e347 Use tempest.lib code in tempest
Now that tempest_lib code is copied back into tempest, stop using
tempest_lib in tempest, and start using the copied code.

Remove the dependency to tempest_lib from requirements, and drop
the script to use tempest_lib in tempest. Add os-testr to the
test-requirements.

Partially implements bp tempest-lib-reintegration

Change-Id: I21ab5fe6349f72c98ac9f960a29bf62e813f8b1b
2016-02-24 10:43:59 -05:00
Matthew Treinish
72e8376175
Remove virtualenv management scripts from oslo-incubator
This commit removes the old virtualenv management scripts from oslo
incubator. They were just wrappers around virtualenv and don't really
work well in a world with global requirements and constraints. This
commit just removes them since there isn't any reason to keep them
around.

Change-Id: Ib7dd6b34533880e48b367732ae3520026a92500e
2016-02-19 11:37:31 -05:00
Jenkins
993a6f58f1 Merge "Revert "Cap Pip<8 due to pip bug"" 2016-02-09 14:02:14 +00:00
caoyue
ab33302bd9 Replace exit() by sys.exit()
exit() is used for interactive shell, it relies on the site module.
sys.exit() is considered good to use in programs.

Trivial-fix

Change-Id: I9e6e7a53581abefbb6cc3ba6ee6b853cd430d456
2016-01-25 12:23:39 +00:00
Allen Gao
0e1f3692ce Revert "Cap Pip<8 due to pip bug"
This reverts commit 308484592d.
This issue has been fixed after pip 8.0.1 released.

TrivialFix

Change-Id: I193ef41c2a9efcab3aea54f4949cf6fb1e9b1084
2016-01-25 18:02:39 +08:00
Xav Paice
308484592d Cap Pip<8 due to pip bug
Due to https://github.com/pypa/pip/issues/3384 in pip v8, this change
puts a cap on pip, in a similar manner to
https://review.openstack.org/#/c/269954/

Change-Id: I20b8b597e8806a7ee21d947eccc98e4ae7e00c3b
Closes-Bug: 1536290
2016-01-21 16:52:03 +13:00
Harshada Mangesh Kakad
b3ecf6534f Use six.StringIO/BytesIO instead of StringIO.StringIO
to keep Python 3.x compatibility, use six.StringIO/BytesIO to
replace StringIO.StringIO

StringIO works for unicode
BytesIO works for bytes

Change-Id: I063d62a4f005084f4088da403f348ad97656ad45
Partial-Bug: #1280100
2015-12-23 01:33:08 -08:00
Jenkins
a8a6f0334b Merge "Add script to use tempest-lib files in tempest" 2015-12-04 17:42:54 +00:00
Matthew Treinish
c1802bc956
Use the idempotent id and tooling from tempest-lib
The idempotent id decorator and the check-uuid script were migrated to
tempest-lib as part of tempest-lib 0.10.0. However, tempest was never
updated to use the library version of idempotent id or check-uuid.
This commit updates tempest to remove it's local copies of the
idempotent_id decorator and the check-uuid tool and use them from
tempest-lib instead.

Also, instead of updating the import of every single test module in
tempest an alias is just added to tempest.test to let all the old
imports work seamlessly. Updating the imports and removing the alias
may or may not be worth it in the future, but that will be a follow
on patch or discussion.

Change-Id: Iae3f491ada8f7b55c1fd817e033d968c2102d11e
2015-12-03 19:20:44 -05:00
ghanshyam
e73744bbd3 Add script to use tempest-lib files in tempest
As many files have been migrated to tempest-lib and
we need to remove those from tempest and make use of lib.

That is easy work but doing it manually takes times.
Automating that process same way it was done for migrating files to lib.

This commit adds new script use_tempest_lib for using the files
from tempest-lib.

In short this script does below steps-

- Get the latest tempest commit-id of file.
- Clone the tempest-lib and fetch the migration commit information
  - Get the tempest commit-id of file which was migrated to lib.
- Check Diff of both (latest commit-id and migrated commit-id).
- If both are same then delete those files from tempest and edit import of
  those file using tempest-lib file.

Sample - Id28c615534235c345e31b895a77a53ceac4af1a4

Change-Id: I8177f47113b6ab9b23af8ff86b87bb946ad549ad
2015-12-03 15:43:29 +09:00
Matthew Treinish
3898474abd
Switch run_tempest.sh to use subunit-trace
This commit switches the run_tempest.sh script to use subunit-trace
instead of the colorizer output filter. Since the colorizer isn't used
anymore it is removed.

The colorizer output has several issues. Firstly the color is actually
meaningless. It's based on arbitrary run times for the tests but don't
actually indicate anything useful. Additionally, it lacks a lot of the
debugging information that subunit-trace provides. Things like skip
messages, the worker breakdown, etc. are missing with the colorizor
output. The last aspect is that switching to use subunit-trace will
provide a consistent output format for running tempest whether you
use tox or run_tempest.sh.

Change-Id: I26e9c45f8e674bdb889b6bc65ad30c4abf6e7655
2015-11-23 13:09:51 -05:00
Ken'ichi Ohmichi
7616d19360 Fix H404/405 violations for tools
There is a lot of H404/405 violations in Tempest now, and that leads
difficult to migrate the code to tempest-lib or the other projects'
repos. This patch fixes these violations for tools for enabling
H404/405 rules on Tempest.

Change-Id: I4e1d78532d8f96edd5fd28a7bd2d62181fe9144b
2015-11-18 07:04:12 +00:00
David Paterson
0bf52d497b Fix init command
Init command was not working properly as it could not locate the
configuration sample file.
Steps taken:
- Move config-generator.tempest.conf to etc so it will be installed
with pbr.
- Update all files with new path to config-generator-tempest.conf as
needed.
- Refactor init command so if it detects we are not in a virtual
environment, try to find the global config directory /etc/tempest.
If that fails fallback to [sys.prefix]/etc/tempest.

Closes-Bug: #1491058
Closes-Bug: #1490670
Change-Id: I960bc711ff78ac2b0441ef63dff8ec4fb268fd3a
2015-09-23 09:42:07 -04:00
Matthew Treinish
c8a39b488b
Stop gating on sample generation check
This removes the check on in tree sample generation to ensure the file
matches the config options. However, to counter the loss of the in-tree
the tempest init command will now automatically generate a new sample
config in the local etc dir when run. This will include all of the oslo
options that were previously included in the in-tree sample file.
Additionally, the docs now contain the complete text of the generated
sample output. These also get published on docs.openstack.org on each
commit to tempest. Between this 2 alternatives there shouldn't be a need
to keep the file in tree anymore, especially given the weekly headaches
with new oslo releases changing the config option help breaking the job.

Change-Id: Ie3bc368329d240ca9604cc2f9b2cac544b3ee715
Depends-On: I4064a041a965ed2419b68efc8dc31fce37b24cfd
2015-08-04 11:00:11 -04:00
ghanshyam
f5360ff14f Sync venv scripts from oslo-incubator
Currently install_venv.py and with_venv.sh create virtual
environment in .venv directory and there is no way to specify
the virtual environment path explicitly.

This has been fixed in oslo-incubator.
This patch sync those virtual environment scripts from oslo-incubator.
Also make run_test.sh and run_tempest.sh scripts to use 'VENV' env variable.

Change-Id: I55311cff4dea1f6c5adaa2a6d51eeee2f6cc71ea
Closes-Bug: 1370086
2015-07-10 11:53:58 +09:00
Joe Gordon
4f10e45db7 Clarify how to resolve a uuid collision
Previously 'tox -epep8' just says run 'tox -v -euuidgen' but without
manually removing the duplicate uuid it fails to fix things and says run
'tox -v -euuidgen' again.

Change-Id: I1948c2d038d89af8dcfe69d01dcc49d024ae3210
2015-03-27 13:15:44 -04:00
Marc Koderer
07a5a6f58d Make check_uuid executable
Check_uuid is a script and not a module. This makes it executable
from CLI.

Change-Id: I1c5b6a79e412471f27bc9779dd821a52ec83c645
2015-03-17 09:14:55 +01:00
Marc Koderer
fb199c0516 Add additional checks for attributes
Some decorators (like dtt) do have a func attribute
but no "attr" and "value" attribute defined.
This leads to errors during pep8 gate check (see [1]).

[1]: https://review.openstack.org/#/c/153234/4

Closes-Bug: #1431267
Change-Id: Id16bce40e30ecb6bc5bff9b55650acc538ece3a9
2015-03-16 12:04:11 +01:00
Doug Hellmann
583ce2c043 Update all Oslo module use
Use graduated libraries where available and update the incubated modules
that remain.

Update imports for libraries that were being used already to avoid the
'oslo' namespace package.

Update config file for creating example configuration files and provide
a new sample file based on the options defined in the libraries.

Note: because of the number of libraries involved there is no real way
to break this change up into smaller pieces. Fortunately, most of the
changes are just to import statements.

Change-Id: Ia0de957b681cb924a57af98d99a9389ee234ed5b
2015-03-11 21:07:56 +00:00
Chris Hoge
7579c1a40e Add UUIDs to all tempest tests and gate check
Adds uuid4 as a decorator of the form:
@test.idempotent_id('12345678-1234-1234-1234-123456789abc')
to every test in the Tempest tree. Includes a gate check to
ensure the existence and uniqueness of the ids.

Modify check tool to ignore Tempest unit tests.

Change-Id: I19e3c7dd555a3ea09d585fb9091c357a300e6559
Co-Authored-By: Sergey Slipushenko <sslypushenko@mirantis.com>
Implements: bp test-uuid
2015-02-26 16:05:16 -08:00
Chris Hoge
296558cf41 Add UUIDs to tests with tools and checks
This patch adds a new tool to check for existence and uniqueness
of UUIDs across Tempest or any other test repository based on
Tempest. The tool also includes an option to automatically tag
a test repository with UUIDs if they don't exist. The tool
will be used in the gate to ensure UUID existence.

Change-Id: I25aa83c7836f5a607af2aaa4bf862fa72766f799
Co-Authored-By: Sergey Slipushenko <sslypushenko@mirantis.com>
Partially-Implements: bp test-uuid
2015-02-26 13:59:05 -08:00
Matthew Treinish
16b6826b94 Fix slowest test output after test run
This commit fixes the output from pretty_tox.sh so that the testr
slowest output isn't gobbled up by the pipe into subunit-trace.

Change-Id: I11e8fe80c35a1d3a90a4d7c8bf2b36c4114b5cc6
2014-12-18 15:14:39 +00:00
Jenkins
31a91ee725 Merge "Fixed a comment on dirty logs list" 2014-11-11 16:52:04 +00:00
Ihar Hrachyshka
226de0b8f8 Fixed a comment on dirty logs list
We've recently changed default behaviour to forbid any dirty logs for
services outside the list instead of explicitly enabling log checks for
specific services. The comment was not updated to reflect the change
though.

Change-Id: I5d5ba64d156a9b5318cfc2740b2a69d75de3c127
2014-10-31 13:56:36 +01:00
Matthew Treinish
cdb7f549a9 Switch to using subunit-trace from tempest-lib
This commit removes the tempest copy of subunit-trace and uses the
entry-point from tempest-lib instead.

Partially implements bp tempest-library

Change-Id: Ic1aeddc2d0b217c41b6b9914ac3f269b80c4ec8e
2014-10-23 15:07:49 +00:00
Jon Grimm
270bd7f8ce Move tempest to oslo-config-generator
Also fixes tempest reported bug with old generator in oslo-incubator,
when identical options are registered in different groups.

Change-Id: I274f48503c1b61ca09cabc8d8c50245ba1059943
Closes-bug: 1282887
2014-10-16 16:06:43 +00:00
Jenkins
cc6ca0d184 Merge "Make output from check_logs less verbose" 2014-09-20 07:32:35 +00:00
Jenkins
c6dac5315e Merge "Fail tempest if 0 tests are run" 2014-09-16 18:20:49 +00:00
Jenkins
24d946e188 Merge "Remove test start output in pretty tracer" 2014-09-16 07:24:45 +00:00
Matthew Treinish
178cc4a605 Fail tempest if 0 tests are run
This commit adds a check to subunit-trace to fail if the test run
doesn't actually run anything. Despite technically passing without
running anything, if you're passing a regex to testr that runs 0 tests
that is probably an error, so treat it as such. This also uncovered a
bug in the test_wrappers unit test where the test was using a bad test
regex but still passsing. Since we start enforcing it here the unit
test is also updated to use the proper regex.

Change-Id: I109a8cd6eb731e5b8daf5cc2d6a76498ddd48c6d
2014-09-15 20:13:44 +00:00
Clark Boylan
a5c669d7f0 Properly detect grenade in check_logs script
The check logs script was not properly detecting grenade and would fail
grenade jobs if the logs were not clean. We don't want to do this
because the upgrade process is not log clean yet. Correct this by
setting grenade flag to true only if the DEVSTACK_GATE_GRENADE variable
has content.

Change-Id: Id8a339d78d981376b9af3a80aae8e4e18a68a85f
2014-09-03 21:48:49 -07:00
Matthew Treinish
1179205dbc Make output from check_logs less verbose
This commit changes the output of check_logs.py to be a bit more
concise and clear. Instead of printing out every instance of errors
found in the log files it will only print out which individual files
had errors. If a file is marked as not allowing ERRORS or TRACES it
also prints that along with the filename. This should hopefully avoid
confusion around the operation of this script running in the gate.

Change-Id: Ib2a9a9736470cd3b445fb51a95a1a6c6696bb2cc
2014-09-03 21:42:59 -04:00
Robert Collins
b5e4a9872b Remove test start output in pretty tracer
Sean does not want and its currently broken.

Change-Id: I7c3cf17bfd8d273823774cd162ae011161c36d3d
2014-08-18 13:20:07 +12:00
Matthew Treinish
96e9e88dc0 Enable H407,H305,H307,E122 ignore E123
After bumping the hacking version to the 0.9.x series ignores were
added for several rules. This commit fixes the violations for a subset
of these rules and re-enables the checks.

E123 is ignored because this rule is ignored in the default pep8 by
default, due to a lack of consensus around it.

Change-Id: I8f14cd2ca6afc38d3fe8ee758272071111022896
2014-08-12 17:39:16 -04:00
Robert Mizielski
e1d8899945 Correct misspelled words
Correct some misspelled words:
responCe => responSe,
unkown => unkNown
infomation => infoRmation.

Change-Id: If0f8c2b71381be087694654969d7ea1483da7b17
2014-07-15 14:54:58 +00:00
Matthew Treinish
afeb91a5da Use config generator rc instead of wrapper script
This commit removes the generate_sample wrapper script and adds an
rc file for the config generator to specify the alternate path.
This makes console messages from the generator tooling point
consistent with what is actually being used. Before because a wrapper
was used error messages would recommend running the incorrect tool.

Change-Id: I0ac792454777af6e3f1d718836031f9a14850229
2014-06-19 16:54:48 +00:00
Matthew Treinish
4251685fac Update config generator from oslo-incubator
This commit pulls in the latests version of the config generator from
oslo.

Change-Id: I3a2b06dd9ca3285a3d93298ea5727b705f6109e4
2014-06-19 16:45:48 +00:00
Matthew Treinish
53eef72128 Add total runtime to summary output
This commit adds the total runtime in secs to the summary view from
subunit-trace.py

Change-Id: If56f2a048b3d1ab3d51425d5b3560e1f365a6c3a
2014-06-12 23:06:30 -04:00
Matthew Treinish
1d5f32b48e Change how tempest debug logs are displayed
This commit cleans up how tempest failure logs are displayed. We'll
no longer dump the failure information to the console as a test fails.
Instead all the failure logs will be printed after the tests are run.

Change-Id: I7ecdc349d913b43f4fb0505d5c17c66f811774b4
2014-06-10 15:19:49 +00:00
Jenkins
ad681f9655 Merge "Remove tempest_auto_config script" 2014-05-10 13:24:32 +00:00
Matthew Treinish
2adef1a87f Remove tempest_auto_config script
This script isn't being used by anyone to my knowledge nor has it
really been maintained since it was first added. We will be working
on a plan for automating tempest configuration soon and this
doesn't really fit the initial set requirements of requirements. It
also does not have any testing, so I expect that it has bit rotted
and probably doesn't work anymore.

Change-Id: Id896d9877312dfb6b12b54ea1c598a1940cdd9ce
2014-05-09 15:18:39 -04:00
Sean Dague
04867448e1 create test and worker summary report
a summary report that includes counts of the test results so far
as well as a worker report to let us understand how our workers
are balanced at the moment, as well as if we are missing worker
0 in our data (which seems to happen quite often).

Change-Id: Ide0b63b86e87f8a18c2f0e45264692ac3797c791
2014-05-08 10:25:38 -04:00
Jenkins
a5ddec8309 Merge "Removing unnecessary pass instructions" 2014-05-08 13:15:06 +00:00