./run_test.sh was deprecated and I used to
run './run_test.sh -8' to only check changed files instead
of whole nova tree. Add this hint 'tox -e pep8 -- -HEAD'
to help people who don't know this.
Change-Id: I0408d0249064e1db89d17c19a524b43a46159a24
The warning message reported by the invocation of the run_tests.sh
script reports a link which is not available anymore. This quick change
modifies the URL to report the correct one.
Change-Id: I60829bf2bb569e26607178112f226e280d8a9f7a
This removes the run_tests.sh script, replacing it with documentation
on how to use tox itself for testing nova things, as well as links to
learn more about both that, and how to not use tox at all if you
really care.
Why should we do this?
New people coming to the project see a file called run_tests.sh and
use it, because why would you have a file like that in the top of your
directory if that wasn't what you were supposed to use. However it's
not what we want anyone using for upstream contributions. One of the
critical differences between it and tox is that it lacks ``python
hashseed randomization``. This runs every test run with a different
forced hashseed, that exposes bugs in tests early. Test contributors
need to be running that on their tests before submission to shake out
these errors early.
In addition, because we have multiple ways in tree to run tests,
helping people debug their test failures means we have to start at a
step -1 of "how are you running tests". This adds load for helping new
folks.
What about people that want to run tests with specific system or
virtual env python setups for pre deployment testing?
Good news! Our tests comply to python standard testing interfaces, so
are runnable by any test runner (we recommend ``testr`` because that's
what tox calls under the covers). You can just run ``testr run`` at
the top level from your virtuenv or system of choice. And life is
good. The constraints we need / want for inbound tests don't apply in
these cases.
This removes a bit of confusion for new contributors, ensures that new
test contributions receive correct local testing before they are run
in our gate, and only require a very minor change in behavior for
downstream test environments.
Change-Id: I05d85a15c0db2a573abb82cb2f7dc3eeaeac0b41
Redirection set up like this will not work as expected as the
duplication (2>&1 part) is set up before redirection, and thus only
stdout ends up getting redirected.
Change-Id: I02f47ee69b48a38db348573259d471cd3b236231
As part of the split of functional and unit tests we need to isolate
the unit tests into a separate directory for having multiple test
targets in a sane way.
Part of bp:functional-tests-for-nova
Change-Id: Id42ba373c1bda6a312b673ab2b489ca56da8c628
Instead of passing lockutils loading explicitly in the test run
command line, provide the support for individual test classes to ask
that lock_path is setup with the REQUIRES_LOCKING = True class
parameter.
Augment all classes that currently fail without this with
REQUIRES_LOCKING = True. Hopefully many of those can be individually
removed in the future so we can get out of needing external locking in
any of our unit tests.
Change-Id: I56229e93ad61ac823646a4119a035d36c3400173
This patch made unit tests stop running (always runs 0 unit tests) we
should revert and put a check in place to prevent that from happening
so that we don't regress the rest of Nova trying to figure out the
oslo.concurrency issue.
This reverts commit 382294762b08abb1921a77b00608ac941478ce05.
Change-Id: I98b04ac2a0ba67ba8af44867799317fdf95b8aff
"run_tests.sh -p" always checks every file for errors even though you
have probably only changed a handful. This patch adds "run_tests.sh -8"
which only checks the files that were modified in the HEAD commit or
the current working tree.
An unscientific benchmark shows that this can be quite a bit faster:
Before:
$ time ./run_tests.sh -p
Running flake8 ...
./nova/compute/manager.py:77:1: F401 'periodic_task' imported but unused
real 0m37.493s
user 0m37.391s
sys 0m0.099s
After:
$ time ./run_tests.sh -8
Running flake8 on nova/compute/manager.py
./nova/compute/manager.py:77:1: F401 'periodic_task' imported but unused
real 0m0.667s
user 0m0.624s
sys 0m0.036s
Change-Id: I8dafaab40fd0427719040a76c69a8ab05e9e4386
Change I007bd39e4d9aad7539b4b7389eb5189b5d1f10e2 fixed how the test
runner was called, but did not take into account that a different
command line is used when -d flag is passed to the script. We fix that but
accounting for the lockuitls change even when -d is passed.
Change-Id: I7462b70a707fff281fabe9d9f20cbb7bf8d037ef
In recent change of commit 78d62186e5b0388f740d42cb8da5798cd67d7880 tox
test has been switched to use openstack.common.lockutils module for
interprocess locks. However, run_tests.sh is left unchanged, which hit a
regression when running tests with run_tests.sh
This change applies the same change in commit 78d621 for tox.ini to
run_tests.sh, i.e. using openstack.common.lockutils to create temporary
directory for locks and pass it as env variables to all tests.
Closes-Bug: #1264058
Change-Id: I007bd39e4d9aad7539b4b7389eb5189b5d1f10e2
By exposing this testr concurrency option, run_tests.sh can now specify
how many processes to run the tests. If not specified, default 0 utilize
maximum CPUs available.
Change-Id: I1a518ae67db32d9838e4a3584cfc3af14212be3c
run_tests.sh -N -p
it will call the flake8 installed on your host to detect PEP8, and
the flake8 plugin "OpenStack hacking" may not installed on your
host, so this command may not detect the OpenStack Style Commandment
supplied by hacking(e.g H202).
run_tests.sh -p
it will call the flake8 from virtual env, flake8 plugin "OpenStack
hacking" installed in virtual env will be triggered.
The result from "run_tests.sh -p" should be trusted, and jenkins uses
virtual env to run flake8 too.
When "-N" is enabled, emit warning to remind user.
Bug #1215404
Change-Id: Ie503fa34e6a3cf7b1f52f0b53019ddea188dabff
In run_tests.sh, function init_testr will initialize testr if the
directory .testrepository is not existed. Actually, testr will do
the check before run the test:
In Python package testrepository, setuptools_command.py:Testr.run
68 def run(self):
69 """Set up testr repo, then run testr"""
70 if not os.path.isdir(".testrepository"):
71 self._run_testr("init")
So, init_testr can be removed safely.
Fixes Bug #1211098
Change-Id: I4bd2ef3dc107281892732efced268eb3b7c90cbb
run_tests.sh had an option to delete tests.sqlite, but that file is not
used anymore so the run_test options don't do anything.
Change-Id: I1abb257579da34c9dd7698543d8297268276bbc5
* run_tests.sh: tools/colorizer.py expects subunit v1 and not v2.
Convert the subunit stream to v1 if subunit-2to1 is present (this
indicates v2 of subunit is being used) before passing the output on to
tools/colorizer.py.
Fixes bug #1168516
Change-Id: I6be917181fa8d18764db89e741190964546d0632
Declarative approach to packaging, obviates the need for copying files.
Also, removes the need for setup.py to import files from the nova
module.
./run_tests.sh now ensure's that nova.egg-info/entry_points.txt is up to
date before running tests.
Change-Id: I7b7a18e065a62491ade54fbed5d8400db1d4e6c7
This was actually a pretty simple fix. When I switched the base
test result class to testtools I lost the methods that were
printing out the errors at the end. This adds the (colorized!)
methods so errors print successfully.
fixes bug 1159116
Change-Id: Ib09d7e18ddf27015ff735a30137421d865382359
while the colorizer is pretty, it has the problem that fail output
actually gets completely swallowed when it is turned on.
Revert the use in run_tests.sh for now until it can be sorted out
so we don't prevent people from using run_tests.sh to do development.
Fixes bug #1159116
Change-Id: I3e6dfdc03c7b033ff132c00a51a368dae9789266
Testr doesn't directly support running tests with pdb so the --debug
option was pulled from run_tests.sh during the testr switch. Add the
flag back into run_tests.sh and have it run tests with testtools.run
which allows users to import pdb in tests then run them without
interference from testr.
Example usage:
# First edit test file to import pdb.
run_tests.sh -d path.to.test
If the test is not given on the command line all tests will be run.
Change-Id: I4afca9ca997ccb6c575b714b778a9db8306c7891
Actually loading the coverage module was leading to odd concurrency
conditions where it was interfering with mocked calls. This changes
to stub out the coverage class with a fake one so that we don't have
to worry about collisions. It also modifies the coverage class to be
created lazily so it isn't automatically started at import time.
Finally it removes the hack in run_tests.sh ignoring the coverage
tests.
Fixes bug 1131023
Change-Id: I39de434454f8b0903f6311e731e215a93027bc91
Running pep8 results in the following error:
Traceback (most recent call last):
File "tools/hacking.py", line 34, in <module>
import pep8
ImportError: No module named pep8
Change-Id: I31a8bfbffa9ddee306ab414fab367c74d34fdd6b
Tox and run_tests.sh were running PEP8 checks against different
file-sets. This patch refactors the logic to determine which files to
run PEP8 checks on into `tools/run_pep8.sh` where it can be called by
both tox and `run_tests.sh`.
Additional fixes:
Some of our Python XenAPI Dom0 plugins don't end in *.py but should
still be checked by PEP8. This patches fixes the hacking.py violations
in the files and adds them back to the srcfiles list.
Merged tools/unused_imports.sh into tools/run_pep8.sh
Change-Id: Id5edd1acb644ab938beffc3473494a179d9d8cda
This commit adds an update option to run_tests.sh. This option
is used to rerun install_venv.py on an already installed venv.
This will then just update out of date packages with pip.
Change-Id: Iac065f71a64a6a7f7f2ee000ee01be9f78af6754
Previous fix (4c891b9243985bb95d8beff8affa9db470a1c94a)
changed how the args are parsed but didn't update the logic
for handling testr args/opts. As a result args/opts for testr
where not be properly passed.
This fixes corrects that oversite and now properly parses those
args as well.
Change-Id: I418b0478b2a180e497d041fcab25158c52f44bac
Fixes: #1116942
* Implementing the * import detection (it is disabled for now)
* New style relative import testing based on syntax rules
* Old style relative import testing based on module search
* Inspection based solution replaced by PYTHONPATH search
in order to avoid module compile and initialization steps
(code execution) in a syntax checking phase.
This solution is faster and safer, but does not able to recognize
modules added dynamically to the module scope.
Change-Id: Ifc871f4fdbcd4a9a736170ceb4475f4f2cbe66bc
The run_tests.sh script invokes
bash -c "${wrapper} $TESTRTESTS | subunit2pyunit"
where '${wrapper}' expands to 'tools/with_venv.sh'. Due to shell
precedence rules, this only applies to '$TESTRTESTS' not the
'subunit2pyunit' part of the command. The 'subunit2pyunit' binary
though may only be available from the .venv
Change-Id: I5972c8f77b3f50a3713881e6ca1acbaeabbd47d4
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
run_tests.sh, with_venv.sh, and the install_venv.py scripts now
support relocating the venv root to another location.
All 3 scripts now support new envinroment variables to configure
the location of the venv and the tools directory. To maintain
compatability the defaults are set to the current values.
venv_path = Location of the virtualenv directory
Default: $(pwd)
venv_name = Name of the virtualenv directory
Default: .venv
tools_path = Location of the tools directory
Default: $(pwd)
Additionally the run_tests.sh script also takes these value as
arguments and will pass them along accordingly.
--virtual-env-path <path> Location of the virtualenv directory
Default: $(pwd)
--virtual-env-name <name> Name of the virtualenv directory
Default: .venv
--tools-path <dir> Location of the tools directory
Default: $(pwd)
DocImpact
Change-Id: I1be036058227206ecca342f692cd3d6aadb24069
Fixes: bug #1116942
When running through run_tests.sh, filter the testr output through
subunit2pyunit so that incremental progress can be seen. Should be
a step towards folks seeing incremental test progress.
Change-Id: Ic90669acabaa1c8b64aae80349bff3d3deaf59fa
* Includes some general tools/hacking cleanup
* Fix several N302 cases
* Disable N302 until all cases are fixed
Change-Id: Iddba07ff13e10dc41a6930749044bb8c0572d279
Using the setup.py testr command in run_tests.sh will sync run_tests.sh
up with the way tox currently runs the tests and collects coverage.
Change-Id: I21e3965280b3242a860c6be49c4dca44f197b03a
Nova coverage was broken because additional coverage tests were added
that interfere with collecting coverage during unit testing. These new
tests were not properly excluded by the coverage unit test regex. Modify
the regex to exclude any tests of the form '.*test.*coverage.*'. This
allows `tox -ecover` and `run_tests.sh --coverage` to run properly.
Change-Id: Id35f50ec335707c9eadb0e7fb3aa0833661f7333
* Fixes N403, along with docstring test
* Adds N404, multi line start
* Disable N403 and N404 until all cases are fixed
* Remove obsolote '--repeat' flag from tox.ini
Change-Id: Ibf6371efc1cdc764e66f6f7c5c4362440101f963
pep8 and hacking.py (by inheritance) have the ability to run self
tests via docstrings, if we format the comments correctly, and change
the docstring test regex. Include self testing in our normal pep8
tests.
fix the comments to actually be test case compliant
fix nova_import_alphabetical to pass on short import lists (only 2
lines, starting at the beginning of a file)
comment where it doesn't appear that hacking.py tests are working
(though there could be deeper black magic in some of them).
fix a doc string in pep8 self tests that breaks N301
fix the fact that pep8 error strings are supposed to start with
\w\d\d\d, we were just lucking out on some formatting because the
string NOVA happened to be only 4 chars long.
Add a couple of escapes so that we don't recursively fail on our
own fail self tests for comments. (boy that's fun)
Change-Id: I14d5e04cfc092b37ec4967cd7485e9975a5eb820