f3461c76fe
Post-mortem debugging, the ability to drop into a debugger with the execution state that triggered the exception, is very useful in diagnosing failure conditions. Our previous test runner, nose, provided the ability to enable post-mortem debugging on test failures (via --pdb-failure) and errors (via --pdb). testr lacks these options at present, so this change adds support for enabling post-mortem debugging via an environment variable. All test-triggered exceptions will result in a post-mortem debugger being invoked if OS_POST_MORTEM_DEBUG is set to "1" or "True". Implements: blueprint neutron-pm-debug-on-test-failure Change-Id: Iddbe1335b059d062c0286df2ad27aef7728461b7
101 lines
3.5 KiB
Plaintext
101 lines
3.5 KiB
Plaintext
Testing Neutron
|
|
=============================================================
|
|
|
|
Overview
|
|
|
|
The unit tests are meant to cover as much code as possible and should
|
|
be executed without the service running. They are designed to test
|
|
the various pieces of the neutron tree to make sure any new changes
|
|
don't break existing functionality.
|
|
|
|
The functional tests are intended to validate actual system
|
|
interaction. Mocks should be used sparingly, if at all. Care
|
|
should be taken to ensure that existing system resources are not
|
|
modified and that resources created in tests are properly cleaned
|
|
up.
|
|
|
|
Running tests
|
|
|
|
There are two mechanisms for running tests: run_tests.sh and tox.
|
|
Before submitting a patch for review you should always ensure all
|
|
test pass; a tox run is triggered by the jenkins gate executed on
|
|
gerrit for each patch pushed for review.
|
|
|
|
With both mechanisms you can either run the tests in the standard
|
|
environment or create a virtual environment to run them in.
|
|
|
|
By default after running all of the tests, any pep8 errors
|
|
found in the tree will be reported.
|
|
|
|
Running individual tests
|
|
|
|
For running individual test modules or cases, you just need to pass
|
|
the dot-separated path to the module you want as an argument to it.
|
|
|
|
For executing a specific test case, specify the name of the test case
|
|
class separating it from the module path with a colon.
|
|
|
|
For example, the following would run only the JSONV2TestCase tests from
|
|
neutron/tests/unit/test_api_v2.py:
|
|
|
|
$ ./run_tests.sh neutron.tests.unit.test_api_v2:JSONV2TestCase
|
|
|
|
or
|
|
|
|
$ ./tox neutron.tests.unit.test_api_v2:JSONV2TestCase
|
|
|
|
Adding more tests
|
|
|
|
Neutron has a fast growing code base and there is plenty of areas that
|
|
need to be covered by unit and functional tests.
|
|
|
|
To get a grasp of the areas where tests are needed, you can check
|
|
current coverage by running:
|
|
|
|
$ ./run_tests.sh -c
|
|
|
|
Development process
|
|
|
|
It is expected that any new changes that are proposed for merge
|
|
come with tests for that feature or code area. Ideally any bugs
|
|
fixes that are submitted also have tests to prove that they stay
|
|
fixed! In addition, before proposing for merge, all of the
|
|
current tests should be passing.
|
|
|
|
Debugging
|
|
|
|
By default, calls to pdb.set_trace() will be ignored when tests
|
|
are run. For pdb statements to work, invoke run_tests as follows:
|
|
|
|
$ ./run_tests.sh -d [test module path]
|
|
|
|
It's possible to debug tests in a tox environment:
|
|
|
|
$ tox -e venv -- python -m testtools.run [test module path]
|
|
|
|
Tox-created virtual environments (venv's) can also be activated
|
|
after a tox run and reused for debugging:
|
|
|
|
$ tox -e venv
|
|
$ . .tox/venv/bin/activate
|
|
$ python -m testtools.run [test module path]
|
|
|
|
Tox packages and installs the neutron source tree in a given venv
|
|
on every invocation, but if modifications need to be made between
|
|
invocation (e.g. adding more pdb statements), it is recommended
|
|
that the source tree be installed in the venv in editable mode:
|
|
|
|
# run this only after activating the venv
|
|
$ pip install --editable .
|
|
|
|
Editable mode ensures that changes made to the source tree are
|
|
automatically reflected in the venv, and that such changes are not
|
|
overwritten during the next tox run.
|
|
|
|
Post-mortem debugging
|
|
|
|
Setting OS_POST_MORTEM_DEBUG=1 in the shell environment will ensure
|
|
that pdb.post_mortem() will be invoked on test failure:
|
|
|
|
$ OS_POST_MORTEM_DEBUG=1 ./run_tests.sh -d [test module path]
|