Enhance TESTING.rst
Add detail about api testing and provide better visual separation between the different types of testing. The current testing guidelines are mainly about running tests, and this change does little to fix that. The intention is to add detail about writing tests in subsequent changes. Change-Id: I39d0439c91e5c6edb1d48d4da310443c99fb6d9e
This commit is contained in:
Maru Newby
parent
21bef562c2
commit
3e83a26e66
90
TESTING.rst
90
TESTING.rst
|
|
@ -4,17 +4,47 @@ Testing Neutron
|
|||
Overview
|
||||
--------
|
||||
|
||||
The unit tests (neutron/test/unit/) are meant to cover as much code as
|
||||
Neutron relies on different types of testing to ensure its quality, as
|
||||
described below. In addition to in-tree testing, `Tempest`_ is
|
||||
responsible for validating Neutron's integration with other OpenStack
|
||||
components, and `Rally`_ is responsible for benchmarking.
|
||||
|
||||
.. _Tempest: http://docs.openstack.org/developer/tempest/
|
||||
.. _Rally: http://rally.readthedocs.org/en/latest/
|
||||
|
||||
Unit Tests
|
||||
~~~~~~~~~~
|
||||
|
||||
Unit tests (neutron/test/unit/) are meant to cover as much code as
|
||||
possible and should be executed without the service running. They are
|
||||
designed to test the various pieces of the neutron tree to make sure
|
||||
any new changes don't break existing functionality.
|
||||
|
||||
The functional tests (neutron/tests/functional/) are intended to
|
||||
Functional Tests
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
Functional tests (neutron/tests/functional/) are intended to
|
||||
validate actual system interaction. Mocks should be used sparingly,
|
||||
if at all. Care should be taken to ensure that existing system
|
||||
resources are not modified and that resources created in tests are
|
||||
properly cleaned up.
|
||||
|
||||
API Tests
|
||||
~~~~~~~~~
|
||||
|
||||
API tests (neutron/tests/api/) are intended to ensure the function
|
||||
and stability of the Neutron API. As much as possible, changes to
|
||||
this path should not be made at the same time as changes to the code
|
||||
to limit the potential for introducing backwards-incompatible changes.
|
||||
|
||||
Since API tests need to be able to target a deployed Neutron daemon
|
||||
that is not necessarily test-managed, they should not depend on
|
||||
controlling the runtime configuration of the target daemon. API tests
|
||||
should be black-box - no assumptions should be made about
|
||||
implementation. Only the contract defined by Neutron's REST API
|
||||
should be validated, and all interaction with the daemon should be via
|
||||
a REST client.
|
||||
|
||||
Development process
|
||||
-------------------
|
||||
|
||||
|
|
@ -72,8 +102,8 @@ for DevStack at https://github.com/bcwaldon/vagrant_devstack.
|
|||
machine and develop from there.
|
||||
|
||||
|
||||
Running unit tests
|
||||
------------------
|
||||
Running tests
|
||||
-------------
|
||||
|
||||
There are three mechanisms for running tests: run_tests.sh, tox,
|
||||
and nose2. Before submitting a patch for review you should always
|
||||
|
|
@ -142,11 +172,35 @@ versions of Python (2.6, 2.7, 3.3, etc).
|
|||
Testr handles the parallel execution of series of test cases as well as
|
||||
the tracking of long-running tests and other things.
|
||||
|
||||
Running unit tests is as easy as executing this in the root directory of the
|
||||
Neutron source code::
|
||||
For more information on the standard Tox-based test infrastructure used by
|
||||
OpenStack and how to do some common test/debugging procedures with Testr,
|
||||
see this wiki page:
|
||||
|
||||
https://wiki.openstack.org/wiki/Testr
|
||||
|
||||
.. _Testr: https://wiki.openstack.org/wiki/Testr
|
||||
.. _tox: http://tox.readthedocs.org/en/latest/
|
||||
.. _virtualenvs: https://pypi.python.org/pypi/virtualenv
|
||||
|
||||
PEP8 and Unit Tests
|
||||
===================
|
||||
|
||||
Running pep8 and unit tests is as easy as executing this in the root
|
||||
directory of the Neutron source code::
|
||||
|
||||
tox
|
||||
|
||||
To run only pep8::
|
||||
|
||||
tox -e pep8
|
||||
|
||||
To run only the unit tests::
|
||||
|
||||
tox -e py27
|
||||
|
||||
Functional Tests
|
||||
================
|
||||
|
||||
To run functional tests that do not require sudo privileges or
|
||||
specific-system dependencies::
|
||||
|
||||
|
|
@ -171,6 +225,9 @@ to install and configure all of Neutron's package dependencies. It is
|
|||
not necessary to provide this option if devstack has already been used
|
||||
to deploy Neutron to the target host.
|
||||
|
||||
Fullstack Tests
|
||||
===============
|
||||
|
||||
To run all the full-stack tests, you may use: ::
|
||||
|
||||
tox -e dsvm-fullstack
|
||||
|
|
@ -185,21 +242,20 @@ on the gate, running the dsvm-functional suite will also run all
|
|||
full-stack tests (and a new worker won't be assigned specifically for
|
||||
dsvm-fullstack).
|
||||
|
||||
To run the api tests against a live Neutron daemon, deploy tempest and
|
||||
neutron with devstack and then run the following commands: ::
|
||||
API Tests
|
||||
=========
|
||||
|
||||
To run the api tests, deploy tempest and neutron with devstack and
|
||||
then run the following command: ::
|
||||
|
||||
export TEMPEST_CONFIG_DIR=/opt/stack/tempest/etc
|
||||
tox -e api
|
||||
|
||||
For more information on the standard Tox-based test infrastructure used by
|
||||
OpenStack and how to do some common test/debugging procedures with Testr,
|
||||
see this wiki page:
|
||||
If tempest.conf cannot be found at the default location used by
|
||||
devstack (/opt/stack/tempest/etc) it may be necessary to set
|
||||
TEMPEST_CONFIG_DIR before invoking tox: ::
|
||||
|
||||
https://wiki.openstack.org/wiki/Testr
|
||||
|
||||
.. _Testr: https://wiki.openstack.org/wiki/Testr
|
||||
.. _tox: http://tox.readthedocs.org/en/latest/
|
||||
.. _virtualenvs: https://pypi.python.org/pypi/virtualenv
|
||||
export TEMPEST_CONFIG_DIR=[path to dir containing tempest.conf]
|
||||
tox -e api
|
||||
|
||||
|
||||
Running individual tests
|
||||
|
|
|
|||
Loading…
Reference in New Issue