160 lines
4.9 KiB
ReStructuredText
160 lines
4.9 KiB
ReStructuredText
Writing and Running Barbican Tests
|
|
==================================
|
|
|
|
As a part of every code review that is submitted to the Barbican project
|
|
there are a number of gating jobs which aid in the prevention of regression
|
|
issues within Barbican. As a result, a Barbican developer should be familiar
|
|
with running Barbican tests locally.
|
|
|
|
For your convenience we provide the ability to run all tests through
|
|
the ``tox`` utility. If you are unfamiliar with tox please see
|
|
refer to the `tox documentation`_ for assistance.
|
|
|
|
.. _`tox documentation`: https://tox.readthedocs.org/en/latest/
|
|
|
|
Unit Tests
|
|
----------
|
|
|
|
Currently, we provide tox environments for Python 2.7 and 3.5. By default
|
|
all available test environments within the tox configuration will execute
|
|
when calling ``tox``. If you want to run them independently, you can do so
|
|
with the following command:
|
|
|
|
.. code-block:: bash
|
|
|
|
# Executes tests on Python 2.7
|
|
tox -e py27
|
|
|
|
.. note::
|
|
|
|
If you do not have the appropriate Python versions available, consider
|
|
setting up PyEnv to install multiple versions of Python. See the
|
|
documentation regarding :doc:`/setup/dev` for more information.
|
|
|
|
.. note::
|
|
|
|
Individual unit tests can also be run, using the following commands:
|
|
|
|
.. code-block:: bash
|
|
|
|
# runs a single test with the function named
|
|
# test_can_create_new_secret_one_step
|
|
tox -e py27 -- test_can_create_new_secret_one_step
|
|
|
|
# runs only tests in the WhenTestingSecretsResource class and
|
|
# the WhenTestingCAsResource class
|
|
tox -e py27 -- '(WhenTestingSecretsResource|WhenTestingCAsResource)'
|
|
|
|
The function name or class specified must be one located in the
|
|
`barbican/tests` directory.
|
|
|
|
Groups of tests can also be run with a regex match after the ``--``.
|
|
For more information on what can be done with ``testr``, please see:
|
|
http://testrepository.readthedocs.org/en/latest/MANUAL.html
|
|
|
|
You can also setup breakpoints in the unit tests. This can be done by
|
|
adding ``import pdb; pdb.set_trace()`` to the line of the unit test you
|
|
want to examine, then running the following command:
|
|
|
|
.. code-block:: bash
|
|
|
|
# Executes tests on Python 2.7
|
|
tox -e debug
|
|
|
|
.. note::
|
|
|
|
For a list of pdb commands, please see:
|
|
https://docs.python.org/2/library/pdb.html
|
|
|
|
**Python 3.5**
|
|
|
|
In order to run the unit tests within the Python 3.5 unit testing environment
|
|
you need to make sure you have all necessary packages installed.
|
|
|
|
- On Ubuntu/Debian::
|
|
|
|
sudo apt-get install python3-dev
|
|
|
|
- On Fedora 21/RHEL7/CensOS7::
|
|
|
|
sudo yum install python3-devel
|
|
|
|
- On Fedora 22 and higher::
|
|
|
|
sudo dnf install python3-devel
|
|
|
|
You then specify to run the unit tests within the Python 3.5 environment when
|
|
invoking tox
|
|
|
|
.. code-block:: bash
|
|
|
|
# Executes tests on Python 3.5
|
|
tox -e py35
|
|
|
|
Functional Tests
|
|
----------------
|
|
|
|
Unlike running unit tests, the functional tests require Barbican and
|
|
Keystone services to be running in order to execute. For more
|
|
information on :doc:`setting up a Barbican development environment
|
|
</setup/dev>` and using :doc:`Keystone with Barbican </setup/keystone>`,
|
|
see our accompanying project documentation.
|
|
|
|
Once you have the appropriate services running and configured you can execute
|
|
the functional tests through tox.
|
|
|
|
.. code-block:: bash
|
|
|
|
# Execute Barbican Functional Tests
|
|
tox -e functional
|
|
|
|
|
|
By default, the functional tox job will use ``testr`` to execute the
|
|
functional tests as used in the gating job.
|
|
|
|
.. note::
|
|
|
|
In order to run an individual functional test function, you must use the
|
|
following command:
|
|
|
|
.. code-block:: bash
|
|
|
|
# runs a single test with the function named
|
|
# test_secret_create_then_check_content_types
|
|
tox -e functional -- test_secret_create_then_check_content_types
|
|
|
|
# runs only tests in the SecretsTestCase class and
|
|
# the OrdersTestCase class
|
|
tox -e functional -- '(SecretsTestCase|OrdersTestCase)'
|
|
|
|
The function name or class specified must be one located in the
|
|
`functionaltests` directory.
|
|
|
|
Groups of tests can also be run with a regex match after the ``--``.
|
|
For more information on what can be done with ``testr``, please see:
|
|
http://testrepository.readthedocs.org/en/latest/MANUAL.html
|
|
|
|
Remote Debugging
|
|
----------------
|
|
|
|
In order to be able to hit break-points on API calls, you must use remote
|
|
debugging. This can be done by adding ``import rpdb; rpdb.set_trace()`` to
|
|
the line of the API call you wish to test. For example, adding the
|
|
breakpoint in ``def on_post`` in ``barbican.api.controllers.secrets.py``
|
|
will allow you to hit the breakpoint when a ``POST`` is done on the
|
|
secrets URL.
|
|
|
|
.. note::
|
|
|
|
After performing the ``POST`` the application will freeze. In order to use
|
|
``rpdb``, you must open up another terminal and run the following:
|
|
|
|
.. code-block:: bash
|
|
|
|
# enter rpdb using telnet
|
|
telnet localhost 4444
|
|
|
|
Once in rpdb, you can use the same commands as pdb, as seen here:
|
|
https://docs.python.org/2/library/pdb.html
|
|
|