doc: Update testing documentation
The information on how to run functional tests was badly out of date. Correct it. Change-Id: I7e77e2a96ab15f7dede01344cb03b0e61c8589bd Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This commit is contained in:
Stephen Finucane
parent
05a83963f7
commit
11d89450c1
@ -1,31 +0,0 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# NOTE(thowe): There are some issues with OCC envvars that force us to do
|
||||
# this for now.
|
||||
#
|
||||
mkdir -p ~/.config/openstack/
|
||||
FILE=~/.config/openstack/clouds.yaml
|
||||
export OS_IDENTITY_API_VERSION=3 # force v3 identity
|
||||
echo 'clouds:' >$FILE
|
||||
echo ' test_cloud:' >>$FILE
|
||||
env | grep OS_ | tr '=' ' ' | while read k v
|
||||
do
|
||||
k=$(echo $k | sed -e 's/OS_//')
|
||||
k=$(echo $k | tr '[A-Z]' '[a-z]')
|
||||
case "$k" in
|
||||
region_name|*_api_version)
|
||||
echo " $k: $v" >>$FILE
|
||||
esac
|
||||
done
|
||||
echo " auth:" >>$FILE
|
||||
env | grep OS_ | tr '=' ' ' | while read k v
|
||||
do
|
||||
k=$(echo $k | sed -e 's/OS_//')
|
||||
k=$(echo $k | tr '[A-Z]' '[a-z]')
|
||||
case "$k" in
|
||||
region_name|*_api_version)
|
||||
;;
|
||||
*)
|
||||
echo " $k: $v" >>$FILE
|
||||
esac
|
||||
done
|
||||
@ -1,19 +1,37 @@
|
||||
clouds:
|
||||
test_cloud:
|
||||
region_name: RegionOne
|
||||
devstack:
|
||||
auth:
|
||||
auth_url: http://xxx.xxx.xxx.xxx:5000/v2.0/
|
||||
username: demo
|
||||
password: secrete
|
||||
auth_url: http://xxx.xxx.xxx.xxx/identity
|
||||
password: password
|
||||
project_domain_id: default
|
||||
project_name: demo
|
||||
rackspace:
|
||||
cloud: rackspace
|
||||
user_domain_id: default
|
||||
username: demo
|
||||
identity_api_version: '3'
|
||||
region_name: RegionOne
|
||||
volume_api_version: '3'
|
||||
devstack-admin:
|
||||
auth:
|
||||
username: joe
|
||||
password: joes-password
|
||||
project_name: 123123
|
||||
region_name: IAD
|
||||
auth_url: http://xxx.xxx.xxx.xxx/identity
|
||||
password: password
|
||||
project_domain_id: default
|
||||
project_name: admin
|
||||
user_domain_id: default
|
||||
username: admin
|
||||
identity_api_version: '3'
|
||||
region_name: RegionOne
|
||||
volume_api_version: '3'
|
||||
devstack-alt:
|
||||
auth:
|
||||
auth_url: http://xxx.xxx.xxx.xxx/identity
|
||||
password: password
|
||||
project_domain_id: default
|
||||
project_name: alt_demo
|
||||
user_domain_id: default
|
||||
username: alt_demo
|
||||
identity_api_version: '3'
|
||||
region_name: RegionOne
|
||||
volume_api_version: '3'
|
||||
example:
|
||||
image_name: fedora-20.x86_64
|
||||
image_name: cirros-0.5.2-x86_64-disk
|
||||
flavor_name: m1.small
|
||||
network_name: private
|
||||
|
||||
@ -3,25 +3,27 @@ Testing
|
||||
|
||||
The tests are run with `tox <https://tox.readthedocs.org/en/latest/>`_ and
|
||||
configured in ``tox.ini``. The test results are tracked by
|
||||
`testr <https://testrepository.readthedocs.org/en/latest/>`_ and configured
|
||||
in ``.testr.conf``.
|
||||
`stestr <https://stestr.readthedocs.io/en/latest/>`_ and configured
|
||||
in ``.stestr.conf`` and via command line options passed to the ``stestr``
|
||||
executable when it's called by ``tox``.
|
||||
|
||||
|
||||
Unit Tests
|
||||
----------
|
||||
|
||||
Run
|
||||
***
|
||||
Running tests
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
In order to run the entire unit test suite, simply run the ``tox`` command
|
||||
inside of your source checkout. This will attempt to run every test command
|
||||
listed inside of ``tox.ini``, which includes Python 3.8, and a PEP 8 check.
|
||||
listed inside of ``tox.ini``, which includes Python 3.x, and a PEP 8 check.
|
||||
You should run the full test suite on all versions before
|
||||
submitting changes for review in order to avoid unexpected failures in the
|
||||
continuous integration system.::
|
||||
|
||||
(sdk3)$ tox
|
||||
$ tox
|
||||
...
|
||||
py38: commands succeeded
|
||||
py3: commands succeeded
|
||||
pep8: commands succeeded
|
||||
congratulations :)
|
||||
|
||||
@ -29,8 +31,13 @@ During development, it may be more convenient to run a subset of the tests
|
||||
to keep test time to a minimum. You can choose to run the tests only on one
|
||||
version. A step further is to run only the tests you are working on.::
|
||||
|
||||
(sdk3)$ tox -e py38 # Run run the tests on Python 3.8
|
||||
(sdk3)$ tox -e py38 TestContainer # Run only the TestContainer tests on 3.8
|
||||
# Run run the tests on Python 3.9
|
||||
$ tox -e py39
|
||||
# Run only the compute unit tests on Python 3.9
|
||||
$ tox -e py39 openstack.tests.unit.compute
|
||||
# Run only the tests in a specific file on Python 3.9
|
||||
$ tox -e py39 -- -n openstack/tests/unit/compute/test_version.py
|
||||
|
||||
|
||||
Functional Tests
|
||||
----------------
|
||||
@ -41,17 +48,43 @@ public clouds but first and foremost they must be run against OpenStack. In
|
||||
practice, this means that the tests should initially be run against a stable
|
||||
branch of `DevStack <https://docs.openstack.org/devstack/latest/>`_.
|
||||
|
||||
os-client-config
|
||||
****************
|
||||
Configuration
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
To connect the functional tests to an OpenStack cloud we use
|
||||
`os-client-config <http://opendev.org/openstack/os-client-config>`_.
|
||||
To setup os-client-config create a ``clouds.yaml`` file in the root of your
|
||||
source checkout.
|
||||
To connect the functional tests to an OpenStack cloud we require a
|
||||
``clouds.yaml`` file, as discussed in :doc:`/user/config/configuration`.
|
||||
You can place this ``clouds.yaml`` file in the root of your source checkout or
|
||||
in one of the other standard locations, ``$HOME/.config/openstack`` or
|
||||
``/etc/openstack``.
|
||||
|
||||
There must be at least three clouds configured, or rather three accounts
|
||||
configured for the one cloud. These accounts are:
|
||||
|
||||
- An admin account, which defaults to ``devstack-admin`` but is configurable
|
||||
via the ``OPENSTACKSDK_OPERATOR_CLOUD`` environment variable,
|
||||
- A user account, which defaults to ``devstack`` but is configurable
|
||||
via the ``OPENSTACKSDK_DEMO_CLOUD`` environment variable, and
|
||||
- An alternate user account, which defaults to ``devstack-demo`` but is
|
||||
configurable via the ``OPENSTACKSDK_DEMO_CLOUD_ALT`` environment variable
|
||||
|
||||
In addition, you must indicate the names of the flavor and image that should be
|
||||
used for tests. These can be configured via ``functional.flavor_name`` and
|
||||
``functional.image_name`` settings in the ``clouds.yaml`` file.
|
||||
|
||||
Finally, you can configure the timeout for tests using the
|
||||
``OPENSTACKSDK_FUNC_TEST_TIMEOUT`` environment variable (defaults to 300
|
||||
seconds). Some test modules take specific timeout values. For example, all
|
||||
tests in ``openstack.tests.functional.compute`` will check for the
|
||||
``OPENSTACKSDK_FUNC_TEST_TIMEOUT_COMPUTE`` environment variable before checking
|
||||
for ``OPENSTACKSDK_FUNC_TEST_TIMEOUT``.
|
||||
|
||||
.. note::
|
||||
|
||||
Recent versions of DevStack will configure a suitable ``clouds.yaml`` file
|
||||
for you, which will be placed at ``/etc/openstack/clouds.yaml``.
|
||||
|
||||
This is an example of a minimal configuration for a ``clouds.yaml`` that
|
||||
connects the functional tests to a DevStack instance. Note that one cloud
|
||||
under ``clouds`` must be named ``test_cloud``.
|
||||
connects the functional tests to a DevStack instance.
|
||||
|
||||
.. literalinclude:: clouds.yaml
|
||||
:language: yaml
|
||||
@ -59,29 +92,16 @@ under ``clouds`` must be named ``test_cloud``.
|
||||
Replace ``xxx.xxx.xxx.xxx`` with the IP address or FQDN of your DevStack
|
||||
instance.
|
||||
|
||||
You can also create a ``~/.config/openstack/clouds.yaml`` file for your
|
||||
DevStack cloud environment using the following commands. Replace
|
||||
``DEVSTACK_SOURCE`` with your DevStack source checkout.::
|
||||
Running tests
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
(sdk3)$ source DEVSTACK_SOURCE/accrc/admin/admin
|
||||
(sdk3)$ ./create_yaml.sh
|
||||
Functional tests are also run against multiple Python versions. In order to run
|
||||
the entire functional test suite against the default Python 3 version in your
|
||||
environment, run the ``tox -e functional`` command inside of your source
|
||||
checkout. This will attempt to run every tests in the
|
||||
``openstack/tests/functional`` directory. For example::
|
||||
|
||||
Run
|
||||
***
|
||||
|
||||
Functional tests are run against both Python 2 and 3. In order to run the
|
||||
entire functional test suite, run the ``tox -e functional`` and
|
||||
``tox -e functional3`` command inside of your source checkout. This will
|
||||
attempt to run every test command under ``/openstack/tests/functional/``
|
||||
in the source tree. You should run the full functional test suite before
|
||||
submitting changes for review in order to avoid unexpected failures in
|
||||
the continuous integration system.::
|
||||
|
||||
(sdk3)$ tox -e functional
|
||||
$ tox -e functional
|
||||
...
|
||||
functional: commands succeeded
|
||||
congratulations :)
|
||||
(sdk3)$ tox -e functional3
|
||||
...
|
||||
functional3: commands succeeded
|
||||
congratulations :)
|
||||
|
||||
2
tox.ini
2
tox.ini
@ -25,7 +25,7 @@ commands =
|
||||
stestr run {posargs}
|
||||
stestr slowest
|
||||
|
||||
[testenv:functional]
|
||||
[testenv:functional{,-py36,-py37,-py38,-py39}]
|
||||
# Some jobs (especially heat) takes longer, therefore increase default timeout
|
||||
# This timeout should not be smaller, than the longest individual timeout
|
||||
setenv =
|
||||
|
||||
Loading…
Reference in New Issue
Block a user