Merge "Update the Tempest README for new run workflow"
This commit is contained in:
Jenkins
Gerrit Code Review
commit
0a7a8e59ad
137
README.rst
137
README.rst
|
|
@ -37,57 +37,61 @@ Tempest Design Principles that we strive to live by.
|
|||
Quickstart
|
||||
----------
|
||||
|
||||
To run Tempest, you first need to create a configuration file that
|
||||
will tell Tempest where to find the various OpenStack services and
|
||||
other testing behavior switches.
|
||||
To run Tempest, you first need to create a configuration file that will tell
|
||||
Tempest where to find the various OpenStack services and other testing behavior
|
||||
switches. Where the configuration file lives and how you interact with it
|
||||
depends on how you'll be running Tempest. There are 2 methods of using Tempest.
|
||||
The first, which is a newer and recommended workflow treats Tempest as a system
|
||||
installed program. The second older method is to run Tempest assuming your
|
||||
working dir is the actually Tempest source repo, and there are a number of
|
||||
assumptions related to that. For this section we'll only cover the newer method
|
||||
as it is simpler, and quicker to work with.
|
||||
|
||||
The easiest way to create a configuration file is to generate a sample
|
||||
in the ``etc/`` directory ::
|
||||
#. You first need to install Tempest this is done with pip, after you check out
|
||||
the Tempest repo you simply run something like::
|
||||
|
||||
$> cd $TEMPEST_ROOT_DIR
|
||||
$> oslo-config-generator --config-file \
|
||||
tools/config/config-generator.tempest.conf \
|
||||
--output-file etc/tempest.conf
|
||||
$ pip install tempest
|
||||
|
||||
After that, open up the ``etc/tempest.conf`` file and edit the
|
||||
configuration variables to match valid data in your environment.
|
||||
This includes your Keystone endpoint, a valid user and credentials,
|
||||
and reference data to be used in testing.
|
||||
This can be done within a venv, but the assumption for this guide is that
|
||||
the Tempest cli entry point will be in your shell's PATH.
|
||||
|
||||
.. note::
|
||||
#. Installing Tempest will create a /etc/tempest dir which will contain the
|
||||
sample config file packaged with Tempest. The contents of /etc/tempest will
|
||||
be copied to all local working dirs, so if there is any common configuration
|
||||
you'd like to be shared between anyone setting up local Tempest working dirs
|
||||
it's recommended that you copy or rename tempest.conf.sample to tempest.conf
|
||||
and make those changes to that file in /etc/tempest
|
||||
|
||||
If you have a running devstack environment, Tempest will be
|
||||
automatically configured and placed in ``/opt/stack/tempest``. It
|
||||
will have a configuration file already set up to work with your
|
||||
devstack installation.
|
||||
#. Setup a local working Tempest dir. This is done using the tempest init
|
||||
command::
|
||||
|
||||
Tempest is not tied to any single test runner, but `testr`_ is the most commonly
|
||||
used tool. Also, the nosetests test runner is **not** recommended to run Tempest.
|
||||
tempest init cloud-01
|
||||
|
||||
After setting up your configuration file, you can execute the set of Tempest
|
||||
tests by using ``testr`` ::
|
||||
works the same as::
|
||||
|
||||
$> testr run --parallel
|
||||
mkdir cloud-01 && cd cloud-01 && tempest init
|
||||
|
||||
.. _testr: http://testrepository.readthedocs.org/en/latest/MANUAL.html
|
||||
This will create a new directory for running a single Tempest configuration.
|
||||
If you'd like to run Tempest against multiple OpenStack deployments the idea
|
||||
is that you'll create a new working directory for each to maintain separate
|
||||
configuration files and local artifact storage for each.
|
||||
|
||||
To run one single test serially ::
|
||||
#. Then cd into the newly created working dir and also modify the local
|
||||
config files located in the etc/ subdir created by the ``tempest init``
|
||||
command. Tempest is expecting a tempest.conf file in etc/ so if only a
|
||||
sample exists you must rename or copy it to tempest.conf before making
|
||||
any changes to it otherwise Tempest will not know how to load it.
|
||||
|
||||
$> testr run tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server
|
||||
#. Once the configuration is done you're now ready to run Tempest. This can
|
||||
be done with testr directly or any `testr`_ based test runner, like
|
||||
`ostestr`_. For example, from the working dir running::
|
||||
|
||||
Alternatively, you can use the run_tempest.sh script which will create a venv
|
||||
and run the tests or use tox to do the same. Tox also contains several existing
|
||||
job configurations. For example::
|
||||
$ ostestr --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.(api|scenario|thirdparty))'
|
||||
|
||||
$> tox -efull
|
||||
|
||||
which will run the same set of tests as the OpenStack gate. (it's exactly how
|
||||
the gate invokes Tempest) Or::
|
||||
|
||||
$> tox -esmoke
|
||||
|
||||
to run the tests tagged as smoke.
|
||||
will run the same set of tests as the default gate jobs.
|
||||
|
||||
.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
|
||||
.. _ostestr: http://docs.openstack.org/developer/os-testr/
|
||||
|
||||
Configuration
|
||||
-------------
|
||||
|
|
@ -146,3 +150,62 @@ work on Python 3.4. However, because large parts of Tempest are self-verifying
|
|||
there might be uncaught issues running on Python 3.4. So until there is a gating
|
||||
job which does a full Tempest run using Python 3.4 there isn't any guarantee
|
||||
that running Tempest under Python 3.4 is bug free.
|
||||
|
||||
Legacy run method
|
||||
-----------------
|
||||
|
||||
The legacy method of running Tempest is to just treat the Tempest source code
|
||||
as a python unittest repository and run directly from the source repo. When
|
||||
running in this way you still start with a Tempest config file and the steps
|
||||
are basically the same except that it expects you know where the Tempest code
|
||||
lives on your system and requires a bit more manual interaction to get Tempest
|
||||
running. For example, when running Tempest this way things like a lock file
|
||||
directory do not get generated automatically and the burden is on the user to
|
||||
create and configure that.
|
||||
|
||||
To start you need to create a configuration file. The easiest way to create a
|
||||
configuration file is to generate a sample in the ``etc/`` directory ::
|
||||
|
||||
$> cd $TEMPEST_ROOT_DIR
|
||||
$> oslo-config-generator --config-file \
|
||||
tools/config/config-generator.tempest.conf \
|
||||
--output-file etc/tempest.conf
|
||||
|
||||
After that, open up the ``etc/tempest.conf`` file and edit the
|
||||
configuration variables to match valid data in your environment.
|
||||
This includes your Keystone endpoint, a valid user and credentials,
|
||||
and reference data to be used in testing.
|
||||
|
||||
.. note::
|
||||
|
||||
If you have a running devstack environment, Tempest will be
|
||||
automatically configured and placed in ``/opt/stack/tempest``. It
|
||||
will have a configuration file already set up to work with your
|
||||
devstack installation.
|
||||
|
||||
Tempest is not tied to any single test runner, but `testr`_ is the most commonly
|
||||
used tool. Also, the nosetests test runner is **not** recommended to run Tempest.
|
||||
|
||||
After setting up your configuration file, you can execute the set of Tempest
|
||||
tests by using ``testr`` ::
|
||||
|
||||
$> testr run --parallel
|
||||
|
||||
.. _testr: http://testrepository.readthedocs.org/en/latest/MANUAL.html
|
||||
|
||||
To run one single test serially ::
|
||||
|
||||
$> testr run tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server
|
||||
|
||||
Alternatively, you can use the run_tempest.sh script which will create a venv
|
||||
and run the tests or use tox to do the same. Tox also contains several existing
|
||||
job configurations. For example::
|
||||
|
||||
$> tox -efull
|
||||
|
||||
which will run the same set of tests as the OpenStack gate. (it's exactly how
|
||||
the gate invokes Tempest) Or::
|
||||
|
||||
$> tox -esmoke
|
||||
|
||||
to run the tests tagged as smoke.
|
||||
|
|
|
|||
Loading…
Reference in New Issue