Merge "Fix docs markup consistency"
This commit is contained in:
Zuul
Gerrit Code Review
commit
dba764c8f3
26
HACKING.rst
26
HACKING.rst
@ -121,38 +121,38 @@ however they do not need to be tagged as ``compute``.
|
||||
Test fixtures and resources
|
||||
---------------------------
|
||||
Test level resources should be cleaned-up after the test execution. Clean-up
|
||||
is best scheduled using `addCleanup` which ensures that the resource cleanup
|
||||
is best scheduled using ``addCleanup`` which ensures that the resource cleanup
|
||||
code is always invoked, and in reverse order with respect to the creation
|
||||
order.
|
||||
|
||||
Test class level resources should be defined in the `resource_setup` method of
|
||||
the test class, except for any credential obtained from the credentials
|
||||
provider, which should be set-up in the `setup_credentials` method.
|
||||
Cleanup is best scheduled using `addClassResourceCleanup` which ensures that
|
||||
Test class level resources should be defined in the ``resource_setup`` method
|
||||
of the test class, except for any credential obtained from the credentials
|
||||
provider, which should be set-up in the ``setup_credentials`` method.
|
||||
Cleanup is best scheduled using ``addClassResourceCleanup`` which ensures that
|
||||
the cleanup code is always invoked, and in reverse order with respect to the
|
||||
creation order.
|
||||
|
||||
In both cases - test level and class level cleanups - a wait loop should be
|
||||
scheduled before the actual delete of resources with an asynchronous delete.
|
||||
|
||||
The test base class `BaseTestCase` defines Tempest framework for class level
|
||||
fixtures. `setUpClass` and `tearDownClass` are defined here and cannot be
|
||||
The test base class ``BaseTestCase`` defines Tempest framework for class level
|
||||
fixtures. ``setUpClass`` and ``tearDownClass`` are defined here and cannot be
|
||||
overwritten by subclasses (enforced via hacking rule T105).
|
||||
|
||||
Set-up is split in a series of steps (setup stages), which can be overwritten
|
||||
by test classes. Set-up stages are:
|
||||
|
||||
- `skip_checks`
|
||||
- `setup_credentials`
|
||||
- `setup_clients`
|
||||
- `resource_setup`
|
||||
- ``skip_checks``
|
||||
- ``setup_credentials``
|
||||
- ``setup_clients``
|
||||
- ``resource_setup``
|
||||
|
||||
Tear-down is also split in a series of steps (teardown stages), which are
|
||||
stacked for execution only if the corresponding setup stage had been
|
||||
reached during the setup phase. Tear-down stages are:
|
||||
|
||||
- `clear_credentials` (defined in the base test class)
|
||||
- `resource_cleanup`
|
||||
- ``clear_credentials`` (defined in the base test class)
|
||||
- ``resource_cleanup``
|
||||
|
||||
Skipping Tests
|
||||
--------------
|
||||
|
||||
@ -62,9 +62,9 @@ and failure rates on the bottom graph. For example, something like `this URL`_.
|
||||
The Old Way using subunit2sql directly
|
||||
""""""""""""""""""""""""""""""""""""""
|
||||
|
||||
SELECT * from tests where test_id like "%test_id%";
|
||||
(where $test_id is the full test_id, but truncated to the class because of
|
||||
setUpClass or tearDownClass failures)
|
||||
``SELECT * from tests where test_id like "%test_id%";``
|
||||
(where ``$test_id`` is the full test_id, but truncated to the class because of
|
||||
``setUpClass`` or ``tearDownClass`` failures)
|
||||
|
||||
You can access the infra mysql subunit2sql db w/ read-only permissions with:
|
||||
|
||||
@ -74,15 +74,20 @@ You can access the infra mysql subunit2sql db w/ read-only permissions with:
|
||||
* db_name: subunit2sql
|
||||
|
||||
For example if you were trying to remove the test with the id:
|
||||
tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON.test_get_flavor_details_for_deleted_flavor
|
||||
``tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON.test_get_flavor_details_for_deleted_flavor``
|
||||
you would run the following:
|
||||
|
||||
#. run: "mysql -u query -p -h logstash.openstack.org subunit2sql" to connect
|
||||
to the subunit2sql db
|
||||
#. run the query: MySQL [subunit2sql]> select * from tests where test_id like
|
||||
#. run the command: ``mysql -u query -p -h logstash.openstack.org subunit2sql``
|
||||
to connect to the subunit2sql db
|
||||
#. run the query:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
MySQL [subunit2sql]> select * from tests where test_id like \
|
||||
"tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON%";
|
||||
|
||||
which will return a table of all the tests in the class (but it will also
|
||||
catch failures in setUpClass and tearDownClass)
|
||||
catch failures in ``setUpClass`` and ``tearDownClass``)
|
||||
#. paste the output table with numbers and the mysql command you ran to
|
||||
generate it into the etherpad.
|
||||
|
||||
|
||||
@ -15,18 +15,18 @@
|
||||
# under the License.
|
||||
|
||||
"""
|
||||
Utility for creating **accounts.yaml** file for concurrent test runs.
|
||||
Utility for creating ``accounts.yaml`` file for concurrent test runs.
|
||||
Creates one primary user, one alt user, one swift admin, one stack owner
|
||||
and one admin (optionally) for each concurrent thread. The utility creates
|
||||
user for each tenant. The **accounts.yaml** file will be valid and contain
|
||||
user for each tenant. The ``accounts.yaml`` file will be valid and contain
|
||||
credentials for created users, so each user will be in separate tenant and
|
||||
have the username, tenant_name, password and roles.
|
||||
|
||||
**Usage:** ``tempest account-generator [-h] [OPTIONS] accounts_file.yaml``.
|
||||
**Usage:** ``tempest account-generator [-h] [OPTIONS] accounts_file.yaml``
|
||||
|
||||
Positional Arguments
|
||||
--------------------
|
||||
**accounts_file.yaml** (Required) Provide an output accounts yaml file. Utility
|
||||
``accounts_file.yaml`` (Required) Provide an output accounts yaml file. Utility
|
||||
creates a .yaml file in the directory where the command is ran. The appropriate
|
||||
name for the file is *accounts.yaml* and it should be placed in *tempest/etc*
|
||||
directory.
|
||||
@ -40,24 +40,24 @@ through CLI options or environment variables.
|
||||
|
||||
You're probably familiar with these, but just to remind:
|
||||
|
||||
======== ======================== ====================
|
||||
======== ============================ ====================
|
||||
Param CLI Environment Variable
|
||||
======== ======================== ====================
|
||||
Username --os-username OS_USERNAME
|
||||
Password --os-password OS_PASSWORD
|
||||
Project --os-project-name OS_PROJECT_NAME
|
||||
Tenant --os-tenant-name (depr.) OS_TENANT_NAME
|
||||
Domain --os-domain-name OS_DOMAIN_NAME
|
||||
======== ======================== ====================
|
||||
======== ============================ ====================
|
||||
Username ``--os-username`` OS_USERNAME
|
||||
Password ``--os-password`` OS_PASSWORD
|
||||
Project ``--os-project-name`` OS_PROJECT_NAME
|
||||
Tenant ``--os-tenant-name`` (depr.) OS_TENANT_NAME
|
||||
Domain ``--os-domain-name`` OS_DOMAIN_NAME
|
||||
======== ============================ ====================
|
||||
|
||||
Optional Arguments
|
||||
------------------
|
||||
**-h**, **--help** (Optional) Shows help message with the description of
|
||||
* ``-h, --help`` (Optional) Shows help message with the description of
|
||||
utility and its arguments, and exits.
|
||||
|
||||
**-c /etc/tempest.conf**, **--config-file /etc/tempest.conf** (Optional) Path
|
||||
to tempest config file. If not specified, it searches for tempest.conf in these
|
||||
locations:
|
||||
* ``-c, --config-file /etc/tempest.conf`` (Optional) Path
|
||||
to tempest config file. If not specified, it searches for tempest.conf in
|
||||
these locations:
|
||||
|
||||
- ./etc/
|
||||
- /etc/tempest
|
||||
@ -65,36 +65,36 @@ locations:
|
||||
- ~/
|
||||
- /etc/
|
||||
|
||||
**--os-username <auth-user-name>** (Optional) Name used for authentication with
|
||||
the OpenStack Identity service. Defaults to env[OS_USERNAME]. Note: User should
|
||||
have permissions to create new user accounts and tenants.
|
||||
* ``--os-username <auth-user-name>`` (Optional) Name used for authentication
|
||||
with the OpenStack Identity service. Defaults to env[OS_USERNAME]. Note: User
|
||||
should have permissions to create new user accounts and tenants.
|
||||
|
||||
**--os-password <auth-password>** (Optional) Password used for authentication
|
||||
* ``--os-password <auth-password>`` (Optional) Password used for authentication
|
||||
with the OpenStack Identity service. Defaults to env[OS_PASSWORD].
|
||||
|
||||
**--os-project-name <auth-project-name>** (Optional) Project to request
|
||||
* ``--os-project-name <auth-project-name>`` (Optional) Project to request
|
||||
authorization on. Defaults to env[OS_PROJECT_NAME].
|
||||
|
||||
**--os-tenant-name <auth-tenant-name>** (Optional, deprecated) Tenant to
|
||||
* ``--os-tenant-name <auth-tenant-name>`` (Optional, deprecated) Tenant to
|
||||
request authorization on. Defaults to env[OS_TENANT_NAME].
|
||||
|
||||
**--os-domain-name <auth-domain-name>** (Optional) Domain the user and project
|
||||
belong to. Defaults to env[OS_DOMAIN_NAME].
|
||||
* ``--os-domain-name <auth-domain-name>`` (Optional) Domain the user and
|
||||
project belong to. Defaults to env[OS_DOMAIN_NAME].
|
||||
|
||||
**--tag TAG** (Optional) Resources tag. Each created resource (user, project)
|
||||
* ``--tag TAG`` (Optional) Resources tag. Each created resource (user, project)
|
||||
will have the prefix with the given TAG in its name. Using tag is recommended
|
||||
for the further using, cleaning resources.
|
||||
|
||||
**-r CONCURRENCY**, **--concurrency CONCURRENCY** (Optional) Concurrency count
|
||||
* ``-r, --concurrency CONCURRENCY`` (Optional) Concurrency count
|
||||
(default: 1). The number of accounts required can be estimated as
|
||||
CONCURRENCY x 2. Each user provided in *accounts.yaml* file will be in
|
||||
a different tenant. This is required to provide isolation between test for
|
||||
running in parallel.
|
||||
|
||||
**--with-admin** (Optional) Creates admin for each concurrent group
|
||||
* ``--with-admin`` (Optional) Creates admin for each concurrent group
|
||||
(default: False).
|
||||
|
||||
**-i VERSION**, **--identity-version VERSION** (Optional) Provisions accounts
|
||||
* ``-i, --identity-version VERSION`` (Optional) Provisions accounts
|
||||
using the specified version of the identity API. (default: '3').
|
||||
|
||||
To see help on specific argument, please do: ``tempest account-generator
|
||||
|
||||
@ -28,45 +28,48 @@ specified in ``tempest.conf`` is never deleted.
|
||||
Example Run
|
||||
-----------
|
||||
|
||||
**WARNING: If step 1 is skipped in the example below, the cleanup procedure
|
||||
.. warning::
|
||||
|
||||
If step 1 is skipped in the example below, the cleanup procedure
|
||||
may delete resources that existed in the cloud before the test run. This
|
||||
may cause an unwanted destruction of cloud resources, so use caution with
|
||||
this command.**
|
||||
this command.
|
||||
|
||||
``$ tempest cleanup --init-saved-state``
|
||||
Examples::
|
||||
|
||||
``$ # Actual running of Tempest tests``
|
||||
|
||||
``$ tempest cleanup``
|
||||
$ tempest cleanup --init-saved-state
|
||||
$ # Actual running of Tempest tests
|
||||
$ tempest cleanup
|
||||
|
||||
Runtime Arguments
|
||||
-----------------
|
||||
|
||||
**--init-saved-state**: Initializes the saved state of the OpenStack deployment
|
||||
and will output a ``saved_state.json`` file containing resources from your
|
||||
deployment that will be preserved from the cleanup command. This should be
|
||||
done prior to running Tempest tests.
|
||||
* ``--init-saved-state``: Initializes the saved state of the OpenStack
|
||||
deployment and will output a ``saved_state.json`` file containing resources
|
||||
from your deployment that will be preserved from the cleanup command. This
|
||||
should be done prior to running Tempest tests.
|
||||
|
||||
**--delete-tempest-conf-objects**: If option is present, then the command will
|
||||
delete the admin project in addition to the resources associated with them on
|
||||
clean up. If option is not present, the command will delete the resources
|
||||
associated with the Tempest and alternate Tempest users and projects but will
|
||||
not delete the projects themselves.
|
||||
* ``--delete-tempest-conf-objects``: If option is present, then the command
|
||||
will delete the admin project in addition to the resources associated with
|
||||
them on clean up. If option is not present, the command will delete the
|
||||
resources associated with the Tempest and alternate Tempest users and
|
||||
projects but will not delete the projects themselves.
|
||||
|
||||
**--dry-run**: Creates a report (``./dry_run.json``) of the projects that will
|
||||
be cleaned up (in the ``_projects_to_clean`` dictionary [1]_) and the global
|
||||
objects that will be removed (domains, flavors, images, roles, projects,
|
||||
and users). Once the cleanup command is executed (e.g. run without
|
||||
parameters), running it again with **--dry-run** should yield an empty report.
|
||||
* ``--dry-run``: Creates a report (``./dry_run.json``) of the projects that
|
||||
will be cleaned up (in the ``_projects_to_clean`` dictionary [1]_) and the
|
||||
global objects that will be removed (domains, flavors, images, roles,
|
||||
projects, and users). Once the cleanup command is executed (e.g. run without
|
||||
parameters), running it again with ``--dry-run`` should yield an empty
|
||||
report.
|
||||
|
||||
**--help**: Print the help text for the command and parameters.
|
||||
* ``--help``: Print the help text for the command and parameters.
|
||||
|
||||
.. [1] The ``_projects_to_clean`` dictionary in ``dry_run.json`` lists the
|
||||
projects that ``tempest cleanup`` will loop through to delete child
|
||||
objects, but the command will, by default, not delete the projects
|
||||
themselves. This may differ from the ``projects`` list as you can clean
|
||||
the Tempest and alternate Tempest users and projects but they will not be
|
||||
deleted unless the **--delete-tempest-conf-objects** flag is used to
|
||||
deleted unless the ``--delete-tempest-conf-objects`` flag is used to
|
||||
force their deletion.
|
||||
|
||||
"""
|
||||
|
||||
@ -19,11 +19,11 @@ Test Selection
|
||||
==============
|
||||
Tempest run has several options:
|
||||
|
||||
* **--regex/-r**: This is a selection regex like what testr uses. It will run
|
||||
* ``--regex, -r``: This is a selection regex like what testr uses. It will run
|
||||
any tests that match on re.match() with the regex
|
||||
* **--smoke/-s**: Run all the tests tagged as smoke
|
||||
* ``--smoke, -s``: Run all the tests tagged as smoke
|
||||
|
||||
There are also the **--blacklist-file** and **--whitelist-file** options that
|
||||
There are also the ``--blacklist-file`` and ``--whitelist-file`` options that
|
||||
let you pass a filepath to tempest run with the file format being a line
|
||||
separated regex, with '#' used to signify the start of a comment on a line.
|
||||
For example::
|
||||
@ -44,21 +44,21 @@ something like::
|
||||
When combined with a whitelist file all the regexes from the file and the CLI
|
||||
regexes will be ORed.
|
||||
|
||||
You can also use the **--list-tests** option in conjunction with selection
|
||||
You can also use the ``--list-tests`` option in conjunction with selection
|
||||
arguments to list which tests will be run.
|
||||
|
||||
You can also use the **--load-list** option that lets you pass a filepath to
|
||||
You can also use the ``--load-list`` option that lets you pass a filepath to
|
||||
tempest run with the file format being in a non-regex format, similar to the
|
||||
tests generated by the **--list-tests** option. You can specify target tests
|
||||
tests generated by the ``--list-tests`` option. You can specify target tests
|
||||
by removing unnecessary tests from a list file which is generated from
|
||||
**--list-tests** option.
|
||||
``--list-tests`` option.
|
||||
|
||||
Test Execution
|
||||
==============
|
||||
There are several options to control how the tests are executed. By default
|
||||
tempest will run in parallel with a worker for each CPU present on the machine.
|
||||
If you want to adjust the number of workers use the **--concurrency** option
|
||||
and if you want to run tests serially use **--serial/-t**
|
||||
If you want to adjust the number of workers use the ``--concurrency`` option
|
||||
and if you want to run tests serially use ``--serial/-t``
|
||||
|
||||
Running with Workspaces
|
||||
-----------------------
|
||||
@ -82,7 +82,7 @@ Test Output
|
||||
===========
|
||||
By default tempest run's output to STDOUT will be generated using the
|
||||
subunit-trace output filter. But, if you would prefer a subunit v2 stream be
|
||||
output to STDOUT use the **--subunit** flag
|
||||
output to STDOUT use the ``--subunit`` flag
|
||||
|
||||
Combining Runs
|
||||
==============
|
||||
@ -90,7 +90,7 @@ Combining Runs
|
||||
There are certain situations in which you want to split a single run of tempest
|
||||
across 2 executions of tempest run. (for example to run part of the tests
|
||||
serially and others in parallel) To accomplish this but still treat the results
|
||||
as a single run you can leverage the **--combine** option which will append
|
||||
as a single run you can leverage the ``--combine`` option which will append
|
||||
the current run's results with the previous runs.
|
||||
"""
|
||||
|
||||
|
||||
@ -21,17 +21,14 @@ API calls are made inside of a test and in what order they are called.
|
||||
Runtime Arguments
|
||||
-----------------
|
||||
|
||||
**--subunit, -s**: (Optional) The path to the subunit file being parsed,
|
||||
* ``--subunit, -s``: (Optional) The path to the subunit file being parsed,
|
||||
defaults to stdin
|
||||
|
||||
**--non-subunit-name, -n**: (Optional) The file_name that the logs are being
|
||||
* ``--non-subunit-name, -n``: (Optional) The file_name that the logs are being
|
||||
stored in
|
||||
|
||||
**--output-file, -o**: (Optional) The path where the JSON output will be
|
||||
* ``--output-file, -o``: (Optional) The path where the JSON output will be
|
||||
written to. This contains more information than is present in stdout.
|
||||
|
||||
**--ports, -p**: (Optional) The path to a JSON file describing the ports being
|
||||
used by different services
|
||||
* ``--ports, -p``: (Optional) The path to a JSON file describing the ports
|
||||
being used by different services
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
@ -26,28 +26,28 @@ Outputs the name and path of all known tempest workspaces
|
||||
|
||||
register
|
||||
--------
|
||||
Registers a new tempest workspace via a given --name and --path
|
||||
Registers a new tempest workspace via a given ``--name`` and ``--path``
|
||||
|
||||
rename
|
||||
------
|
||||
Renames a tempest workspace from --old-name to --new-name
|
||||
Renames a tempest workspace from ``--old-name`` to ``--new-name``
|
||||
|
||||
move
|
||||
----
|
||||
Changes the path of a given tempest workspace --name to --path
|
||||
Changes the path of a given tempest workspace ``--name`` to ``--path``
|
||||
|
||||
remove
|
||||
------
|
||||
Deletes the entry for a given tempest workspace --name
|
||||
Deletes the entry for a given tempest workspace ``--name``
|
||||
|
||||
--rmdir Deletes the given tempest workspace directory
|
||||
``--rmdir`` Deletes the given tempest workspace directory
|
||||
|
||||
General Options
|
||||
===============
|
||||
|
||||
**--workspace_path**: Allows the user to specify a different location for the
|
||||
workspace.yaml file containing the workspace definitions
|
||||
instead of ~/.tempest/workspace.yaml
|
||||
* ``--workspace_path``: Allows the user to specify a different location for the
|
||||
workspace.yaml file containing the workspace definitions instead of
|
||||
``~/.tempest/workspace.yaml``
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
@ -15,6 +15,7 @@ multiple OpenStack services to exercise the touch points between them.
|
||||
Any scenario test should have a real-life use case. An example would be:
|
||||
|
||||
- "As operator I want to start with a blank environment":
|
||||
|
||||
1. upload a glance image
|
||||
2. deploy a vm from it
|
||||
3. ssh to the guest
|
||||
|
||||
Loading…
Reference in New Issue
Block a user