Fix indentation in docs
This commit fixes indentation in tempest docs. These indentations are not necessary and it causes a weird html outputs. Change-Id: I9c8714558a3327b7ad0b0ab0d3fdc7e770c3c75b
This commit is contained in:
Masayuki Igawa
parent
10d9e73349
commit
b78b923e5a
@ -121,8 +121,8 @@ fix it. When it will happen, we will deal with it on a case-by-case basis.
|
||||
|
||||
When to approve
|
||||
---------------
|
||||
* Every patch needs two +2s before being approved.
|
||||
* Its ok to hold off on an approval until a subject matter expert reviews it
|
||||
* If a patch has already been approved but requires a trivial rebase to merge,
|
||||
* Every patch needs two +2s before being approved.
|
||||
* Its ok to hold off on an approval until a subject matter expert reviews it
|
||||
* If a patch has already been approved but requires a trivial rebase to merge,
|
||||
you do not have to wait for a second +2, since the patch has already had
|
||||
two +2s.
|
||||
|
||||
@ -17,10 +17,10 @@ Test Credentials
|
||||
Tempest allows for configuring a set of admin credentials in the ``auth``
|
||||
section, via the following parameters:
|
||||
|
||||
#. ``admin_username``
|
||||
#. ``admin_password``
|
||||
#. ``admin_project_name``
|
||||
#. ``admin_domain_name``
|
||||
#. ``admin_username``
|
||||
#. ``admin_password``
|
||||
#. ``admin_project_name``
|
||||
#. ``admin_domain_name``
|
||||
|
||||
Admin credentials are not mandatory to run Tempest, but when provided they
|
||||
can be used to:
|
||||
@ -47,9 +47,9 @@ In order for Tempest to be able to talk to your OpenStack deployment you need
|
||||
to provide it with information about how it communicates with keystone.
|
||||
This involves configuring the following options in the ``identity`` section:
|
||||
|
||||
- ``auth_version``
|
||||
- ``uri``
|
||||
- ``uri_v3``
|
||||
- ``auth_version``
|
||||
- ``uri``
|
||||
- ``uri_v3``
|
||||
|
||||
The ``auth_version`` option is used to tell Tempest whether it should be using
|
||||
Keystone's v2 or v3 api for communicating with Keystone. The two uri options are
|
||||
@ -74,11 +74,11 @@ three sets of username, password, and project names for a primary user,
|
||||
an admin user, and an alternate user. To enable and use dynamic credentials you
|
||||
only need to configure two things:
|
||||
|
||||
#. A set of admin credentials with permissions to create users and
|
||||
#. A set of admin credentials with permissions to create users and
|
||||
projects. This is specified in the ``auth`` section with the
|
||||
``admin_username``, ``admin_project_name``, ``admin_domain_name`` and
|
||||
``admin_password`` options
|
||||
#. To enable dynamic credentials in the ``auth`` section with the
|
||||
#. To enable dynamic credentials in the ``auth`` section with the
|
||||
``use_dynamic_credentials`` option.
|
||||
|
||||
This is also currently the default credential provider enabled by Tempest, due
|
||||
@ -115,7 +115,7 @@ like with dynamic credentials.
|
||||
|
||||
To enable and use locking test accounts you need do a few things:
|
||||
|
||||
#. Create an accounts.yaml file which contains the set of pre-existing
|
||||
#. Create an accounts.yaml file which contains the set of pre-existing
|
||||
credentials to use for testing. To make sure you don't have a credentials
|
||||
starvation issue when running in parallel make sure you have at least two
|
||||
times the number of worker processes you are using to execute Tempest
|
||||
@ -123,13 +123,13 @@ To enable and use locking test accounts you need do a few things:
|
||||
|
||||
You can check the accounts.yaml.sample file packaged in Tempest for the yaml
|
||||
format.
|
||||
#. Provide Tempest with the location of your accounts.yaml file with the
|
||||
#. Provide Tempest with the location of your accounts.yaml file with the
|
||||
``test_accounts_file`` option in the ``auth`` section
|
||||
|
||||
*NOTE: Be sure to use a full path for the file; otherwise Tempest will
|
||||
likely not find it.*
|
||||
|
||||
#. Set ``use_dynamic_credentials = False`` in the ``auth`` group
|
||||
#. Set ``use_dynamic_credentials = False`` in the ``auth`` group
|
||||
|
||||
It is worth pointing out that each set of credentials in the accounts.yaml
|
||||
should have a unique project. This is required to provide proper isolation
|
||||
@ -162,8 +162,8 @@ For Tempest to be able to create servers you need to specify flavors that it
|
||||
can use to boot the servers with. There are two options in the Tempest config
|
||||
for doing this:
|
||||
|
||||
#. ``flavor_ref``
|
||||
#. ``flavor_ref_alt``
|
||||
#. ``flavor_ref``
|
||||
#. ``flavor_ref_alt``
|
||||
|
||||
Both of these options are in the ``compute`` section of the config file and take
|
||||
in the flavor id (not the name) from Nova. The ``flavor_ref`` option is what
|
||||
@ -181,8 +181,8 @@ Images
|
||||
Just like with flavors, Tempest needs to know which images to use for booting
|
||||
servers. There are two options in the compute section just like with flavors:
|
||||
|
||||
#. ``image_ref``
|
||||
#. ``image_ref_alt``
|
||||
#. ``image_ref``
|
||||
#. ``image_ref_alt``
|
||||
|
||||
Both options are expecting an image id (not name) from Nova. The ``image_ref``
|
||||
option is what will be used for booting the majority of servers in Tempest.
|
||||
@ -192,13 +192,13 @@ those tests will be skipped.
|
||||
|
||||
There are also options in the ``scenario`` section for images:
|
||||
|
||||
#. ``img_file``
|
||||
#. ``img_dir``
|
||||
#. ``aki_img_file``
|
||||
#. ``ari_img_file``
|
||||
#. ``ami_img_file``
|
||||
#. ``img_container_format``
|
||||
#. ``img_disk_format``
|
||||
#. ``img_file``
|
||||
#. ``img_dir``
|
||||
#. ``aki_img_file``
|
||||
#. ``ari_img_file``
|
||||
#. ``ami_img_file``
|
||||
#. ``img_container_format``
|
||||
#. ``img_disk_format``
|
||||
|
||||
However, unlike the other image options, these are used for a very small subset
|
||||
of scenario tests which are uploading an image. These options are used to tell
|
||||
@ -261,7 +261,7 @@ booting.
|
||||
|
||||
To set a fixed network name simply:
|
||||
|
||||
#. Set the ``fixed_network_name`` option in the ``compute`` group
|
||||
#. Set the ``fixed_network_name`` option in the ``compute`` group
|
||||
|
||||
In the case that the configured fixed network name can not be found by a user
|
||||
network list call, it will be treated like one was not provided except that a
|
||||
@ -329,9 +329,9 @@ connecting to and remotely accessing the created servers.
|
||||
|
||||
To enable remote access to servers, there are 3 options at a minimum that are used:
|
||||
|
||||
#. ``run_validation``
|
||||
#. ``connect_method``
|
||||
#. ``auth_method``
|
||||
#. ``run_validation``
|
||||
#. ``connect_method``
|
||||
#. ``auth_method``
|
||||
|
||||
The ``run_validation`` is used to enable or disable ssh connectivity for
|
||||
all tests (with the exception of scenario tests which do not have a flag for
|
||||
@ -370,9 +370,9 @@ API calls for each service, it needs to know how that project is defined in the
|
||||
service catalog. There are three options for each service section to accomplish
|
||||
this:
|
||||
|
||||
#. ``catalog_type``
|
||||
#. ``endpoint_type``
|
||||
#. ``region``
|
||||
#. ``catalog_type``
|
||||
#. ``endpoint_type``
|
||||
#. ``region``
|
||||
|
||||
Setting ``catalog_type`` and ``endpoint_type`` should normally give Tempest
|
||||
enough information to determine which endpoint it should pull from the service
|
||||
|
||||
@ -44,9 +44,9 @@ ensure that we don't break backwards compatibility. For patches that change
|
||||
existing interfaces we have to be careful to make sure we don't break any
|
||||
external consumers. Some common red flags are:
|
||||
|
||||
* a change to an existing API requires a change outside the library directory
|
||||
* a change to an existing API requires a change outside the library directory
|
||||
where the interface is being consumed
|
||||
* a unit test has to be significantly changed to make the proposed change pass
|
||||
* a unit test has to be significantly changed to make the proposed change pass
|
||||
|
||||
Testing
|
||||
'''''''
|
||||
|
||||
@ -33,9 +33,9 @@ project guidelines.
|
||||
Tempest will cover only integration testing of applicable microversions with
|
||||
below exceptions:
|
||||
|
||||
#. Test covers a feature which is important for interoperability. This covers tests requirement
|
||||
#. Test covers a feature which is important for interoperability. This covers tests requirement
|
||||
from Defcore.
|
||||
#. Test needed to fill Schema gaps.
|
||||
#. Test needed to fill Schema gaps.
|
||||
Tempest validates API responses with defined JSON schema. API responses can be different on
|
||||
each microversion and the JSON schemas need to be defined separately for the microversion.
|
||||
While implementing new integration tests for a specific microversion, there
|
||||
@ -69,7 +69,7 @@ below exceptions:
|
||||
needs to add JSON schema for v2.3, v2.4, v2.6.
|
||||
In that case if only 1 or 2 tests can verify all new schemas then we do not need
|
||||
separate tests for each new schemas. In worst case, we have to add 3 separate tests.
|
||||
#. Test covers service behavior at large scale with involvement of more deep layer like hypervisor
|
||||
#. Test covers service behavior at large scale with involvement of more deep layer like hypervisor
|
||||
etc not just API/DB layer. This type of tests will be added case by case basis and
|
||||
with project team consultation about why it cannot be covered on project side and worth to test
|
||||
in Tempest.
|
||||
|
||||
@ -29,18 +29,18 @@ Proposing a test removal
|
||||
|
||||
In the proposal etherpad we'll be looking for answers to 3 questions
|
||||
|
||||
#. The tests proposed for removal must have equiv. coverage in a different
|
||||
#. The tests proposed for removal must have equiv. coverage in a different
|
||||
project's test suite (whether this is another gating test project, or an in
|
||||
tree functional test suite). For API tests preferably the other project will
|
||||
have a similar source of friction in place to prevent breaking api changes
|
||||
so that we don't regress and let breaking api changes slip through the
|
||||
gate.
|
||||
#. The test proposed for removal has a failure rate < 0.50% in the gate over
|
||||
#. The test proposed for removal has a failure rate < 0.50% in the gate over
|
||||
the past release (the value and interval will likely be adjusted in the
|
||||
future)
|
||||
|
||||
.. _`prong #3`:
|
||||
#. There must not be an external user/consumer of tempest
|
||||
#. There must not be an external user/consumer of tempest
|
||||
that depends on the test proposed for removal
|
||||
|
||||
The answers to 1 and 2 are easy to verify. For 1 just provide a link to the new
|
||||
@ -68,22 +68,22 @@ setUpClass or tearDownClass failures)
|
||||
|
||||
You can access the infra mysql subunit2sql db w/ read-only permissions with:
|
||||
|
||||
* hostname: logstash.openstack.org
|
||||
* username: query
|
||||
* password: query
|
||||
* db_name: subunit2sql
|
||||
* hostname: logstash.openstack.org
|
||||
* username: query
|
||||
* password: query
|
||||
* db_name: subunit2sql
|
||||
|
||||
For example if you were trying to remove the test with the id:
|
||||
tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON.test_get_flavor_details_for_deleted_flavor
|
||||
you would run the following:
|
||||
|
||||
#. run: "mysql -u query -p -h logstash.openstack.org subunit2sql" to connect
|
||||
#. run: "mysql -u query -p -h logstash.openstack.org subunit2sql" to connect
|
||||
to the subunit2sql db
|
||||
#. run the query: MySQL [subunit2sql]> select * from tests where test_id like
|
||||
#. run the query: MySQL [subunit2sql]> select * from tests where test_id like
|
||||
"tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON%";
|
||||
which will return a table of all the tests in the class (but it will also
|
||||
catch failures in setUpClass and tearDownClass)
|
||||
#. paste the output table with numbers and the mysql command you ran to
|
||||
#. paste the output table with numbers and the mysql command you ran to
|
||||
generate it into the etherpad.
|
||||
|
||||
Eventually a cli interface will be created to make that a bit more friendly.
|
||||
@ -131,11 +131,11 @@ Exceptions to this procedure
|
||||
For the most part all tempest test removals have to go through this procedure
|
||||
there are a couple of exceptions though:
|
||||
|
||||
#. The class of testing has been decided to be outside the scope of tempest.
|
||||
#. A revert for a patch which added a broken test, or testing which didn't
|
||||
#. The class of testing has been decided to be outside the scope of tempest.
|
||||
#. A revert for a patch which added a broken test, or testing which didn't
|
||||
actually run in the gate (basically any revert for something which
|
||||
shouldn't have been added)
|
||||
#. Tests that would become out of scope as a consequence of an API change,
|
||||
#. Tests that would become out of scope as a consequence of an API change,
|
||||
as described in `API Compatibility`_.
|
||||
Such tests cannot live in Tempest because of the branchless nature of
|
||||
Tempest. Such test must still honor `prong #3`_.
|
||||
@ -143,11 +143,11 @@ there are a couple of exceptions though:
|
||||
For the first exception type the only types of testing in tree which have been
|
||||
declared out of scope at this point are:
|
||||
|
||||
* The CLI tests (which should be completely removed at this point)
|
||||
* Neutron Adv. Services testing (which should be completely removed at this
|
||||
* The CLI tests (which should be completely removed at this point)
|
||||
* Neutron Adv. Services testing (which should be completely removed at this
|
||||
point)
|
||||
* XML API Tests (which should be completely removed at this point)
|
||||
* EC2 API/boto tests (which should be completely removed at this point)
|
||||
* XML API Tests (which should be completely removed at this point)
|
||||
* EC2 API/boto tests (which should be completely removed at this point)
|
||||
|
||||
For tests that fit into this category the only criteria for removal is that
|
||||
there is equivalent testing elsewhere.
|
||||
@ -159,12 +159,12 @@ Starting in the liberty cycle tempest has defined a set of projects which
|
||||
are defined as in scope for direct testing in tempest. As of today that list
|
||||
is:
|
||||
|
||||
* Keystone
|
||||
* Nova
|
||||
* Glance
|
||||
* Cinder
|
||||
* Neutron
|
||||
* Swift
|
||||
* Keystone
|
||||
* Nova
|
||||
* Glance
|
||||
* Cinder
|
||||
* Neutron
|
||||
* Swift
|
||||
|
||||
anything that lives in tempest which doesn't test one of these projects can be
|
||||
removed assuming there is equivalent testing elsewhere. Preferably using the
|
||||
|
||||
@ -36,12 +36,12 @@ methods.
|
||||
In standard unittest the lifecycle of a TestCase can be described in the
|
||||
following phases:
|
||||
|
||||
#. setUpClass
|
||||
#. setUp
|
||||
#. Test Execution
|
||||
#. tearDown
|
||||
#. doCleanups
|
||||
#. tearDownClass
|
||||
#. setUpClass
|
||||
#. setUp
|
||||
#. Test Execution
|
||||
#. tearDown
|
||||
#. doCleanups
|
||||
#. tearDownClass
|
||||
|
||||
setUpClass
|
||||
----------
|
||||
@ -54,10 +54,10 @@ setup for interacting with the configured cloud.
|
||||
To accomplish this you do **not** define a setUpClass function, instead there
|
||||
are a number of predefined phases to setUpClass that are used. The phases are:
|
||||
|
||||
* skip_checks
|
||||
* setup_credentials
|
||||
* setup_clients
|
||||
* resource_setup
|
||||
* skip_checks
|
||||
* setup_credentials
|
||||
* setup_clients
|
||||
* resource_setup
|
||||
|
||||
which is executed in that order. Cleanup of resources provisioned during
|
||||
the resource_setup must be scheduled right after provisioning using
|
||||
|
||||
@ -14,7 +14,7 @@ multiple OpenStack services to exercise the touch points between them.
|
||||
|
||||
Any scenario test should have a real-life use case. An example would be:
|
||||
|
||||
- "As operator I want to start with a blank environment":
|
||||
- "As operator I want to start with a blank environment":
|
||||
1. upload a glance image
|
||||
2. deploy a vm from it
|
||||
3. ssh to the guest
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user