Fix indentation in docs
This commit fixes indentation in tempest docs. These indentations are not necessary and it causes a weird html outputs. Change-Id: I9c8714558a3327b7ad0b0ab0d3fdc7e770c3c75b
This commit is contained in:
Masayuki Igawa
parent
10d9e73349
commit
b78b923e5a
@ -9,14 +9,14 @@ Tempest Specific Commandments
|
||||
------------------------------
|
||||
|
||||
- [T102] Cannot import OpenStack python clients in tempest/api &
|
||||
tempest/scenario tests
|
||||
tempest/scenario tests
|
||||
- [T104] Scenario tests require a services decorator
|
||||
- [T105] Tests cannot use setUpClass/tearDownClass
|
||||
- [T106] vim configuration should not be kept in source files.
|
||||
- [T107] Check that a service tag isn't in the module path
|
||||
- [T108] Check no hyphen at the end of rand_name() argument
|
||||
- [T109] Cannot use testtools.skip decorator; instead use
|
||||
decorators.skip_because from tempest.lib
|
||||
decorators.skip_because from tempest.lib
|
||||
- [T110] Check that service client names of GET should be consistent
|
||||
- [T111] Check that service client names of DELETE should be consistent
|
||||
- [T112] Check that tempest.lib should not import local tempest code
|
||||
|
||||
@ -121,8 +121,8 @@ fix it. When it will happen, we will deal with it on a case-by-case basis.
|
||||
|
||||
When to approve
|
||||
---------------
|
||||
* Every patch needs two +2s before being approved.
|
||||
* Its ok to hold off on an approval until a subject matter expert reviews it
|
||||
* If a patch has already been approved but requires a trivial rebase to merge,
|
||||
you do not have to wait for a second +2, since the patch has already had
|
||||
two +2s.
|
||||
* Every patch needs two +2s before being approved.
|
||||
* Its ok to hold off on an approval until a subject matter expert reviews it
|
||||
* If a patch has already been approved but requires a trivial rebase to merge,
|
||||
you do not have to wait for a second +2, since the patch has already had
|
||||
two +2s.
|
||||
|
||||
@ -17,10 +17,10 @@ Test Credentials
|
||||
Tempest allows for configuring a set of admin credentials in the ``auth``
|
||||
section, via the following parameters:
|
||||
|
||||
#. ``admin_username``
|
||||
#. ``admin_password``
|
||||
#. ``admin_project_name``
|
||||
#. ``admin_domain_name``
|
||||
#. ``admin_username``
|
||||
#. ``admin_password``
|
||||
#. ``admin_project_name``
|
||||
#. ``admin_domain_name``
|
||||
|
||||
Admin credentials are not mandatory to run Tempest, but when provided they
|
||||
can be used to:
|
||||
@ -47,9 +47,9 @@ In order for Tempest to be able to talk to your OpenStack deployment you need
|
||||
to provide it with information about how it communicates with keystone.
|
||||
This involves configuring the following options in the ``identity`` section:
|
||||
|
||||
- ``auth_version``
|
||||
- ``uri``
|
||||
- ``uri_v3``
|
||||
- ``auth_version``
|
||||
- ``uri``
|
||||
- ``uri_v3``
|
||||
|
||||
The ``auth_version`` option is used to tell Tempest whether it should be using
|
||||
Keystone's v2 or v3 api for communicating with Keystone. The two uri options are
|
||||
@ -74,12 +74,12 @@ three sets of username, password, and project names for a primary user,
|
||||
an admin user, and an alternate user. To enable and use dynamic credentials you
|
||||
only need to configure two things:
|
||||
|
||||
#. A set of admin credentials with permissions to create users and
|
||||
projects. This is specified in the ``auth`` section with the
|
||||
``admin_username``, ``admin_project_name``, ``admin_domain_name`` and
|
||||
``admin_password`` options
|
||||
#. To enable dynamic credentials in the ``auth`` section with the
|
||||
``use_dynamic_credentials`` option.
|
||||
#. A set of admin credentials with permissions to create users and
|
||||
projects. This is specified in the ``auth`` section with the
|
||||
``admin_username``, ``admin_project_name``, ``admin_domain_name`` and
|
||||
``admin_password`` options
|
||||
#. To enable dynamic credentials in the ``auth`` section with the
|
||||
``use_dynamic_credentials`` option.
|
||||
|
||||
This is also currently the default credential provider enabled by Tempest, due
|
||||
to its common use and ease of configuration.
|
||||
@ -115,21 +115,21 @@ like with dynamic credentials.
|
||||
|
||||
To enable and use locking test accounts you need do a few things:
|
||||
|
||||
#. Create an accounts.yaml file which contains the set of pre-existing
|
||||
credentials to use for testing. To make sure you don't have a credentials
|
||||
starvation issue when running in parallel make sure you have at least two
|
||||
times the number of worker processes you are using to execute Tempest
|
||||
available in the file. (If running serially the worker count is 1.)
|
||||
#. Create an accounts.yaml file which contains the set of pre-existing
|
||||
credentials to use for testing. To make sure you don't have a credentials
|
||||
starvation issue when running in parallel make sure you have at least two
|
||||
times the number of worker processes you are using to execute Tempest
|
||||
available in the file. (If running serially the worker count is 1.)
|
||||
|
||||
You can check the accounts.yaml.sample file packaged in Tempest for the yaml
|
||||
format.
|
||||
#. Provide Tempest with the location of your accounts.yaml file with the
|
||||
``test_accounts_file`` option in the ``auth`` section
|
||||
You can check the accounts.yaml.sample file packaged in Tempest for the yaml
|
||||
format.
|
||||
#. Provide Tempest with the location of your accounts.yaml file with the
|
||||
``test_accounts_file`` option in the ``auth`` section
|
||||
|
||||
*NOTE: Be sure to use a full path for the file; otherwise Tempest will
|
||||
likely not find it.*
|
||||
*NOTE: Be sure to use a full path for the file; otherwise Tempest will
|
||||
likely not find it.*
|
||||
|
||||
#. Set ``use_dynamic_credentials = False`` in the ``auth`` group
|
||||
#. Set ``use_dynamic_credentials = False`` in the ``auth`` group
|
||||
|
||||
It is worth pointing out that each set of credentials in the accounts.yaml
|
||||
should have a unique project. This is required to provide proper isolation
|
||||
@ -162,8 +162,8 @@ For Tempest to be able to create servers you need to specify flavors that it
|
||||
can use to boot the servers with. There are two options in the Tempest config
|
||||
for doing this:
|
||||
|
||||
#. ``flavor_ref``
|
||||
#. ``flavor_ref_alt``
|
||||
#. ``flavor_ref``
|
||||
#. ``flavor_ref_alt``
|
||||
|
||||
Both of these options are in the ``compute`` section of the config file and take
|
||||
in the flavor id (not the name) from Nova. The ``flavor_ref`` option is what
|
||||
@ -181,8 +181,8 @@ Images
|
||||
Just like with flavors, Tempest needs to know which images to use for booting
|
||||
servers. There are two options in the compute section just like with flavors:
|
||||
|
||||
#. ``image_ref``
|
||||
#. ``image_ref_alt``
|
||||
#. ``image_ref``
|
||||
#. ``image_ref_alt``
|
||||
|
||||
Both options are expecting an image id (not name) from Nova. The ``image_ref``
|
||||
option is what will be used for booting the majority of servers in Tempest.
|
||||
@ -192,13 +192,13 @@ those tests will be skipped.
|
||||
|
||||
There are also options in the ``scenario`` section for images:
|
||||
|
||||
#. ``img_file``
|
||||
#. ``img_dir``
|
||||
#. ``aki_img_file``
|
||||
#. ``ari_img_file``
|
||||
#. ``ami_img_file``
|
||||
#. ``img_container_format``
|
||||
#. ``img_disk_format``
|
||||
#. ``img_file``
|
||||
#. ``img_dir``
|
||||
#. ``aki_img_file``
|
||||
#. ``ari_img_file``
|
||||
#. ``ami_img_file``
|
||||
#. ``img_container_format``
|
||||
#. ``img_disk_format``
|
||||
|
||||
However, unlike the other image options, these are used for a very small subset
|
||||
of scenario tests which are uploading an image. These options are used to tell
|
||||
@ -261,7 +261,7 @@ booting.
|
||||
|
||||
To set a fixed network name simply:
|
||||
|
||||
#. Set the ``fixed_network_name`` option in the ``compute`` group
|
||||
#. Set the ``fixed_network_name`` option in the ``compute`` group
|
||||
|
||||
In the case that the configured fixed network name can not be found by a user
|
||||
network list call, it will be treated like one was not provided except that a
|
||||
@ -329,9 +329,9 @@ connecting to and remotely accessing the created servers.
|
||||
|
||||
To enable remote access to servers, there are 3 options at a minimum that are used:
|
||||
|
||||
#. ``run_validation``
|
||||
#. ``connect_method``
|
||||
#. ``auth_method``
|
||||
#. ``run_validation``
|
||||
#. ``connect_method``
|
||||
#. ``auth_method``
|
||||
|
||||
The ``run_validation`` is used to enable or disable ssh connectivity for
|
||||
all tests (with the exception of scenario tests which do not have a flag for
|
||||
@ -370,9 +370,9 @@ API calls for each service, it needs to know how that project is defined in the
|
||||
service catalog. There are three options for each service section to accomplish
|
||||
this:
|
||||
|
||||
#. ``catalog_type``
|
||||
#. ``endpoint_type``
|
||||
#. ``region``
|
||||
#. ``catalog_type``
|
||||
#. ``endpoint_type``
|
||||
#. ``region``
|
||||
|
||||
Setting ``catalog_type`` and ``endpoint_type`` should normally give Tempest
|
||||
enough information to determine which endpoint it should pull from the service
|
||||
|
||||
@ -44,9 +44,9 @@ ensure that we don't break backwards compatibility. For patches that change
|
||||
existing interfaces we have to be careful to make sure we don't break any
|
||||
external consumers. Some common red flags are:
|
||||
|
||||
* a change to an existing API requires a change outside the library directory
|
||||
where the interface is being consumed
|
||||
* a unit test has to be significantly changed to make the proposed change pass
|
||||
* a change to an existing API requires a change outside the library directory
|
||||
where the interface is being consumed
|
||||
* a unit test has to be significantly changed to make the proposed change pass
|
||||
|
||||
Testing
|
||||
'''''''
|
||||
|
||||
@ -33,46 +33,46 @@ project guidelines.
|
||||
Tempest will cover only integration testing of applicable microversions with
|
||||
below exceptions:
|
||||
|
||||
#. Test covers a feature which is important for interoperability. This covers tests requirement
|
||||
from Defcore.
|
||||
#. Test needed to fill Schema gaps.
|
||||
Tempest validates API responses with defined JSON schema. API responses can be different on
|
||||
each microversion and the JSON schemas need to be defined separately for the microversion.
|
||||
While implementing new integration tests for a specific microversion, there
|
||||
may be a gap in the JSON schemas (caused by previous microversions) implemented
|
||||
in Tempest.
|
||||
Filling that gap while implementing the new integration test cases is not efficient due to
|
||||
many reasons:
|
||||
#. Test covers a feature which is important for interoperability. This covers tests requirement
|
||||
from Defcore.
|
||||
#. Test needed to fill Schema gaps.
|
||||
Tempest validates API responses with defined JSON schema. API responses can be different on
|
||||
each microversion and the JSON schemas need to be defined separately for the microversion.
|
||||
While implementing new integration tests for a specific microversion, there
|
||||
may be a gap in the JSON schemas (caused by previous microversions) implemented
|
||||
in Tempest.
|
||||
Filling that gap while implementing the new integration test cases is not efficient due to
|
||||
many reasons:
|
||||
|
||||
* Hard to review
|
||||
* Sync between multiple integration tests patches which try to fill the same schema gap at same
|
||||
time
|
||||
* Might delay the microversion change on project side where project team wants Tempest
|
||||
tests to verify the results.
|
||||
* Hard to review
|
||||
* Sync between multiple integration tests patches which try to fill the same schema gap at same
|
||||
time
|
||||
* Might delay the microversion change on project side where project team wants Tempest
|
||||
tests to verify the results.
|
||||
|
||||
Tempest will allow to fill the schema gaps at the end of each cycle, or more
|
||||
often if required.
|
||||
Schema gap can be filled with testing those with a minimal set of tests. Those
|
||||
tests might not be integration tests and might be already covered on project
|
||||
side also.
|
||||
This exception is needed because:
|
||||
Tempest will allow to fill the schema gaps at the end of each cycle, or more
|
||||
often if required.
|
||||
Schema gap can be filled with testing those with a minimal set of tests. Those
|
||||
tests might not be integration tests and might be already covered on project
|
||||
side also.
|
||||
This exception is needed because:
|
||||
|
||||
* Allow to create microversion response schema in Tempest at the same time that projects are
|
||||
implementing their API microversions. This will make implementation easier for adding
|
||||
required tests before a new microversion change can be merged in the corresponding project
|
||||
and hence accelerate the development of microversions.
|
||||
* New schema must be verified by at least one test case which exercises such schema.
|
||||
* Allow to create microversion response schema in Tempest at the same time that projects are
|
||||
implementing their API microversions. This will make implementation easier for adding
|
||||
required tests before a new microversion change can be merged in the corresponding project
|
||||
and hence accelerate the development of microversions.
|
||||
* New schema must be verified by at least one test case which exercises such schema.
|
||||
|
||||
For example:
|
||||
If any projects implemented 4 API microversion say- v2.3, v2.4, v2.5, v2.6
|
||||
Assume microversion v2.3, v2.4, v2.6 change the API Response which means Tempest
|
||||
needs to add JSON schema for v2.3, v2.4, v2.6.
|
||||
In that case if only 1 or 2 tests can verify all new schemas then we do not need
|
||||
separate tests for each new schemas. In worst case, we have to add 3 separate tests.
|
||||
#. Test covers service behavior at large scale with involvement of more deep layer like hypervisor
|
||||
etc not just API/DB layer. This type of tests will be added case by case basis and
|
||||
with project team consultation about why it cannot be covered on project side and worth to test
|
||||
in Tempest.
|
||||
For example:
|
||||
If any projects implemented 4 API microversion say- v2.3, v2.4, v2.5, v2.6
|
||||
Assume microversion v2.3, v2.4, v2.6 change the API Response which means Tempest
|
||||
needs to add JSON schema for v2.3, v2.4, v2.6.
|
||||
In that case if only 1 or 2 tests can verify all new schemas then we do not need
|
||||
separate tests for each new schemas. In worst case, we have to add 3 separate tests.
|
||||
#. Test covers service behavior at large scale with involvement of more deep layer like hypervisor
|
||||
etc not just API/DB layer. This type of tests will be added case by case basis and
|
||||
with project team consultation about why it cannot be covered on project side and worth to test
|
||||
in Tempest.
|
||||
|
||||
Project Scope For Microversion Testing
|
||||
""""""""""""""""""""""""""""""""""""""
|
||||
@ -294,72 +294,72 @@ Microversion tests implemented in Tempest
|
||||
|
||||
* Compute
|
||||
|
||||
* `2.1`_
|
||||
* `2.1`_
|
||||
|
||||
.. _2.1: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id1
|
||||
.. _2.1: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id1
|
||||
|
||||
* `2.2`_
|
||||
* `2.2`_
|
||||
|
||||
.. _2.2: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id2
|
||||
.. _2.2: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id2
|
||||
|
||||
* `2.10`_
|
||||
* `2.10`_
|
||||
|
||||
.. _2.10: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id9
|
||||
.. _2.10: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id9
|
||||
|
||||
* `2.20`_
|
||||
* `2.20`_
|
||||
|
||||
.. _2.20: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id18
|
||||
.. _2.20: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id18
|
||||
|
||||
* `2.25`_
|
||||
* `2.25`_
|
||||
|
||||
.. _2.25: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-mitaka
|
||||
.. _2.25: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-mitaka
|
||||
|
||||
* `2.32`_
|
||||
* `2.32`_
|
||||
|
||||
.. _2.32: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id29
|
||||
.. _2.32: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id29
|
||||
|
||||
* `2.37`_
|
||||
* `2.37`_
|
||||
|
||||
.. _2.37: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id34
|
||||
.. _2.37: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id34
|
||||
|
||||
* `2.42`_
|
||||
* `2.42`_
|
||||
|
||||
.. _2.42: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-ocata
|
||||
.. _2.42: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-ocata
|
||||
|
||||
* `2.47`_
|
||||
* `2.47`_
|
||||
|
||||
.. _2.47: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id42
|
||||
.. _2.47: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id42
|
||||
|
||||
* `2.48`_
|
||||
* `2.48`_
|
||||
|
||||
.. _2.48: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id43
|
||||
.. _2.48: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id43
|
||||
|
||||
* Volume
|
||||
|
||||
* `3.3`_
|
||||
* `3.3`_
|
||||
|
||||
.. _3.3: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id3
|
||||
.. _3.3: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id3
|
||||
|
||||
* `3.9`_
|
||||
* `3.9`_
|
||||
|
||||
.. _3.9: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id9
|
||||
.. _3.9: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id9
|
||||
|
||||
* `3.11`_
|
||||
* `3.11`_
|
||||
|
||||
.. _3.11: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id11
|
||||
.. _3.11: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id11
|
||||
|
||||
* `3.12`_
|
||||
* `3.12`_
|
||||
|
||||
.. _3.12: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id12
|
||||
.. _3.12: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id12
|
||||
|
||||
* `3.14`_
|
||||
* `3.14`_
|
||||
|
||||
.. _3.14: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id14
|
||||
.. _3.14: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id14
|
||||
|
||||
* `3.19`_
|
||||
* `3.19`_
|
||||
|
||||
.. _3.19: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id18
|
||||
.. _3.19: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id18
|
||||
|
||||
* `3.20`_
|
||||
* `3.20`_
|
||||
|
||||
.. _3.20: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id19
|
||||
.. _3.20: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id19
|
||||
|
||||
@ -29,19 +29,19 @@ Proposing a test removal
|
||||
|
||||
In the proposal etherpad we'll be looking for answers to 3 questions
|
||||
|
||||
#. The tests proposed for removal must have equiv. coverage in a different
|
||||
project's test suite (whether this is another gating test project, or an in
|
||||
tree functional test suite). For API tests preferably the other project will
|
||||
have a similar source of friction in place to prevent breaking api changes
|
||||
so that we don't regress and let breaking api changes slip through the
|
||||
gate.
|
||||
#. The test proposed for removal has a failure rate < 0.50% in the gate over
|
||||
the past release (the value and interval will likely be adjusted in the
|
||||
future)
|
||||
#. The tests proposed for removal must have equiv. coverage in a different
|
||||
project's test suite (whether this is another gating test project, or an in
|
||||
tree functional test suite). For API tests preferably the other project will
|
||||
have a similar source of friction in place to prevent breaking api changes
|
||||
so that we don't regress and let breaking api changes slip through the
|
||||
gate.
|
||||
#. The test proposed for removal has a failure rate < 0.50% in the gate over
|
||||
the past release (the value and interval will likely be adjusted in the
|
||||
future)
|
||||
|
||||
.. _`prong #3`:
|
||||
#. There must not be an external user/consumer of tempest
|
||||
that depends on the test proposed for removal
|
||||
.. _`prong #3`:
|
||||
#. There must not be an external user/consumer of tempest
|
||||
that depends on the test proposed for removal
|
||||
|
||||
The answers to 1 and 2 are easy to verify. For 1 just provide a link to the new
|
||||
test location. If you are linking to the tempest removal patch please also put
|
||||
@ -68,23 +68,23 @@ setUpClass or tearDownClass failures)
|
||||
|
||||
You can access the infra mysql subunit2sql db w/ read-only permissions with:
|
||||
|
||||
* hostname: logstash.openstack.org
|
||||
* username: query
|
||||
* password: query
|
||||
* db_name: subunit2sql
|
||||
* hostname: logstash.openstack.org
|
||||
* username: query
|
||||
* password: query
|
||||
* db_name: subunit2sql
|
||||
|
||||
For example if you were trying to remove the test with the id:
|
||||
tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON.test_get_flavor_details_for_deleted_flavor
|
||||
you would run the following:
|
||||
|
||||
#. run: "mysql -u query -p -h logstash.openstack.org subunit2sql" to connect
|
||||
to the subunit2sql db
|
||||
#. run the query: MySQL [subunit2sql]> select * from tests where test_id like
|
||||
"tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON%";
|
||||
which will return a table of all the tests in the class (but it will also
|
||||
catch failures in setUpClass and tearDownClass)
|
||||
#. paste the output table with numbers and the mysql command you ran to
|
||||
generate it into the etherpad.
|
||||
#. run: "mysql -u query -p -h logstash.openstack.org subunit2sql" to connect
|
||||
to the subunit2sql db
|
||||
#. run the query: MySQL [subunit2sql]> select * from tests where test_id like
|
||||
"tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON%";
|
||||
which will return a table of all the tests in the class (but it will also
|
||||
catch failures in setUpClass and tearDownClass)
|
||||
#. paste the output table with numbers and the mysql command you ran to
|
||||
generate it into the etherpad.
|
||||
|
||||
Eventually a cli interface will be created to make that a bit more friendly.
|
||||
Also a dashboard is in the works so we don't need to manually run the command.
|
||||
@ -131,23 +131,23 @@ Exceptions to this procedure
|
||||
For the most part all tempest test removals have to go through this procedure
|
||||
there are a couple of exceptions though:
|
||||
|
||||
#. The class of testing has been decided to be outside the scope of tempest.
|
||||
#. A revert for a patch which added a broken test, or testing which didn't
|
||||
actually run in the gate (basically any revert for something which
|
||||
shouldn't have been added)
|
||||
#. Tests that would become out of scope as a consequence of an API change,
|
||||
as described in `API Compatibility`_.
|
||||
Such tests cannot live in Tempest because of the branchless nature of
|
||||
Tempest. Such test must still honor `prong #3`_.
|
||||
#. The class of testing has been decided to be outside the scope of tempest.
|
||||
#. A revert for a patch which added a broken test, or testing which didn't
|
||||
actually run in the gate (basically any revert for something which
|
||||
shouldn't have been added)
|
||||
#. Tests that would become out of scope as a consequence of an API change,
|
||||
as described in `API Compatibility`_.
|
||||
Such tests cannot live in Tempest because of the branchless nature of
|
||||
Tempest. Such test must still honor `prong #3`_.
|
||||
|
||||
For the first exception type the only types of testing in tree which have been
|
||||
declared out of scope at this point are:
|
||||
|
||||
* The CLI tests (which should be completely removed at this point)
|
||||
* Neutron Adv. Services testing (which should be completely removed at this
|
||||
point)
|
||||
* XML API Tests (which should be completely removed at this point)
|
||||
* EC2 API/boto tests (which should be completely removed at this point)
|
||||
* The CLI tests (which should be completely removed at this point)
|
||||
* Neutron Adv. Services testing (which should be completely removed at this
|
||||
point)
|
||||
* XML API Tests (which should be completely removed at this point)
|
||||
* EC2 API/boto tests (which should be completely removed at this point)
|
||||
|
||||
For tests that fit into this category the only criteria for removal is that
|
||||
there is equivalent testing elsewhere.
|
||||
@ -159,12 +159,12 @@ Starting in the liberty cycle tempest has defined a set of projects which
|
||||
are defined as in scope for direct testing in tempest. As of today that list
|
||||
is:
|
||||
|
||||
* Keystone
|
||||
* Nova
|
||||
* Glance
|
||||
* Cinder
|
||||
* Neutron
|
||||
* Swift
|
||||
* Keystone
|
||||
* Nova
|
||||
* Glance
|
||||
* Cinder
|
||||
* Neutron
|
||||
* Swift
|
||||
|
||||
anything that lives in tempest which doesn't test one of these projects can be
|
||||
removed assuming there is equivalent testing elsewhere. Preferably using the
|
||||
|
||||
@ -36,12 +36,12 @@ methods.
|
||||
In standard unittest the lifecycle of a TestCase can be described in the
|
||||
following phases:
|
||||
|
||||
#. setUpClass
|
||||
#. setUp
|
||||
#. Test Execution
|
||||
#. tearDown
|
||||
#. doCleanups
|
||||
#. tearDownClass
|
||||
#. setUpClass
|
||||
#. setUp
|
||||
#. Test Execution
|
||||
#. tearDown
|
||||
#. doCleanups
|
||||
#. tearDownClass
|
||||
|
||||
setUpClass
|
||||
----------
|
||||
@ -54,10 +54,10 @@ setup for interacting with the configured cloud.
|
||||
To accomplish this you do **not** define a setUpClass function, instead there
|
||||
are a number of predefined phases to setUpClass that are used. The phases are:
|
||||
|
||||
* skip_checks
|
||||
* setup_credentials
|
||||
* setup_clients
|
||||
* resource_setup
|
||||
* skip_checks
|
||||
* setup_credentials
|
||||
* setup_clients
|
||||
* resource_setup
|
||||
|
||||
which is executed in that order. Cleanup of resources provisioned during
|
||||
the resource_setup must be scheduled right after provisioning using
|
||||
@ -65,7 +65,7 @@ the addClassResourceCleanp helper. The resource cleanups stacked this way
|
||||
are executed in reverse order during tearDownClass, before the cleanup of
|
||||
test credentials takes place. An example of a TestCase which defines all
|
||||
of these would be::
|
||||
|
||||
|
||||
from tempest.common import waiters
|
||||
from tempest import config
|
||||
from tempest.lib.common.utils import test_utils
|
||||
|
||||
@ -14,11 +14,11 @@ multiple OpenStack services to exercise the touch points between them.
|
||||
|
||||
Any scenario test should have a real-life use case. An example would be:
|
||||
|
||||
- "As operator I want to start with a blank environment":
|
||||
1. upload a glance image
|
||||
2. deploy a vm from it
|
||||
3. ssh to the guest
|
||||
4. create a snapshot of the vm
|
||||
- "As operator I want to start with a blank environment":
|
||||
1. upload a glance image
|
||||
2. deploy a vm from it
|
||||
3. ssh to the guest
|
||||
4. create a snapshot of the vm
|
||||
|
||||
|
||||
Why are these tests in Tempest?
|
||||
|
||||
Loading…
Reference in New Issue
Block a user