From b78b923e5a9c8f8a21cad2c9aed4bd12c28d1a9b Mon Sep 17 00:00:00 2001 From: Masayuki Igawa Date: Fri, 17 Nov 2017 16:12:37 +0900 Subject: [PATCH] Fix indentation in docs This commit fixes indentation in tempest docs. These indentations are not necessary and it causes a weird html outputs. Change-Id: I9c8714558a3327b7ad0b0ab0d3fdc7e770c3c75b --- HACKING.rst | 4 +- REVIEWING.rst | 10 +- doc/source/configuration.rst | 86 ++++++++--------- doc/source/library.rst | 6 +- doc/source/microversion_testing.rst | 140 ++++++++++++++-------------- doc/source/test_removal.rst | 86 ++++++++--------- doc/source/write_tests.rst | 22 ++--- tempest/scenario/README.rst | 10 +- 8 files changed, 182 insertions(+), 182 deletions(-) diff --git a/HACKING.rst b/HACKING.rst index c942cb1432..57f0409ba8 100644 --- a/HACKING.rst +++ b/HACKING.rst @@ -9,14 +9,14 @@ Tempest Specific Commandments ------------------------------ - [T102] Cannot import OpenStack python clients in tempest/api & - tempest/scenario tests + tempest/scenario tests - [T104] Scenario tests require a services decorator - [T105] Tests cannot use setUpClass/tearDownClass - [T106] vim configuration should not be kept in source files. - [T107] Check that a service tag isn't in the module path - [T108] Check no hyphen at the end of rand_name() argument - [T109] Cannot use testtools.skip decorator; instead use - decorators.skip_because from tempest.lib + decorators.skip_because from tempest.lib - [T110] Check that service client names of GET should be consistent - [T111] Check that service client names of DELETE should be consistent - [T112] Check that tempest.lib should not import local tempest code diff --git a/REVIEWING.rst b/REVIEWING.rst index 5e08a6b2fb..766d0c63f7 100644 --- a/REVIEWING.rst +++ b/REVIEWING.rst @@ -121,8 +121,8 @@ fix it. When it will happen, we will deal with it on a case-by-case basis. When to approve --------------- - * Every patch needs two +2s before being approved. - * Its ok to hold off on an approval until a subject matter expert reviews it - * If a patch has already been approved but requires a trivial rebase to merge, - you do not have to wait for a second +2, since the patch has already had - two +2s. +* Every patch needs two +2s before being approved. +* Its ok to hold off on an approval until a subject matter expert reviews it +* If a patch has already been approved but requires a trivial rebase to merge, + you do not have to wait for a second +2, since the patch has already had + two +2s. diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst index 8f2865ad59..e5f70d2fff 100644 --- a/doc/source/configuration.rst +++ b/doc/source/configuration.rst @@ -17,10 +17,10 @@ Test Credentials Tempest allows for configuring a set of admin credentials in the ``auth`` section, via the following parameters: - #. ``admin_username`` - #. ``admin_password`` - #. ``admin_project_name`` - #. ``admin_domain_name`` +#. ``admin_username`` +#. ``admin_password`` +#. ``admin_project_name`` +#. ``admin_domain_name`` Admin credentials are not mandatory to run Tempest, but when provided they can be used to: @@ -47,9 +47,9 @@ In order for Tempest to be able to talk to your OpenStack deployment you need to provide it with information about how it communicates with keystone. This involves configuring the following options in the ``identity`` section: - - ``auth_version`` - - ``uri`` - - ``uri_v3`` +- ``auth_version`` +- ``uri`` +- ``uri_v3`` The ``auth_version`` option is used to tell Tempest whether it should be using Keystone's v2 or v3 api for communicating with Keystone. The two uri options are @@ -74,12 +74,12 @@ three sets of username, password, and project names for a primary user, an admin user, and an alternate user. To enable and use dynamic credentials you only need to configure two things: - #. A set of admin credentials with permissions to create users and - projects. This is specified in the ``auth`` section with the - ``admin_username``, ``admin_project_name``, ``admin_domain_name`` and - ``admin_password`` options - #. To enable dynamic credentials in the ``auth`` section with the - ``use_dynamic_credentials`` option. +#. A set of admin credentials with permissions to create users and + projects. This is specified in the ``auth`` section with the + ``admin_username``, ``admin_project_name``, ``admin_domain_name`` and + ``admin_password`` options +#. To enable dynamic credentials in the ``auth`` section with the + ``use_dynamic_credentials`` option. This is also currently the default credential provider enabled by Tempest, due to its common use and ease of configuration. @@ -115,21 +115,21 @@ like with dynamic credentials. To enable and use locking test accounts you need do a few things: - #. Create an accounts.yaml file which contains the set of pre-existing - credentials to use for testing. To make sure you don't have a credentials - starvation issue when running in parallel make sure you have at least two - times the number of worker processes you are using to execute Tempest - available in the file. (If running serially the worker count is 1.) +#. Create an accounts.yaml file which contains the set of pre-existing + credentials to use for testing. To make sure you don't have a credentials + starvation issue when running in parallel make sure you have at least two + times the number of worker processes you are using to execute Tempest + available in the file. (If running serially the worker count is 1.) - You can check the accounts.yaml.sample file packaged in Tempest for the yaml - format. - #. Provide Tempest with the location of your accounts.yaml file with the - ``test_accounts_file`` option in the ``auth`` section + You can check the accounts.yaml.sample file packaged in Tempest for the yaml + format. +#. Provide Tempest with the location of your accounts.yaml file with the + ``test_accounts_file`` option in the ``auth`` section - *NOTE: Be sure to use a full path for the file; otherwise Tempest will - likely not find it.* + *NOTE: Be sure to use a full path for the file; otherwise Tempest will + likely not find it.* - #. Set ``use_dynamic_credentials = False`` in the ``auth`` group +#. Set ``use_dynamic_credentials = False`` in the ``auth`` group It is worth pointing out that each set of credentials in the accounts.yaml should have a unique project. This is required to provide proper isolation @@ -162,8 +162,8 @@ For Tempest to be able to create servers you need to specify flavors that it can use to boot the servers with. There are two options in the Tempest config for doing this: - #. ``flavor_ref`` - #. ``flavor_ref_alt`` +#. ``flavor_ref`` +#. ``flavor_ref_alt`` Both of these options are in the ``compute`` section of the config file and take in the flavor id (not the name) from Nova. The ``flavor_ref`` option is what @@ -181,8 +181,8 @@ Images Just like with flavors, Tempest needs to know which images to use for booting servers. There are two options in the compute section just like with flavors: - #. ``image_ref`` - #. ``image_ref_alt`` +#. ``image_ref`` +#. ``image_ref_alt`` Both options are expecting an image id (not name) from Nova. The ``image_ref`` option is what will be used for booting the majority of servers in Tempest. @@ -192,13 +192,13 @@ those tests will be skipped. There are also options in the ``scenario`` section for images: - #. ``img_file`` - #. ``img_dir`` - #. ``aki_img_file`` - #. ``ari_img_file`` - #. ``ami_img_file`` - #. ``img_container_format`` - #. ``img_disk_format`` +#. ``img_file`` +#. ``img_dir`` +#. ``aki_img_file`` +#. ``ari_img_file`` +#. ``ami_img_file`` +#. ``img_container_format`` +#. ``img_disk_format`` However, unlike the other image options, these are used for a very small subset of scenario tests which are uploading an image. These options are used to tell @@ -261,7 +261,7 @@ booting. To set a fixed network name simply: - #. Set the ``fixed_network_name`` option in the ``compute`` group +#. Set the ``fixed_network_name`` option in the ``compute`` group In the case that the configured fixed network name can not be found by a user network list call, it will be treated like one was not provided except that a @@ -329,9 +329,9 @@ connecting to and remotely accessing the created servers. To enable remote access to servers, there are 3 options at a minimum that are used: - #. ``run_validation`` - #. ``connect_method`` - #. ``auth_method`` +#. ``run_validation`` +#. ``connect_method`` +#. ``auth_method`` The ``run_validation`` is used to enable or disable ssh connectivity for all tests (with the exception of scenario tests which do not have a flag for @@ -370,9 +370,9 @@ API calls for each service, it needs to know how that project is defined in the service catalog. There are three options for each service section to accomplish this: - #. ``catalog_type`` - #. ``endpoint_type`` - #. ``region`` +#. ``catalog_type`` +#. ``endpoint_type`` +#. ``region`` Setting ``catalog_type`` and ``endpoint_type`` should normally give Tempest enough information to determine which endpoint it should pull from the service diff --git a/doc/source/library.rst b/doc/source/library.rst index 074d642e42..14415ae44a 100644 --- a/doc/source/library.rst +++ b/doc/source/library.rst @@ -44,9 +44,9 @@ ensure that we don't break backwards compatibility. For patches that change existing interfaces we have to be careful to make sure we don't break any external consumers. Some common red flags are: - * a change to an existing API requires a change outside the library directory - where the interface is being consumed - * a unit test has to be significantly changed to make the proposed change pass +* a change to an existing API requires a change outside the library directory + where the interface is being consumed +* a unit test has to be significantly changed to make the proposed change pass Testing ''''''' diff --git a/doc/source/microversion_testing.rst b/doc/source/microversion_testing.rst index acf55930ee..7189312834 100644 --- a/doc/source/microversion_testing.rst +++ b/doc/source/microversion_testing.rst @@ -33,46 +33,46 @@ project guidelines. Tempest will cover only integration testing of applicable microversions with below exceptions: - #. Test covers a feature which is important for interoperability. This covers tests requirement - from Defcore. - #. Test needed to fill Schema gaps. - Tempest validates API responses with defined JSON schema. API responses can be different on - each microversion and the JSON schemas need to be defined separately for the microversion. - While implementing new integration tests for a specific microversion, there - may be a gap in the JSON schemas (caused by previous microversions) implemented - in Tempest. - Filling that gap while implementing the new integration test cases is not efficient due to - many reasons: +#. Test covers a feature which is important for interoperability. This covers tests requirement + from Defcore. +#. Test needed to fill Schema gaps. + Tempest validates API responses with defined JSON schema. API responses can be different on + each microversion and the JSON schemas need to be defined separately for the microversion. + While implementing new integration tests for a specific microversion, there + may be a gap in the JSON schemas (caused by previous microversions) implemented + in Tempest. + Filling that gap while implementing the new integration test cases is not efficient due to + many reasons: - * Hard to review - * Sync between multiple integration tests patches which try to fill the same schema gap at same - time - * Might delay the microversion change on project side where project team wants Tempest - tests to verify the results. + * Hard to review + * Sync between multiple integration tests patches which try to fill the same schema gap at same + time + * Might delay the microversion change on project side where project team wants Tempest + tests to verify the results. - Tempest will allow to fill the schema gaps at the end of each cycle, or more - often if required. - Schema gap can be filled with testing those with a minimal set of tests. Those - tests might not be integration tests and might be already covered on project - side also. - This exception is needed because: + Tempest will allow to fill the schema gaps at the end of each cycle, or more + often if required. + Schema gap can be filled with testing those with a minimal set of tests. Those + tests might not be integration tests and might be already covered on project + side also. + This exception is needed because: - * Allow to create microversion response schema in Tempest at the same time that projects are - implementing their API microversions. This will make implementation easier for adding - required tests before a new microversion change can be merged in the corresponding project - and hence accelerate the development of microversions. - * New schema must be verified by at least one test case which exercises such schema. + * Allow to create microversion response schema in Tempest at the same time that projects are + implementing their API microversions. This will make implementation easier for adding + required tests before a new microversion change can be merged in the corresponding project + and hence accelerate the development of microversions. + * New schema must be verified by at least one test case which exercises such schema. - For example: - If any projects implemented 4 API microversion say- v2.3, v2.4, v2.5, v2.6 - Assume microversion v2.3, v2.4, v2.6 change the API Response which means Tempest - needs to add JSON schema for v2.3, v2.4, v2.6. - In that case if only 1 or 2 tests can verify all new schemas then we do not need - separate tests for each new schemas. In worst case, we have to add 3 separate tests. - #. Test covers service behavior at large scale with involvement of more deep layer like hypervisor - etc not just API/DB layer. This type of tests will be added case by case basis and - with project team consultation about why it cannot be covered on project side and worth to test - in Tempest. + For example: + If any projects implemented 4 API microversion say- v2.3, v2.4, v2.5, v2.6 + Assume microversion v2.3, v2.4, v2.6 change the API Response which means Tempest + needs to add JSON schema for v2.3, v2.4, v2.6. + In that case if only 1 or 2 tests can verify all new schemas then we do not need + separate tests for each new schemas. In worst case, we have to add 3 separate tests. +#. Test covers service behavior at large scale with involvement of more deep layer like hypervisor + etc not just API/DB layer. This type of tests will be added case by case basis and + with project team consultation about why it cannot be covered on project side and worth to test + in Tempest. Project Scope For Microversion Testing """""""""""""""""""""""""""""""""""""" @@ -294,72 +294,72 @@ Microversion tests implemented in Tempest * Compute - * `2.1`_ + * `2.1`_ - .. _2.1: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id1 + .. _2.1: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id1 - * `2.2`_ + * `2.2`_ - .. _2.2: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id2 + .. _2.2: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id2 - * `2.10`_ + * `2.10`_ - .. _2.10: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id9 + .. _2.10: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id9 - * `2.20`_ + * `2.20`_ - .. _2.20: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id18 + .. _2.20: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id18 - * `2.25`_ + * `2.25`_ - .. _2.25: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-mitaka + .. _2.25: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-mitaka - * `2.32`_ + * `2.32`_ - .. _2.32: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id29 + .. _2.32: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id29 - * `2.37`_ + * `2.37`_ - .. _2.37: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id34 + .. _2.37: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id34 - * `2.42`_ + * `2.42`_ - .. _2.42: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-ocata + .. _2.42: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-ocata - * `2.47`_ + * `2.47`_ - .. _2.47: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id42 + .. _2.47: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id42 - * `2.48`_ + * `2.48`_ - .. _2.48: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id43 + .. _2.48: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id43 * Volume - * `3.3`_ + * `3.3`_ - .. _3.3: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id3 + .. _3.3: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id3 - * `3.9`_ + * `3.9`_ - .. _3.9: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id9 + .. _3.9: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id9 - * `3.11`_ + * `3.11`_ - .. _3.11: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id11 + .. _3.11: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id11 - * `3.12`_ + * `3.12`_ - .. _3.12: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id12 + .. _3.12: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id12 - * `3.14`_ + * `3.14`_ - .. _3.14: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id14 + .. _3.14: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id14 - * `3.19`_ + * `3.19`_ - .. _3.19: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id18 + .. _3.19: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id18 - * `3.20`_ + * `3.20`_ - .. _3.20: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id19 + .. _3.20: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html#id19 diff --git a/doc/source/test_removal.rst b/doc/source/test_removal.rst index 07c3046b0f..b57e98f893 100644 --- a/doc/source/test_removal.rst +++ b/doc/source/test_removal.rst @@ -29,19 +29,19 @@ Proposing a test removal In the proposal etherpad we'll be looking for answers to 3 questions - #. The tests proposed for removal must have equiv. coverage in a different - project's test suite (whether this is another gating test project, or an in - tree functional test suite). For API tests preferably the other project will - have a similar source of friction in place to prevent breaking api changes - so that we don't regress and let breaking api changes slip through the - gate. - #. The test proposed for removal has a failure rate < 0.50% in the gate over - the past release (the value and interval will likely be adjusted in the - future) +#. The tests proposed for removal must have equiv. coverage in a different + project's test suite (whether this is another gating test project, or an in + tree functional test suite). For API tests preferably the other project will + have a similar source of friction in place to prevent breaking api changes + so that we don't regress and let breaking api changes slip through the + gate. +#. The test proposed for removal has a failure rate < 0.50% in the gate over + the past release (the value and interval will likely be adjusted in the + future) - .. _`prong #3`: - #. There must not be an external user/consumer of tempest - that depends on the test proposed for removal + .. _`prong #3`: +#. There must not be an external user/consumer of tempest + that depends on the test proposed for removal The answers to 1 and 2 are easy to verify. For 1 just provide a link to the new test location. If you are linking to the tempest removal patch please also put @@ -68,23 +68,23 @@ setUpClass or tearDownClass failures) You can access the infra mysql subunit2sql db w/ read-only permissions with: - * hostname: logstash.openstack.org - * username: query - * password: query - * db_name: subunit2sql +* hostname: logstash.openstack.org +* username: query +* password: query +* db_name: subunit2sql For example if you were trying to remove the test with the id: tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON.test_get_flavor_details_for_deleted_flavor you would run the following: - #. run: "mysql -u query -p -h logstash.openstack.org subunit2sql" to connect - to the subunit2sql db - #. run the query: MySQL [subunit2sql]> select * from tests where test_id like - "tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON%"; - which will return a table of all the tests in the class (but it will also - catch failures in setUpClass and tearDownClass) - #. paste the output table with numbers and the mysql command you ran to - generate it into the etherpad. +#. run: "mysql -u query -p -h logstash.openstack.org subunit2sql" to connect + to the subunit2sql db +#. run the query: MySQL [subunit2sql]> select * from tests where test_id like + "tempest.api.compute.admin.test_flavors_negative.FlavorsAdminNegativeTestJSON%"; + which will return a table of all the tests in the class (but it will also + catch failures in setUpClass and tearDownClass) +#. paste the output table with numbers and the mysql command you ran to + generate it into the etherpad. Eventually a cli interface will be created to make that a bit more friendly. Also a dashboard is in the works so we don't need to manually run the command. @@ -131,23 +131,23 @@ Exceptions to this procedure For the most part all tempest test removals have to go through this procedure there are a couple of exceptions though: - #. The class of testing has been decided to be outside the scope of tempest. - #. A revert for a patch which added a broken test, or testing which didn't - actually run in the gate (basically any revert for something which - shouldn't have been added) - #. Tests that would become out of scope as a consequence of an API change, - as described in `API Compatibility`_. - Such tests cannot live in Tempest because of the branchless nature of - Tempest. Such test must still honor `prong #3`_. +#. The class of testing has been decided to be outside the scope of tempest. +#. A revert for a patch which added a broken test, or testing which didn't + actually run in the gate (basically any revert for something which + shouldn't have been added) +#. Tests that would become out of scope as a consequence of an API change, + as described in `API Compatibility`_. + Such tests cannot live in Tempest because of the branchless nature of + Tempest. Such test must still honor `prong #3`_. For the first exception type the only types of testing in tree which have been declared out of scope at this point are: - * The CLI tests (which should be completely removed at this point) - * Neutron Adv. Services testing (which should be completely removed at this - point) - * XML API Tests (which should be completely removed at this point) - * EC2 API/boto tests (which should be completely removed at this point) +* The CLI tests (which should be completely removed at this point) +* Neutron Adv. Services testing (which should be completely removed at this + point) +* XML API Tests (which should be completely removed at this point) +* EC2 API/boto tests (which should be completely removed at this point) For tests that fit into this category the only criteria for removal is that there is equivalent testing elsewhere. @@ -159,12 +159,12 @@ Starting in the liberty cycle tempest has defined a set of projects which are defined as in scope for direct testing in tempest. As of today that list is: - * Keystone - * Nova - * Glance - * Cinder - * Neutron - * Swift +* Keystone +* Nova +* Glance +* Cinder +* Neutron +* Swift anything that lives in tempest which doesn't test one of these projects can be removed assuming there is equivalent testing elsewhere. Preferably using the diff --git a/doc/source/write_tests.rst b/doc/source/write_tests.rst index 5a2876ee0c..49af95a306 100644 --- a/doc/source/write_tests.rst +++ b/doc/source/write_tests.rst @@ -36,12 +36,12 @@ methods. In standard unittest the lifecycle of a TestCase can be described in the following phases: - #. setUpClass - #. setUp - #. Test Execution - #. tearDown - #. doCleanups - #. tearDownClass +#. setUpClass +#. setUp +#. Test Execution +#. tearDown +#. doCleanups +#. tearDownClass setUpClass ---------- @@ -54,10 +54,10 @@ setup for interacting with the configured cloud. To accomplish this you do **not** define a setUpClass function, instead there are a number of predefined phases to setUpClass that are used. The phases are: - * skip_checks - * setup_credentials - * setup_clients - * resource_setup +* skip_checks +* setup_credentials +* setup_clients +* resource_setup which is executed in that order. Cleanup of resources provisioned during the resource_setup must be scheduled right after provisioning using @@ -65,7 +65,7 @@ the addClassResourceCleanp helper. The resource cleanups stacked this way are executed in reverse order during tearDownClass, before the cleanup of test credentials takes place. An example of a TestCase which defines all of these would be:: - + from tempest.common import waiters from tempest import config from tempest.lib.common.utils import test_utils diff --git a/tempest/scenario/README.rst b/tempest/scenario/README.rst index c1dcccc0b1..ad300c2266 100644 --- a/tempest/scenario/README.rst +++ b/tempest/scenario/README.rst @@ -14,11 +14,11 @@ multiple OpenStack services to exercise the touch points between them. Any scenario test should have a real-life use case. An example would be: - - "As operator I want to start with a blank environment": - 1. upload a glance image - 2. deploy a vm from it - 3. ssh to the guest - 4. create a snapshot of the vm +- "As operator I want to start with a blank environment": + 1. upload a glance image + 2. deploy a vm from it + 3. ssh to the guest + 4. create a snapshot of the vm Why are these tests in Tempest?