openstack
/
tempest
Code Issues Proposed changes
tempest/tempest/tests
History
Balazs Gibizer dfb304355b Introduce @serial test execution decorator 
Tempest provides a LockFixture to avoid two potentially interfering
tests to run in parallel. However, this solution does not scale when
we want to separate a set of tests from many other test cases. For
example, host aggregate and availability zone testing needs compute
hosts without any nova servers to be able to test moving computes
between aggregates but a lot of other tests are creating nova
servers. To fully separate these aggregate tests from the rest of
the tempest test cases, this patch proposes a @serial class decorator
to mark a test class to be run totally independently of any other test
classes.

Under the hood, the @serial decorator is implemented with a tempest-wide
interprocess read-write lock. The serial test classes always take the
write lock, while the non-serial classes take the read lock. The lock
allows in many readers OR a single writer. So the serial tests are run
independently from the rest.

To minimize the time a serial test blocks other tempest tests run in
parallel, this patch also introduced a serial_tests test directory to
store the serial tests. The current test ordering in a fresh env
uses alphabetical order so the serial tests will run at the end of
the execution not randomly in the middle. The gate uses fresh VMs
for every run so we can rely on this optimization there. In local
envs where tests are re-run, the subsequent runs will be ordered at
runtime by stestr. Therfore, a longer runtime might be observed due to
locking, but the correctness of the test execution is still kept.

Related-Bug: #821732
Change-Id: I0181517edab75f586464a38c4811417f888783b1
 		2023-01-18 02:45:43 +00:00
..
api [part1] Remove six 2021-02-24 12:45:22 +00:00
cmd Remove compute api_extensions config option 2022-05-05 18:15:29 -05:00
common Merge "Added functionality to wait for port creation" 2023-01-09 17:43:30 +00:00
files Change openstack-dev to openstack-discuss 2018-12-04 06:39:07 -05:00
lib Merge "Decorate volume.base functions - fix cleanup" 2023-01-10 16:33:34 +00:00
README.rst Doc: fix markups, capitalization and add 2 REVIEWING advices 2017-07-11 20:26:32 +02:00
__init__.py Add unittest framework + tests for wrapper scripts 2013-08-27 11:55:13 -04:00
base.py Use unittest.mock instead of third party mock 2020-05-17 17:22:58 +00:00
fake_config.py Add test coverage CMD cleanup_service 2018-09-27 17:02:18 +00:00
fake_tempest_plugin.py Service Clients registration interface for plugins 2016-07-26 22:52:33 +01:00
test_base_test.py Support scope in dynamic cred for specific roles 2021-03-06 17:14:22 +00:00
test_config.py Revert "Move dscv and ca_certs to config section service_clients" 2016-08-19 14:20:40 +00:00
test_decorators.py Remove compute api_extensions config option 2022-05-05 18:15:29 -05:00
test_hacking.py Use LOG.warning instead of deprecated LOG.warn 2022-01-19 13:38:21 +09:00
test_imports.py Use unittest.mock instead of third party mock 2020-05-17 17:22:58 +00:00
test_list_tests.py [part1] Remove six 2021-02-24 12:45:22 +00:00
test_microversions.py [part1] Remove six 2021-02-24 12:45:22 +00:00
test_tempest_plugin.py Add a validation resources fixture 2017-09-12 12:37:24 -06:00
test_test.py Introduce @serial test execution decorator 2023-01-18 02:45:43 +00:00
utils.py Unit tests: mock some time.sleep and time.time 2016-03-03 14:33:45 +01:00

README.rst

Tempest Field Guide to Unit tests

What are these tests?

Unit tests are the self checks for Tempest. They provide functional verification and regression checking for the internal components of Tempest. They should be used to just verify that the individual pieces of Tempest are working as expected. They should not require an external service to be running and should be able to run solely from the Tempest tree.

Why are these tests in Tempest?

These tests exist to make sure that the mechanisms that we use inside of Tempest are valid and remain functional. They are only here for self validation of Tempest.

Scope of these tests

Unit tests should not require an external service to be running or any extra configuration to run. Any state that is required for a test should either be mocked out or created in a temporary test directory. (see test_wrappers.py for an example of using a temporary test directory)