Balazs Gibizer
dfb304355b
Tempest provides a LockFixture to avoid two potentially interfering tests to run in parallel. However, this solution does not scale when we want to separate a set of tests from many other test cases. For example, host aggregate and availability zone testing needs compute hosts without any nova servers to be able to test moving computes between aggregates but a lot of other tests are creating nova servers. To fully separate these aggregate tests from the rest of the tempest test cases, this patch proposes a @serial class decorator to mark a test class to be run totally independently of any other test classes. Under the hood, the @serial decorator is implemented with a tempest-wide interprocess read-write lock. The serial test classes always take the write lock, while the non-serial classes take the read lock. The lock allows in many readers OR a single writer. So the serial tests are run independently from the rest. To minimize the time a serial test blocks other tempest tests run in parallel, this patch also introduced a serial_tests test directory to store the serial tests. The current test ordering in a fresh env uses alphabetical order so the serial tests will run at the end of the execution not randomly in the middle. The gate uses fresh VMs for every run so we can rely on this optimization there. In local envs where tests are re-run, the subsequent runs will be ordered at runtime by stestr. Therfore, a longer runtime might be observed due to locking, but the correctness of the test execution is still kept. Related-Bug: #821732 Change-Id: I0181517edab75f586464a38c4811417f888783b1
50 lines
1.9 KiB
Python
50 lines
1.9 KiB
Python
# Copyright 2013 IBM Corp.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
# not use this file except in compliance with the License. You may obtain
|
|
# a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
# License for the specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
import os
|
|
import unittest
|
|
|
|
from tempest.test_discover import plugins
|
|
|
|
|
|
def load_tests(loader, tests, pattern):
|
|
ext_plugins = plugins.TempestTestPluginManager()
|
|
|
|
suite = unittest.TestSuite()
|
|
base_path = os.path.split(os.path.dirname(os.path.abspath(__file__)))[0]
|
|
base_path = os.path.split(base_path)[0]
|
|
# Load local tempest tests
|
|
for test_dir in ['api', 'scenario', 'serial_tests']:
|
|
full_test_dir = os.path.join(base_path, 'tempest', test_dir)
|
|
if not pattern:
|
|
suite.addTests(loader.discover(full_test_dir,
|
|
top_level_dir=base_path))
|
|
else:
|
|
suite.addTests(loader.discover(full_test_dir, pattern=pattern,
|
|
top_level_dir=base_path))
|
|
|
|
plugin_load_tests = ext_plugins.get_plugin_load_tests_tuple()
|
|
if not plugin_load_tests:
|
|
return suite
|
|
|
|
# Load any installed plugin tests
|
|
for plugin in plugin_load_tests:
|
|
test_dir, top_path = plugin_load_tests[plugin]
|
|
if not pattern:
|
|
suite.addTests(loader.discover(test_dir, top_level_dir=top_path))
|
|
else:
|
|
suite.addTests(loader.discover(test_dir, pattern=pattern,
|
|
top_level_dir=top_path))
|
|
return suite
|