Tempest provides a LockFixture to avoid two potentially interfering
tests to run in parallel. However, this solution does not scale when
we want to separate a set of tests from many other test cases. For
example, host aggregate and availability zone testing needs compute
hosts without any nova servers to be able to test moving computes
between aggregates but a lot of other tests are creating nova
servers. To fully separate these aggregate tests from the rest of
the tempest test cases, this patch proposes a @serial class decorator
to mark a test class to be run totally independently of any other test
Under the hood, the @serial decorator is implemented with a tempest-wide
interprocess read-write lock. The serial test classes always take the
write lock, while the non-serial classes take the read lock. The lock
allows in many readers OR a single writer. So the serial tests are run
independently from the rest.
To minimize the time a serial test blocks other tempest tests run in
parallel, this patch also introduced a serial_tests test directory to
store the serial tests. The current test ordering in a fresh env
uses alphabetical order so the serial tests will run at the end of
the execution not randomly in the middle. The gate uses fresh VMs
for every run so we can rely on this optimization there. In local
envs where tests are re-run, the subsequent runs will be ordered at
runtime by stestr. Therfore, a longer runtime might be observed due to
locking, but the correctness of the test execution is still kept.
from comments when it was last touched it looks like workarounds for
unittest2 might be able to be dropped.
simplify the workaround logic around unittest2 TestCase logic
In FIPS mode, using RSA keys for ssh is fine as long as SHA-1 is
not used for the signature algorithm. Unfortunately, the version
of cirros used in OpenStack CI does not have a version of dropbear
that supports SHA-2 signatures. So, any connections from a FIPS
enabled machine will fail as the cirros instance will only support
ssh-rsa (SHA-1 signatures).
To get around this, we add a new option to specify the key type
(validation.ssh_key_type). This will allow the addition of other
key types in future if needed.
Tempest now supports 'rsa' and 'ecdsa' key types.
We also add a fips job to the experimental queue to test the usage
of the new key type.
Commit Ib301466bb0a1996997c63599918fa96059a927bd added the usage
of oslo_utils md5 api which was added only in release 4.7.0.
Updating the requirements to accomodate this.
In addition, changing the requirements for other dependent libraries
We have recently observed that ssh-keygen by default
create OPENSSH format for keys and paramiko got the
support for decrypting it from paramiko 2.7.0
We have recenlty seen failures in whitebox plugin but
considering this can also be observe in tempest as well
it's good to update the version here too.
We have jsonschema capped at a fairly old version. Other than some
specific releases, it looks like keeping it below 3.0 was added in
I943fd68b9fab3bce1764305a5058df5339470757 without really any explanation
In order to update to a 3.x release we need to:
1. Remove the cap from global-requirements.txt (see Depends-On), leaving
upper-constraints.txt at a 2.x release
2. Remove the cap from all consumers (this change)
3. Release a new version of consumers that are published to pypi
4. Update upper-constraints.txt with those new releases
5. Update jsonschema in upper-constraints.txt to a 3.X release
6. Test consumers with the change from 5.
7. [Optional] fix issues in consumers that arise from 6.
8. Merge the change from 5.
Co-Authored-by: Sean McGinnis <email@example.com>
This commit switches the Tempest CLI commands to internally use stestr
instead of testrepository. At this point in time the testrepository
project is effectively unmaintained and stestr was a fork started to
have an actively maintained test runner. It also focuses on being a
dedicated python test runner, instead of an abstract test runner
interface for any tests that emit subunit.
Besides the bug fixes and other improvements included with stestr, this
switch provides a number of advantages for tempest. Primarily stestr has
a real python API for invoking the test runner directly from python. This
means we can simplify the wrapper code to simply call a function instead
of building out a set of CLI arguments and passing that to the CLI
Co-Authored-By: Matthew Treinish <firstname.lastname@example.org>
Since the commit 482e3ce6ab had
just removed the NegativeAutoTest Framework, which was the only
user of package testscenarios, testscenarios is no longer used by
tempest. This patch suggests to remove testscenarios from the
package denpendency defined file requirments.txt.
That new tox target checks that we don't have missing requirements in
requirements.txt AND that we don't have superfluous (unused) requirements
That patch is copy/pasted from I014375a5ecfebc723541c2f2db6c2e0fba636aa2
which tried to do the same for Nova (unmerged yet for an unknown reason).
This change also includes the fixes to requirements.txt needed
to pass the new tests.