DeprecationWarnings for jsonschema has been fixed at below path.
https://review.opendev.org/c/openstack/tempest/+/875264
But it is required to update the minimum jsonschema version >= 4.16.0
for including a below commit.
ee024ffc61
Closes-Bug: #2008490
Change-Id: I0f459d7f74406e30af725c751f3b7cd1a6a931a6
Tempest provides a LockFixture to avoid two potentially interfering
tests to run in parallel. However, this solution does not scale when
we want to separate a set of tests from many other test cases. For
example, host aggregate and availability zone testing needs compute
hosts without any nova servers to be able to test moving computes
between aggregates but a lot of other tests are creating nova
servers. To fully separate these aggregate tests from the rest of
the tempest test cases, this patch proposes a @serial class decorator
to mark a test class to be run totally independently of any other test
classes.
Under the hood, the @serial decorator is implemented with a tempest-wide
interprocess read-write lock. The serial test classes always take the
write lock, while the non-serial classes take the read lock. The lock
allows in many readers OR a single writer. So the serial tests are run
independently from the rest.
To minimize the time a serial test blocks other tempest tests run in
parallel, this patch also introduced a serial_tests test directory to
store the serial tests. The current test ordering in a fresh env
uses alphabetical order so the serial tests will run at the end of
the execution not randomly in the middle. The gate uses fresh VMs
for every run so we can rely on this optimization there. In local
envs where tests are re-run, the subsequent runs will be ordered at
runtime by stestr. Therfore, a longer runtime might be observed due to
locking, but the correctness of the test execution is still kept.
Related-Bug: #821732
Change-Id: I0181517edab75f586464a38c4811417f888783b1
Tempest was using:
-python module xml.etree[1],
It was vulnerable to different atacks.
Instead of xml.etree.ElementTree tempest is now using
defusedxml.ElementTree which is more secure.
[1] https://bandit.readthedocs.io/en/1.7.0/blacklists/blacklist_calls.html B313
Change-Id: I50a8ab3c3be2decccd7480ecf00f1a3e4a75f172
from comments when it was last touched it looks like workarounds for
unittest2 might be able to be dropped.
related: https://github.com/mtreinish/stestr/pull/265
simplify the workaround logic around unittest2 TestCase logic
Change-Id: Ibac9d0c2fa2f30605dd44ee58b84946464ea6449
In FIPS mode, using RSA keys for ssh is fine as long as SHA-1 is
not used for the signature algorithm. Unfortunately, the version
of cirros used in OpenStack CI does not have a version of dropbear
that supports SHA-2 signatures. So, any connections from a FIPS
enabled machine will fail as the cirros instance will only support
ssh-rsa (SHA-1 signatures).
To get around this, we add a new option to specify the key type
(validation.ssh_key_type). This will allow the addition of other
key types in future if needed.
Tempest now supports 'rsa' and 'ecdsa' key types.
We also add a fips job to the experimental queue to test the usage
of the new key type.
Change-Id: Ib59eb8432fa1a2813b3047955157d1b3d24a55f8
Commit Ib301466bb0a1996997c63599918fa96059a927bd added the usage
of oslo_utils md5 api which was added only in release 4.7.0.
Updating the requirements to accomodate this.
In addition, changing the requirements for other dependent libraries
Change-Id: Ia86208975cdc2e4261cb6cb20547d76ddb3ec571
We have recently observed that ssh-keygen by default
create OPENSSH format for keys and paramiko got the
support for decrypting it from paramiko 2.7.0[1]
We have recenlty seen failures in whitebox plugin but
considering this can also be observe in tempest as well
it's good to update the version here too.
[1]: http://www.paramiko.org/changelog.html
Change-Id: I3603bbc9787e99fe864a4ec63d49fea6b90a45a8
We have jsonschema capped at a fairly old version. Other than some
specific releases, it looks like keeping it below 3.0 was added in
I943fd68b9fab3bce1764305a5058df5339470757 without really any explanation
why.
In order to update to a 3.x release we need to:
1. Remove the cap from global-requirements.txt (see Depends-On), leaving
upper-constraints.txt at a 2.x release
2. Remove the cap from all consumers (this change)
3. Release a new version of consumers that are published to pypi
4. Update upper-constraints.txt with those new releases
5. Update jsonschema in upper-constraints.txt to a 3.X release
(See: https://review.openstack.org/649789)
6. Test consumers with the change from 5.
7. [Optional] fix issues in consumers that arise from 6.
8. Merge the change from 5.
Change-Id: I95b5520ca0e8da7b2a7663a272bc740b8c809592
Co-Authored-by: Sean McGinnis <sean.mcginnis@gmail.com>
Depends-On: https://review.openstack.org/649669
This commit switches the Tempest CLI commands to internally use stestr
instead of testrepository. At this point in time the testrepository
project is effectively unmaintained and stestr was a fork started to
have an actively maintained test runner. It also focuses on being a
dedicated python test runner, instead of an abstract test runner
interface for any tests that emit subunit.
Besides the bug fixes and other improvements included with stestr, this
switch provides a number of advantages for tempest. Primarily stestr has
a real python API for invoking the test runner directly from python. This
means we can simplify the wrapper code to simply call a function instead
of building out a set of CLI arguments and passing that to the CLI
processor.
Co-Authored-By: Matthew Treinish <mtreinish@kortar.org>
Depends-On: Ic1fa3a98b6bcd151c489b078028687892655a19b
Depends-On: I3855aad5ce129ec8ccb87c05f7aa709b74070efe
Depends-On: https://review.openstack.org/529490/
Change-Id: I6f5fa7796c576b71c4a0dde66896974a8039a848
Since the commit 482e3ce6ab had
just removed the NegativeAutoTest Framework, which was the only
user of package testscenarios, testscenarios is no longer used by
tempest. This patch suggests to remove testscenarios from the
package denpendency defined file requirments.txt.
Change-Id: Id2d4ab7c54223044551e4684e9ae1eee130f01c0
That new tox target checks that we don't have missing requirements in
requirements.txt AND that we don't have superfluous (unused) requirements
in requirements.txt.
That patch is copy/pasted from I014375a5ecfebc723541c2f2db6c2e0fba636aa2
which tried to do the same for Nova (unmerged yet for an unknown reason).
This change also includes the fixes to requirements.txt needed
to pass the new tests.
Change-Id: Iff37af95269c38e4d435a53a21e5d60e2c5a3fa8
This `call_until_true()` is handy and could be used in Tempest plugins.
Let's move it to tempest/lib.
Also add some unit tests.
Change-Id: Ie379030baa336239e6027c8f3cdbeb74c561f66b
Ghanshyam