stable/zed has the new defaults and scope available but
they were disabled by default. Let's test those by enabling
in stable/zed also so that we know if they work fine in
integrated way or not
Change-Id: Id2bd58854f60f38dccf0ef805de284c7e95db2f3
Now we have stable/2023.1 branch ready for devstack
and so does for all service projects.
This commit adds the Tempest testing for stable/2023.1
by adding new jobs running on stable/2023.1 version of
openstack.
Change-Id: I5994c6fb56e01c9697d4d80bcbff2939ef2eff04
This reverts commit 517563fde8.
Reason for revert: we did few improvement in job/tests run, let's see if timeout is solved or less frequent.
Change-Id: Ib29fb6d817b801e1472fc22fe761a2309f473dc0
Recently we are seeing a lot of job timeout(bug#2004780)
and we see many tests taking time and also number of tests
increasing over time. This is to prepare the list of extra tests
(here extra tests means the tests which are covered by the other
API or scenario tests) which we do not need to run in every
integrated jobs. Instead, we can run them in a separete job(s).
Currently I am adding admin (except keystone) and negative tests
in the 'extra tests' list but we can add more tests here which
we think are covered in some other tests.
As negative tests are important for interop, adding those extra
tests coverage for stable branch job also but running them in
periodic run only.
Related-Bug: #2004780
Change-Id: Id02221df0d6180519751c63e890851bd59fdafa0
tempest-slow-py3 job run all the slow test serially which
takes lot of time and end up job timeout. This preparing
tempest-slow-parallel job which will run slow tests parallelly
in periodic run. Based on the results, later we can make
tempest-slow-py3 job to run tests in parallel.
Also, run tempest-full-parallel in periodic and based on the
result we can run tempest-full-py3 job scenario tests in parallel.
Relavant-Bug: #2004780
Change-Id: I876dacb40daa384cddc8faae3200cd3d39506ddc
We have a few jobs running gate even they are non voting
and run in periodic as well. They do not need to run in
every change but running/checking them in periodic run
is enough coverage. Below are the jobs moving to periodic:
* tempest-full-py3-ipv6
We do run tempest-ipv6-only job as voting in gate to
cover the ipv6 run and tempest-full-py3-ipv6 job can
run perdiocially to test the full tempest on ipv6.
*tempest-full-centos-9-stream
we already discussed and agreed in TC also that centos stream
testing is best effort and can be in periodic or non voting.
*tempest-full-test-account-no-admin-py3
Checking if tempest can be run without admin in periodic and not
on every change is enough.
* tempest-full-yoga
We do run all supported stable branch jobs periodically and running only
latest and oldest supported in check pipeline should be enough to catch
any breaking change on stable branches.
Relavant-Bug: #2004780
Change-Id: I8a2da7288e3f2264ce3cc39115c1d807b21fff95
When I converted this from a decorator to a context manager, I didn't
remove the fn parameter, nor was it even doing the thing I expected
on the second and later iterations because it doesn't fail in my local
environment. Apparently we're not running this test in
the tempest gate, so this adds to experimental so we can at least
run it on command.
Change-Id: Ia72b50f7f7bf64fe0ddd3f1a415b1807ff264b66
The commit marks the job temporarily as non-voting to unblock the
CI. There are multiple patches waiting to be merged which address
other bugs (e.g. the timeout issues) and this job fails at ~50%
rate which makes the merging of other patches comlicated.
Related-Bug: #1998916
Change-Id: I4ef3a6e5c4bbef93d355bfa42589fdb60db43663
We have many services (Nova, Neutron, Glance etc) implemented the
new RBAC (project scope and project personas). For these services,
all tests should pass as projects personas (project reader) does
not impact existing testing/usage.
keystone has system scope adopted in their policy for now which
we need to make it work for project scope also and until then
we will see test failing.
This commit adds a new tempest full job which enable the scope
and new defaults of RBAC for applicable services.
Depends-On: https://review.opendev.org/c/openstack/neutron/+/865040
Change-Id: Ib8f2f0e25205edba332fb9bd2a73012016d45061
As per 2023.1 cycle testing runtime and community goal[1] we need to migrate
upstream CI/CD to Ubuntu Jammy(22.04). Most of the Tempest jobs will be migrate
automatically once devstack base job start running on Jammy(Depends-On).
As we need to run at least one job on Ubuntu Focal(20.04), I am adding
a new job tempest-full-ubuntu-focal.
Few job like tempest-full-multinode-py3 and plugin sanity job needs to override
the nodeset. multinode job needs to run on Focal till stable/zed and on Jammy
from master(2023.1) onwards. Stable branch jobs testing stable branch needs to
keep running on their supported distro version which is Focal.
Depends-On: https://review.opendev.org/c/openstack/devstack/+/860795
Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/861116
[1] https://governance.openstack.org/tc/goals/selected/migrate-ci-jobs-to-ubuntu-jammy.html
Change-Id: I2e37603b098880b778100613800496b01167b22f
Now we have stable/zed branch ready for devstack and so does
for all service projects.
This commit adds the Tempest testing for stable/zed by adding
new jobs running on stable/zed version of openstack.
Change-Id: I6a331a6f2466aa4e4d9f607660b87fa60a63bf4b
tempest-multinode-full-py3 is voting job running
in check pipeline only, we should run it in gate
pipeline also.
Change-Id: Ifdd7e48b31ef29c54b2bfa805e1eb3197589f925
Nova does not support few operation for exmaple
migration, unshelve etc cross cell and we need to
skip such operation testing for cross cell job.
nova-multi-cell job is multi cell job which will be
helpful to check the new tests in tempest.
Adding it in experimental queue so that we can run it
on demand basis.
Change-Id: Ib59f00f78921afd00a7dce26cfd453288eea462c
We merged a broken test case in Icdb3398061d0274fec8b4e10d955a4b00edf4524
as nova-live-migration job was non voting and we missed that the job
failed on the patch.
Let's make the job voting to avoid this in the future.
Change-Id: Ibe8c439da6c1ab6657b6f2618c3a72a074e24305
As 3.10 is next python version we will be testing in
next cycle (after Zed), let's add it in our testing in advance.
Change-Id: I1d60c21473c7b657a3f4ad57525c4561f2cfcfd5
This marks the end of support for stable/victoria
which is EM now[1]. This remove the stable/victoria
jobs and also adds relesenote to mark the end of
support of Victoria.
[1] https://releases.openstack.org/
Change-Id: Ia63cdc682a32a2bcfd748f924368db6028b64d8c
py3.6 and py3.7 jobs are failing due to oslo dropped the
py3.6|7 support. We will add them back once bug#1975036
is resolved
Tempest plugin sanity job is broken due to failure in
murano-tempest-plugin. To make this patch merge which
will unblock the gate, I am making plugin sanity job as
non voting. We will revert that once below fix is merged
tempest-full-centos-9-stream job is also failing which fix
is also up and waiting for this patch to merge first, so making
them non voting.
devstack-plugin-ceph-tempest-py3 job is also mostly failing, let's
make it temporarily non voting in order to avoid endless rechecks
- https://review.opendev.org/c/openstack/murano-tempest-plugin/+/843040
Related-Bug: #1975036
Change-Id: Ibcaf1bce80521e05d76199ddebccb33488360644
To cover the stable branch testing on branchless Tempest
master gate, we test tempest-full version of stable branches.
Let's test the tempest-slow job also for stables branches.
All supported stable branches jobs are added in periodic
stable pipeline. As slow jobs are slow, it will make check pipelines
too heavy if we add all the stable branches slow jobs so
I am adding slow jobs only for the oldest supported stable job
which should cover the tempest master working on most
of the stable.
This will cover the Tempest slow tests stability for stable branches.
Change-Id: Idb8f178fa4fb4b669f3a163d5c5a9517b9d43c9a
This marks the end of support for stable/ussuri
which is EM now[1]. This remove the stable/ussuri
jobs which are failing due to bug#1968798 and also
adds relesenote to mark the end of support of Ussuri.
[1] https://releases.openstack.org/
Change-Id: Ieeb4da8859d44470c5cb67a62f2dcfd7786daf3a
Now we have stable/yoga branch ready for devstack and so does
for all service projects.
This commit adds the Tempest testing for stable/yoga by adding
new jobs running on stable/yoga version of openstack.
Change-Id: Id731c26a0f134428094e8d830fb615695dffe10b
ServerStableDeviceRescueTest also perform the attach_volume
on rescue server and in cleanup detach_volume. As described in
the bug#1960346 we need to wait for server readiness before
detach volume called.
Also making centos stream 9 job as voting.
Closes-Bug: #1960346
Change-Id: Ia213297b13f42d39213dea9a3b2cfee561cdcf28
centos 9 stream is testing runtime for Yoga so
let's add it in tempest gate. This job is failing
due to bug#1960346 and it will be helpful to know the
job status when we add the fixes with below series:
- https://review.opendev.org/q/topic:wait_until_sshable_pingable
Related-Bug: #1960346
Change-Id: Ib91f67fb9a592e91649e06bb2821f452b36b49f0
stable/ussuri job was made non voting due to
bug#1959467
- https://review.opendev.org/c/openstack/tempest/+/827258
As bug is fixed now, let's revert it to voting again.
Change-Id: I834300ee8d9ee8952284b55f53d97456a1ca6032
[1] Added random int to flavor name but that resulted into
TypeError: can only concatenate str (not "int") to str.
Concatenate using % operator to fix it.
Also make tempest-full-ussuri-py3 job non-voting temporary
to unblock gate, can be reverted once [2] is fixed.
[1] https://review.opendev.org/c/openstack/tempest/+/826946
[2] https://launchpad.net/bugs/1959600
Related-Bug: #1959467
Change-Id: I76e15b6b52bb44fec550ead510186c644c4a1c7f
In FIPS mode, using RSA keys for ssh is fine as long as SHA-1 is
not used for the signature algorithm. Unfortunately, the version
of cirros used in OpenStack CI does not have a version of dropbear
that supports SHA-2 signatures. So, any connections from a FIPS
enabled machine will fail as the cirros instance will only support
ssh-rsa (SHA-1 signatures).
To get around this, we add a new option to specify the key type
(validation.ssh_key_type). This will allow the addition of other
key types in future if needed.
Tempest now supports 'rsa' and 'ecdsa' key types.
We also add a fips job to the experimental queue to test the usage
of the new key type.
Change-Id: Ib59eb8432fa1a2813b3047955157d1b3d24a55f8
As set out in the governance repo for Xena CentOS 8 stream with py36 is
a supported platform and runtime for the release:
https://governance.openstack.org/tc/reference/runtimes/xena.html
As a result the Nova team want to run CentOS 8 stream based jobs within
the integrated-gate-compute template.
An additional tempest-full-py3-centos-8-stream job is added to Tempest's
check and gate queues to ensure coverage here.
Both jobs are given additional swap to workaround bug #1949606, a
behaviour change in QEMU >= 5.0.0 when using [libvirt]virt_type=qemu
that causes additional memory to be consumed by each running instance.
This workaround of additional swap will be removed in the future once
Nova is able to workaround this itself through a new libvirt domain
configurable.
Change-Id: I4b074dca58960b2622dde15ad7d9a91b3ba0c096
As Tempest is branchless and needs to support older
supported stable branches, let's test it with all the
python version supported in those supported stable branches.
For that, we need to explicitly dd the py jobs instead of using
the template. This way, we will explicitly know that we are testing
all supported python versions when the new release template bumps the
min python version in the template.
In Yoga testing, we are making py3.9 job to run as voting, so
adding py3.9 job as voting.
Change-Id: I422da8da06509e529f4091e21cdc3d392d3cf910
Now we have stable/xena branch ready for devstack
and so does for all service projects.
This commit adds the Tempest testing for stable/xena
by adding new jobs running on stable/xena version of
openstack.
Change-Id: If6f188aebe48f17d8a343c958cdeb5507421a40b
API microversions are inhertied in nature from features points
of view, means higher microversion will have all the features/
changes done in lower microversion.
In Tempest we write the microversion tests by capping
the min and max microversion so that they can request
the API with correct microversion.
But for non microversion tests we do not test if they are
run-able for all the configured microversion in Tempest config
file. To test it at some extend this commit adds a experimental
job to run the API tests (compute and volume) with 'latest'
microversion to know if that run successfully or need modification.
This job is experimental as now as I expect lot of test failure
with the 'latest' microversion and as we keep fixing those and all
the tests pass then we can move it to voting job in check/gate pipeline.
Change-Id: Ie840dd0997519d7e11a661853f9376f43c6e7905
Currently tempest-ipv6-only job run on all the files change
present in /tools dir which is not required. For example
when we update the tempest plugin sanity scripts present in
/tools dir then this job run which is unneccessary.
Optimizing the tools dir file as per related job required to run.
Change-Id: Ia1d8a8a6dd22bec02e9f93598ddfd3d110b6c009
Ghanshyam Mann
yatinkarel