Commit Graph

127 Commits

Author SHA1 Message Date
Ghanshyam
95358523e4 Define the Integrated-gate-object-storage gate template
tempest-full job run all services API and scenario tests (non slow)
and this job is part of Integrated-gate gate template which run on
multiple services (nova, neutron, cinder, glance, placement) check
and gate pipeline.

But there are many unrelated tests run on each project gate. For example,
neutron gate run cinder only tests and fail due to cinder bugs. There is
no point of breaking the neutron gate due to cinder only test failure and
slow down the neutron development phase.

This is what we dicussed in Train PTG in denver to improve the integrated
gate[1] and came up with idea of split the integated-gate template into
much logical way so that only dependent services tests run on their gate.

One way was to move the service only tests to service tempest plugin side
but Tempest is used by interop as well as on production cloud testing so
moving the test is not good idea.

This commit propose the idea of defining the 'Integrated-gate-object-storage'
gate template for swift gate which will run only Swift, Cinder and glance tests.

ML discussion: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html

Next step will be to replace the Integrated-gate-py3 template with new
Integrated-gate-object-storage template on swift gate.

[1] https://etherpad.openstack.org/p/qa-train-ptg

Change-Id: I70851d41174462b83cc8c525cecdb1b3d65e4ec7
2019-07-16 04:37:49 +00:00
Ghanshyam
ee7fd1f7e9 Define the Integrated-gate-compute gate template
tempest-full job run all services API and scenario tests (non slow)
and this job is part of Integrated-gate gate template which run on
multiple services (nova, neutron, cinder, glance, placement) check
and gate pipeline.

But there are many unrelated tests run on each project gate. For example,
neutron gate run cinder only tests and fail due to cinder bugs. There is
no point of breaking the neutron gate due to cinder only test failure and
slow down the neutron development phase.

This is what we dicussed in Train PTG in denver to improve the integrated
gate[1] and came up with idea of split the integated-gate template into
much logical way so that only dependent services tests run on their gate.

One way was to move the service only tests to service tempest plugin side
but Tempest is used by interop as well as on production cloud testing so
moving the test is not good idea.

This commit propose the idea of defining the 'Integrated-gate-compute'
gate template for nova gate which will run only nova, neutron, cinder
and glance tests.

ML discussion: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html

Next step will be to replace the Integrated-gate-py3 template with new
Integrated-gate-compute template on nova gate.

It will also help to disable the non-nova related serviecs and tests which
cannot be disabled from tempest-full job.

[1] https://etherpad.openstack.org/p/qa-train-ptg

Change-Id: I807d4cac6214395df292a6955686ea44677c3481
2019-07-16 04:37:34 +00:00
Ghanshyam
31609dc1da Define the Integrated-gate-placement gate template
tempest-full job run all services API and scenario tests (non slow)
and this job is part of Integrated-gate gate template which run on
multiple services (nova, neutron, cinder, glance, placement) check
and gate pipeline.

But there are many unrelated tests run on each project gate. For example,
neutron gate run cinder only tests and fail due to cinder bugs. There is
no point of breaking the neutron gate due to cinder only test failure and
slow down the neutron development phase.

This is what we dicussed in Train PTG in denver to improve the integrated
gate[1] and came up with idea of split the integated-gate template into
much logical way so that only dependent services tests run on their gate.

One way was to move the service only tests to service tempest plugin side
but Tempest is used by interop as well as on production cloud testing so
moving the test is not good idea.

This commit propose the idea of defining the 'Integrated-gate-placement'
gate template for placement gate which will run only nova and neutron tests.

ML discussion: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html

Next step will be to replace the Integrated-gate-py3 template with new
Integrated-gate-placement template on placement gate.

[1] https://etherpad.openstack.org/p/qa-train-ptg

Change-Id: I200197e2fb57879aaeb382266d6309f0d1137b0c
2019-07-16 04:37:16 +00:00
Zuul
aa30a65c9e Merge "Define the Integrated-gate-storage gate template" 2019-07-12 04:14:04 +00:00
Zuul
b3f8338289 Merge "Remove networking-ansible from tempest plugins list" 2019-07-09 16:12:54 +00:00
Ghanshyam Mann
a699445d28 Define the Integrated-gate-storage gate template
tempest-full job run all services API and scenario tests (non slow)
and this job is part of Integrated-gate gate template which run on
multiple services (nova, neutron, cinder, glance, placement) check
and gate pipeline.

But there are many unrelated tests run on each project gate. For example,
neutron gate run cinder only tests and fail due to cinder bugs. There is
no point of breaking the neutron gate due to cinder only test failure and
slow down the neutron development phase.

This is what we dicussed in Train PTG in denver to improve the integrated
gate[1] and came up with idea of split the integated-gate template into
much logical way so that only dependent services tests run on their gate.

One way was to move the service only tests to service tempest plugin side
but Tempest is used by interop as well as on production cloud testing so
moving the test is not good idea.

This commit propose the idea of defining the 'Integrated-gate-storage'
gate template for cinder gate which will run Cinder, Glance, Swift and
Nova tests.

ML discussion: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html

Next step will be to replace the Integrated-gate-py3 template with new
Integrated-gate-storage template on cinder and glance gate.

[1] https://etherpad.openstack.org/p/qa-train-ptg

Change-Id: Iae7d26325228aad0edb94f7e191c8fd0d60e1362
2019-07-09 13:55:04 +00:00
Ghanshyam
5072a24080 Define the Integrated-gate-networking gate template
tempest-full job run all services API and scenario tests (non slow)
and this job is part of Integrated-gate gate template which run on
multiple services (nova, neutron, cinder, glance, placement) check
and gate pipeline.

But there are many unrelated tests run on each project gate. For example,
neutron gate run cinder only tests and fail due to cinder bugs. There is
no point of breaking the neutron gate due to cinder only test failure and
slow down the neutron development phase.

This is what we dicussed in Train PTG in denver to improve the integrated
gate[1] and came up with idea of split the integated-gate template into
much logical way so that only dependent services tests run on their gate.

One way was to move the service only tests to service tempest plugin side
but Tempest is used by interop as well as on production cloud testing so
moving the test is not good idea.

This commit propose the idea of defining the 'Integrated-gate-networking'
gate template for neutron gate which will run only nova and neutron tests.

ML discussion: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html

Next step will be to replace the Integrated-gate-py3 template with new
Integrated-gate-networking template on neutron gate.

[1] https://etherpad.openstack.org/p/qa-train-ptg

Change-Id: Ifca8172f2f929105af4bad8ab72b05c2c45a5f58
2019-07-05 08:04:23 +00:00
Masayuki Igawa
012dc98fb2
Remove networking-ansible from tempest plugins list
This commit removes x/networking-ansible from the tempest plugins list.
The original patch[0] was abandoned because networking-ansible took a
different approach[1] for the testing. So, it's not a tempest plugin
anymore.

[0] https://review.opendev.org/#/c/634647/
[1] https://review.opendev.org/#/c/634918/

Change-Id: I554c438f27a7032f31e56eac2aac110e08bb3d7d
2019-07-04 16:20:15 +09:00
Zuul
3eb3c29e97 Merge "Minimum bandwidth allocation with placement" 2019-06-21 07:52:34 +00:00
Zuul
cd77312a7a Merge "Publish non-openstack namespace tempest plugins" 2019-06-12 17:02:29 +00:00
Masayuki Igawa
e36fe67e86
Publish non-openstack namespace tempest plugins
This commit changes to publish non-openstack namespace tempest plugins
to the tempest registry doc page. We missed some tempest plugins such as
airship's tempest-plugin because of the recent namespace change. And
also, this commit adds two projects[0] to .zuul.yaml because it had
been added already.

This commit republish them on the doc page, and add two projects to the
blacklist to pass the sanity tests.

Change-Id: Ieecb8989ff8959f1b2c0e5f2f6b8ea2c3ebbcaa9
2019-06-10 11:08:06 +09:00
Lajos Katona
c87a06b3c2 Minimum bandwidth allocation with placement
Add basic scenario test for placement based minimum bandwidth
allocation.

Change-Id: Ie274f199ce33199a6fc8c4d6846a853522a90d3a
Depends-On: https://review.openstack.org/574783
Depends-On: https://review.openstack.org/636360
Depends-On: https://review.opendev.org/660924
Depends-On: https://review.opendev.org/663270
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
2019-06-07 12:42:07 +00:00
Matt Riedemann
85dfbf6ef9 Remove nova-cells-v1 job from experimental queue
The nova-cells-v1 job was removed from nova in the Train
release with change I2031bf2cc914d134e5ed61156d2cfc621f006e0b
so we can remove it from tempest's experimental queue.

Change-Id: Ie759315bf7a4440eaae91b5ee4b672234692dd25
2019-05-24 15:13:49 -04:00
Zuul
6555d7c98d Merge "Add cyborg-tempest-plugin to required-projects and remove octavia" 2019-05-17 20:57:48 +00:00
Masayuki Igawa
7f5dd855c6
Add cyborg-tempest-plugin to required-projects and remove octavia
This commit adds opendev.org/openstack/cyborg-tempest-plugin to
required-projects to pass the plugin-sanity-check job. It's a new
tempest plugin. It doesn't work properly on master yet. So, I put it on
the BLACKLIST.

And this commit also removes octavia because octavia tempest tests were
already migrated to octavia-tempest-plugin, and there is no tempest test
in the octavia repo. The removal patch[0] in octavia was already merged.

[0] https://review.opendev.org/#/c/659516/

Change-Id: Ied13dbf774472e1c36dca4ccca157104a81fbf9d
2019-05-17 08:24:15 +09:00
Andreas Jaeger
39fe695177 Remove legacy-periodic-tempest-dsvm-neutron-full-ocata
This job does not run - and our logs shows no invocation at all,
see
http://zuul.openstack.org/builds?job_name=legacy-periodic-tempest-dsvm-neutron-full-ocata

Remove it.

Change-Id: Ib71d0a1d4e4650f07c8162f848def09724f1e987
2019-05-15 19:39:42 +02:00
OpenDev Sysadmins
682965e392 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:29:35 +00:00
ghanshyam
77a1933bd3 Run integration jobs for requirements.txt changes
currently we do not run the gate jobs if any change is for
test|requirements.txt only which can break the things.
Example- https://review.openstack.org/#/c/649804/

Because Tempest does not have functional testing which might
be enough for teting the requirement change, we start running
the all gate jobs for such change.

Change-Id: I54384f5c664673ef32937cd1d7c4a46c10c18f9b
2019-04-06 17:55:21 +00:00
Zuul
a415f6b1cc Merge "Add Tempest gate job for stable/stein" 2019-04-02 00:30:25 +00:00
Zuul
89733c6ac3 Merge "Remove the Ocata stable branch jobs from master gate" 2019-03-27 20:09:06 +00:00
Zuul
1a2f37013a Merge "Use nova zuulv3 job for nova v2 API" 2019-03-26 22:38:51 +00:00
ghanshyam
5b98592b9a Add Tempest gate job for stable/stein
Now we have stable/stein branch ready for devstack
and so does for all service projects.

This commit adds the Tempest testing for stable/stein
by adding new jobs running on stable/stein version of
openstack.
Depends-On: https://review.openstack.org/#/c/647867/

Change-Id: Ia22988821b5c786a0b25a980cb4ca329dac54bac
2019-03-26 19:58:50 +00:00
ghanshyam
787e694269 Remove the Ocata stable branch jobs from master gate
Tempest 20.0.0 is released now, which marks the end of support
for Ocata release - https://review.openstack.org/#/c/645867/

We do not need to run the Ocata stable job om master gate now.
Depends-On: https://review.openstack.org/#/c/645867

Change-Id: I883f3c98201631acdadeaab6678a796a85662e44
2019-03-22 21:12:51 +00:00
Zuul
e83940d21a Merge "Remove unused and duplicate nodeset defined in Tempest" 2019-03-21 04:41:16 +00:00
Zuul
baceb5ef9d Merge "Move tempest-tox-plugin-sanity-check to Bionic" 2019-03-16 05:27:36 +00:00
ghanshyam
ac6224ee63 Use nova zuulv3 job for nova v2 API
Nova has migrated the legacy-tempest-dsvm-nova-v20-api
job to zuulv3 native - "nova-tempest-v2-api"

This commit use the new job in tempest gate.

Change-Id: I81a07d8fbb529b0ac55ce40c50a6e5348e1f43df
2019-03-15 00:13:47 +00:00
Matt Riedemann
3858ced89c Set ENABLE_VOLUME_MULTIATTACH=true on tempest-slow subnode localrc
Before change Icb971831c8d4fe5f940d9e7993d53f1c3765e30f in devstack
in Rocky, the ENABLE_VOLUME_MULTIATTACH variable is used to control
whether or not the UCA is enabled on the node and defaults to False.
The tempest-slow job is multi-node and runs on stable branches since
change Ie5492603a12ca4ae56baa744bf650ae4d8036256 and as a result
is configuring the UCA on the subnode but not the controller, so
nova-compute on each node is running with different versions of QEMU
which can cause live migration failures if trying to live migrate
from a newer version of QEMU to an older version.

This fixes the issue by configuring the subnode with the correct
localrc value for ENABLE_VOLUME_MULTIATTACH.

Change-Id: Idea94dde6973805995c1678fcfb6b0e93025492a
Closes-Bug: #1819910
2019-03-13 10:18:08 -04:00
ghanshyam
a066fcdc26 Remove unused and duplicate nodeset defined in Tempest
Below nodesets defined in Tempest are removed now:

- openstack-bionic-node This is unused after I5a24c697103090f6712d6e9168abe6c86f7c380c
- openstack-opensuse150-node is duplicate of devstack-single-node-opensuse-150

Depends-On: https://review.openstack.org/#/c/641746/

Change-Id: I39c03bfa52191a0b1acf6a73fa31b3c071ee60c1
2019-03-07 23:11:52 +00:00
Zuul
c975f08e9b Merge "Make tempest-slow run on stable branch also" 2019-03-07 22:41:37 +00:00
ghanshyam
28edda63f2 Move tempest-tox-plugin-sanity-check to Bionic
This commit makes tempest-tox-plugin-sanity-check job
to run on Bionic. This job only run on master Tempest gate
so no need to worry about its xenial version.

Change-Id: I6d8c63a62416b35e9d203cbeabc9868aa6a95f44
2019-03-07 17:46:11 +00:00
Zuul
ae046a31a3 Merge "Improved tempest plugin sanity check script" 2019-03-06 13:45:06 +00:00
Zuul
d819399bdb Merge "Make tempest-multinode-full job for stable branch" 2019-03-02 03:51:22 +00:00
ghanshyam
00945fb93e Make tempest-slow run on stable branch also
tempest-slow has been added for stable branches also
- I5cb829beebcc59de682cba927f5b01793ec70d00

let's make this job definition to allow to run on stable branches also.

Change-Id: Ie5492603a12ca4ae56baa744bf650ae4d8036256
2019-02-25 12:32:51 +00:00
ghanshyam
42c9a8d00f Make tempest-multinode-full job for stable branch
tempest-multinode-full should be available to run on
all stable branch not just master. All the devstack
changes are backported already.

tempest-multinode-full will use Bionic node from
stein onwards and xenial for all other stable branches.

Change-Id: I94f66028df58a69aa7c26e8a03428bf48596d25c
2019-02-25 12:28:27 +00:00
Chandan Kumar
97b1ad78ac
Improved tempest plugin sanity check script
* To use a fresh venv, use `--clear` option when creating a virtualenv
* Removes the project as the script fails to uninstall the dependencies.
* Added missing tempest plugins
* Log the stdout of each plugin in a file
* Adds some plugins to BLACKLIST to avoid errors

Change-Id: I5068d96105a22cbfd33f5900f4bdc21b47a65c06
2019-02-25 13:09:08 +09:00
ZhongShengping
f83799a7fe add python 3.7 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.7.

See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135626.html

Change-Id: I13a6f591cd43331a2690d7f08f73c28692f6886a
Story: #2004073
Task: #27445
2019-02-19 17:06:20 +08:00
Zuul
30cfcf8478 Merge "Enable volume multiattach tests in tempest-full/slow jobs" 2019-02-09 04:52:08 +00:00
Matt Riedemann
7581e99a05 Enable volume multiattach tests in tempest-full/slow jobs
The volume multiattach tests originally required special
devstack configuration in Queens for the Ubuntu Cloud Archive,
but that is no longer necessary. This change enables the volume
multiattach tests in the tempest-full(-py3) and tempest-slow jobs
so we can drop the nova-multiattach job, which is mostly redundant
coverage of the tempest.api.compute.* tests, and reduce the total
number of jobs we run against nova/cinder/tempest changes.

Due to intermittent bug 1807723 when running the
test_volume_swap_with_multiattach test with two compute services,
that test is conditionally skipped if there is more than one compute.
This is probably no great loss in test coverage for now given
swapping multiattach volumes is likely rarely used (see bug 1775418).

Related ML thread:

http://lists.openstack.org/pipermail/openstack-dev/2018-October/135299.html

Depends-On: https://review.openstack.org/634977

Change-Id: I522a15ba3dbfee5d8ef417e43288a12319abf6ff
2019-02-05 11:50:20 -05:00
Ghanshyam Mann
e0d85b2055 Make interop check job as voting
'interop-tempest-consistency' job
has been added to check the test renaming
compatibility with interop usage.

This job was added as n-v to check the stability
and it is running fine since it was added so
making this as voting job in check pipeline.

Change-Id: I088f64d87b04b5db2319ccd3b3d02821a06f35e9
2019-02-05 03:02:40 +00:00
Zuul
60fd2c3019 Merge "Add tempest-multinode-full-py3 job definition" 2019-02-01 15:03:44 +00:00
Slawek Kaplonski
e5d93a4a54 Add tempest-slow-py3 job definition
This job is based on tempest-slow job but it runs
on Python 3.

This patch also removes tempest-slow job from gate pipeline.
Running only tempest-slow-py3 job there is sufficient.
To keep py2 coverage tempest-slow job will be still run
in check queue together with tempest-slow-py3.

Change-Id: I9fad35c6327fac30a25e940f7fec8515995c68c8
2019-01-31 12:21:50 +01:00
Slawek Kaplonski
0df2f91d19 Add tempest-multinode-full-py3 job definition
This job is based on tempest-multinode-full job but it
runs on Python 3.

Change-Id: Id387452730133d687ee13e3a38c6bc392a4bba3e
2019-01-30 23:30:02 +00:00
ghanshyam
b294a17e34 Use renamed template 'integrated-gate-py3'
'integrated-gate-py35' template is going to be
renamed to 'integrated-gate-py3' in https://review.openstack.org/#/c/626078/

Integrated jobs are running on Bionic now where python 3.6 is available.
Which means gate jobs in 'integrated-gate-py35' template are
running on python 3.6 not on 3.5 which makes this template name confusing.

depends on commit rename the 'integrated-gate-py35' to 'integrated-gate-py3'
so that it can convey that template will use available python 3 version
in used distro. For example: 3.5 in xenial and 3.6 in bionic.

This commit starts using the new template name so that old
template name can be removed.

Depends-On: https://review.openstack.org/#/c/626078/
Change-Id: Ic5264b3b79d0a4caaa7f466f8686d38f54fe1ffe
2018-12-19 04:06:36 +00:00
Zuul
9bb03a9782 Merge "Add irrelevant-files for grenade-py3 job" 2018-12-13 03:32:35 +00:00
Zuul
76fb8c3bbb Merge "Move nova-cells-v1 job to the experimental queue" 2018-12-12 23:29:09 +00:00
ghanshyam
109c4f15d0 Add irrelevant-files for grenade-py3 job
Adding irrelevant-files for new job grenade-py3 to avoid
running this integrated testing job on doc or unit test changes.

Change-Id: I2918b6563a65729e70323ca5986fc7a66e2bcc94
2018-12-12 12:30:16 +00:00
Zuul
a63b904cc7 Merge "Migrate tempest provided base jobs to bionic" 2018-12-12 08:13:56 +00:00
Matt Riedemann
023a683d3c Move nova-cells-v1 job to the experimental queue
Nova is moving the nova-cells-v1 job to its experimental
queue so we should do the same in Tempest.

Depends-On: https://review.openstack.org/623538

Change-Id: I277f3a4bea69501ea4c07118c552343d5ea38f6a
Related-Bug: #1807407
2018-12-07 10:42:02 -05:00
ghanshyam
f9a4c77b8f Migrate tempest provided base jobs to bionic
All the base jobs provided by Tempest are migrated to run in bionic
from stein onwards. As Tempest is branchless, we need to keep running or
available those base job running on xenial for stein before which is
stable/rocky, stable/queens and stable/pike.

The selection of the correct nodeset for this happens mostly
automatically when we avoid selecting nodesets explicitly because we
inherit them from the correct branches of devstack which have the
matching job variants defined.

There are only two places left where we need to explicitly override the
default nodesets:

1. tempest-multinode-full
Here we need to select a nodeset with two nodes. Since this job runs
only on master, we simply select two Bionic nodes instead of two Xenial
nodes now.

2. Stable branch testing jobs like tempest-full-rocky
These do not use branch selectors, but "override-checkout" instead, so
they inherit Bionic nodesets from devstack master branch. We need to
force them to run on Xenial for backwards compatibility.

Drop the tempest-full-py36 job because it will be identical to the
tempest-full-py3 job now.

Depends-On: https://review.openstack.org/610977
Depends-On: https://review.openstack.org/611594
Change-Id: I5a24c697103090f6712d6e9168abe6c86f7c380c
2018-11-21 08:58:52 +00:00
Felipe Monteiro
68273ac78a Fix required projects for sanity plugin check job
Currently, the sanity plugin check job is failing because
of errors like the following:

  ERROR! /home/zuul/src/git.openstack.org/openstack/cinder-tempest-plugin not found
  In Zuul v3 all repositories used need to be declared
  in the 'required-projects' parameter on the job.
  To fix this issue, add:

    openstack/cinder-tempest-plugin [0]

So this adds missing projects like tempest plugin repos to get
the job passing.

Included in this patch set:

* Add missing repos to required-projects
* Blacklist networking-plumgrid since it is failing because
  it hasn't been supported in years [1]
* Remove trio2o from required-projects because it is already
  blacklisted
* Add irrelevant-files for sanity check job

[0] http://logs.openstack.org/96/615396/4/check/tempest-tox-plugin-sanity-check/a18e9a9/ara-report/
[1] https://github.com/openstack/networking-plumgrid

Change-Id: Ifee972d87ae83759000db925449764a60a98f7ba
2018-11-07 19:24:11 -05:00