92 Commits

Author SHA1 Message Date
Zuul
d40125ef7f Merge "add python 3.7 unit test job" 2019-03-15 14:15:41 +00:00
Matt Riedemann
f9a6321c7b Customize irrelevant-files for nova-live-migration job
I noticed change Iea6288fe6d341ee92f87a35e0b0a59fe564ab96c
was not running the nova-live-migration job even though
it was making changes to nova/tests/live_migration/hooks/run_tests.sh.
The reason is the nova-live-migration job irrelevant-files were
excluding changes to nova/tests/*.

This copies the nova-grenade-live-migration irrelevant-files list
to the nova-live-migration job and defines it as a variable so it
can be re-used in the nova-grenade-live-migration job definition.

Change-Id: I753fda1a83b340f4699c049158e6744b099f55d8
2019-03-14 10:05:23 -04:00
ghanshyam
452fd2d91b Migrate legacy jobs to Ubuntu Bionic
We have migrated the zuulv3 job to Bionic during Dec/Jan month.
 - http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.html
 - https://etherpad.openstack.org/p/devstack-bionic
But that effort does not move all gate job to Bionic as there are
large amount of jobs are still legacy jobs. All the legacy jobs still
use Xenial as nodeset.

As per the decided runtime for Stein, we need to test everything on openstack
CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html

Below patch move the legacy base jobs to bionic which will move the derived jobs
automatically to bionic. These jobs are modified with branch variant so that they will use
Bionic node from stein onwards and xenial for all other stable branches
until stable/rocky.
- https://review.openstack.org/#/c/639096

This commit remove the overridden nodeset from legacy jobs
so that it will start using the nodeset defined in parent job.

More Details: 
- https://etherpad.openstack.org/p/legacy-job-bionic
- http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003614.html

Depends-On: https://review.openstack.org/#/c/639096

Change-Id: I3266b6fd26ec86f429b91b69d82c8e99f80cb6ca
2019-03-13 23:54:38 +00:00
ghanshyam
d74a5b23a5 Disable the tls-proxy in nova-next & fix nova-tox-functional-py35 parent
While moving the legacy job nova-next on bionic, tls-proxy
did not work and leads to nova-next job fail.

To proceed further on Bionic migration which is blocked by nova-next failure,
this commit temporary disable the tls-proxy service until bug#1819794 is fixed.

Also this updates the parent of nova-tox-functional-py35 from openstack-tox
to openstack-tox-functional-py35 in order to handle the upcoming change
of the infra CI default node type from ubuntu-xenial to ubuntu-bionic.

The python3.5 binary is not provided on ubuntu-bionic and the shared
"py35" job definitions in the openstack-zuul-jobs repository have been
patched to force them to run on ubuntu-xenial [1]. We should inherit
from one of these jobs for jobs that rely on python3.5.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003746.html

Related-Bug: #1819794

Change-Id: Ie46311fa9195b8f359bfc3f61514fc7f70d78084
2019-03-13 23:46:19 +00:00
Corey Bryant
21357c76ba add python 3.7 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.7.

See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135626.html

Change-Id: I36f641a7c281a7d8e168c507dae7109132bcefde
Story: #2004073
Task: #27433
2019-03-13 10:36:55 -04:00
Zuul
151deebde8 Merge "Do not run tempest.scenario.test_network* tests in nova-next" 2019-03-09 14:18:16 +00:00
Zuul
eaaade924d Merge "Make nova-grenade-live-migration voting and gating" 2019-03-07 19:10:56 +00:00
Zuul
fb4308e33a Merge "Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree" 2019-03-07 18:58:39 +00:00
Matt Riedemann
f4387e4b66 Do not run tempest.scenario.test_network* tests in nova-next
The test_network* scenario tests in tempest take a really long
time. For example, the tempest.scenario.test_network_v6.TestGettingAddress
tests in one job (that timed out) took ~35 minutes.

We already run test network scenario tests in the tempest-slow job
so let's just let that job handle these and exclude them from
nova-next.

Change-Id: I9c7fc0f0b0937f04c5b3ab9c5e8fff21c4232b86
2019-03-06 10:12:26 -05:00
Matt Riedemann
c613df13cb Make nova-grenade-live-migration voting and gating
Per the mailing list [1] we want to make this job
voting and gate since it provides valuable test coverage
for mixed-compute service version live migration.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003341.html

Change-Id: I7b213930b180878629eee984b6aaef3b6e5aec07
2019-02-28 21:40:34 -05:00
Matt Riedemann
52613c01ce Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree
This moves the legacy-grenade-dsvm-neutron-multinode-live-migration
job from openstack-zuul-jobs in-tree. Nova is the only project
that runs this job and this allows nova to directly manage it.

The job will still use the same irrelevant-files list and is still
non-voting for the time being and only runs in the check queue.

It has been renamed to "nova-grenade-live-migration".

This change is meant to be backported to rocky, queens and pike
to allow us to drop legacy-grenade-dsvm-neutron-multinode-live-migration
from openstack-zuul-jobs where the job is branchless but restricted
to pike+ branches.

A follow up master-only (stein) change will make the job voting
and gate per the plan in the mailing list [1].

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003341.html

Change-Id: Ie9b61775dbb92b10237688eaddaca606c1c73a23
2019-02-28 21:40:30 -05:00
Matt Riedemann
00d9a3a2e6 Test live migration with config drive
Tempest live migration tests do not create the server
with config drive so we do not, by default, have test
coverage of live migrating an instance with a config
drive. This change forces nova to create a config
drive for all instances in the nova-live-migration
job which will also run evacuate tests with a config
drive attached.

Change-Id: I8157b9e6e9da492a225cbb50a20f434f83a5fecb
Related-Bug: #1770640
2019-02-28 10:54:01 -05:00
Zuul
ab1bce0ed6 Merge "Fix irrelevant-files for legacy-grenade-dsvm-neutron-multinode-live-migration" 2019-02-23 00:44:01 +00:00
Zuul
3548cf5921 Merge "Drop the integrated-gate (py27) template" 2019-02-20 04:53:27 +00:00
Matt Riedemann
6d803746de Fix irrelevant-files for legacy-grenade-dsvm-neutron-multinode-live-migration
The change to irrelevant-files in I902e459093af9b82f9033d58cffcb2a628f5ec39
made it such that the job will *only* run on changes to the
nova/tests/live_migration/ path which was not the intent. This fixes
the regex to not run the job on changes to nova/tests/ unless those
changes are in the nova/tests/live_migration/ directory.

This would probably be easier to reason about long-term if the
nova/tests/live_migration/ directory was moved under nova/gate/
and then we could just use the standard irrelevant-files list. That
refactoring is left for another day though.

Change-Id: Iff2fca18a779e70deecf8e57279e2daf9bc5529a
Closes-Bug: #1816152
2019-02-15 12:23:14 -05:00
Zuul
8c2dd6166f Merge "Change nova-next tempest test regex" 2019-02-15 02:05:43 +00:00
Matt Riedemann
1adf60466e Drop the integrated-gate (py27) template
We're currently running essentially redundant tempest
and grenade jobs on all changes in the check and gate
queues because we use the integrated-gate and
integrated-gate-py3 job templates which add the
following jobs:

* integrated-gate:
  - tempest-full
  - neutron-grenade

* integrated-gate-py3:
  - tempest-full-py3
  - grenade-py3

This change removes the integrated-gate template which
drops the tempest-full and neutron-grenade jobs from
nova changes. We will still have py27 coverage in unit
and functional test jobs which should be sufficient, and
both tempest and devstack still use the integrated-gate
template so they should detect any py27-specific breaking
changes.

Related ML thread:

http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002388.html

Change-Id: I93e938277454a1fc203b3d930ec1bc1eceac0a1e
2019-02-14 14:23:08 +00:00
Zuul
7e3ad4eec6 Merge "Change nova-next job to run with python3" 2019-02-13 18:52:58 +00:00
Matt Riedemann
e209fc5ed0 Change nova-next tempest test regex
Before this change, nova-next ran the same tests
as the tempest-full job which was all non-slow API
and scenario tests.

To avoid running redundant tests on the same nova
change, this patch changes the tempest run regex
to run only compute API tests along with all
scenario tests and includes slow tests (by not
excluding them like tempest-full does).

By removing the non-compute API tests we should
still be able to keep this job running in a reasonable
time even though the slow tests are added.

As discussed in https://review.openstack.org/606978/,
test_volume_swap_with_multiattach will be run again
with this change since it otherwise won't run in
tempest-slow since that is a multi-node job and the
test was blocked from running in multi-node jobs until
bug 1807723 is fixed. In other words, this gives us
volume multi-attach testing of swap volumes again since
nova-next is a single-node job.

Change-Id: Icbc06849dfcc9f41c7aaf7de109e036a993de7c7
2019-02-13 08:22:17 -05:00
Zuul
0bed18ffbb Merge "Fix legacy-grenade-dsvm-neutron-multinode-live-migration" 2019-02-12 23:08:43 +00:00
Zuul
782d742e60 Merge "Drop nova-multiattach job" 2019-02-09 22:46:34 +00:00
Matt Riedemann
f2db43d57a Fix legacy-grenade-dsvm-neutron-multinode-live-migration
Since change I81301eeecc7669a169deeb1e2c5d298a595aab94 in
devstack, nova-cpu.conf is a copy of nova.conf minus
database access. Grenade jobs also run nova-compute with
nova-cpu.conf anyway so we can just drop the conditional
which was otherwise messing up the config file that the
ceph script would write rbd configuration which is why
live block migration tests with ceph were failing.

While in here, the zuul job configuration is updated so
that changes to nova/tests/live_migration/ can be
self-testing.

Change-Id: I902e459093af9b82f9033d58cffcb2a628f5ec39
Closes-Bug: #1813216
2019-02-06 15:44:35 -05:00
Matt Riedemann
e9dd8c0f48 Drop nova-multiattach job
The dependent tempest change enables the volume multiattach
tests in the tempest-full and tempest-slow jobs, on which
nova already gates, which allows us to drop the special
nova-multiattach job which is mostly redundant test coverage
of the other tempest.api.compute.* tests, and allows us to
run one fewer job on nova/cinder/tempest changes in Stein.

The docs are updated to reflect the source of the testing
now.

Also depends on cinder dropping its usage of the nova-multiattach
job before we can drop the job definition from nova.

Depends-On: https://review.openstack.org/606978
Depends-On: https://review.openstack.org/606985
Change-Id: I744afa1df9a6ed8e0eba4310b20c33755ce1ba88
2019-02-05 09:13:02 -05:00
Matt Riedemann
ab72dd4096 Change nova-next job to run with python3
The way of the future is python 3 so it makes sense
that our nova-next job, which is meant to test future
things and advanced configuration, would run under python3.

Change-Id: Ie33efb41de19837b488d175900d9a4666f073bce
2019-02-04 11:36:25 -05:00
Ghanshyam Mann
36efe194fe Switch tempest-slow to be run on python 3
This patch switches tempest-slow to new
tempest-slow-py3 job.

For coverage of tempest-slow tests on py2,
Tempest check pipeline will continue running
this job on check pipeline. As this tempest-slow
job is not from integrated gate template, it is
ok to run only py3 job on project side.

Depends-On: https://review.openstack.org/633983
Change-Id: I614dea09547c8f86f667b1daeb18f69fea7937fb
2019-01-31 11:15:34 +00:00
Iury Gregory Melo Ferreira
ef06819b19 Rename Ironic jobs
-remove dsvm from the name of the jobs to match zuulv3 pattern
-ironic-tempest-dsvm-ipa-wholedisk-agent_ipmitool-tinyipa-multinode
is now ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode

Depends-On: https://review.openstack.org/#/c/629173/
Change-Id: I2a394bc5a3ee94befcce71120e39464370b88455
2019-01-09 15:26:37 +01:00
Matt Riedemann
f54b43c8b9 Run nova-lvm job on nova/privsep/* changes
As seen from bug 1808247, there is something about
the [libvirt]/image_type=lvm configuration that
tickles privsep in ways that our normal integrated
gate jobs don't, so this adds nova/privsep/* to the
whitelist of files to trigger the nova-lvm job.

Change-Id: I18f749e6089f776fcca386daea0e479b5382a44b
Related-Bug: #1808247
2019-01-03 17:21:22 -05:00
Zuul
1dc94d51c5 Merge "Use renamed template 'integrated-gate-py3'" 2018-12-22 01:43:50 +00:00
Zuul
1d8fc8e5ac Merge "Add irrelevant-files for grenade-py3 jobs" 2018-12-21 11:52:05 +00:00
ghanshyam
a6a5dce78c Use renamed template 'integrated-gate-py3'
'integrated-gate-py35' template is going to be
renamed to 'integrated-gate-py3' in https://review.openstack.org/#/c/626078/

Integrated jobs are running on Bionic now where python 3.6 is available.
Which means gate jobs in 'integrated-gate-py35' template are
running on python 3.6 not on 3.5 which makes this template name confusing.

depends on commit rename the 'integrated-gate-py35' to 'integrated-gate-py3'
so that it can convey that template will use available python 3 version
in used distro. For example: 3.5 in xenial and 3.6 in bionic and so on.

This commit starts using the new template name so that old
template name can be removed.

Depends-On: https://review.openstack.org/#/c/626078/
Change-Id: I7f2bc476d0be7994feef781cb254c8b0c53b8b5b
2018-12-19 04:34:36 +00:00
Zuul
22b15df551 Merge "Add py36/py37 functional jobs to the experimental queue" 2018-12-17 17:39:07 +00:00
Zuul
2cf77492ff Merge "Add python 3.7 unit and functional tox jobs" 2018-12-17 16:36:04 +00:00
Matt Riedemann
a4fccaeb29 Add py36/py37 functional jobs to the experimental queue
Change-Id: I2afc65ea0f19a416a878139e238d54287b4b51c4
2018-12-13 10:41:46 -05:00
Chris Dent
2ea552e019 Add python 3.7 unit and functional tox jobs
Without these, if you try to run tox -epy37,functional-py37 you'll
get a successful tox run, but no actual tests are run, which is
rather misleading. Given the generaly availability of python 3.7
this is a bad thing.

Running the tests under python 3.7 identified a few minor tests
failures, also fixed here. Each is a result of a change in behavior in
python 3.7:

* printable unicode changes with a new Unicode 11-based unicodedata
  package
* intentionally raising StopIteration in a generator is now considered a
  RuntimeError, 'return' should be used instead
* an exception message is different beween python 3 and python 2, and the
  guard for it was mapping python 3.5 and 3.6 but not 3.7.

zuul configuration is adjusted to add an experimental job for python 3.7
unit. A functional test job is not added, because we don't have 3.6 yet,
and we probably want to get through that first.

Closes-Bug: #1807976
Closes-Bug: #1807970
Change-Id: I37779a12d3b36eb3dc7e2733d07fe0ed23ab3da6
2018-12-13 10:41:46 -05:00
Ghanshyan Mann
5565dd3636 Add irrelevant-files for grenade-py3 jobs
grenade-py3 job is part of integrated-gate-py35
template now[1]

Adding corresponding irrelevant-files for grenade-py3
also same as for other integrated jobs to avoid
running it on test etc only changes
Example- https://review.openstack.org/#/c/576712/

[1] http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/project-templates.yaml#n1397

Change-Id: I80580449198698edb9a90f2e3df42f225d7deb51
2018-12-13 22:39:35 +09:00
Chris Dent
787bb33606 Use external placement in functional tests
Adjust the fixtures used by the functional tests so they
use placement database and web fixtures defined by placement
code. To avoid making redundant changes, the solely placement-
related unit and functional tests are removed, but the placement
code itself is not (yet).

openstack-placement is required by the functional tests. It is not
added to test-requirements as we do not want unit tests to depend
on placement in any way, and we enforce this by not having placement
in the test env.

The concept of tox-siblings is used to ensure that the
placement requirement will be satisfied correctly if there is a
depends-on. To make this happen, the functional jobs defined in
.zuul.yaml are updated to require openstack/placement.

tox.ini has to be updated to use a envdir that is the same
name as job. Otherwise the tox siblings role in ansible cannot work.

The handling of the placement fixtures is moved out of nova/test.py
into the functional tests that actually use it because we do not
want unit tests (which get the base test class out of test.py) to
have anything to do with placement. This requires adjusting some
test files to use absolute import.

Similarly, a test of the comparison function for the api samples tests
is moved into functional, because it depends on placement functionality,

TestUpgradeCheckResourceProviders in unit.cmd.test_status is moved into
a new test file: nova/tests/functional/test_nova_status.py. This is done
because it requires the PlacementFixture, which is only available to
functional tests. A MonkeyPatch is required in the test to make sure that
the right context managers are used at the right time in the command
itself (otherwise some tables do no exist). In the test itself, to avoid
speaking directly to the placement database, which would require
manipulating the RequestContext objects, resource providers are now
created over the API.

Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Change-Id: Idaed39629095f86d24a54334c699a26c218c6593
2018-12-12 18:46:49 +00:00
Zuul
2c7aa78980 Merge "Move nova-cells-v1 to experimental queue" 2018-12-11 01:23:35 +00:00
Matt Riedemann
e02fbb53d5 Move nova-cells-v1 to experimental queue
Cells v1 has been deprecated since Pike. CERN
has been running with cells v2 since Queens.
The cells v1 job used to be the only thing that
ran with nova-network, but we switched the job
to use neutron in Rocky:

  I9de6b710baffdffcd1d7ab19897d5776ef27ae7e

The cells v1 job also suffers from intermittent
test failures, like with snapshot tests.

Given the deprecated nature of cells v1 we should
just move it to the experimental queue so that it
can be run on-demand if desired but does not gate
on all nova changes, thus further moving along its
eventual removal.

This change also updates the cells v1 status doc
and adds some documentation about the different
job queues that nova uses for integration testing.

Change-Id: I74985f1946fffd0ae4d38604696d0d4656b6bf4e
Closes-Bug: #1807407
2018-12-07 10:59:37 -05:00
Takashi NATSUME
311110e81b Remove Placement API reference
The placement project has published the API reference
in its own repository and the related jobs for the nova project
has been removed since Ia4680f24d78af1260f2f0106a458b78a079c1287.

So remove the files and definitions related to
the placement API reference in the nova repository.

Change-Id: Ia43b958a28e7e763e7ecb29e06f8d21d2b9a850f
2018-11-28 03:38:41 +00:00
Matt Riedemann
8327011f91 Add post-test hook for testing evacuate
This adds a post-test bash script to test evacuate
in a multinode job.

This performs two tests:

1. A negative test where we inject a fault by stopping
   libvirt prior to the evacuation and wait for the
   server to go to ERROR status.

2. A positive where we restart libvirt, wait for the
   compute service to be enabled and then evacuate
   the server and wait for it to be ACTIVE.

For now we hack this into the nova-live-migration
job, but it should probably live in a different job
long-term.

Change-Id: I9b7c9ad6b0ab167ba4583681efbbce4b18941178
2018-10-25 16:15:56 -04:00
Zuul
8b627fe68c Merge "Remove the CachingScheduler" 2018-10-25 13:22:53 +00:00
Zuul
7701eefb81 Merge "Rename tempest-nova job to follow conventions" 2018-10-23 11:08:25 +00:00
Zuul
55405ef31e Merge "Convert legacy-tempest-dsvm-neutron-src-oslo.versionedobjects job" 2018-10-23 08:29:39 +00:00
Zuul
f97c1ad1e3 Merge "Remove duplicate legacy-tempest-dsvm-multinode-full job" 2018-10-23 02:24:52 +00:00
Zuul
b9dc5cc1a1 Merge "Zuul: Update barbican experimental job" 2018-10-23 01:37:28 +00:00
Andreas Jaeger
8718c9718f Rename tempest-nova job to follow conventions
Follow the naming conventions from
https://docs.openstack.org/infra/manual/drivers.html#consistent-naming-for-jobs-with-zuul-v3
- a job defined in nova repo needs to start with nova, not tempest.

Thus rename tempest-nova to nova-tempest.

Change-Id: I457c9993d1af8b01503c9d4c61caff6f26951f52
2018-10-22 06:06:55 +02:00
Andreas Jaeger
2cb74d32dd Convert legacy-tempest-dsvm-neutron-src-oslo.versionedobjects job
Migrate experimental job legacy-tempest-dsvm-neutron-src-oslo.versionedobjects
to Zuul v3 native. We can just use the tempest-full job here, adding
required-projects will automatically check out the library from source
instead of installing it from pip.

Since this is the only user of the legacy job, define the job in the
nova repository.

Change-Id: I51155755a565a9a38084906a57407e87083fb362
2018-10-22 06:06:19 +02:00
Zuul
d7e76f8cad Merge "Migrate nova v2.0 legacy job to zuulv3" 2018-10-19 08:19:45 +00:00
Matt Riedemann
25dadb94db Remove the CachingScheduler
The CachingScheduler has been deprecated since Pike [1].
It does not use the placement service and as more of nova
relies on placement for managing resource allocations,
maintaining compabitility for the CachingScheduler is
exorbitant.

The release note in this change goes into much more detail
about why the FilterScheduler + Placement should be a
sufficient replacement for the original justification
for the CachingScheduler along with details on how to migrate
from the CachingScheduler to the FilterScheduler.

Since the [scheduler]/driver configuration option does allow
loading out-of-tree drivers and the scheduler driver interface
does have the USES_ALLOCATION_CANDIDATES variable, it is
possible that there are drivers being used which are also not
using the placement service. The release note also explains this
but warns against it. However, as a result some existing
functional tests, which were using the CachingScheduler, are
updated to still test scheduling without allocations being
created in the placement service.

Over time we will likely remove the USES_ALLOCATION_CANDIDATES
variable in the scheduler driver interface along with the
compatibility code associated with it, but that is left for
a later change.

[1] Ia7ff98ff28b7265058845e46b277317a2bfc96d2

Change-Id: I1832da2190be5ef2b04953938860a56a43e8cddf
2018-10-18 17:55:36 -04:00
ghanshyam
0e14434dbc Remove duplicate legacy-tempest-dsvm-multinode-full job
Experimental job 'legacy-tempest-dsvm-multinode-full'
is duplicate of 'tempest-multinode-full' job which is running
in check pipeline as n-v job.

This commit remove the legacy duplicate job from experimental pipeline.

Change-Id: Id4f51ae3125d6c8e3fa66244673c55693234da5c
2018-10-16 10:46:08 +00:00