126 Commits

Author SHA1 Message Date
Balazs Gibizer
2cf9a5f9fa Add cold migrate and resize to nova-grenade-multinode
Changes in [1] could potentially break a mixed-compute-version
environment as we don't have grenade coverage for cold migrate and
resize. This adds that coverage to the nova-grenade-multinode
job.

[1]https://review.opendev.org/#/c/655721/10

Change-Id: I81372d610ddf8abb473621deb6e7cb68eb000fee
2019-08-30 15:35:46 -04:00
Balazs Gibizer
7f8dd04108 Rename the nova-grenade-live-migration job to nova-grenade-multinode
We are planning to add non-live-migration multinode tests to this job
so first we rename it to be more generic.

Change-Id: I7571ff508fcccba40cf2307a8788c1850d2d9fd8
2019-08-30 15:35:46 -04:00
Balazs Gibizer
3c1d9dab85 Move live_migration test hooks under gate/
This patch resolves a TODO in the .zuul.yaml about using common
irrelevant files in our dsvm jobs. To be able to do that we need to move
the test hooks from nova/tests/live_migraton under gate/.

Change-Id: I4e5352fd1a99ff2b4134a734eac6626be772caf1
2019-08-29 14:45:48 -04:00
Eric Fried
d91c1626ff Process [compute] in $NOVA_CPU_CONF in nova-next
Due to bug 1802143, devstack was ignoring variables set in the
$NOVA_CPU_CONF section of local_conf. Now that that bug has been fixed
(see the Depends-On) we can put those back in their rightful place.

This patch moves the [compute]resource_provider_association_refresh
setting, introduced via [1], accordingly.

[1] https://review.opendev.org/616033

Depends-On: https://review.opendev.org/672805
Change-Id: Ifa50748b6d6e91ac954d0a7a6060505fd45716d4
Related-Bug: #1802143
2019-08-23 14:11:48 +00:00
Zuul
3a5f8498f4 Merge "Make nova-multi-cell job voting and gating" 2019-08-09 22:54:22 +00:00
Matt Riedemann
a8f502c04f Make nova-multi-cell job voting and gating
Compared to tempest-full-py3 and tempest-slow-py3
the nova-multi-cell job has been relatively stable
for the past couple of months so let's make it voting
so we don't regress multi-cell functionality.

Change-Id: I11081177abfe2fb69fbd8fc17464c9f08962ab4c
2019-08-03 08:45:03 -04:00
Matt Riedemann
8823280724 Add nova-osprofiler-redis job to experimental queue
This adds a nova-specific variant of the
tempest-smoke-py3-osprofiler-redis job to nova's
experimental queue. This could be useful in comparing
before and after results on nova changes that are
related to performance, although that's admittedly
hard in a CI environment where we could be running on
different node providers (we could pin that though?)
and differing levels of noisy neighbors during each
job run.

Change-Id: I2dcb5c1e8289eaf7977b38a3d26e74183823c42b
2019-08-02 13:31:24 -04:00
Zuul
0fc5b35758 Merge "(Re-)enable vnc console tests in nova-multi-cell job" 2019-08-02 13:04:58 +00:00
Zuul
78ce2e89e5 Merge "Replace "integrated-gate-py3" template with new "integrated-gate-compute"" 2019-07-30 19:42:41 +00:00
Zuul
7313b10e48 Merge "Run 'tempest-ipv6-only' job in gate" 2019-07-30 18:49:21 +00:00
Matt Riedemann
4188d08c20 Disable cinder-backup service in nova-next job
We don't run any tests that require the cinder backup
service in the nova-next job nor do we care to so save
some time in stacking and space on the node by not
installing that service.

Change-Id: I6991a231366908920847fa1b8375ece8d8b35268
2019-07-24 14:05:44 +00:00
Matt Riedemann
cee072b962 Convert nova-next to a zuul v3 job
For the most part this should be a pretty straight-forward
port of the run.yaml. The most complicated thing is executing
the post_test_hook.sh script. For that, a new post-run playbook
and role are added.

The relative path to devstack scripts in post_test_hook.sh itself
had to drop the 'new' directory since we are no longer executing
the script through devstack-gate anymore the 'new' path does not
exist.

Change-Id: Ie3dc90862c895a8bd9bff4511a16254945f45478
2019-07-23 11:32:35 -04:00
Ghanshyam Mann
f582ef397c Run 'tempest-ipv6-only' job in gate
As part of Train community goal 'Support IPv6-Only Deployments and Testing'[1],
Tempest has defined the new job 'tempest-ipv6-only'(adding
in Depends-On patch) which will deploy services on IPv6 and run smoke
tests and IPv6 related tests present in Tempest.

This job will be part of Nova, Neutron, Cinder, Keystone, Glance, Swift
gate.

Verification structure will be:
- 'devstack-IPv6' deploy the service on IPv6
- 'devstack-tempest-ipv6' run will verify the IPv6-only setting and listen address
- 'tempest-ipv6-only' will run the smoke + IPv6 related test case.

This commit adds the new job 'tempest-ipv6-only' run on gate.

Story: #2005477
Task: #35909

Depends-On: https://review.opendev.org/#/c/671231/

[1] https://governance.openstack.org/tc/goals/train/ipv6-support-and-testing.html

Change-Id: I21cba258e4ad55681468c66060f4e0c1f4becbd4
2019-07-20 17:41:04 +00:00
Ghanshyam Mann
153cb54e67 Replace "integrated-gate-py3" template with new "integrated-gate-compute"
tempest-full-py3 job run all services API and scenario tests (non slow)
and this job is part of Integrated-gate-py3 gate template which run on
multiple services (nova, neutron, cinder, glance, placement, swift, keystone)
check and gate pipeline.
Due to all services tests run in single tempest-full-py3 job, it has
been not so stable. Many times unrelated service test break other
project gate.

But not all those services are dependent of Nova. Dependent in term of
both way, not all services used by Nova or use Nova. Nova gate
does not need to run all those independent services tests. It can run only
Nova, Neutron, Cinder and Glance related tests.

Idea here is to make integrated-gate testing more stable without losing
the test coverage.

Tempest has new "integrated-gate-compute" template which will run only
Nova, Neutron, Cinder and Glance related tests[1].

This commit replaces the "integrated-gate-py3" template with new
"integrated-gate-compute".

ML discussion: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html

[1] https://opendev.org/openstack/tempest/src/branch/master/.zuul.yaml#L594

Change-Id: I11dd1a94054ed29f70330906c1d95967b3de718c
2019-07-19 03:09:44 +00:00
Matt Riedemann
33e9740d00 Add neutron-tempest-iptables_hybrid job to experimental queue
Due to regression bug 1833902 we should have the option to
run the neutron job that tests the OVS hybrid plug configuration
on specific patches.

We could eventually consider running this in the check queue
like neutron-tempest-linuxbridge with a strict set of irrelevant
files but that would require some thought, i.e. we'd want to run
the job on changes to nova/compute/manager.py since the network
vif model bind events code is used in the ComputeManager and is
specific to OVS hybrid plug ports. Rather than get bogged down in
thinking up what that set is, this change takes the easy path and
just throws the job into the on-demand experimental queue.

Change-Id: Id5172d97b349abe59b47d1284e97943091652419
Related-Bug: #1833902
2019-07-02 10:38:31 +03:00
Matt Riedemann
a24d883e8c (Re-)enable vnc console tests in nova-multi-cell job
This reverts commit 47b73030cba7d67fe308c5b89c2a64e93d74bcd6.

The dependent devstack change fixes the issue so we can enable
vnc console tempest testing in the nova-multi-cell job.

Depends-On: https://review.opendev.org/661391/

Change-Id: Iad71b6e15bb91e51080cd9f940410dc9af791963
Related-Bug: #1830417
2019-06-25 11:42:06 -04:00
Zuul
f2b58bb20b Merge "Make all functional tests reusable by other projects" 2019-06-05 07:20:42 +00:00
Zuul
2ea6e6f8db Merge "Follow up for counting quota usage from placement" 2019-06-04 05:54:29 +00:00
melanie witt
5e74915064 Follow up for counting quota usage from placement
This addresses comments from the series:

  * Remove usage-specific info from docstring
  * Add note to nova-next job description "changelog"
  * Add info about data migration to config option help
  * Consolidate code under count_usage_from_placement conditional
  * Consolidate variables for checking data migration doneness
  * Remove hard-coded user_id and project_id from func test
  * Re-word code comment about checking data migration doneness

Related to blueprint count-quota-usage-from-placement

Change-Id: Ida2de9256fcc9e092fb9977b8ac067fc1472c316
2019-05-31 21:23:58 +00:00
Chris Dent
f2a2a6b1f7 Make all functional tests reusable by other projects
In I73f9dacd912331d209a6fa04eaf5591d94d720f2 the functional-py36
was made reusable by other projects by adding explicitly adding
nova to projects and explicitly stating the zuul work dir.

This change adds the other functional jobs.

Change-Id: Iaf2c1f87ae4a4ec2ecf6cc19220c6d15f8580321
Story: 2005562
2019-05-29 10:26:31 +00:00
Matt Riedemann
47b73030cb Skip novnc tests in multi-cell job until bug 1830417 is fixed
NoVNCConsoleTestJSON.test_novnc is intermittently failing
in the multi-cell job when trying to get a vnc console for
an instance in cell2. It looks like requests aren't getting
routed to the novnc-cell2 service properly, so let's skip
novnc tests until the bug is fixed since we want to stabilize
this job to make it voting.

Change-Id: I74a4b392ccec66a7a80a117d1cdf83a418c7dcdc
Related-Bug: #1830417
2019-05-24 14:23:05 -04:00
Zuul
c5b83c3fbc Merge "Enable n-novnc in nova-multi-cell job" 2019-05-20 20:34:36 +00:00
ZhongShengping
0973fbe21f Update Python 3 test runtimes for Train
This goal is to implement the process set out in the 2018-10-24 Python
Update Process TC resolution[1], for the Train cycle to ensure unit
testing is in place for all of the Tested Runtimes for Train[2].
In practice, this generally means adding unit tests for Python 3.7 and dropping
unit tests for Python 3.5. Using the Zuul template for Train will ensure that
all projects that support Python3 will be tested against the agreed runtime
versions, and make it easier to update them in future.

[1]https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html
[2]https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I095770fabafda1a7735c2711d4df061a6dc2bb78
Depends-On: https://review.opendev.org/#/c/641878/
2019-05-09 17:35:43 +08:00
Chris Dent
2860bef70a Make nova-tox-functional-py36 reusable
We'd like placement to be able to run this job itself. Instead of
duplicating it, we can simply use it directly. However, according
to discussions with the infra gang, to ensure that works properly
we need to make openstack/nova a required project and explicitly
set zuul_work_dir. These are the default values already when
used from nova itself.

Change-Id: I73f9dacd912331d209a6fa04eaf5591d94d720f2
Story: 2005562
2019-05-04 10:50:56 -06:00
Zuul
9e9c498ea2 Merge "Add nova-multi-cell job" 2019-04-30 21:18:42 +00:00
Matt Riedemann
6632468782 Enable n-novnc in nova-multi-cell job
With the dependent devstack fix we should be able to run
the n-novnc service in a multi-cell setup.

Depends-On: https://review.opendev.org/649473/

Change-Id: I6d2a564840a5ac4f4f7e803619d9962653bb5d6c
Related-Bug: #1822873
2019-04-29 17:10:56 -04:00
Matt Riedemann
d2e58cb490 Add nova-multi-cell job
This adds a non-voting nova-multi-cell job which
is a 2-node job with 2 non-cell0 cells (cell1 and cell2).
The compute on the controller will be in cell1 and the
subnode compute will be in cell2.

Until cross-cell resize is supported, cold migration
has to be disabled since we can't cold migrate to
the same host (and there is only one compute host per
cell). As a result, resize to the same host is enabled
(it is normally disabled in the tempest multinode job).
The parent job enables live migration testing but since
we only have one host per cell and don't support live
migration across cells, the live migration tests are
also disabled.

The job is also setup to run with python3 since that
is the way of the future.

Non-essential services, like cinder-backup, are disabled
since we (1) won't need it for the tests we run and
(2) would like to save as much room on the hosts as possible
since we're running multiple cell conductors on the
controller host.

Since this is new and its already uncovered at least one
bug (bug 1826472), the job is non-voting in the check queue
for now until we're happy with its stability.

Related to blueprint cross-cell-resize

Change-Id: I8ad3f8bcbecf0f6b1d0f57e9f61ff21bd7fbe365
2019-04-29 17:10:51 -04:00
Matt Riedemann
929cd32505 Don't run tempest/devstack jobs on nova/test.py only changes
We already skip running the devstack/tempest integration jobs
for changes which only touch nova/tests/* but not the nova/test.py
module itself, so this adds that filter.

Change-Id: I8984e5b68ccb7d327b7ce16c50f19e46e769699a
2019-04-25 10:29:45 -04:00
OpenDev Sysadmins
fc3890667e OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:45:52 +00:00
Zuul
67bfb86fdf Merge "Dropping the py35 testing" 2019-04-16 08:22:36 +00:00
Ghanshyam Mann
6321f01f0c Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: Ia1289f038b92c36dbdd79bcdf12d7cb95a3e5aa9
Signed-off-by: zhangyangyang <zhangyangyang@unionpay.com>
2019-04-15 03:07:18 +00:00
Stephen Finucane
7c7798ad52 Remove cells v1 jobs
We're going to start ripping out cells v1. First step is removing the
tests for this soon-to-be-removed feature.

Part of blueprint remove-cells-v1

Change-Id: I2031bf2cc914d134e5ed61156d2cfc621f006e0b
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-04-09 17:15:37 +01:00
Chris Dent
dc4ed2d048 Add placement as required project to functional py36 and 37
The functional tests are configured to install placement from git
so that they test with placement-from-master. In the py27 and py35
functional tests, placement is a part of required-projects, which
means that depends-on to a particular patch of placement can work
via tox-siblings[1].

This patch adds the same functionality for py36 and py37 functional
tests, which currently run only in the experimental queue.

[1] https://docs.openstack.org/infra/manual/zuulv3.html#installation-of-sibling-requirements

Change-Id: Ia40aba6d3036fea2bc62fdf76e8e221b552f1cdc
2019-04-01 19:28:18 +00:00
Matt Riedemann
95e782dfd8 Stop running tempest-multinode-full
The job was added to nova with change:

  05ab017907376cc262887104556b01fee18db0c2

As I explained in the mailing list [1], we should
have sufficient test coverage in other voting jobs
that run against nova changes with the use of the
tempest-full-py3, tempest-slow-py3, nova-next, and
nova-(grenade)live-migration jobs. The tempest-full
and nova-next jobs are single-node jobs which run
compute API tests and some scenario tests but no
slow tests and no live migration. The tempest-slow
job is multinode and only runs slow tests. The
live migration jobs run live migration tests over
two nodes (and nova-live-migration also runs
evacuate tests). So tempest-multinode-full is
probably mostly redundant, a waste of time and
resources, especially since it's non-voting.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004139.html

Change-Id: Id2f748d4b119002292bda45689472a68254079ee
2019-03-21 23:20:30 -04:00
Zuul
d40125ef7f Merge "add python 3.7 unit test job" 2019-03-15 14:15:41 +00:00
Matt Riedemann
f9a6321c7b Customize irrelevant-files for nova-live-migration job
I noticed change Iea6288fe6d341ee92f87a35e0b0a59fe564ab96c
was not running the nova-live-migration job even though
it was making changes to nova/tests/live_migration/hooks/run_tests.sh.
The reason is the nova-live-migration job irrelevant-files were
excluding changes to nova/tests/*.

This copies the nova-grenade-live-migration irrelevant-files list
to the nova-live-migration job and defines it as a variable so it
can be re-used in the nova-grenade-live-migration job definition.

Change-Id: I753fda1a83b340f4699c049158e6744b099f55d8
2019-03-14 10:05:23 -04:00
ghanshyam
452fd2d91b Migrate legacy jobs to Ubuntu Bionic
We have migrated the zuulv3 job to Bionic during Dec/Jan month.
 - http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.html
 - https://etherpad.openstack.org/p/devstack-bionic
But that effort does not move all gate job to Bionic as there are
large amount of jobs are still legacy jobs. All the legacy jobs still
use Xenial as nodeset.

As per the decided runtime for Stein, we need to test everything on openstack
CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html

Below patch move the legacy base jobs to bionic which will move the derived jobs
automatically to bionic. These jobs are modified with branch variant so that they will use
Bionic node from stein onwards and xenial for all other stable branches
until stable/rocky.
- https://review.openstack.org/#/c/639096

This commit remove the overridden nodeset from legacy jobs
so that it will start using the nodeset defined in parent job.

More Details: 
- https://etherpad.openstack.org/p/legacy-job-bionic
- http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003614.html

Depends-On: https://review.openstack.org/#/c/639096

Change-Id: I3266b6fd26ec86f429b91b69d82c8e99f80cb6ca
2019-03-13 23:54:38 +00:00
ghanshyam
d74a5b23a5 Disable the tls-proxy in nova-next & fix nova-tox-functional-py35 parent
While moving the legacy job nova-next on bionic, tls-proxy
did not work and leads to nova-next job fail.

To proceed further on Bionic migration which is blocked by nova-next failure,
this commit temporary disable the tls-proxy service until bug#1819794 is fixed.

Also this updates the parent of nova-tox-functional-py35 from openstack-tox
to openstack-tox-functional-py35 in order to handle the upcoming change
of the infra CI default node type from ubuntu-xenial to ubuntu-bionic.

The python3.5 binary is not provided on ubuntu-bionic and the shared
"py35" job definitions in the openstack-zuul-jobs repository have been
patched to force them to run on ubuntu-xenial [1]. We should inherit
from one of these jobs for jobs that rely on python3.5.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003746.html

Related-Bug: #1819794

Change-Id: Ie46311fa9195b8f359bfc3f61514fc7f70d78084
2019-03-13 23:46:19 +00:00
Corey Bryant
21357c76ba add python 3.7 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.7.

See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135626.html

Change-Id: I36f641a7c281a7d8e168c507dae7109132bcefde
Story: #2004073
Task: #27433
2019-03-13 10:36:55 -04:00
Zuul
151deebde8 Merge "Do not run tempest.scenario.test_network* tests in nova-next" 2019-03-09 14:18:16 +00:00
Zuul
eaaade924d Merge "Make nova-grenade-live-migration voting and gating" 2019-03-07 19:10:56 +00:00
Zuul
fb4308e33a Merge "Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree" 2019-03-07 18:58:39 +00:00
Matt Riedemann
f4387e4b66 Do not run tempest.scenario.test_network* tests in nova-next
The test_network* scenario tests in tempest take a really long
time. For example, the tempest.scenario.test_network_v6.TestGettingAddress
tests in one job (that timed out) took ~35 minutes.

We already run test network scenario tests in the tempest-slow job
so let's just let that job handle these and exclude them from
nova-next.

Change-Id: I9c7fc0f0b0937f04c5b3ab9c5e8fff21c4232b86
2019-03-06 10:12:26 -05:00
Matt Riedemann
c613df13cb Make nova-grenade-live-migration voting and gating
Per the mailing list [1] we want to make this job
voting and gate since it provides valuable test coverage
for mixed-compute service version live migration.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003341.html

Change-Id: I7b213930b180878629eee984b6aaef3b6e5aec07
2019-02-28 21:40:34 -05:00
Matt Riedemann
52613c01ce Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree
This moves the legacy-grenade-dsvm-neutron-multinode-live-migration
job from openstack-zuul-jobs in-tree. Nova is the only project
that runs this job and this allows nova to directly manage it.

The job will still use the same irrelevant-files list and is still
non-voting for the time being and only runs in the check queue.

It has been renamed to "nova-grenade-live-migration".

This change is meant to be backported to rocky, queens and pike
to allow us to drop legacy-grenade-dsvm-neutron-multinode-live-migration
from openstack-zuul-jobs where the job is branchless but restricted
to pike+ branches.

A follow up master-only (stein) change will make the job voting
and gate per the plan in the mailing list [1].

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003341.html

Change-Id: Ie9b61775dbb92b10237688eaddaca606c1c73a23
2019-02-28 21:40:30 -05:00
Matt Riedemann
00d9a3a2e6 Test live migration with config drive
Tempest live migration tests do not create the server
with config drive so we do not, by default, have test
coverage of live migrating an instance with a config
drive. This change forces nova to create a config
drive for all instances in the nova-live-migration
job which will also run evacuate tests with a config
drive attached.

Change-Id: I8157b9e6e9da492a225cbb50a20f434f83a5fecb
Related-Bug: #1770640
2019-02-28 10:54:01 -05:00
Zuul
ab1bce0ed6 Merge "Fix irrelevant-files for legacy-grenade-dsvm-neutron-multinode-live-migration" 2019-02-23 00:44:01 +00:00
Zuul
3548cf5921 Merge "Drop the integrated-gate (py27) template" 2019-02-20 04:53:27 +00:00
Matt Riedemann
6d803746de Fix irrelevant-files for legacy-grenade-dsvm-neutron-multinode-live-migration
The change to irrelevant-files in I902e459093af9b82f9033d58cffcb2a628f5ec39
made it such that the job will *only* run on changes to the
nova/tests/live_migration/ path which was not the intent. This fixes
the regex to not run the job on changes to nova/tests/ unless those
changes are in the nova/tests/live_migration/ directory.

This would probably be easier to reason about long-term if the
nova/tests/live_migration/ directory was moved under nova/gate/
and then we could just use the standard irrelevant-files list. That
refactoring is left for another day though.

Change-Id: Iff2fca18a779e70deecf8e57279e2daf9bc5529a
Closes-Bug: #1816152
2019-02-15 12:23:14 -05:00
Zuul
8c2dd6166f Merge "Change nova-next tempest test regex" 2019-02-15 02:05:43 +00:00