61 Commits

Author SHA1 Message Date
Mark Goddard
c16597aa2d Add seed VM provisioning CI job
Adds the kayobe-seed-vm-centos8 CI job to configure the Zuul VM as a
seed hypervisor, and use nested virt to provision a seed VM.  This
ensures that the seed hypervisor code paths are tested.

The job uses a Cirros image for the seed VM rather than the usual CentOS
cloud image. This is to reduce bandwidth required to download the image.
It does mean that the resulting seed VM cannot be used as a seed, but
nested virt would make this slow and unreliable anyway. Cirros does not
load cdrom drivers by default, so we add the configdrive as a disk
rather than a cdrom device.

Depends-On: https://review.opendev.org/617161

Change-Id: I2268a1ddf9a2870c713f32a40689e1686365aabd
Story: 2001655
Task: 6683
2020-06-16 17:19:47 +01:00
Zuul
8c312d0a35 Merge "CI: set previous_release to ussuri" 2020-06-16 12:50:07 +00:00
Will Szumski
0cc3703797 CI: set previous_release to ussuri
This controls which release we are upgrading from.

Change-Id: I16c86ce90d2b5a689e69681f12d0dbcb5c4cd6be
2020-06-12 13:51:28 +01:00
Will Szumski
9c36feb4ed Revert "Use OpenStack Ussuri release"
This reverts commit ffce6fa6411667f1474c33d845ccc4ac72765689.

Change-Id: I00f1b7c7d0182a80e0fb212099f67e92187ca603
2020-06-12 13:41:06 +01:00
4b960d94b8 Add Python3 victoria unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for victoria.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: Ie2c272d1f185083537274adb534001b2d8664154
2020-06-10 12:48:58 +00:00
Mark Goddard
ffce6fa641 Use OpenStack Ussuri release
Switch all dependencies to use the OpenStack Ussuri release.

This commit should be reverted on the master branch once the
Kayobe stable/ussuri branch has been cut and RC1 released.

Change-Id: Id126ea109e6dfcba09b524129876985547ad368f
2020-05-29 15:42:07 +01:00
Mark Goddard
b9d76f6ef5 Remove support for CentOS 7 and Python 2
* Always use Python 3
* Drop code paths for CentOS 7
* Drop support for Yum
* Remove support for host NTP daemon, always use chrony
* Switch references from 'yum_install_epel' to 'dnf_install_epel'
* Remove overcloud host image workaround for tagged VLAN admin network
* Remove the kayobe.utils.yum_install function, which is unused

Change-Id: I368f6edafed9779658798fc342116b4c1b3ffd48
Story: 2006574
Task: 39481
2020-05-28 10:25:51 +01:00
Zuul
558276a8a6 Merge "CI: Add overcloud host configure job" 2020-04-24 00:03:24 +00:00
Mark Goddard
8bed623571 Fix passwords.yml generation with vault encryption on Python 3
Kayobe generates passwords.yml for Kolla Ansible, and can encrypt it
using the vault password. Previously this was failing on Python 3 due to
passing a string to file.write() which expects bytes.

This change fixes the issue by encoding the password string passed to
file.write().

This allows us to run the ansible role tests under Python 3.

Change-Id: I33813f79984a46f1967ef3aee455dcfbe7eb93da
Story: 2006574
Task: 39481
2020-04-20 15:42:22 +01:00
Mark Goddard
92a437f63c CI: Add overcloud host configure job
Tests various non-default configuration:

* Custom users
* Network interfaces, VLANs, bridges, bonds
* Software RAID
* LVM & docker devicemapper
* timezone
* Package mirrors
* yum-cron / DNF automatic

This improved test coverage allows us to be more confident about these
features working on CentOS 8.

Change-Id: I36148e4356deb7d5ec00d8d3ebeb2d3932ff4f94
Story: 2006574
Task: 38938
2020-04-16 15:44:49 +00:00
Mark Goddard
0628beaa01 CentOS 8: Enable seed upgrade job
Kolla CentOS 8 Train Bifrost image is now available.

Change-Id: If2b06f80fa2817377ed881aa61ef73e4be38df55
Story: 2006574
Task: 39047
2020-03-25 14:56:57 +00:00
Mark Goddard
a28fd66b6e CentOS 8: Enable overcloud upgrade job
Missing bifrost image for Train, so no seed upgrade job yet.

Depends-On: https://review.opendev.org/711067

Change-Id: I0e88683f775769c1a80879685b0e7a2983599b08
Story: 2006574
Task: 39047
2020-03-23 10:32:57 +00:00
Mark Goddard
b0359e37df CentOS 8: Add seed and overcloud CI jobs, drop CentOS 7 jobs
CentOS 8 removes interfaces from their bridge during ifdown, and removes
the bridge if there are no interfaces left. When Kayobe bounces veth
links plugged into the bridge, it causes the bridge which has the IP we
are using for SSH to be removed. Use a dummy interface in CI to avoid
this problem.

Kolla-ansible has dropped all CentOS 7 jobs on master now, and prechecks
only allow CentOS 7 hosts. Drop all CentOS 7 jobs. We will have to run
without upgrade jobs in place until Train supports CentOS 8.

Depends-On: https://review.opendev.org/695881

Change-Id: I7c1a885b36445e33d4db1b1c8533db28a644b4a1
Story: 2006574
Task: 38870
2020-03-11 16:36:44 +00:00
Mark Goddard
f1a4b8cafc CI: set previous_release to train
This controls which release we are upgrading from.

Change-Id: I6b56a56e94f0b00b042c693a18ec1685724c7683
Depends-On: https://review.opendev.org/709189
2020-02-21 16:42:42 +00:00
Pierre Riteau
c9474bf9b0 Revert "Use OpenStack Train release"
This reverts commit a93b85ba07113a77a6c0db498a59cef3bca15b94.

The local Python executable for Kolla Ansible is changed to Python 3
because Kolla Ansible master no longer supports Python 2.

Change-Id: I768ce8db9cec1c70d94f271997bbcc64d370403e
2020-02-14 14:54:23 +00:00
Ghanshyam Mann
c0a1216afa [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

kayobe is ready with python 3 and ok to drop the
python 2.7 support.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: Idbbdd96ff23846d29833c5997d1e32e65b5169f2
2020-02-11 10:12:39 +00:00
Zuul
c41b2d714f Merge "CI: Increase job run attempts" 2020-01-06 10:06:14 +00:00
Will Szumski
a93b85ba07 Use OpenStack Train release
Switch all dependencies to use the OpenStack Train release.

This commit should be reverted on the master branch once the
Kayobe stable/train branch has been cut and RC1 released.

Change-Id: I6006e14df27a180f1438f9eb01b9ebefdddaff72
2019-12-12 11:55:56 +01:00
Mark Goddard
4e89518d0c CI: Increase job run attempts
Attempts affect pre failures.  This means we can increase stability of
jobs by rejecting nodes that fail pre without failing runs at the same
time (unless we are really unlucky and hit b0rken nodes 5 times in a
row).

This change is adapted from I17b7f878c742fa8db66f738526855a02ab9f1905 in
kolla-ansible.

Change-Id: I010b2fdd7c39e3314a9e1c59072d877f635f7522
2019-10-21 15:53:05 +01:00
Zuul
886fe4fd76 Merge "CI: Remove rocky pin from molecule requirements" 2019-10-02 17:44:58 +00:00
Zuul
185e93d653 Merge "CI: set previous_release to stein" 2019-10-01 17:18:39 +00:00
Andreas Jaeger
aea7117764 Fix after x/kayobe rename
Update .git review and zuul configuration.

Change-Id: I8efee18cf28344a6af44acbe1ca1d4470f7258fe
2019-09-16 16:26:27 +02:00
Mark Goddard
0765c09368 CI: Remove rocky pin from molecule requirements
Change-Id: I6f56ba2d6e3c5e5f91e83bd12d8a09c04845cd62
2019-09-12 10:46:22 +01:00
Mark Goddard
266b087244 CI: set previous_release to stein
This controls which release we are upgrading from.

Change-Id: I76f0db99cbfc0dea8cf16f0db370c8865e7dae03
2019-09-12 10:43:32 +01:00
Zuul
4169f2468f Merge "Revert "Use OpenStack Stein release"" 2019-09-03 18:50:58 +00:00
Mark Goddard
691ddeb614 Execute kayobe in verbose mode in CI
Also refactors the CI job playbooks to create the previous git repo
clones for upgrade jobs in the kayobe-base job.

Change-Id: I10c2b872887a47fab1a373fc0525dcd6fc78f7e2
2019-08-22 14:36:01 +00:00
Mark Goddard
ebe13df6f7 Revert "Use OpenStack Stein release"
This reverts commit b46b2b72bbec2a076a8175fc0aff96cbdefcabff.

Switch dependencies and upper constraints back to master branch for
development after cutting stable/stein branch.

Change-Id: Ie7f588aab59e57efe240a58a3bd0a6f09fbcefb7
2019-08-22 13:52:42 +01:00
Mark Goddard
4286c32299 Publish docs and release notes on openstack.o.o
Kayobe is now a kolla deliverable [1], so should use the official
site for publishing release notes and docs.

Links will be updated once these have been published.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007250.html

Change-Id: I9cc0bf08aa69386a18278f240c751a4d9956f926
2019-08-03 19:02:20 +01:00
Mark Goddard
b46b2b72bb Use OpenStack Stein release
Switch all dependencies to use the OpenStack Stein release.

This commit should be reverted on the master branch once the
Kayobe stable/stein branch has been cut and RC1 released.

Change-Id: I50a39994c8ffa0c7d65e026ae765973541423476
2019-05-29 17:20:25 +01:00
Andreas Jaeger
f1e02e9367 Remove duplicate docs job
Both docs-on-readthedocs and build-openstack-sphinx-docs contain a build
job - a different one: the current openstack-tox-docs and the obsolete
build-openstack-sphinx-docs. Remove the obsolete template and thus
avoid building docs twice for each change.

Change-Id: I358641d29304815eec6c699ae7e20da1e069443d
2019-05-25 17:55:51 +02:00
Zuul
9890e4bbc1 Merge "DRY out Zuul job configuration" 2019-05-21 15:53:29 +00:00
Mark Goddard
6c5fbcf942 Remove release notes readthedocs webhook job
This was found not to work, and prevents releases from being made.

The cause of the issue is that the trigger-readthedocs-webhook job (in
project-config) is marked as 'final', meaning that a non-trusted Zuul
config source can't override its variables. Instead, you're supposed to
use the trigger-readthedocs-webhook project template, but only one
instance of this can be instantiated by design.

Let's revert to manually updating the release notes for now.

Change-Id: I271c972c7fdde23085f3026137806bb1e3048e5e
2019-05-20 15:41:05 +01:00
Mark Goddard
ca3b5a7701 DRY out Zuul job configuration
Adds a new abstract parent job for the following kayobe deploy jobs:

kayobe-overcloud-centos
kayobe-overcloud-upgrade-centos
kayobe-seed-centos
kayobe-seed-upgrade-centos

This parent includes commonly used variables, and pre-run and post-run
playbooks for preparing and executing diagnostics, plus other
initialisation.

Change-Id: I9ab89fbc49bb32d86af56d50ec3914740e65d0f8
2019-05-07 15:59:25 +00:00
Zuul
cada311d4c Merge "Add python 3.6 jobs" 2019-04-26 17:57:36 +00:00
OpenDev Sysadmins
a65511a3d8 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:50:44 +00:00
Mark Goddard
eda8246b7b Use master version of dependencies
Use master branch of:

* Kolla
* Kolla ansible
* Bifrost
* IPA
* OpenStack services
* Requirements

Also, use the infra docker registry mirror at port 8082 which does not
have any path component in its URL, and so works with Docker CE.

Depends-On: https://review.openstack.org/#/c/651254/
Change-Id: Icdb62aa0083d9272a08421138719cf5f6b52ec5a
Story: 2001864
Task: 14316
2019-04-09 16:55:02 +00:00
Mark Goddard
da8059644c Add python 3.6 jobs
Also change to use the py3 tox environment rather than specifying a
version explicitly.

Change-Id: I05d5d6f5718b39dc95b40efe44e9ebb3fb5b1311
Story: 2004959
Task: 30035
2019-03-15 13:00:40 +00:00
Mark Goddard
4f5e87dbf2 Update release notes for stable/rocky
Also updates the release.sh script to support tagging release candidates
and development milestones. Adds zuul configuration to trigger a
Readthedocs webhook for the release notes.

Change-Id: Ia784b34fb0b740998de72599d4921f9303e41cac
2019-02-15 11:23:43 +00:00
Mark Goddard
7593a8b925 Test upgrading seed services in CI
Adds the kayobe-seed-upgrade-centos job, which performs an upgrade of
the seed services from the previous release to the current release.

Change-Id: Ia3eb39cf81cb3618fd94c4456bd576b52098c946
Story: 2004308
Task: 27873
2018-12-21 15:21:47 +00:00
Mark Goddard
09bcc6f28d Make seed deploy job voting again
The upstream fix to bifrost is now included in the rocky kolla images.

Change-Id: I0ca2e0a2d07a7be7d570ed63fceb2e82cf2d6173
2018-12-07 16:18:10 +00:00
Mark Goddard
0e329810a6 Bump zuul job timeouts
We are seeing a few overcloud deploy/upgrade jobs timing out. The jobs
appear to be progressing, so killing them isn't particularly helping.
Minimal random sampling suggests this affects the OVH cloud.

Change-Id: Ibce5457edf7a5f06e834aa587b5d4b5b6c69b8b6
2018-11-21 19:02:12 +00:00
Will Miller
d0e9c50fd2 Add tenks-deploy.sh dev script
tenks-deploy.sh deploys a minimal virtualized baremetal test cluster
locally. It also adds it to the overcloud-base CI job. To make the new
CI job work, we need to configure the firewall on the test machine to
allow the baremetal machines to communicate with the openstack services.

Change-Id: I7487a2606cf0bac71c5c63db41b2b518a6f6398b
Depends-On: https://review.openstack.org/#/c/615939
Depends-On: https://review.openstack.org/#/c/618003
Story: 2004297
Task: 27850
2018-11-20 18:53:09 +00:00
Mark Goddard
935d3cef6a Update dependencies to Rocky
Use stable/rocky branch of:

* Kolla
* Kolla ansible
* Bifrost
* IPA
* OpenStack services
* Requirements

Also updates Kolla Ansible inventory template.

The seed deploy job has been made non-voting and non-gating, because we
are waiting for bifrost change https://review.openstack.org/#/c/618740
to merge, be released, and for the kolla bifrost image to use the new
package.

Change-Id: Id5e7fdbd196f96e1e75ffc68bc93aab18fa38aa7
Story: 2001864
Task: 27798
Depends-On: I58e4f951d4a3dd89e0784fd82d8a62dbba374f79
2018-11-19 14:37:33 +00:00
Mark Goddard
6266312fa1 Test upgrades in CI
There is currently no coverage of upgrades in CI, which leaves us open
to regression in this infrequently tested but crucial area. This change
adds the required scripts and Zuul configuration.

A control plane is first deployed using the Kayobe stable/pike branch
and associated default configuration. The control plane is tested by
deploying then deleting a server instance. An upgrade to Queens is
performed, using the Kayobe master branch, or code in review if
applicable. The upgraded control plane is tested by deploying then
deleting a second server instance.

A workaround was required to restart the nova_compute service after the
upgrade, since the SIGHUP sent to it by Kolla Ansible during upgrade
appeared to be putting it into a degraded state.

A future improvement to this test could be to leave a server instance
running during the upgrade.

Change-Id: I0e595524e39d1131fe3ec6aaf2aeec3ff3d6a536
Story: 2003472
Task: 24732
2018-11-05 12:02:31 +00:00
Zuul
72cff1bef8 Merge "Add a zuul template to update readthedocs" 2018-11-02 17:20:03 +00:00
Mark Goddard
1244dc79b5 Improve irrelevant-files for Zuul jobs
Change-Id: I35a54f48bc8234425dda639af37c4029536d9666
2018-11-02 11:02:30 +00:00
Mark Goddard
01a00ec569 Add a zuul template to update readthedocs
This only affects the kayobe documentation, not the release notes.

A webhook is configured for the kayobe readthedocs project which has ID
50407.

See
http://lists.openstack.org/pipermail/openstack-dev/2018-August/132836.html
for further information.

Change-Id: Ibfccf30f51ab2f63743bf6e2b61bf9a3883cff23
2018-10-10 19:26:05 +01:00
Zuul
fa681dc2c1 Merge "Include kayobe in zuul project list" 2018-10-09 16:34:55 +00:00
Andreas Jaeger
9f3e8997fa Cleanup zuul jobs
Use templates instead of individual jobs.

Remoe usage of releasenotes templates since it publishes content to
docs.o.o and only official OpenStack projects can publish there, use
instead just the build jobs.

Change-Id: I3046569efd586707150fb4cc753fd55979470946
2018-09-29 19:15:52 +02:00
Mark Goddard
dd9266f5b8 Include kayobe in zuul project list
This ensures other projects can use zuul jobs defined in kayobe.

Change-Id: Ie8a3dc3dfba1ce0bd78ac676e3c4553461f45a05
2018-09-28 12:06:36 +01:00