153 Commits

Author SHA1 Message Date
Mark Goddard
05c09523fa ironic: default to ipxe booting
Enable the Ironic ipxe boot interface by default, following a similar
change in Ironic [1].

Drop the kolla_enable_ironic_ipxe flag, following a similar change in
Kolla Ansible [2]. Both PXE and iPXE are now enabled by default. Users
may revert to using PXE for ironic inspector's dnsmasq, by setting
ironic_dnsmasq_serve_ipxe to false in etc/kayobe/kolla/globals.yml.

[1] https://review.opendev.org/c/openstack/ironic/+/816824
[2] https://review.opendev.org/c/openstack/kolla-ansible/+/834512/

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/832159
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/834511
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/837069

Change-Id: Ifb80bd15a20c9cfb8fbc6e0f6ac23baae631a18e
2022-04-12 11:57:21 +00:00
Mark Goddard
a7ee3ac5c8 CI: separate image builds into a non-voting job
Disk and container image builds tend to be fairly unreliable.
With 3 voting seed jobs all building images, this can introduce
instability into the CI jobs.

This change adds a non-voting kayobe-seed-images-centos8s job, which
does the following:

* Builds IPA images
* Builds an overcloud host image
* Builds a base container image

Similar Rocky and Ubuntu jobs are added to the experimental pipeline,
and may be run by commenting 'check experimental' in gerrit.

The existing kayobe-seed-* jobs no longer build images.

Change-Id: Idecda342f3ab86733e8d59061458d44af834dbb0
2022-03-31 08:52:17 +00:00
Zuul
0cd0f05781 Merge "libvirt: support SASL authentication" 2022-03-29 21:13:19 +00:00
Zuul
4bb2aa8f29 Merge "libvirt: deploy libvirt on the host" 2022-03-29 21:13:12 +00:00
Mark Goddard
c9c0019d7e Use jinja2.pass_context instead of contextfilter
The contextfilter decorator was deprecated in jinja2 3.0.0, and has been
dropped in 3.1.0. This results in the following warning, and failed
attempts to use filters:

    [WARNING]: Skipping plugin (networks.py) as it seems to be invalid:
    module 'jinja2' has no attribute 'contextfilter'

This change switches to use the pass_context decorator. The minimum
version of Jinja2 is raised to 3 to ensure pass_context is present.

This change also includes some changes to address issues with image
builds in CI, caused by CentOS Scream.

1. disable IPA image builds in seed deploy jobs

IPA image builds will be split out into a separate job. For now, disable
them.

2. disable overcloud host image builds in seed deploy jobs

Overcloud host image builds will be split out into a separate job. For
now, disable them.

Depends-On: https://review.opendev.org/c/openstack/kayobe/+/835279
Change-Id: If657bf5b0117812d3c53942464cc41cf86cc8ad5
2022-03-29 13:59:56 +01:00
Mark Goddard
f4493e41ff libvirt: support SASL authentication
Adds support for SASL authentication of libvirt TCP and TLS connections
when using a compute host libvirt daemon.

In line with the dependent Kolla Ansible patch, we enable SASL by
default, and use DIGEST-MD5 with TCP and SCRAM-SHA-256 with TLS.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/833022
Depends-On: https://github.com/stackhpc/ansible-role-libvirt-host/pull/52

Story: 2009858
Task: 44735

Change-Id: Id3972c24022aeb6421494c3cccdc8e7cbce802e6
2022-03-24 13:44:48 +00:00
Mark Goddard
c4b74f4801 libvirt: deploy libvirt on the host
In some cases it may be desirable to run libvirt daemon on the host. For
example, when mixing host and container OS distributions.

This change makes it possible to disable the nova_libvirt container, by
setting kolla_enable_nova_libvirt_container to false.

The stackhpc.libvirt-host role is used in order to install and configure
a libvirt daemon on compute hosts when
kolla_enable_nova_libvirt_container is false.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/825357
Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/829225
Depends-On: https://github.com/stackhpc/ansible-role-libvirt-host/pull/51

Story: 2009858
Task: 44495

Change-Id: I73fef63fb886a9d543d2f4231fb009523495edb3
2022-03-24 13:44:48 +00:00
Mark Goddard
5c661b888e Ubuntu: add support for Apt configuration
This change adds support for configuration of Apt package manager in
/etc/apt/apt.conf.d/. This allows adding arbitrary global configuration
options for Apt. Options can be added in different files, allowing for
different filename-based priorities.

CI tests and documentation are provided.

Story: 2009655
Task: 43987

Change-Id: I9d7d18851359e97cd01b4c2287bf79110796b25a
2022-03-23 06:48:56 +00:00
Mark Goddard
c603be2536 Ubuntu: add support for Apt repository configuration
This change adds support for configuring Apt repositories on Ubuntu
hosts during host configuration.

Repositories are configured in a single file
(/etc/apt/sources.list.d/kayobe.sources), using the modern deb822
format [1]. This format is more flexible and readable than the original
single-line format, particularly if multiple options are used.

Using a single file allows us to more easily keep the set of
repositories in sync, since Ansible doesn't make it easy to clean things
up.

Support is added for marking repositories as signed by a particular GPG
key. This approach is now preferred over the deprecated [2] apt-key
tool, which resulted in a set of globally trusted keys.

It is also possible to disable the repositories in
/etc/apt/sources.list via apt_disable_sources_list. This allows for
replacing the standard repositories with a local mirror.

CI tests and documentation are provided.

[1] https://manpages.ubuntu.com/manpages/focal/en/man5/sources.list.5.html
[2] https://manpages.ubuntu.com/manpages/groovy/man8/apt-key.8.html

Story: 2009655
Task: 43818

Change-Id: I3f821937b0930a0ac9341178de7ae5123d82b957
2022-03-23 06:47:17 +00:00
Zuul
4616c87010 Merge "Add support for Rocky Linux 8" 2022-03-20 22:20:24 +00:00
Michal Nasiadka
8e55ea08a4 Add support for Rocky Linux 8
Change-Id: If7d6e58b19f98ccb7cc4c209e458cb6f4f4765ad
2022-03-18 15:04:21 +00:00
Maksim Malchuk
1de4f2a4a3 Skip IP address allocation and configuration if needed
Sometimes some hosts should be configured with an interface without any
IP address set (e.g. bridged interface) and to achieve that this change
adds the new attribute 'no_ip' for the network configuration. Also the
change contain a test for this.

Change-Id: I2c9dfeca7f0d37a96f9cbd9df51d94098cf07258
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2022-03-18 11:39:40 +01:00
Zuul
5fdf643807 Merge "CI: Don't download Cirros or IPA in seed jobs" 2022-03-16 11:41:09 +00:00
Zuul
5d996709a6 Merge "CI: Move to pytest-testinfra" 2022-03-16 01:34:36 +00:00
Zuul
554f23074e Merge "CI: fix TLS job by freeing up memory" 2022-03-16 01:34:33 +00:00
Mark Goddard
d77a30db36 CI: Don't download Cirros or IPA in seed jobs
We build IPA images and a deployment image in the seed jobs, so we don't
need to download Cirros or IPA images. Also, these downloads depend on
external resources which may make jobs less reliable.

For seed upgrade jobs, disable IPA and deployment image downloads.

Change-Id: Ib59c8bc2d8938eca18c943bb2e66ed185152a739
2022-03-15 13:24:42 +00:00
Mark Goddard
acf6d0d51f CI: Disable container image builds on Ubuntu
The kayobe-seed-ubuntu-focal job is currently fairly unreliable, often
failing to build the base container image.

We are not using the mirrors provided by OpenDev infra, which may be
making these builds less reliable.

This change disables container image builds in CI on Ubuntu. It should
be reverted if they are made more reliable.

Change-Id: I648fa6423ad9ff43120c7808f080b0359ad8621c
2022-03-15 09:52:38 +00:00
Pierre Riteau
b0c1d4c53a CI: fix TLS job by freeing up memory
When TLS is enabled, extra RAM usage is causing the OOM killer to
terminate Tenks VMs, which are using large RSS amounts (around 1.5 GB).

Disable Heat and Horizon to free up enough memory to make the job pass.

Change-Id: If483a6a6fb6d5b2c9b6b7dbd22939b0b46599538
2022-03-11 17:38:09 +01:00
Michal Nasiadka
a20fc47f0c CI: Move to pytest-testinfra
Change-Id: I615707976454a91c8f6aecc5eda1852def7197d4
2022-03-07 14:32:48 +01:00
Zuul
678b915aff Merge "CI: remove qemu-utils installation" 2022-03-03 18:00:32 +00:00
Zuul
a245d5019c Merge "CI: stop setting libvirt_host_require_vt in overrides" 2022-03-03 13:16:25 +00:00
Pierre Riteau
e453f3c570 CI: remove qemu-utils installation
nova-libvirt images now include qemu-utils on master [1] and xena [2].

[1] https://review.opendev.org/c/openstack/kolla/+/830401
[2] https://review.opendev.org/c/openstack/kolla/+/831411

Change-Id: I8f5f93340642d055cce7ef306d942e75b10c86a9
2022-03-03 10:01:21 +01:00
Mark Goddard
a0665cd9c6 CI: stop using zuul as kayobe_ansible_user in TLS jobs
Previously we were using the zuul user in the TLS jobs. This was due to
a permissions issue when accessing the CA certificate in kayobe-config
in the zuul user's home directory.

This change reverts to the default of using the stack user for the TLS
jobs. In order to make this work, the generated CA cert chain is added
to the trust store.

Change-Id: I875f8976df75dee68ba00842fe624c29cc1b123c
2022-03-02 13:34:12 +00:00
Zuul
082f9c5477 Merge "CI: test fact caching" 2022-02-28 13:09:51 +00:00
Zuul
01e5834ed1 Merge "CI: enable libvirt TLS in TLS job" 2022-02-23 18:00:52 +00:00
Pierre Riteau
959bef6745 CI: test fact caching
Also synchronise Ansible settings between Kayobe and Kolla Ansible.

Change-Id: Idaea4a984391a8cd05a5b0eee254ac6bad531a3e
2022-02-23 09:29:43 +01:00
Mark Goddard
3efc551e7e Install Ansible collections for kolla-ansible
This change uses the new Galaxy requirements file in Kolla Ansible to
install the openstack.kolla collection.

Cross-project dependencies on ansible-collection-kolla are supported.

Story: 2009854
Task: 44504

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/819430
Change-Id: Iac185dd2bbbca128c6cf71b2734e94b3e1c6133b
2022-02-22 09:23:35 +00:00
Mark Goddard
06c6191799 Install openstack.kolla collection
This patch adds the openstack.kolla collection to the Galaxy
requirements. It is installed from the OpenDev git repository. The
collection is not yet used by Kayobe.

Zuul cross-project dependencies on the ansible-collection-kolla
repository are supported (and used in this commit).

Story: 2009854
Task: 44503

Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/820165/
Change-Id: I91cbac839f816a00ac54bc4a350f44b5ae457cc3
2022-02-22 09:23:14 +00:00
Mark Goddard
5cf03e9870 CI: stop setting libvirt_host_require_vt in overrides
It is now set globally in kayobe-config-dev.

Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/829225
Change-Id: Ib46834fd226c3f59146bbaada53c8d079481d315
2022-02-21 14:14:30 +00:00
Zuul
e027e5fc10 Merge "CI: test ironic UEFI boot mode in overcloud TLS jobs" 2022-02-21 13:59:11 +00:00
Mark Goddard
883027afb0 CI: Enable bare metal testing for Ubuntu
Previously we were seeing issues with ipmitool and virtualbmc on Ubuntu:

  Error setting Chassis Boot Parameter 5\nError setting Chassis Boot Parameter 0

The dependent change fixes these issues, and this change enables bare
metal testing in Ubuntu CI.

Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/829006

Change-Id: I96827fc32c1594ca9a0535e259929c49d3f0e704
2022-02-14 09:16:59 +00:00
Mark Goddard
c00085da69 CI: enable libvirt TLS in TLS job
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/826723
Change-Id: I52ffa0b23ce2ae54fbfe5879a25b128ca1864195
2022-02-10 20:14:29 +00:00
Mark Goddard
b57b94bbad CI: test ironic UEFI boot mode in overcloud TLS jobs
Set the Ironic boot mode to legacy BIOS explicitly in Tenks config in
anticipation of an upcoming change to the default boot mode.

Override the boot mode to UEFI in the overcloud TLS job to improve
coverage. This requires enabling iPXE booting.

Depends-On: https://review.opendev.org/c/openstack/tenks/+/827479/

Change-Id: Id1b4e9775c834b8b97e086241ee8b247977225a2
2022-02-08 11:16:03 +00:00
Mark Goddard
e0dec91689 Disable EPEL by default
EPEL is no longer required for a default installation. Let's disable it.

Also clean up the install_epel variable from Kolla Ansible globals.yml
template, since it never existed.

Story: 2009757
Task: 44227

Change-Id: I96eb4685f997e85ad2ee5318640d58d0287a016d
2021-12-23 13:04:12 +00:00
Piotr Parczewski
b5f0a17acf CI: Use correct TD agent repository version
Change-Id: Ic8c212111301d6b144e3aa0fe124b022b0296ec6
2021-12-07 14:35:18 +01:00
Zuul
efbb3082d7 Merge "Build overcloud host image directly with DIB" 2021-11-26 22:57:10 +00:00
Pierre Riteau
c35f112a14 Build overcloud host image directly with DIB
As a first step towards supporting multiple overcloud disk images, this
change introduces a new command to build a disk image directly with DIB:
`kayobe overcloud host image build`.

It also disables building a root disk image during Bifrost bootstrap if
overcloud_dib_build_host_images is set to true.

Change-Id: I93d242889e225b4e60254f6b9cc5eeb457294ac8
Story: 2002098
Task: 41693
2021-11-26 09:28:00 +01:00
Zuul
e61d8f24a6 Merge "[release] Set previous_release to xena" 2021-11-25 19:22:59 +00:00
Zuul
52d7523c0c Merge "CI: always return host configure test results" 2021-11-19 11:33:19 +00:00
Zuul
e55fab9674 Merge "CI: add Infra VM jobs" 2021-11-11 22:20:57 +00:00
Mark Goddard
4bf7cdf1e3 [release] Set previous_release to xena
Also remove some Wallaby upgrade compatibility code in CI playbooks

Change-Id: I84e336bec37eac12c659c7a9d9fc4ddd191a379d
2021-11-09 17:31:53 +00:00
Zuul
7000b794fe Merge "CI: Disable heat in upgrade jobs to save disk space" 2021-10-27 16:02:08 +00:00
Mark Goddard
104e383fc7 CI: add Infra VM jobs
Depends-On: https://review.opendev.org/c/openstack/kayobe/+/812670
Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/805239
Depends-On: https://review.opendev.org/c/openstack/kayobe/+/813212

Change-Id: I7dbd643fec72223bfda20cbbc37809ae78073804
2021-10-26 09:35:17 +01:00
Pierre Riteau
06a51ceaec CI: Disable heat in upgrade jobs to save disk space
We see frequent failures of upgrade jobs on stable branches due to lack
of disk space. Disable heat in an attempt to free up extra space, since
we do not need it for our testing.

Change-Id: I20c99ca9dd6e78b041e4662c9b1b54eac904e3c9
2021-10-22 09:54:00 +02:00
Zuul
b150f57a4e Merge "CI: add host configure tests for firewalld" 2021-10-21 14:21:28 +00:00
Zuul
0ff94df353 Merge "CI: enable DNF tests on all CentOS versions" 2021-10-20 13:07:37 +00:00
Zuul
c47376febe Merge "CI: enable DNF tests on CentOS Stream 8" 2021-10-20 13:07:27 +00:00
Mark Goddard
67126eb5f2 CI: add host configure tests for firewalld
Follow up adding tests for Id60e25e129e323f3c07e702bb81a11efc530fb3e.

Change-Id: Ieb6d6e4d491b4ceb44d5fe364f0da215ac303fe1
2021-10-12 09:24:48 +01:00
Mark Goddard
c2ae611ee8 CI: always return host configure test results
Currently, the overcloud host configure test results are only returned
on success, when they are least useful. This is because the zuul_return
task is not reached when the test task fails. This change reorders the
tasks to fix the issue.

Change-Id: Ie572eda567782cc377767bd8d925145e2ad4bd61
2021-10-07 10:26:52 +01:00
Michał Nasiadka
769c2efb97 Add support for configuring tuned profile
This is only supported on CentOS for now due to limitations of the
Ansible role used to configure tuned.

Change-Id: Ie07c5f467975f8da2f720e70c94cea6285981d72
Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
Story: 2007853
Task: 40155
2021-10-06 17:49:24 +02:00