2290 Commits

Author SHA1 Message Date
Mark Goddard
07f8e41bbb Revert "[release] Use OpenStack Yoga release"
This reverts commit 88dd02dc92ca4b544aec18bfbb1d1d93954c55fe.

Reason for revert: Master is now Zed

Change-Id: I9a18c41dbdc369a7a719632d6e41e2373784f29e
2022-04-14 08:09:18 +00:00
Mark Goddard
88dd02dc92 [release] Use OpenStack Yoga release
Switch all dependencies to use the OpenStack Yoga release.

This commit should be reverted on the master branch once the
Kayobe stable/yoga branch has been cut and RC1 released.

Change-Id: Ib7495c1bf79de8b1ea67e4a8652345c22e9d1a3e
2022-04-07 13:51:18 +01:00
Zuul
7b8c2fa6a6 Merge "Fix Ansible inventory generation when reusing group names" 2022-04-07 12:25:06 +00:00
Zuul
31951b6d59 Merge "Adds kolla_ansible_venv_ansible" 2022-04-07 10:02:47 +00:00
Pierre Riteau
0c190a22f0 Fix Ansible inventory generation when reusing group names
When a kolla-ansible group is composed of a kayobe group with the same
name and of at least one other kayobe group, kayobe would generate an
invalid Ansible inventory such as:

[compute:children]
controllers
compute

Because the top-level group should already be defined, we only need to
add as children the kayobe groups using different names.

Change-Id: I88bdf1e3d0c08271ac8938ae2f9ac3f9fee1efa5
Story: 2009927
Task: 44798
2022-04-07 08:02:02 +00:00
Zuul
60d74df815 Merge "CI: separate image builds into a non-voting job" 2022-04-06 12:49:57 +00:00
Mark Goddard
8b3494e2a4 Revert "CI: Revert from UEFI to BIOS mode in TLS jobs"
This reverts commit b2e8e715538b4f7dcfda7d657497818828a66dae.

Reason for revert: Other measures were taken to reduce memory overhead.

Change-Id: I4987480b4ed1bd4bd95a0ae72861f471cc229e20
2022-04-06 07:00:38 +00:00
Mark Goddard
60fc11fa26 [release] Sync Kolla Ansible inventory for Yoga
Syncs with Kolla Ansible inventory for Yoga release, at commit
1a20c2348830eb189026b7d1d799ed0fa435aeeb.

Change-Id: I119c200c0c61e7996ba9996d480d07cb0a273b60
2022-04-01 11:16:06 +00:00
Mark Goddard
a7ee3ac5c8 CI: separate image builds into a non-voting job
Disk and container image builds tend to be fairly unreliable.
With 3 voting seed jobs all building images, this can introduce
instability into the CI jobs.

This change adds a non-voting kayobe-seed-images-centos8s job, which
does the following:

* Builds IPA images
* Builds an overcloud host image
* Builds a base container image

Similar Rocky and Ubuntu jobs are added to the experimental pipeline,
and may be run by commenting 'check experimental' in gerrit.

The existing kayobe-seed-* jobs no longer build images.

Change-Id: Idecda342f3ab86733e8d59061458d44af834dbb0
2022-03-31 08:52:17 +00:00
Pierre Riteau
c2186b1bd3 [release] Sync Kolla Ansible feature flags for Yoga
Change-Id: I174623bbcfeb35c59cd4c155f36078800d4c35cd
2022-03-31 07:29:35 +02:00
Will Szumski
1375517d2b Adds kolla_ansible_venv_ansible
This adds a variable that allows you to modify the version of ansible
installed in the kolla-ansible virtualenv. This is useful if you want
to use a customised version of ansible.

Change-Id: I319dd51ed3221826f820fbc0ae3639b89e9c82ea
2022-03-30 15:36:37 +01:00
Zuul
91f2d9da2d Merge "Fix custom config idempotence" 2022-03-30 11:16:45 +00:00
Zuul
0cd0f05781 Merge "libvirt: support SASL authentication" 2022-03-29 21:13:19 +00:00
Zuul
4bb2aa8f29 Merge "libvirt: deploy libvirt on the host" 2022-03-29 21:13:12 +00:00
Mark Goddard
c9c0019d7e Use jinja2.pass_context instead of contextfilter
The contextfilter decorator was deprecated in jinja2 3.0.0, and has been
dropped in 3.1.0. This results in the following warning, and failed
attempts to use filters:

    [WARNING]: Skipping plugin (networks.py) as it seems to be invalid:
    module 'jinja2' has no attribute 'contextfilter'

This change switches to use the pass_context decorator. The minimum
version of Jinja2 is raised to 3 to ensure pass_context is present.

This change also includes some changes to address issues with image
builds in CI, caused by CentOS Scream.

1. disable IPA image builds in seed deploy jobs

IPA image builds will be split out into a separate job. For now, disable
them.

2. disable overcloud host image builds in seed deploy jobs

Overcloud host image builds will be split out into a separate job. For
now, disable them.

Depends-On: https://review.opendev.org/c/openstack/kayobe/+/835279
Change-Id: If657bf5b0117812d3c53942464cc41cf86cc8ad5
2022-03-29 13:59:56 +01:00
Mark Goddard
f4493e41ff libvirt: support SASL authentication
Adds support for SASL authentication of libvirt TCP and TLS connections
when using a compute host libvirt daemon.

In line with the dependent Kolla Ansible patch, we enable SASL by
default, and use DIGEST-MD5 with TCP and SCRAM-SHA-256 with TLS.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/833022
Depends-On: https://github.com/stackhpc/ansible-role-libvirt-host/pull/52

Story: 2009858
Task: 44735

Change-Id: Id3972c24022aeb6421494c3cccdc8e7cbce802e6
2022-03-24 13:44:48 +00:00
Mark Goddard
c4b74f4801 libvirt: deploy libvirt on the host
In some cases it may be desirable to run libvirt daemon on the host. For
example, when mixing host and container OS distributions.

This change makes it possible to disable the nova_libvirt container, by
setting kolla_enable_nova_libvirt_container to false.

The stackhpc.libvirt-host role is used in order to install and configure
a libvirt daemon on compute hosts when
kolla_enable_nova_libvirt_container is false.

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/825357
Depends-On: https://review.opendev.org/c/openstack/kayobe-config-dev/+/829225
Depends-On: https://github.com/stackhpc/ansible-role-libvirt-host/pull/51

Story: 2009858
Task: 44495

Change-Id: I73fef63fb886a9d543d2f4231fb009523495edb3
2022-03-24 13:44:48 +00:00
Zuul
24b2da7cbc Merge "Ubuntu: add support for Apt configuration" 2022-03-23 17:18:16 +00:00
Zuul
2251d041d2 Merge "Ubuntu: add support for Apt repository configuration" 2022-03-23 10:55:51 +00:00
Zuul
0f6752b016 Merge "CI: pin pytest-metadata<2 for molecule" 2022-03-23 07:15:17 +00:00
Mark Goddard
5c661b888e Ubuntu: add support for Apt configuration
This change adds support for configuration of Apt package manager in
/etc/apt/apt.conf.d/. This allows adding arbitrary global configuration
options for Apt. Options can be added in different files, allowing for
different filename-based priorities.

CI tests and documentation are provided.

Story: 2009655
Task: 43987

Change-Id: I9d7d18851359e97cd01b4c2287bf79110796b25a
2022-03-23 06:48:56 +00:00
Mark Goddard
c603be2536 Ubuntu: add support for Apt repository configuration
This change adds support for configuring Apt repositories on Ubuntu
hosts during host configuration.

Repositories are configured in a single file
(/etc/apt/sources.list.d/kayobe.sources), using the modern deb822
format [1]. This format is more flexible and readable than the original
single-line format, particularly if multiple options are used.

Using a single file allows us to more easily keep the set of
repositories in sync, since Ansible doesn't make it easy to clean things
up.

Support is added for marking repositories as signed by a particular GPG
key. This approach is now preferred over the deprecated [2] apt-key
tool, which resulted in a set of globally trusted keys.

It is also possible to disable the repositories in
/etc/apt/sources.list via apt_disable_sources_list. This allows for
replacing the standard repositories with a local mirror.

CI tests and documentation are provided.

[1] https://manpages.ubuntu.com/manpages/focal/en/man5/sources.list.5.html
[2] https://manpages.ubuntu.com/manpages/groovy/man8/apt-key.8.html

Story: 2009655
Task: 43818

Change-Id: I3f821937b0930a0ac9341178de7ae5123d82b957
2022-03-23 06:47:17 +00:00
Mark Goddard
f3bca3e0f6 CI: pin pytest-metadata<2 for molecule
The release of pytest-metadata 2.0.0 appears to have broken the molecule tests.

  AttributeError: 'Config' object has no attribute '_metadata'

Change-Id: I60cf89ce2f8c236b7febb92d437abf7e21bb7acd
2022-03-22 20:09:21 +00:00
Mark Goddard
48a6dcc82d Ubuntu: support host package update
Currently, the 'kayobe * host package update' commands do not work on
Ubuntu, since the main task is skipped.

This change adds support for running the commands against Ubuntu hosts,
passing parameters to the package module based on the OS distribution.

Change-Id: I879df0ea3f357c2bb45a5e7331a3f2569eac63e9
Story: 2009685
Task: 43988
2022-03-22 09:11:28 +00:00
Zuul
4616c87010 Merge "Add support for Rocky Linux 8" 2022-03-20 22:20:24 +00:00
Mark Goddard
994af643e6 ironic: switch to kernel_append_params
Since I4fb42d376636dc363cd86950ed37de4a3d28df73, kolla-ansible sets
kernel_append_params in ironic.conf, instead of the deprecated
pxe_append_params. Make the same change in the Kayobe ironic.conf
template, to avoid getting shadowed by the kolla-ansible default.

This also fixes the overcloud TLS job, which started failing because it
lost the ipa-insecure kernel option, making the Ironic API callback
invalid.

Change-Id: Id7e4bd7f199ad9dcb21d5db082e7a187cb310df9
2022-03-18 20:42:25 +00:00
Michal Nasiadka
8e55ea08a4 Add support for Rocky Linux 8
Change-Id: If7d6e58b19f98ccb7cc4c209e458cb6f4f4765ad
2022-03-18 15:04:21 +00:00
Mark Goddard
8de9d53681 Fix custom config idempotence
When some custom config files are used, generation of local
configuration for Kolla Ansible is not idempotent. This happens because
an executable permission is applied recursively in a directory, then
reverted on regular files.

Change-Id: I8cc9531570b76a8282a95c4036324e9d1025d7cb
2022-03-18 14:22:39 +00:00
Maksim Malchuk
1de4f2a4a3 Skip IP address allocation and configuration if needed
Sometimes some hosts should be configured with an interface without any
IP address set (e.g. bridged interface) and to achieve that this change
adds the new attribute 'no_ip' for the network configuration. Also the
change contain a test for this.

Change-Id: I2c9dfeca7f0d37a96f9cbd9df51d94098cf07258
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2022-03-18 11:39:40 +01:00
Zuul
a1ed4a05bb Merge "Use naming convention to infer VLAN tagging" 2022-03-17 02:01:15 +00:00
Zuul
bf4f58489f Merge "Fix link formatting in release note" 2022-03-16 18:46:57 +00:00
Will Szumski
dd956189ed Use naming convention to infer VLAN tagging
Any interface not matching the convention: <interface>.<vlan id> will be
assumed to untagged when generating the config drive. The link type will
be set to `phy` in `network_data.json` instead of `vlan`. This prevents
invalid config drive configuration from being generated.

Change-Id: I00e7615581c7a214e0ee8e8f04cacdbb3ea4131a
Story: 2009910
Task: 44736
2022-03-16 16:30:19 +01:00
Pierre Riteau
2d16a70e00 CI: Enable overcloud TLS job again
Change If483a6a6fb6d5b2c9b6b7dbd22939b0b46599538 which fixed the job
should have included this bit.

Change-Id: I22e2e6d361bc15fe7aa47ff833b3c5a525266221
2022-03-16 15:40:19 +01:00
Zuul
5fdf643807 Merge "CI: Don't download Cirros or IPA in seed jobs" 2022-03-16 11:41:09 +00:00
Pierre Riteau
1d4046ef4b Fix link formatting in release note
Change-Id: I3d6a703f74a662ca28022fe44ca936c192c7e89e
2022-03-16 11:42:08 +01:00
Zuul
0c3fd4b5e2 Merge "Only create patch links on overcloud hosts" 2022-03-16 01:49:30 +00:00
Zuul
4f76cae4ab Merge "Bump MichaelRigart.interfaces to v1.13.1" 2022-03-16 01:34:38 +00:00
Zuul
5d996709a6 Merge "CI: Move to pytest-testinfra" 2022-03-16 01:34:36 +00:00
Zuul
554f23074e Merge "CI: fix TLS job by freeing up memory" 2022-03-16 01:34:33 +00:00
Zuul
b463568434 Merge "CI: Disable container image builds on Ubuntu" 2022-03-15 20:26:33 +00:00
Mark Goddard
07eb06755e CI: Pin ansible-lint to <6
The molecule job is failing with the following since the release of
ansible-lint 6.0.0:

    ModuleNotFoundError: No module named 'ansiblelint.prerun'

Pin ansible-lint to <6.

Also ignore changes to molecule-requirements.txt for jobs that don't use
it.

Change-Id: I00ea1b4cd57a22a1a0939ddc463ef336da2f0902
2022-03-15 15:10:01 +00:00
Mark Goddard
d77a30db36 CI: Don't download Cirros or IPA in seed jobs
We build IPA images and a deployment image in the seed jobs, so we don't
need to download Cirros or IPA images. Also, these downloads depend on
external resources which may make jobs less reliable.

For seed upgrade jobs, disable IPA and deployment image downloads.

Change-Id: Ib59c8bc2d8938eca18c943bb2e66ed185152a739
2022-03-15 13:24:42 +00:00
Mark Goddard
acf6d0d51f CI: Disable container image builds on Ubuntu
The kayobe-seed-ubuntu-focal job is currently fairly unreliable, often
failing to build the base container image.

We are not using the mirrors provided by OpenDev infra, which may be
making these builds less reliable.

This change disables container image builds in CI on Ubuntu. It should
be reverted if they are made more reliable.

Change-Id: I648fa6423ad9ff43120c7808f080b0359ad8621c
2022-03-15 09:52:38 +00:00
Pierre Riteau
b0c1d4c53a CI: fix TLS job by freeing up memory
When TLS is enabled, extra RAM usage is causing the OOM killer to
terminate Tenks VMs, which are using large RSS amounts (around 1.5 GB).

Disable Heat and Horizon to free up enough memory to make the job pass.

Change-Id: If483a6a6fb6d5b2c9b6b7dbd22939b0b46599538
2022-03-11 17:38:09 +01:00
Pierre Riteau
5fb6362e25 CI: Disable TLS job until fixed
The overcloud job with TLS is failing frequently with the Tenks VM
triggering the OOM killer. Disable until we find a reliable fix.

Change-Id: If42d1f0580d8305a36dd4fd4431680559eb5d68f
2022-03-11 10:19:31 +01:00
Will Szumski
bb9da9fb24 Only create patch links on overcloud hosts
These are only neccesary on network and compute nodes.

Co-authored-by: Mark Goddard <mark@stackhpc.com>

Change-Id: Id5af3969da63150e892201f7518f50a3da73e852
Story: 2009911
Task: 44740
2022-03-10 16:51:38 +00:00
Mark Goddard
a786ef20f7 Bump MichaelRigart.interfaces to v1.13.1
The update of MichaelRigart.interfaces to v1.13.0
introduced a regression due to
https://github.com/michaelrigart/ansible-role-interfaces/pull/118.

Configuration for active interfaces could be removed and not recreated,
if not present in the kayobe network interface configuration. This
affects the Kayobe development environment, in which the configuration
does not touch the host's real network interfaces. That would lead to
DHCP leases not being renewed, and consequent loss of access to the
host.

This change fixes the issue by bumping to v1.13.1, which reverts the
change.

Change-Id: I03da729e1d05783af2e290237485f6035d2ddcdc
2022-03-10 09:18:15 +00:00
Michal Nasiadka
a20fc47f0c CI: Move to pytest-testinfra
Change-Id: I615707976454a91c8f6aecc5eda1852def7197d4
2022-03-07 14:32:48 +01:00
Zuul
678b915aff Merge "CI: remove qemu-utils installation" 2022-03-03 18:00:32 +00:00
Zuul
f3f4dbb78c Merge "Add support for root filesystem UUID customisation" 2022-03-03 15:49:47 +00:00