2308 Commits

Author SHA1 Message Date
James E. Blair
59d7af0e67 Add per-build WinRM cert generation
This adds roles that, similar to add-build-sshkey, create a per-build
WinRM certificate, install it on remote windows nodes, and then switch
to using the certificate in Ansible for authentication.  A second role
is included which can clean up the cert which is useful for static
nodes.

Since winrm certificates must be acessible within the bubblewrap
container, these roles can be used to restrict the system-wide winrm
cert to trusted playbooks while untrusted playbooks will only have access
to the per-build cert (with appropriate configuration of the executor).

Change-Id: I4efe25594c2f543886a000aa02fb0a38683a43cb
2022-04-13 15:04:51 -07:00
Zuul
57df8b9d6d Merge "[multi-node-bridge] Allow to skip openvswitch installation" 2022-03-21 03:26:11 +00:00
Clark Boylan
21341d2d47 Fix encrypt files stat validation
The input to encrypt files may be a list of paths so our validation has
to evaluate and state each list entry separately. Without this we fail
beacuse the list of paths is treated like a single path and that does
not stat resulting in early failure.

Change-Id: Ibe3f6b162c3adad928708464ea03ddded2f4c683
2022-03-17 10:56:13 -07:00
Jeremy Stanley
e88f592c4d encrypt-file: roll back extended file stat
Change I8e7bc38c68c224795630b90a1b989098a7661491 switched the
"Validate input file" task to use a path stat, but when a list of
files is passed into encrypt_file this fails because the list is
treated as a string. Switch back to the more naive "is undefined"
check until it can be properly redone with a loop or similar and
appropriate tests added.

Change-Id: I83e665bc890aec42462776e0079457bb4506552f
2022-03-17 16:17:01 +00:00
Zuul
096e8bd9c1 Merge "encrypt-file: always import expiring keys" 2022-03-17 03:15:57 +00:00
Zuul
cebc28432c Merge "ensure-podman: add containernetworking-plugins" 2022-03-16 03:36:03 +00:00
8989684719 [multi-node-bridge] Allow to skip openvswitch installation
In neutron-multinode jobs where we use ovs/ovn from source,
we want to use multi-node-bridge role only for bridge
configuration but not for ovs installation. In the job
we install ovn and openvswitch before calling this role to
configure the bridges.

Adding a role var 'install_ovs' to allow skipping ovs
installation and service start, it's default to true so
no change in current behavior of the role.

It's an alternative approach to [1].

[1] https://review.opendev.org/c/zuul/zuul-jobs/+/762650

Related-Bug: #1904117
Change-Id: I64942679520681bdf7f953c0a3c7fc0d13e77856
2022-03-09 18:23:41 +05:30
Zuul
00c8ae9204 Merge "[ensure-python] Improve check for CentOS/RHEL 9 packages" 2022-03-03 17:37:39 +00:00
Szymon Datko
fab437bbfe [ensure-python] Improve check for CentOS/RHEL 9 packages
The value of python_version is float by default.
The existing comparison will only work for strings.
This commit changes it to work in both cases.

Change-Id: Ib13c33f38b611a38525d2803474b4ae1e05413f7
2022-03-02 13:16:58 +00:00
Zuul
a0a0b00166 Merge "Fix ensure-pip test on Debian Buster" 2022-03-02 04:17:07 +00:00
Dr. Jens Harbott
024f377921 Fix ensure-pip test on Debian Buster
Our wheel mirrors contain pluggy-1.0.0-py2.py3-none-any.whl which pip
sees as a sign to install it under py2.7 although the wheel is only
meant for >=py3.6. Add a cap to the test command to install some older
version instead.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I32d2c29988f92124e7dcdf462b2dd07cfd68b4d2
2022-03-01 21:10:28 +01:00
Zuul
b67fc88b0c Merge "[ensure-python] Fix for CentOS/RHEL 9 packages" 2022-03-01 16:05:50 +00:00
Ian Wienand
92105159e7 ensure-podman: add containernetworking-plugins
I think this package is technically optional because you can run as
non-root without it.  But since this is a generic role that is just
supposed to give you a working podman, include it.

Somehow we forgot to remove the podman centos-8 (not stream) job in
previous cleanups; remove it here.

Change-Id: I0a87545d29a5ea49a2d9d98c09467c89e7232d4b
2022-03-01 12:44:50 +11:00
Ian Wienand
c20a2435a6 configure-mirrors: fix stripped newline
I74b9de7092f182c942a58ac7a46b9fbd791889de hit a common ansible gotcha
where it likes to strip the trailing newline after a {% endif %}.
This has resulted in invalid lines in our sources.list.

Unfortunately we miss this because it still exits with 0.  Add a
simple test looking for warning output.

Change-Id: I46d393a5e67d10a52c4dcca803176ff368a4b5bd
2022-03-01 11:10:30 +11:00
Ian Wienand
e160f59e0e encrypt-file: become when installing packages
This needs to become: yes to install packages

Change-Id: I4632d48253b78852200f8fa014567ef4e7f64676
2022-02-21 03:51:33 +00:00
Zuul
69cf896ae2 Merge "encrypt-file : role to encrypt a file" 2022-02-20 21:17:40 +00:00
Zuul
fcd6baae7c Merge "Add patch to list of build dependencies" 2022-02-18 21:36:02 +00:00
Ian Wienand
f2cfa6c356 encrypt-file: always import expiring keys
If a key is in our existing keyring has an expiry date (or, has
expired), always import the provided value again as it may be
refreshing the value.

Add an expiring key to test the matching; although on an ephemeral
node we're importing always anyway.

Also update the file test to a stat -- this is better than a weird
error from gpg later.

Change-Id: I8e7bc38c68c224795630b90a1b989098a7661491
2022-02-19 08:05:50 +11:00
Ian Wienand
ad7093c17b encrypt-file : role to encrypt a file
This is a role that takes some ASCII gpg keys, and encrypts a file
with them.

Change-Id: If2fe7921ff051a1c5d0589f5e32fba26d30ae96c
2022-02-19 08:05:40 +11:00
jgilaber
1f3920f6eb Add patch to list of build dependencies
This commit adds the patch package to the list of python build
dependencies for pyenv. This is not always necessary, but some python
versions (for example 3.7.12) require a patch to be applied before
installing and the process fails if the patch package is not installed.

Change-Id: I605fc1c9255e98692a6a901cfc509b7fdb873ae1
2022-02-17 10:16:25 +01:00
Jonathan Rosser
5d01b68574 Allow some configure-mirrors repositories to be disabled
Out-of-the-box installs of centos-8 do not enable the PowerTools
or HighAvailability repositories. Debian/Ubuntu do not enable
the backports repository by default.

Having these repos enabled by default in CI has led to merging
broken code for OpenStack deployment tooling which is attempting
to also manage the presence/absence of these repositories. It
is challenging to remove these repositories on the running node
because the repo URL (rather than just the name) is required as
input to the apt_repository and yum_repository ansible modules.

This patch adds a role default variable to configure-mirrors to
allow a job to opt out of these extra repositories. The default
is set to 'True' to allow existing jobs to work as before.

Change-Id: I74b9de7092f182c942a58ac7a46b9fbd791889de
2022-02-16 20:39:31 +00:00
Szymon Datko
272f091cf9 [ensure-python] Fix for CentOS/RHEL 9 packages
This commit introduces a conditional for RPM package name
when running on a recent CentOS/RHEL system.

Change-Id: I716cc97a8dace02c874d2e39dc90d6a42808ebf6
2022-02-16 14:27:14 +01:00
Ian Wienand
e3b7e0dcf6 ensure-sphinx: upgrade pip
Change I98c3da5b02a4ac7fb9d7bd8e00170762e77b9f40 caused a regression,
because previously virtualenv would be pulling the latest pip, whereas
"pip -m venv" is using the system-vendored pip, which is older.

Upgrade pip to the latest in the sphinx environment to maintain the
status-quo.

Change-Id: I1004c2727379f73eafc8b32a3e14842200ad342c
2022-02-09 11:44:06 +11:00
Zuul
9d69672a57 Merge "Make CentOS9 jobs voting" 2022-02-08 09:19:52 +00:00
Ian Wienand
239ace0ebf ensure-virtualenv: Don't support on CentOS 9-stream
It looks like 9-stream don't want to support virtualenv.  Fail the
ensure-virtualenv role on this platform, and account for this in
testing.

People should use ensure-pip and venv which is portable everywhere.

Change-Id: Ifae93c1eeb96792aa26a624574d595d77cb58c4b
2022-02-04 14:45:11 +11:00
Ian Wienand
ab4042e60a ensure-sphinx: Use python3
All our platforms are Python 3 now, so we can simplify this.  We can
remove the virtualenv dependency as ensure-pip installs venv and sets
variables for us to use it when making a virtualenv.

Change-Id: I98c3da5b02a4ac7fb9d7bd8e00170762e77b9f40
2022-02-04 08:03:25 +11:00
Zuul
9b89ebcebd Merge "Change RDO train repository for Centos 8 stream" 2022-02-03 01:00:29 +00:00
Zuul
f2f18a29f1 Merge "Add CentOS Stream 9 to configure-mirrors role" 2022-02-03 01:00:25 +00:00
Alfredo Moralejo
62032362e2 Make CentOS9 jobs voting
After mergint [1] we should be able to make all the CS9 jobs voting.

[1] https://review.opendev.org/c/zuul/zuul-jobs/+/826603

Change-Id: Ifeb334794d63ece8640d1f6cbd03885d2bbb7c56
2022-02-02 17:28:11 +01:00
Dariusz Smigiel
1f76342ccf Pull correct upstream pip version
Recent release of pip removed support for Python 3.6 [1]

Configure default url for pip, depending on supported
Python version.

[1]: https://pip.pypa.io/en/stable/news/#v22-0

Change-Id: I01faeba1dff864de31b3bbf9ddbe0a4d454fdbe9
2022-02-01 08:33:58 -08:00
Alfredo Moralejo
64fd772a32 Change RDO train repository for Centos 8 stream
After removing Centos 8 repositories [1], the repository has been
set to buildlogs server, which provides testing packages.
This commit is switching back repository to mirror.centos.org once we
have openvswitch in centos network and AFS mirrors.

[1] https://www.centos.org/centos-linux-eol/

Change-Id: I6e0dd26312c593a5c7fe5525a733be8e26faec9d
2022-02-01 11:39:09 +01:00
Alfredo Moralejo
62cc9b24e1 Install OVS from RDO Train Testing repository for CS8
CentOS Linux 8 repos have been removed from centos mirrors. We were
using that repo to install OVS in multi-node from train repo.

This patch is moving multi-node rolo to install ovs from the RDO testing
repo for train until we have it pushed to the official mirrors for CS8.

Change-Id: Id57e8e90903a0d468c6d3094fbbaf4840a368e2a
2022-01-31 11:11:58 +01:00
Alfredo Moralejo
25f110c399 Add CentOS Stream 9 to configure-mirrors role
So that we use AFS mirrors for CS9 nodes.

Closes-Bug: #1959181
Change-Id: I2da3a5e8a45598c6b3ded132ea61b86b4480e262
2022-01-27 15:05:46 +01:00
Alfredo Moralejo
6c3e76af1a pin oauthlib version for python2.7
oauthlib 3.1.1 has removed support for python2 but it's being pulled
into tox-py27 jobs.

Change-Id: I39ac626219d94bbb74f77e460a18da5721ba25e7
2022-01-27 14:35:06 +01:00
Zuul
13167d7a21 Merge "Introduce iptables_package var" 2022-01-19 15:59:39 +00:00
Chandan Kumar (raukadah)
6aa268834c Introduce iptables_package var
on RHEL-9 there is no iptables package, we need to install
iptables-nft package here.

In CentOS Stream-9 and Fedora-34 onwards iptables-nft package
is available.[1]

But we also need to support other distros, so we are introducing
iptables_packages var and distro specific var files (having different
name) for installing iptables package.

[1]. https://pkgs.org/download/iptables-nft

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I8d5d3182996fc1e83b7f4f7eb99cf4c347d6ef1f
2022-01-19 15:11:08 +05:30
Szymon Datko
ced6cd132f Add support for RPM packages
This commit extends the installation of Python interpreter
from system packages to also support RPM packages (assuming
the playbook is run in an environment with proper repositories).

Also the obsolete note about supporting only Debian distribution
is removed, as the `ensure-python` role already supports other
platforms (via pyenv and stow).

Change-Id: Id6ef8ec3537952348e0a7b233bd26b57613da327
2022-01-18 16:00:22 +01:00
Clark Boylan
88c1c731e6 Remove centos-8 testing
CentOS 8 EOL'd at the end of 2021. OpenDev is beginning the process of
removing the image and starting with some job cleanups like the job
cleanups in this change. Users can use CentOS 8 Stream or some other
RHEL alike.

Change-Id: I9dab417b28a0503e1409e70953ed275f2c6f8c30
2022-01-11 12:52:08 -08:00
Clark Boylan
9d51ae5ea8 Update ansible-lint pin to fix rich dep
Older ansible-lint didn't pin the 'rich' dep and that dep updated and
broke ansible-lint. Newer ansible-lint fixed this. Update ansible-lint
to get around this problem.

Change-Id: Ic798398ab68239d4f13cb806a92038fd4946d25a
2022-01-10 11:44:21 -08:00
James E. Blair
90c427d630 Switch docs theme to RTD
To match change I2870450ffd02f55509fcc1297d050b09deafbfb9 in Zuul.

This does not use the versioning feature due to the nature of this repo.

This also corrects a reference which is now an error.

Change-Id: Ia1d31df932b447f11bc588925de9974d4f6dfc7d
2021-12-16 06:48:49 -08:00
Sagi Shnaidman
9a29f54cea Print version of installed podman
Change-Id: Ic3a5d43b726d638d22a9916f128bbcded1b8065a
2021-12-11 11:21:27 +00:00
Zuul
eb8106a155 Merge "Add fips version of jobs needed for OpenStack" 2021-12-10 15:40:09 +00:00
Clark Boylan
0206d6671a Try to fix broken stestr command discovery
When processing subunit streams we attempt to discover where the
(s)testr commands are located. For some reason the output of our scripts
that do this very occasionally emit a newline before the command path.
When this happens we were using the blank line as the command path which
fails as that empty command results in our command arguments being
interpreted as the command path.

Attempt to address this by trimming the stdout instead of taking the
first line. This should remove the leading and trailing newlines and
give us only the command itself.

Change-Id: Id651e019cf3d0b7ab37fdf9df04be249ea7f7af6
2021-12-08 10:51:55 -08:00
Clark Boylan
c697816786 Remove tumbleweed jobs
OpenDev doesn't currently have an image built for tumblweed. Rather than
try and fix this image up and get it running again OpenDev has proposed
it simply be removed. In preparation for that and to avoid running jobs
for a system that doesn't currently have an image we remove the jobs
here.

It was hoped that tumbleweed would be a good forward looking platform
but it seems to need a lot more care than we are able to give it.
Additionally Zuul is probably the major consumer of the platform and it
doesn't see much consumption by other projects.

Change-Id: Iad351b6da1f08183bc4c49c049c303a9491af844
2021-12-08 10:00:06 -08:00
Zuul
538abe9ac4 Merge "ensure-dstat-graph: clone from opendev" 2021-12-07 20:54:39 +00:00
Ian Wienand
7493032793 ensure-dstat-graph: clone from opendev
We have integrated these changes into a new opendev.org managed
repository.

Change-Id: Ie163ae95efe572e60a5b8b6444ecc7a35011ece8
2021-12-07 13:04:11 +11:00
Ian Wienand
219afd0227 Update Fedora 34 to 35
Change-Id: I84f9e052f488217f8a63791556de99469451a1d9
2021-11-22 10:32:59 +11:00
Zuul
0b4bb29b5a Merge "Revert "Add no_log to the log uploading module parameters"" 2021-11-16 19:18:20 +00:00
Artem Goncharov
2f1951c23c Revert "Add no_log to the log uploading module parameters"
This reverts commit 10ee6ebf016fa00c9eb8cd15a09832982b08fe48.

Reason for revert: log url get broken

Change-Id: Ifb8db77c2534a92bbb19fde1bfbf31562ed04daf
2021-11-16 19:06:33 +00:00
Zuul
88e8bf4b2e Merge "Add no_log to the log uploading module parameters" 2021-11-16 16:45:20 +00:00