2373 Commits

Author SHA1 Message Date
82d321c6fd ensure-sphix: upgrade setuptools in venv
With [1] pip was ensured to be latest, but for
cases like ubuntu bionic which has too old
setuptools(39.0.1) and that doesn't support version
from file, the support only available with
setuptools-39.2.0[2].

This patch proposes to upgrade setuptools
along with pip to support dependencies which
set's version in a file. Other option is to
pass option --use-deprecated legacy-resolver
to pip install or downgrade pip to working version.

This will resolve issue with translation jobs
in neutron which fails during installation of
pyroute2[3].

[1] https://review.opendev.org/c/zuul/zuul-jobs/+/828441
[2] https://github.com/pypa/setuptools/pull/1359/commits/a960ee1c3
[3] https://zuul.openstack.org/builds?job_name=propose-translation-update&project=openstack%2Fneutron

Change-Id: I23475a48fa9cc924962f2f4b5edc7aeb5dd6261c
2022-04-13 20:35:46 +05:30
Zuul
17c41cbff3 Merge "build-docker-image: Fix buildset registry pushes" 2022-04-11 23:23:31 +00:00
Zuul
9761561e5a Merge "Add upload-logs-ibm role" 2022-04-11 22:30:23 +00:00
James E. Blair
a8b4bc6ff3 Add upload-logs-ibm role
This role uploads logs to IBM Cloud object storage.

Change-Id: Ibe1131f863a64051b427fcb03b126b1577c4843a
2022-04-11 14:20:49 -07:00
Zuul
3eb75563fe Merge "Fix a bug in s3 log uploader with .gz files" 2022-04-11 20:04:48 +00:00
Mohammed Naser
613a6ce85f build-docker-image: Fix buildset registry pushes
If we build an image with `docker_registry` set, it will fail to
tag the image for the buildset registry since we do not prefix the
image with `docker_registry` when retagging it (if set).

This patch resolves this by adding that prefix if it is set so
it will refer to an image that exists.

Change-Id: Iba29161156c8e8ff4f79a92771456cf105d780fe
2022-04-11 15:22:00 -04:00
Zuul
06b04a2111 Merge "run-buildset-registry: Drop extra install packages task" 2022-04-04 18:08:43 +00:00
Zuul
70c30231cd Merge "Add tox-py310 job" 2022-04-04 13:51:04 +00:00
Zuul
75b52f8774 Merge "ensure-pip: fix typo in ensure_pip_virtualenv_command documentation" 2022-04-04 07:26:18 +00:00
Zuul
bbd83b763f Merge "ensure-kubernetes: fix missing 02-crio.conf" 2022-04-04 06:24:04 +00:00
Jens Harbott
cd3c622abb Fix test jobs
- voluptuous dropped support for py27. see also [0]
- drop centos-8 jobs for which we no longer have images

[0] https://review.opendev.org/c/opendev/glean/+/836194

Change-Id: I6626a5a59625640d1c0b4e9026743441059177e0
2022-04-02 13:47:50 +02:00
Mohammed Naser
8d61aac55f run-buildset-registry: Drop extra install packages task
It seems like we dropped usage of Python's passlib at some point
which made the two lists similar.  Since Python 2 is been gone for
quite some tiem and the dependencies are the same anyways, we can
just drop that extra task and merge it into one.

Change-Id: Ifa3d8bd21346a560230fb6de60e985baaaee0a92
2022-03-24 20:01:37 -04:00
Mohammed Naser
65a3a8245b ensure-kubernetes: fix missing 02-crio.conf
There was some recent changes to Minikube that added a hard
dependency on that file being there which are not caught by
upstream testing since they don't test with the "none" VM
driver.

This adds the file to workaround the issue so that it can
complete the bootstrap of the cluster.

Change-Id: I3869350ae620721ea4e755e50facd0291a7772d3
2022-03-24 20:01:30 -04:00
Sorin Sbarnea
f5dbe71c49 Add tox-py310 job
Change-Id: Ib6a06893c0b4c7e5c988acf7315b17367b330d9e
2022-03-24 13:25:04 +01:00
Zuul
57df8b9d6d Merge "[multi-node-bridge] Allow to skip openvswitch installation" 2022-03-21 03:26:11 +00:00
Clark Boylan
21341d2d47 Fix encrypt files stat validation
The input to encrypt files may be a list of paths so our validation has
to evaluate and state each list entry separately. Without this we fail
beacuse the list of paths is treated like a single path and that does
not stat resulting in early failure.

Change-Id: Ibe3f6b162c3adad928708464ea03ddded2f4c683
2022-03-17 10:56:13 -07:00
Jeremy Stanley
e88f592c4d encrypt-file: roll back extended file stat
Change I8e7bc38c68c224795630b90a1b989098a7661491 switched the
"Validate input file" task to use a path stat, but when a list of
files is passed into encrypt_file this fails because the list is
treated as a string. Switch back to the more naive "is undefined"
check until it can be properly redone with a loop or similar and
appropriate tests added.

Change-Id: I83e665bc890aec42462776e0079457bb4506552f
2022-03-17 16:17:01 +00:00
Zuul
096e8bd9c1 Merge "encrypt-file: always import expiring keys" 2022-03-17 03:15:57 +00:00
Zuul
cebc28432c Merge "ensure-podman: add containernetworking-plugins" 2022-03-16 03:36:03 +00:00
8989684719 [multi-node-bridge] Allow to skip openvswitch installation
In neutron-multinode jobs where we use ovs/ovn from source,
we want to use multi-node-bridge role only for bridge
configuration but not for ovs installation. In the job
we install ovn and openvswitch before calling this role to
configure the bridges.

Adding a role var 'install_ovs' to allow skipping ovs
installation and service start, it's default to true so
no change in current behavior of the role.

It's an alternative approach to [1].

[1] https://review.opendev.org/c/zuul/zuul-jobs/+/762650

Related-Bug: #1904117
Change-Id: I64942679520681bdf7f953c0a3c7fc0d13e77856
2022-03-09 18:23:41 +05:30
Zuul
00c8ae9204 Merge "[ensure-python] Improve check for CentOS/RHEL 9 packages" 2022-03-03 17:37:39 +00:00
Szymon Datko
fab437bbfe [ensure-python] Improve check for CentOS/RHEL 9 packages
The value of python_version is float by default.
The existing comparison will only work for strings.
This commit changes it to work in both cases.

Change-Id: Ib13c33f38b611a38525d2803474b4ae1e05413f7
2022-03-02 13:16:58 +00:00
Zuul
a0a0b00166 Merge "Fix ensure-pip test on Debian Buster" 2022-03-02 04:17:07 +00:00
Dr. Jens Harbott
024f377921 Fix ensure-pip test on Debian Buster
Our wheel mirrors contain pluggy-1.0.0-py2.py3-none-any.whl which pip
sees as a sign to install it under py2.7 although the wheel is only
meant for >=py3.6. Add a cap to the test command to install some older
version instead.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I32d2c29988f92124e7dcdf462b2dd07cfd68b4d2
2022-03-01 21:10:28 +01:00
Zuul
b67fc88b0c Merge "[ensure-python] Fix for CentOS/RHEL 9 packages" 2022-03-01 16:05:50 +00:00
Ian Wienand
92105159e7 ensure-podman: add containernetworking-plugins
I think this package is technically optional because you can run as
non-root without it.  But since this is a generic role that is just
supposed to give you a working podman, include it.

Somehow we forgot to remove the podman centos-8 (not stream) job in
previous cleanups; remove it here.

Change-Id: I0a87545d29a5ea49a2d9d98c09467c89e7232d4b
2022-03-01 12:44:50 +11:00
Ian Wienand
c20a2435a6 configure-mirrors: fix stripped newline
I74b9de7092f182c942a58ac7a46b9fbd791889de hit a common ansible gotcha
where it likes to strip the trailing newline after a {% endif %}.
This has resulted in invalid lines in our sources.list.

Unfortunately we miss this because it still exits with 0.  Add a
simple test looking for warning output.

Change-Id: I46d393a5e67d10a52c4dcca803176ff368a4b5bd
2022-03-01 11:10:30 +11:00
Ian Wienand
4d4e386bad ensure-pip: fix typo in ensure_pip_virtualenv_command documentation
Fix a typo in the documentation for this variable.

Change-Id: I8da3d5e512e8fddccf6bd6a39078579c7433fba9
2022-02-28 14:52:30 +11:00
Ian Wienand
e160f59e0e encrypt-file: become when installing packages
This needs to become: yes to install packages

Change-Id: I4632d48253b78852200f8fa014567ef4e7f64676
2022-02-21 03:51:33 +00:00
Zuul
69cf896ae2 Merge "encrypt-file : role to encrypt a file" 2022-02-20 21:17:40 +00:00
Zuul
fcd6baae7c Merge "Add patch to list of build dependencies" 2022-02-18 21:36:02 +00:00
Ian Wienand
f2cfa6c356 encrypt-file: always import expiring keys
If a key is in our existing keyring has an expiry date (or, has
expired), always import the provided value again as it may be
refreshing the value.

Add an expiring key to test the matching; although on an ephemeral
node we're importing always anyway.

Also update the file test to a stat -- this is better than a weird
error from gpg later.

Change-Id: I8e7bc38c68c224795630b90a1b989098a7661491
2022-02-19 08:05:50 +11:00
Ian Wienand
ad7093c17b encrypt-file : role to encrypt a file
This is a role that takes some ASCII gpg keys, and encrypts a file
with them.

Change-Id: If2fe7921ff051a1c5d0589f5e32fba26d30ae96c
2022-02-19 08:05:40 +11:00
jgilaber
1f3920f6eb Add patch to list of build dependencies
This commit adds the patch package to the list of python build
dependencies for pyenv. This is not always necessary, but some python
versions (for example 3.7.12) require a patch to be applied before
installing and the process fails if the patch package is not installed.

Change-Id: I605fc1c9255e98692a6a901cfc509b7fdb873ae1
2022-02-17 10:16:25 +01:00
Jonathan Rosser
5d01b68574 Allow some configure-mirrors repositories to be disabled
Out-of-the-box installs of centos-8 do not enable the PowerTools
or HighAvailability repositories. Debian/Ubuntu do not enable
the backports repository by default.

Having these repos enabled by default in CI has led to merging
broken code for OpenStack deployment tooling which is attempting
to also manage the presence/absence of these repositories. It
is challenging to remove these repositories on the running node
because the repo URL (rather than just the name) is required as
input to the apt_repository and yum_repository ansible modules.

This patch adds a role default variable to configure-mirrors to
allow a job to opt out of these extra repositories. The default
is set to 'True' to allow existing jobs to work as before.

Change-Id: I74b9de7092f182c942a58ac7a46b9fbd791889de
2022-02-16 20:39:31 +00:00
Szymon Datko
272f091cf9 [ensure-python] Fix for CentOS/RHEL 9 packages
This commit introduces a conditional for RPM package name
when running on a recent CentOS/RHEL system.

Change-Id: I716cc97a8dace02c874d2e39dc90d6a42808ebf6
2022-02-16 14:27:14 +01:00
Ian Wienand
e3b7e0dcf6 ensure-sphinx: upgrade pip
Change I98c3da5b02a4ac7fb9d7bd8e00170762e77b9f40 caused a regression,
because previously virtualenv would be pulling the latest pip, whereas
"pip -m venv" is using the system-vendored pip, which is older.

Upgrade pip to the latest in the sphinx environment to maintain the
status-quo.

Change-Id: I1004c2727379f73eafc8b32a3e14842200ad342c
2022-02-09 11:44:06 +11:00
Zuul
9d69672a57 Merge "Make CentOS9 jobs voting" 2022-02-08 09:19:52 +00:00
Ian Wienand
239ace0ebf ensure-virtualenv: Don't support on CentOS 9-stream
It looks like 9-stream don't want to support virtualenv.  Fail the
ensure-virtualenv role on this platform, and account for this in
testing.

People should use ensure-pip and venv which is portable everywhere.

Change-Id: Ifae93c1eeb96792aa26a624574d595d77cb58c4b
2022-02-04 14:45:11 +11:00
Ian Wienand
ab4042e60a ensure-sphinx: Use python3
All our platforms are Python 3 now, so we can simplify this.  We can
remove the virtualenv dependency as ensure-pip installs venv and sets
variables for us to use it when making a virtualenv.

Change-Id: I98c3da5b02a4ac7fb9d7bd8e00170762e77b9f40
2022-02-04 08:03:25 +11:00
Zuul
9b89ebcebd Merge "Change RDO train repository for Centos 8 stream" 2022-02-03 01:00:29 +00:00
Zuul
f2f18a29f1 Merge "Add CentOS Stream 9 to configure-mirrors role" 2022-02-03 01:00:25 +00:00
Alfredo Moralejo
62032362e2 Make CentOS9 jobs voting
After mergint [1] we should be able to make all the CS9 jobs voting.

[1] https://review.opendev.org/c/zuul/zuul-jobs/+/826603

Change-Id: Ifeb334794d63ece8640d1f6cbd03885d2bbb7c56
2022-02-02 17:28:11 +01:00
Dariusz Smigiel
1f76342ccf Pull correct upstream pip version
Recent release of pip removed support for Python 3.6 [1]

Configure default url for pip, depending on supported
Python version.

[1]: https://pip.pypa.io/en/stable/news/#v22-0

Change-Id: I01faeba1dff864de31b3bbf9ddbe0a4d454fdbe9
2022-02-01 08:33:58 -08:00
Alfredo Moralejo
64fd772a32 Change RDO train repository for Centos 8 stream
After removing Centos 8 repositories [1], the repository has been
set to buildlogs server, which provides testing packages.
This commit is switching back repository to mirror.centos.org once we
have openvswitch in centos network and AFS mirrors.

[1] https://www.centos.org/centos-linux-eol/

Change-Id: I6e0dd26312c593a5c7fe5525a733be8e26faec9d
2022-02-01 11:39:09 +01:00
Alfredo Moralejo
62cc9b24e1 Install OVS from RDO Train Testing repository for CS8
CentOS Linux 8 repos have been removed from centos mirrors. We were
using that repo to install OVS in multi-node from train repo.

This patch is moving multi-node rolo to install ovs from the RDO testing
repo for train until we have it pushed to the official mirrors for CS8.

Change-Id: Id57e8e90903a0d468c6d3094fbbaf4840a368e2a
2022-01-31 11:11:58 +01:00
Alfredo Moralejo
25f110c399 Add CentOS Stream 9 to configure-mirrors role
So that we use AFS mirrors for CS9 nodes.

Closes-Bug: #1959181
Change-Id: I2da3a5e8a45598c6b3ded132ea61b86b4480e262
2022-01-27 15:05:46 +01:00
Alfredo Moralejo
6c3e76af1a pin oauthlib version for python2.7
oauthlib 3.1.1 has removed support for python2 but it's being pulled
into tox-py27 jobs.

Change-Id: I39ac626219d94bbb74f77e460a18da5721ba25e7
2022-01-27 14:35:06 +01:00
Zuul
13167d7a21 Merge "Introduce iptables_package var" 2022-01-19 15:59:39 +00:00
Chandan Kumar (raukadah)
6aa268834c Introduce iptables_package var
on RHEL-9 there is no iptables package, we need to install
iptables-nft package here.

In CentOS Stream-9 and Fedora-34 onwards iptables-nft package
is available.[1]

But we also need to support other distros, so we are introducing
iptables_packages var and distro specific var files (having different
name) for installing iptables package.

[1]. https://pkgs.org/download/iptables-nft

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I8d5d3182996fc1e83b7f4f7eb99cf4c347d6ef1f
2022-01-19 15:11:08 +05:30