1803 Commits

Author SHA1 Message Date
Ian Wienand
ab4042e60a ensure-sphinx: Use python3
All our platforms are Python 3 now, so we can simplify this.  We can
remove the virtualenv dependency as ensure-pip installs venv and sets
variables for us to use it when making a virtualenv.

Change-Id: I98c3da5b02a4ac7fb9d7bd8e00170762e77b9f40
2022-02-04 08:03:25 +11:00
Zuul
9b89ebcebd Merge "Change RDO train repository for Centos 8 stream" 2022-02-03 01:00:29 +00:00
Zuul
f2f18a29f1 Merge "Add CentOS Stream 9 to configure-mirrors role" 2022-02-03 01:00:25 +00:00
Dariusz Smigiel
1f76342ccf Pull correct upstream pip version
Recent release of pip removed support for Python 3.6 [1]

Configure default url for pip, depending on supported
Python version.

[1]: https://pip.pypa.io/en/stable/news/#v22-0

Change-Id: I01faeba1dff864de31b3bbf9ddbe0a4d454fdbe9
2022-02-01 08:33:58 -08:00
Alfredo Moralejo
64fd772a32 Change RDO train repository for Centos 8 stream
After removing Centos 8 repositories [1], the repository has been
set to buildlogs server, which provides testing packages.
This commit is switching back repository to mirror.centos.org once we
have openvswitch in centos network and AFS mirrors.

[1] https://www.centos.org/centos-linux-eol/

Change-Id: I6e0dd26312c593a5c7fe5525a733be8e26faec9d
2022-02-01 11:39:09 +01:00
Alfredo Moralejo
62cc9b24e1 Install OVS from RDO Train Testing repository for CS8
CentOS Linux 8 repos have been removed from centos mirrors. We were
using that repo to install OVS in multi-node from train repo.

This patch is moving multi-node rolo to install ovs from the RDO testing
repo for train until we have it pushed to the official mirrors for CS8.

Change-Id: Id57e8e90903a0d468c6d3094fbbaf4840a368e2a
2022-01-31 11:11:58 +01:00
Alfredo Moralejo
25f110c399 Add CentOS Stream 9 to configure-mirrors role
So that we use AFS mirrors for CS9 nodes.

Closes-Bug: #1959181
Change-Id: I2da3a5e8a45598c6b3ded132ea61b86b4480e262
2022-01-27 15:05:46 +01:00
Zuul
13167d7a21 Merge "Introduce iptables_package var" 2022-01-19 15:59:39 +00:00
Chandan Kumar (raukadah)
6aa268834c Introduce iptables_package var
on RHEL-9 there is no iptables package, we need to install
iptables-nft package here.

In CentOS Stream-9 and Fedora-34 onwards iptables-nft package
is available.[1]

But we also need to support other distros, so we are introducing
iptables_packages var and distro specific var files (having different
name) for installing iptables package.

[1]. https://pkgs.org/download/iptables-nft

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I8d5d3182996fc1e83b7f4f7eb99cf4c347d6ef1f
2022-01-19 15:11:08 +05:30
Szymon Datko
ced6cd132f Add support for RPM packages
This commit extends the installation of Python interpreter
from system packages to also support RPM packages (assuming
the playbook is run in an environment with proper repositories).

Also the obsolete note about supporting only Debian distribution
is removed, as the `ensure-python` role already supports other
platforms (via pyenv and stow).

Change-Id: Id6ef8ec3537952348e0a7b233bd26b57613da327
2022-01-18 16:00:22 +01:00
James E. Blair
90c427d630 Switch docs theme to RTD
To match change I2870450ffd02f55509fcc1297d050b09deafbfb9 in Zuul.

This does not use the versioning feature due to the nature of this repo.

This also corrects a reference which is now an error.

Change-Id: Ia1d31df932b447f11bc588925de9974d4f6dfc7d
2021-12-16 06:48:49 -08:00
Sagi Shnaidman
9a29f54cea Print version of installed podman
Change-Id: Ic3a5d43b726d638d22a9916f128bbcded1b8065a
2021-12-11 11:21:27 +00:00
Clark Boylan
0206d6671a Try to fix broken stestr command discovery
When processing subunit streams we attempt to discover where the
(s)testr commands are located. For some reason the output of our scripts
that do this very occasionally emit a newline before the command path.
When this happens we were using the blank line as the command path which
fails as that empty command results in our command arguments being
interpreted as the command path.

Attempt to address this by trimming the stdout instead of taking the
first line. This should remove the leading and trailing newlines and
give us only the command itself.

Change-Id: Id651e019cf3d0b7ab37fdf9df04be249ea7f7af6
2021-12-08 10:51:55 -08:00
Ian Wienand
7493032793 ensure-dstat-graph: clone from opendev
We have integrated these changes into a new opendev.org managed
repository.

Change-Id: Ie163ae95efe572e60a5b8b6444ecc7a35011ece8
2021-12-07 13:04:11 +11:00
Zuul
0b4bb29b5a Merge "Revert "Add no_log to the log uploading module parameters"" 2021-11-16 19:18:20 +00:00
Artem Goncharov
2f1951c23c Revert "Add no_log to the log uploading module parameters"
This reverts commit 10ee6ebf016fa00c9eb8cd15a09832982b08fe48.

Reason for revert: log url get broken

Change-Id: Ifb8db77c2534a92bbb19fde1bfbf31562ed04daf
2021-11-16 19:06:33 +00:00
Zuul
88e8bf4b2e Merge "Add no_log to the log uploading module parameters" 2021-11-16 16:45:20 +00:00
Zuul
6ff50fb0d4 Merge "Fix zuul-jobs-test-upload-logs-s3 job" 2021-11-16 16:34:48 +00:00
Artem Goncharov
10ee6ebf01 Add no_log to the log uploading module parameters
Use no_log parameter for potentially sensitive log uploading modules

Change-Id: I6264f1b028db0e4f1f74ae557847c146c00295da
2021-11-16 16:29:31 +00:00
Artem Goncharov
e7d7575eaf Fix zuul-jobs-test-upload-logs-s3 job
Since the security update we are not able to modify
zuul.executor.log_root variable in the role testing.
Replace this with explicit log folder creation on the test node.

Change-Id: Ie6cff6f9e9c5594167ddda6cd345f9c3d9f2f470
2021-11-16 16:20:33 +00:00
Gonéri Le Bouder
3cba437a15
enable-fips: Fedora also support FIPS
Fedora supports FIPS. There is no version check because the feature is
available on all the supported Fedora releases.

Change-Id: I924fb565a4d70e7c93a215e9e0a5b2b80bced52a
2021-10-28 15:20:59 -04:00
Gonéri Le Bouder
72581dcaac
use-buildset-registry: remove useless import
The module was not used and Python does not provide any `test` module since
py39.

Change-Id: Ia69a632bd4fff05da644c58d557912bc877a8135
2021-10-28 12:48:06 -04:00
Zuul
0bfa518718 Merge "ensure-dstat-graph: pull updated branch" 2021-10-25 23:31:59 +00:00
Zuul
9ff0c50c6e Merge "ensure-zookeeper: eliminate the tmpfs size limit" 2021-10-24 23:14:42 +00:00
James E. Blair
5a83a49dac ensure-zookeeper: eliminate the tmpfs size limit
Zuul's use of this role has exceeded the 500M cap.  Let's go ahead
and remove it since it's arbitrary anyway, and Zuul is likely the
primary user of the role.

Change-Id: Ib0bc72f6c909e44bc04af7a8fbe40ef8415addcf
2021-10-22 15:32:03 -07:00
James E. Blair
aeeb946ebf Update dstat to support bionic and others
Change-Id: Ida88ebbc6c169878adbb8e4f06da05eac813a04b
2021-10-22 12:51:41 -07:00
Ian Wienand
94d7d42f14 ensure-dstat-graph: pull updated branch
This updated branch fixes a few issues and should ensure display with
pcp-dstat (I9234b226242f145db9c235797649202aa530ec74)

Add an override so we pull it

Change-Id: I0d5e1b567c364a9e6c7aa0b95de17abffaef0434
2021-10-22 18:31:10 +11:00
James E. Blair
61d00fec6f Use pcp instead of dstand and return dstat graph artifact
Dstat is dead, but pcp supplies a compatible interface.

Installation can be unreliable, so let's ignore failures since
this isn't generally critical.

Change-Id: I9234b226242f145db9c235797649202aa530ec74
2021-10-21 15:39:54 -07:00
Zuul
984147663c Merge "build-python-release: Add flag for whether to build a wheel or not" 2021-10-20 20:32:00 +00:00
Zuul
e3519ce469 Merge "multi-node-bridge: repos to install ovs in C9" 2021-10-20 12:45:36 +00:00
Sandeep Yadav
a795073493 multi-node-bridge: repos to install ovs in C9
With this patch, Adding repos to install openvswitch
in Centos9 stream.

For Centos9 stream nodes we need to enable 2 repos:-
* nfv repo - Ussuri+ openvswitch is shipped as part of NFV SIG
* rdo xena repo - rdo-openvswitch(which provides openvswitch wrapper)
is available in OpenStack repo

Closes-Bug: #1947665
Change-Id: If5a32b30637296a070f2e75ba8feb6aa45d8ecac
2021-10-20 11:17:41 +05:30
Ian Wienand
db23e0af70 ensure-podman: support Debian bullseye
The package is included in the latest stable, Debian bullseye.  Add an
install match, and test it.

Change-Id: I62d2582e8adf2bedb2ea7d41b4b22a48c9b2f274
2021-10-15 11:38:58 +11:00
Ian Wienand
72d4e506d5 intermediate-registry: handle socat warning out
Zuul switched to a new base image, and it seems the new socat puts out
a warning (something like

 ...  socat[489590] W ioctl(5, IOCTL_VM_SOCKETS_GET_LOCAL_CID, ...): Inappropriate ioctl for device

for reference).

Grep the output so we only get the line about what port it is
listening on.

Change-Id: I74fb86a9158b45e6601ee1fbc199ba80cd4991fe
2021-10-14 16:04:46 +11:00
Bhagyashri Shewale
84b86437c5 Handled TypeError while installing any sibling python packages
The below error is coming on mol-centos7 jobs while
installing sibling for python2 packeges on centos7

Error: TypeError: initial_value must be unicode or None, not str

This patch handled the TypeError while installing sibling
for python packeges for centos7.

Closes-Bug: #1946641
Change-Id: Ie8058cca92d099e50af19b95b4c417c5a665da0d
2021-10-13 12:52:03 +05:30
Gonéri Le Bouder
ad56b0b4eb
build-container-image: improve the reliabilty
build-container-image may ends up pushing large files. Network is not
always as reliable as we want. This commit ensures we retry several
times in case of failure.

Change-Id: Ieaf92a8c3531e24ab5e41783b540de1df806c02f
2021-10-08 11:38:11 -04:00
Zuul
d5e4d55c15 Merge "ensure-rust: rework global install" 2021-10-08 01:40:58 +00:00
Zuul
08e37ff140 Merge "Revert "Revert "Include tox_extra_args in tox siblings tasks""" 2021-10-06 17:31:10 +00:00
Zuul
bbf4c1bbaa Merge "ensure-twine: Avoid Reinstalling twine if present" 2021-10-04 20:58:04 +00:00
Zuul
ec6b89b0d4 Merge "ensure-tox: Avoid Reinstalling tox venv if present" 2021-10-04 20:48:06 +00:00
Ian Wienand
67ae84ce39 ensure-rust: rework global install
I think the secondary "rustup default" was intended to make sure the
Zuul user sets up to use the installed rust toolchain
(I32f9b285904a7036f9a80ada8a49fa9cf31b5163) but actually results in a
re-download of components and another local installation.  This isn't
really the intention, and also doubles the time spent installing.

From the linked comment, it seems like we're not doing our global
install correctly; even putting it in /usr doesn't avoid the need for
RUST_HOME to be set.  Take it's suggestion and install out-of-the-way
in /opt, use a small /usr/local/bin wrapper to call with correct env
vars set and then setup the installed global binary names to be called
via that.

Change-Id: I28ef747b809a17664305bfd9754022251390647b
2021-10-04 11:43:22 +11:00
Jeremy Stanley
c02c28a982 Revert "Revert "Include tox_extra_args in tox siblings tasks""
This reverts commit 3668acc723d5b54cdaf789842a3c6e236bb6adb7.

Change-Id: I643b82c7dc6fd78ff67678b56c639a6c45fd0061
2021-10-01 16:03:13 +00:00
Jeremy Stanley
1861faf567 More exact section matching for tox showconfig
With the original verbose output support in change
Iafeb88eaf9a596603ad4d2134a4574345d5189ab we looked for lines from
tox --showconfig output starting with an opening '[' but verbosity
also causes output from pip install activity to be included if a
tox.requires entry causes it to install packages before continuing
and these lines are prefixed by a process ID number wrapped in
brackets with the command string after that, which prematurely
triggered our search for the start of the INI content. Add a
stipulation that the first INI output line also end in ']' in order
to skip over those additional prepended lines.

Change-Id: If29e5a9abe3b92a145d87f5efc1b93350ea3908a
2021-10-01 16:03:12 +00:00
Jeremy Stanley
3668acc723 Revert "Include tox_extra_args in tox siblings tasks"
This reverts commit 8b1cc73ee30428517f9524624b9eb309e9b14c1f.

Apparently tox.requires adds lines like this to verbose showconfig
output:

    using tox-3.24.4 from /.../tox/__init__.py (pid 2919)

Which is confusing the current parser in the siblings module. Roll
this back while we work on a fix and regression test.

Change-Id: If3b1d48b36a5d32fddfdabd9c0ec1b81dd6453f2
2021-09-30 17:54:17 +00:00
Zuul
f0d72b612b Merge "Include tox_extra_args in tox siblings tasks" 2021-09-30 15:51:42 +00:00
Zuul
e6cabd2f69 Merge "Support verbose showconfig in tox siblings" 2021-09-30 15:51:41 +00:00
Zuul
8c35f29ae1 Merge "Add tox_config_file rolevar to tox" 2021-09-30 15:37:15 +00:00
Jeremy Stanley
8b1cc73ee3 Include tox_extra_args in tox siblings tasks
When the tox role was introduced, a tox_extra_args rolevar was
included allowing the tox command line to be extended with arbitrary
options. When siblings functionality was added, tox_extra_args did
not get included in its separate tox invocations. If a project has a
particular situation where some aspect of tox's functionality must
be overridden in order to work, doing so through tox_extra_args
needs to apply to every tox invocation, including siblings installs.

Change-Id: Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
2021-09-17 20:34:37 +00:00
Jeremy Stanley
bc8776bad4 Support verbose showconfig in tox siblings
Unfortunately, when tox combines --showconfig with verbosity options
like -vv, some non-config output gets streamed to stdout before the
configuration is emitted. Filter this preamble in
tox_install_sibling_packages by discarding any initial lines of
output before the first section heading.

Also extend get_envlist() to deal with the fact that additional
verbosity adds a [tox] section in the --showconfig output, which it
was previously relying on to determine whether the config had been
filtered to a subset of env sections. Instead also check the
tox.args string to determine whether a -e option was passed on the
command line.

Change-Id: Iafeb88eaf9a596603ad4d2134a4574345d5189ab
2021-09-17 20:34:37 +00:00
Jeremy Stanley
3d449a07df Add tox_config_file rolevar to tox
In some situations, projects may not keep a tox.ini in the root
directory of their repository, or may even have multiple tox
configuration files. Allow the location and name of the config to be
overridden explicitly through the use of a new rolevar.

Change-Id: I1927142e6d9fa75e96902ae001c8ca98d69c7443
2021-09-17 20:34:37 +00:00
Sorin Sbarnea
f97385ebf9 Make default tox run more strict about interpreter version
Avoid false-positive CI testing with tox where misconfigured tox
projects end-up skipping tests or running with different python
version than the required one.

While use of this option on development machine may be ok, when
executed in CI context, we never want to be relaxed about what
we test and which versions of python we use.

I seen projects running with wrong version of python for months
before someone discovered that a different version of python
was used on CI.

Change-Id: I5be9bce86833db11afd7072e477ccaf42658bf99
2021-09-08 08:19:08 +00:00