Commit Graph

2243 Commits

Author SHA1 Message Date
Ian Wienand
94d7d42f14 ensure-dstat-graph: pull updated branch
This updated branch fixes a few issues and should ensure display with
pcp-dstat (I9234b226242f145db9c235797649202aa530ec74)

Add an override so we pull it

Change-Id: I0d5e1b567c364a9e6c7aa0b95de17abffaef0434
2021-10-22 18:31:10 +11:00
James E. Blair
61d00fec6f Use pcp instead of dstand and return dstat graph artifact
Dstat is dead, but pcp supplies a compatible interface.

Installation can be unreliable, so let's ignore failures since
this isn't generally critical.

Change-Id: I9234b226242f145db9c235797649202aa530ec74
2021-10-21 15:39:54 -07:00
Zuul
bd60fbd744 Merge "Add FIPS enable multinode job definition" 2021-10-20 23:54:04 +00:00
Zuul
984147663c Merge "build-python-release: Add flag for whether to build a wheel or not" 2021-10-20 20:32:00 +00:00
Douglas Viroel
9107f3ee7d Add FIPS enable multinode job definition
This patch adds a new multinode job definition that enables
FIPS mode prior to multinode configuration.
In order to enable FIPS mode, the OS boot procedure need to be
changed to enable the appropriate kernel flag. This modification
has effect only after system reboot.
The default behavior of this job is to always enable FIPS mode.

Change-Id: I6f1365837d9ed2ba82c391a20f9094c9ef0e6c4e
Signed-off-by: Douglas Viroel <dviroel@redhat.com>
2021-10-20 11:20:52 -03:00
Zuul
e3519ce469 Merge "multi-node-bridge: repos to install ovs in C9" 2021-10-20 12:45:36 +00:00
Sandeep Yadav
a795073493 multi-node-bridge: repos to install ovs in C9
With this patch, Adding repos to install openvswitch
in Centos9 stream.

For Centos9 stream nodes we need to enable 2 repos:-
* nfv repo - Ussuri+ openvswitch is shipped as part of NFV SIG
* rdo xena repo - rdo-openvswitch(which provides openvswitch wrapper)
is available in OpenStack repo

Closes-Bug: #1947665
Change-Id: If5a32b30637296a070f2e75ba8feb6aa45d8ecac
2021-10-20 11:17:41 +05:30
Ian Wienand
db23e0af70 ensure-podman: support Debian bullseye
The package is included in the latest stable, Debian bullseye.  Add an
install match, and test it.

Change-Id: I62d2582e8adf2bedb2ea7d41b4b22a48c9b2f274
2021-10-15 11:38:58 +11:00
Ian Wienand
72d4e506d5 intermediate-registry: handle socat warning out
Zuul switched to a new base image, and it seems the new socat puts out
a warning (something like

 ...  socat[489590] W ioctl(5, IOCTL_VM_SOCKETS_GET_LOCAL_CID, ...): Inappropriate ioctl for device

for reference).

Grep the output so we only get the line about what port it is
listening on.

Change-Id: I74fb86a9158b45e6601ee1fbc199ba80cd4991fe
2021-10-14 16:04:46 +11:00
Bhagyashri Shewale
84b86437c5 Handled TypeError while installing any sibling python packages
The below error is coming on mol-centos7 jobs while
installing sibling for python2 packeges on centos7

Error: TypeError: initial_value must be unicode or None, not str

This patch handled the TypeError while installing sibling
for python packeges for centos7.

Closes-Bug: #1946641
Change-Id: Ie8058cca92d099e50af19b95b4c417c5a665da0d
2021-10-13 12:52:03 +05:30
Gonéri Le Bouder
ad56b0b4eb
build-container-image: improve the reliabilty
build-container-image may ends up pushing large files. Network is not
always as reliable as we want. This commit ensures we retry several
times in case of failure.

Change-Id: Ieaf92a8c3531e24ab5e41783b540de1df806c02f
2021-10-08 11:38:11 -04:00
Zuul
d5e4d55c15 Merge "ensure-rust: rework global install" 2021-10-08 01:40:58 +00:00
Zuul
08e37ff140 Merge "Revert "Revert "Include tox_extra_args in tox siblings tasks""" 2021-10-06 17:31:10 +00:00
Zuul
bbf4c1bbaa Merge "ensure-twine: Avoid Reinstalling twine if present" 2021-10-04 20:58:04 +00:00
Zuul
ec6b89b0d4 Merge "ensure-tox: Avoid Reinstalling tox venv if present" 2021-10-04 20:48:06 +00:00
Ian Wienand
67ae84ce39 ensure-rust: rework global install
I think the secondary "rustup default" was intended to make sure the
Zuul user sets up to use the installed rust toolchain
(I32f9b285904a7036f9a80ada8a49fa9cf31b5163) but actually results in a
re-download of components and another local installation.  This isn't
really the intention, and also doubles the time spent installing.

From the linked comment, it seems like we're not doing our global
install correctly; even putting it in /usr doesn't avoid the need for
RUST_HOME to be set.  Take it's suggestion and install out-of-the-way
in /opt, use a small /usr/local/bin wrapper to call with correct env
vars set and then setup the installed global binary names to be called
via that.

Change-Id: I28ef747b809a17664305bfd9754022251390647b
2021-10-04 11:43:22 +11:00
Jeremy Stanley
c02c28a982 Revert "Revert "Include tox_extra_args in tox siblings tasks""
This reverts commit 3668acc723.

Change-Id: I643b82c7dc6fd78ff67678b56c639a6c45fd0061
2021-10-01 16:03:13 +00:00
Jeremy Stanley
1861faf567 More exact section matching for tox showconfig
With the original verbose output support in change
Iafeb88eaf9a596603ad4d2134a4574345d5189ab we looked for lines from
tox --showconfig output starting with an opening '[' but verbosity
also causes output from pip install activity to be included if a
tox.requires entry causes it to install packages before continuing
and these lines are prefixed by a process ID number wrapped in
brackets with the command string after that, which prematurely
triggered our search for the start of the INI content. Add a
stipulation that the first INI output line also end in ']' in order
to skip over those additional prepended lines.

Change-Id: If29e5a9abe3b92a145d87f5efc1b93350ea3908a
2021-10-01 16:03:12 +00:00
Jeremy Stanley
31a470245b Test tox.requires doesn't break sibling installs
Update the alternate tox.ini files in test-playbooks/python to
trigger configuration we saw expose a bug in the siblings lib's
showconfig parser when a tox.requires value resulted in pip
activity. Update its minversion to the earliest tox which provides
the tox.requires configuration option, for clarity.

While we're there, switch to using tox_config_file instead of
overriding zuul_work_dir since that's now an option for this role
and was previously not exercised in the test job.

Change-Id: I521580da978903cd116c872447b1c028c470e63d
2021-10-01 12:46:14 +00:00
Jeremy Stanley
3668acc723 Revert "Include tox_extra_args in tox siblings tasks"
This reverts commit 8b1cc73ee3.

Apparently tox.requires adds lines like this to verbose showconfig
output:

    using tox-3.24.4 from /.../tox/__init__.py (pid 2919)

Which is confusing the current parser in the siblings module. Roll
this back while we work on a fix and regression test.

Change-Id: If3b1d48b36a5d32fddfdabd9c0ec1b81dd6453f2
2021-09-30 17:54:17 +00:00
Zuul
7fcb90c59d Merge "Deprecate EOL Python releases and OS versions" 2021-09-30 17:33:30 +00:00
Zuul
266f3e3642 Merge "Explicit tox_extra_args in zuul-jobs-test-tox" 2021-09-30 15:51:44 +00:00
Zuul
f0d72b612b Merge "Include tox_extra_args in tox siblings tasks" 2021-09-30 15:51:42 +00:00
Zuul
e6cabd2f69 Merge "Support verbose showconfig in tox siblings" 2021-09-30 15:51:41 +00:00
Zuul
8c35f29ae1 Merge "Add tox_config_file rolevar to tox" 2021-09-30 15:37:15 +00:00
Zuul
8e85bcdce8 Merge "Pin to funcparserlib prerelease for new SetupTools" 2021-09-30 15:35:13 +00:00
Zuul
8b86f270d7 Merge "Stop testing playbooks/roles on Ubuntu Xenial" 2021-09-30 14:51:03 +00:00
Zuul
3c18483b01 Merge "Make default tox run more strict about interpreter version" 2021-09-29 17:44:02 +00:00
Jeremy Stanley
483838ceca Deprecate EOL Python releases and OS versions
Update the deprecation policy to indicate that zuul-jobs is no
longer tested with EOL platforms. Also explicitly switch the minimum
Python 3 documented to 3.6, and add a note to the tox-py34 and
tox-py35 jobs mentioning that they're no longer directly tested.
Move those jobs to the deprecated jobs list as well, to help
reinforce the point that their continued use is not recommended.

Change-Id: I2edbf8ea010caf7a7641e0d88f360965fc0b96ab
2021-09-23 17:35:32 +00:00
Zuul
91468c150c Merge "Pin protobuf<3.18 for Python<3.6" 2021-09-18 07:03:28 +00:00
Jeremy Stanley
22689ba524 Explicit tox_extra_args in zuul-jobs-test-tox
In change Iafeb88eaf9a596603ad4d2134a4574345d5189ab we fixed a bug
with handling of `tox --showconfig -vv` in the siblings tasks for
the tox role. This arose because of the desire to add tox_extra_args
to all tox invocations in Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
because we currently set a default of '-vv' for the tox_extra_args
rolevar. This fix could easily regress in the future if we were to
ever change the default value, so explicitly test with it in the
zuul-jobs-test-tox job by adding it to the python/tox test playbook.

Change-Id: Ib6ddf7d188904193a5ecd520acac1d676e5d78ae
2021-09-17 20:34:37 +00:00
Jeremy Stanley
8b1cc73ee3 Include tox_extra_args in tox siblings tasks
When the tox role was introduced, a tox_extra_args rolevar was
included allowing the tox command line to be extended with arbitrary
options. When siblings functionality was added, tox_extra_args did
not get included in its separate tox invocations. If a project has a
particular situation where some aspect of tox's functionality must
be overridden in order to work, doing so through tox_extra_args
needs to apply to every tox invocation, including siblings installs.

Change-Id: Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
2021-09-17 20:34:37 +00:00
Jeremy Stanley
bc8776bad4 Support verbose showconfig in tox siblings
Unfortunately, when tox combines --showconfig with verbosity options
like -vv, some non-config output gets streamed to stdout before the
configuration is emitted. Filter this preamble in
tox_install_sibling_packages by discarding any initial lines of
output before the first section heading.

Also extend get_envlist() to deal with the fact that additional
verbosity adds a [tox] section in the --showconfig output, which it
was previously relying on to determine whether the config had been
filtered to a subset of env sections. Instead also check the
tox.args string to determine whether a -e option was passed on the
command line.

Change-Id: Iafeb88eaf9a596603ad4d2134a4574345d5189ab
2021-09-17 20:34:37 +00:00
Jeremy Stanley
3d449a07df Add tox_config_file rolevar to tox
In some situations, projects may not keep a tox.ini in the root
directory of their repository, or may even have multiple tox
configuration files. Allow the location and name of the config to be
overridden explicitly through the use of a new rolevar.

Change-Id: I1927142e6d9fa75e96902ae001c8ca98d69c7443
2021-09-17 20:34:37 +00:00
Jeremy Stanley
f8a60d416f Pin to funcparserlib prerelease for new SetupTools
SetupTools 58 dropped support for its old use_2to3 option, which has
started surfacing a number of ancient Python packages in need of
updates. In this case, the last full release of funcparserlib (which
is a transitive dependency by way of blockdiag by way of
sphinxcontrib-blockdiag) was in 2013, but luckily they have an alpha
release which we can pin explicitly and pull in as a temporary
workaround to get docs builds going again.

Change-Id: I6903eeac2c479e2da795c1dbd215cdee33d09fd7
2021-09-17 20:34:37 +00:00
Jeremy Stanley
9a653e69a6 Stop testing playbooks/roles on Ubuntu Xenial
Ubuntu Xenial reached end of standard support almost 6 months ago.
Its default python3 is 3.5, which new versions of many libraries are
dropping support for as that hit EOL earlier this month. We are now
faced with a situation where we cannot test the fetch-sphinx-tarball
role on Xenial because latest SetupTools breaks old funcparserlib
(indirectly needed for our blockdiag and seqdiag elements in
zuul-jobs' docker-image.rst), and the only funcparserlib release
which is installable has dropped Python 3.5 support.

Cease running Ubuntu Xenial platform and Python 3.5 tests for
changes to zuul-jobs now. This is a soft removal, since we're not
actually removing Xenial platform checks from playbooks or roles
yet, nor removing the tox-py35 job definition.

Change-Id: I46b9b887427133147481e92a1e7c523e6592fd2d
2021-09-17 20:34:37 +00:00
Jeremy Stanley
f8f1d1c271 Pin protobuf<3.18 for Python<3.6
Our Google Cloud log upload role relies on google-cloud-storage,
which in turn needs protobuf. Unfortunately, protobuf dropped Python
2.7 and 3.5 support in its 3.18.0 release, so we use an environment
marker to pin it in our test requirements.

Change-Id: I89caf4e36850fc4a912b76d75f368144ddb5e15f
2021-09-16 19:17:41 +00:00
Jeremy Stanley
a3caa9ed5b Preinstall PBR in ensure-pip test playbook
Work around lack of SNI support in old distutils versions shipped
with Python on platforms like CentOS 7 and Ubuntu 16.04 LTS by
installing PBR first so that distutils won't be compelled to do so.
Warehouse (PyPI) ceased supporting clients without SNI support in
March of this year.

Change-Id: Ic741d9a87c2ca3a5249cd03c3cbd38e2ad1f46a1
2021-09-09 17:46:50 +00:00
Sorin Sbarnea
f97385ebf9 Make default tox run more strict about interpreter version
Avoid false-positive CI testing with tox where misconfigured tox
projects end-up skipping tests or running with different python
version than the required one.

While use of this option on development machine may be ok, when
executed in CI context, we never want to be relaxed about what
we test and which versions of python we use.

I seen projects running with wrong version of python for months
before someone discovered that a different version of python
was used on CI.

Change-Id: I5be9bce86833db11afd7072e477ccaf42658bf99
2021-09-08 08:19:08 +00:00
Zuul
2cedd93c02 Merge "ensure-docker: enable centos-8-stream testing" 2021-08-31 06:05:05 +00:00
Zuul
b6ffaaecbb Merge "ensure-podman: enable testing of centos-8-stream" 2021-08-31 06:05:04 +00:00
Monty Taylor
23867c2cf8 Update binfmt support image used
The multiarch/qemu-user-static image seems to be more successful.

Change-Id: Iea8c1b9fdd7e9dff51e9dfe9702216e73199ff06
2021-08-26 09:17:59 -05:00
Ian Wienand
6e4120a2ba build-docker-image: fix indentation of documentation
Introduced with I862819959c77a557199f64b4d42109bc7915959c

Change-Id: I88d6a4e200cd508172aaf2fd3a76fcf4c5596ff8
2021-08-25 12:39:30 +10:00
Ian Wienand
b950178733 build-docker-image: Add flag to use BuildKit
It seems like BuildKit is the next generation, but not likely to be
enabled by default soon (https://github.com/moby/moby/issues/40379).
Add a flag so people who want to use its features can easily opt-in.

Change-Id: I862819959c77a557199f64b4d42109bc7915959c
2021-08-25 10:25:14 +10:00
Sorin Sbarnea
76d5a24433 ensure-podman: enable testing of centos-8-stream
Change-Id: Ibe28bcaec2efd30b905578d6b2b4b00186ccd524
2021-08-20 17:09:15 +01:00
Sorin Sbarnea
f38122369a ensure-docker: enable centos-8-stream testing
Change-Id: I7c3134d4c73611e6f1591e76f9b749d37def4d10
2021-08-20 17:06:31 +01:00
Clark Boylan
ce31c2c678 Find (s)testr more reliably
We have seen instances where type -p (s)testr seems to return with a
leading blank line which confuses ansible later when trying to use the
first line of output as the path to (s)testr. Address this by chomping
with grep -v ^$. Additionally use type -P instead of -p to ensure we
always get a path even when the command may be an alias or builtin.

Change-Id: Ibffe1e1499eca18ef5dc3904fe222a55242b827d
2021-08-11 10:04:39 -07:00
Zuul
7bd7aa5c9a Merge "Remove success-url" 2021-08-06 20:53:54 +00:00
Xinliang Liu
e3327efd38 Fix install podman error on Ubuntu aarch64 Bionic
The official podman ubuntu install guide[1] tells to install podman from
Kubic project repo for ubuntu Bionic and Focal. And project atomic PPA
repo[2] is deprecated.

But Kubic repo only provides x86_64 deb packages for Bionic. For non x86_64
platforms use project atomic PPA repo on Bionic.

Also add a job zuul-jobs-test-ensure-podman-ubuntu-focal.

[1] https://podman.io/getting-started/installation
[2] https://launchpad.net/~projectatomic/+archive/ubuntu/ppa

Change-Id: I402adf1866e4bb8f3b388216bc48b9927e1388b1
2021-08-05 10:40:14 +00:00
James E. Blair
f37ecd4bd6 Remove success-url
This option is ignored by Zuul as of release 4.7.0

This change is safe to merge now, as it won't break older versions.

Change-Id: I29d943462524b44cd04943df42838944fd43535d
2021-07-29 16:42:59 -07:00