141 Commits

Author SHA1 Message Date
Zuul
d12b99d6e7 Merge "packer: namespace test jobs correctly" 2020-05-26 12:41:27 +00:00
Sorin Sbarnea
536e1f7f8c Make gentoo multinode job nv
Gentoo multinode job is consistently failing. To disable only this
job with current test framework, we need to adapt the test framework, so
that disabling of specific jobs is possible.

Also includes minor documentation fix which triggers affected jobs.

Co-Authored-By: Andreas Jaeger <aj@suse.com>
Change-Id: Ida9d1714f1f6c3f4fbee469ef940147f969d9466
2020-05-26 09:45:20 +02:00
Albin Vass
45c2b5c3a4 packer: namespace test jobs correctly
Change-Id: Ia87f5050da7d22cd4f5016179795cf65e7df911e
2020-05-24 10:25:52 +02:00
Albin Vass
86041d9914 Don't require tox_envlist
Since tox_envlist has a default value it cannot be undefined
so the fail task will never run. Instead handle the case when
tox_envlist is an empty string by getting the default configured
envlist from tox. Also handle the casewhen tox_envlist is 'ALL'.

This also updates tox_install_sibling_packages to correctly
handle multiple testenvs and uses configuration supplied by
'tox --showconfig -e <envlist>' instead of guessing where the
envdir and logdir are located.

We also cannot run tox inside python because it gets complicated
to know which tox_executable we should call during the python test
cases so run these commands in ansible and pass the output to
tox_install_sibling_packages.

Since role params have higher precedence than set_fact we set an
internal _tox_envlist fact that is a comma separated list of testenvs
that should be run.

Change-Id: I9e5a1b041f653cbcff7b8ed62e4a95a0a040fdd7
2020-05-19 14:32:26 +02:00
Zuul
9b29d2f555 Merge "Fix broken tox-siblings.yaml test" 2020-05-15 14:19:00 +00:00
Albin Vass
d816de1e93 Fix broken tox-siblings.yaml test
Change-Id: I63b2949d0cddc58f15792072f4c36a099a233008
2020-05-15 15:32:06 +02:00
Zuul
86db3adeb1 Merge "Move artifactory test job" 2020-05-14 18:05:08 +00:00
Zuul
f05687fd78 Merge "Add upload-artifactory role" 2020-05-14 17:26:10 +00:00
James E. Blair
01c2e4cbd6 Move artifactory test job
Move it from artifactory jobs to general jobs to match the
documentation structure.

Change-Id: I4601df21b9c5a332272d89ee4f00fa810250ebee
2020-05-14 10:12:40 -07:00
Tristan Cacqueray
6f940da935 fetch-sphinx-tarball: introduce zuul_use_fetch_output
This change enables using fetch-sphinx-tarball role along with
the fetch-output role. By default the role still synchronizes
artifacts back to the executor.

Change-Id: I7452f34bccdca49e256053f9630f77953b27f99c
2020-05-14 12:03:58 +00:00
Zuul
d17cc7b522 Merge "fetch-sphinx-output: introduce zuul_use_fetch_output" 2020-05-13 18:59:10 +00:00
Albin Vass
664b016fa5 Add upload-artifactory role
Adds a role to let users define a manifest of artifacts located
in zuul-output/artifacts that should be uploaded to pre defined
artifactory instances.

Change-Id: I00dc0302e85ce59b3808f6e62e2bcdadf2e41fde
2020-05-12 20:02:12 +02:00
Tristan Cacqueray
59bacd1518 fetch-sphinx-output: introduce zuul_use_fetch_output
This change enables using fetch-sphinx-output role along with
the fetch-output role. By default the role still synchronizes
artifacts back to the executor.

Change-Id: I81c8a2faf566be694dbf1c59cbbd157dc4df5146
2020-05-12 16:38:26 +00:00
Albin Vass
c9db6cca96 ansible-lint-rules: Fix bad path and filename
Change-Id: I02b937b8d68600062a25731ae5d4b43f33c8c4a9
2020-05-08 19:38:58 +02:00
Ian Wienand
53a2907e55 Remove opensuse-15-plain testing
With the dependent change pip-and-virtualenv is dropped from the
opensuse-15 image, thus we don't need the -plain testing any more.

Change-Id: Iccf70042833093ebeca92c171dfe05dbf14a55aa
Depends-On: https://review.opendev.org/725749
2020-05-06 12:40:40 +10:00
Zuul
47bce7db05 Merge "Add plain nodes to testing" 2020-05-05 23:26:06 +00:00
Ian Wienand
8733cff8db Add plain nodes to testing
This adds more of the plain nodes (without pip-and-virtualenv) to our
testing environment to facilitate the eventual switch in opendev
infra.

Change-Id: I091ab667c5f071ba5844ea567464b1c29052638d
2020-05-06 06:41:56 +10:00
Zuul
25206338a4 Merge "ansible-lint: use matchplay instead of matchtask" 2020-05-05 20:18:14 +00:00
Albin Vass
bee0c6ae2f ansible-lint: use matchplay instead of matchtask
For some reason matchtask doesn't match includes, matchplay does so use
that instead.

Change-Id: I040f7f3394503e92d06c05e8ff671a43b14baebc
2020-05-05 20:42:38 +02:00
Andreas Jaeger
069f4aef37 Add py38 testing
Python 3.8 is current version, replace py37 so that we test
3.5 and 3.8 as lowest/highest.

This follows I4b382bb89847d3ae2b20292d923c3c2211dfb254 for zuul.

Change-Id: I35a1492ff0e34c7951c8ecf216992044fdb3065f
2020-05-05 20:02:16 +02:00
Zuul
618c78f191 Merge "Support multi-arch image builds with docker buildx" 2020-05-05 13:22:18 +00:00
Monty Taylor
63bd307e63 Support multi-arch image builds with docker buildx
Docker has experimental support for building multi-arch
container images with a buildx command. Currently it only
supports pushing to a registry after running and the images
don't end up in the local docker images list. To work around
that, push to the buildset registry then pull back. This
is the inverse of the normal case where we build, then
retag, then push. The end result should be the same.

Change-Id: I6a4c4f9e262add909d2d5c2efa33ec69b9d9364a
2020-05-04 14:02:13 -05:00
Tristan Cacqueray
4903ecd30b dhall-diff: add new job
This change adds a new dhall-diff job to verify generated config
is idempotent.

Change-Id: I96a335dc78c4fa74564b854997433e5be0b5e633
2020-05-03 19:06:37 +00:00
Albin Vass
85ea882974 Only run tests for ensure-bazel when it is updated
Change-Id: I922a16a9c41a215c84d193360d45c0cfe9001639
2020-05-01 23:15:18 +02:00
Zuul
10cd19fdae Merge "ensure-yarn: run ensure-nodejs before" 2020-05-01 17:24:27 +00:00
Zuul
096b1218c3 Merge "haskell-stack-test: add haskell tool stack test" 2020-05-01 16:09:58 +00:00
Zuul
43ad1fb400 Merge "Add Bazel build and ensure roles" 2020-05-01 15:39:41 +00:00
Tristan Cacqueray
cf64b73f78 haskell-stack-test: add haskell tool stack test
This change adds a new job to test stack based haskell project.

Change-Id: Ie3492097b351eec743d98e9ca8bc693c00c11a78
2020-05-01 15:35:33 +00:00
Mohammed Naser
6920617af5 ensure-yarn: run ensure-nodejs before
YARN needs a modern version of NodeJS and if we don't use the role
to get the latest version, we end up failing on most platforms that
ship an old version of it.

This patch also adds testing to validate that ensure-yarn alone works.

Co-Authored-By: Andreas Jaeger <aj@suse.com>
Change-Id: I83ac09bc5751c85c6f1eac550d289a300ca2255e
2020-05-01 17:07:48 +02:00
Mohammed Naser
00cc4aa038 ensure-docker: refactor to use ensure-package-repositories
This patch refactors ensure-docker to use the new ensure-package-repostories
role which cleans up the codebase for it a lot.  It also converges the
Debian and Ubuntu configuration as they were pretty much the same.

Also, it updates the Docker to the proper GPG key as while refactoring,
it was exposed that it was using the wrong key.  Docker uses a different
key for Debian and RHEL based packaging.

Change-Id: Iad430c6a31be0750c1a0f50e12a26df230de9f4e
2020-05-01 15:49:30 +02:00
Mohammed Naser
ff5e5c17f6 ensure-podman: refactor to use ensure--package-repositories
This patch refactors ensure-podman to use the new generic role
to allow it to be cross-platform in the future.

Change-Id: Id9ad0fbaeb9d19d022393ad16a27f11260c45da0
2020-05-01 15:49:30 +02:00
Mohammed Naser
7ea6d11981 ensure-nodejs: refactor to use ensure-package-repositories
This patch refactors ensure-nodejs to use the new generic role
to allow it to be cross-platform in the future.

Change-Id: I859b6b3e33bc22ad4f99828e921ac2ed35a49252
2020-05-01 15:49:30 +02:00
Mohammed Naser
90d9b9b7d9 ensure-kubernetes: refactor to use ensure-package-repositories
This patch refactors ensure-kubernetes to use the new generic role
to allow it to be cross-platform in the future.

Change-Id: Ic90c8973431823814c7e6b122f4ef25a67d64c2f
2020-05-01 15:49:30 +02:00
Paul Albertella
ad7297199d Add Bazel build and ensure roles
* Build role requires build target to be specified via a variable
* Ensure role checks for a specific version of Bazel and downloads
  and installs if missing; defaults to v3.1.0 and downloading from
  bazelbuild Github, but these can be overrriden using variables
* Also installs dependencies for Bazel; only supports Ubuntu/Debian
  at present, but includes scope to extend for other platforms
* Includes test playbook and job

Change-Id: I83f198aaf20c2b3664bea6fc05edd3b4fca13a4f
2020-04-30 18:05:41 +01:00
Zuul
d775467078 Merge "Add ensure-virtualenv" 2020-04-28 20:51:51 +00:00
Ian Wienand
08bbd16c52 Add ensure-virtualenv
This is part of the efforts to remove pip-and-virtualenv from our base
images [1].

There are some users who specifically require the virtualenv command
(perhaps, like dib, they have some code that uses the activate_this.py
mechanisms it provides wich venv does not).

This installs the virtualenv package for the currently running
distribution.

One of the main maintenance issues of pip-and-virtualenv is that tried
to ensure that "virtualenv" created a Python 2 environment always by
default.  Now that we have Python 3 only distributions like current
Fedora, this is not something we can continue to do (even if we wanted
to, which we don't).

What owns virtualenv and what it produces in our heterogeneous
environment is messy, and I think the best we can do is document it as
done here.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html

Change-Id: I97d8bfb970ed2b5aaa02a0813899014c94994066
2020-04-28 06:09:54 +00:00
Tristan Cacqueray
2e82d88589 hlint: add haskell source code suggestions job
This change adds a new job to run hlint on haskell source files.

Change-Id: Ibf76c5552acecd68dfc56c4f31d8045ca5b233f1
2020-04-26 12:05:12 +00:00
Zuul
42edd207f9 Merge "cabal-test: add initial haskell job" 2020-04-24 14:34:59 +00:00
Tristan Cacqueray
d92b39cae2 cabal-test: add initial haskell job
This change adds a new job to test cabal based haskell project.

Change-Id: Idb24eb5f8f9bd1409bae07bf4da415f1d05568e4
2020-04-24 13:12:13 +00:00
Zuul
66b4de8b26 Merge "Add testing of fetch-sphinx-tarball role" 2020-04-23 15:00:43 +00:00
Sorin Sbarnea
bb247a7c2f Switch remaining tests to fedora-31
Previous switch to fedora-31 missed to update few jobs.

Change-Id: Id5b30008797457d54b104713a0db04307b1e48af
2020-04-23 14:06:03 +01:00
Sorin Sbarnea
b906d000fe Add testing of fetch-sphinx-tarball role
We discovered that role fails on some platform because of the missing
bzip2 package.  We can avoid this by using gzip which is much more
likely to exist on machines (and exists in the container images for
most operating systems).

Change-Id: Ia33a0f77343df809cd3c591a4a988e8e3712c133
Co-Authored-By: Mohammed Naser <mnaser@vexxhost.com>
2020-04-23 13:52:48 +01:00
Zuul
9400fa6b65 Merge "Use cached 'tox_executable' in fetch-tox-output" 2020-04-21 19:32:44 +00:00
Albin Vass
0515041465 Use cached 'tox_executable' in fetch-tox-output
Otherwise this role could fail when tox is not installed in PATH.

Also updates the default behaviour to only collect the environments
configured with 'envlist' in tox.ini and collects all environments if
the envlist is set to 'ALL'.

Change-Id: I60bcc25cb853d2db3ee767a719b9ac54e421532e
2020-04-21 12:10:54 +02:00
Ian Wienand
d46c946f9b Make ubuntu-plain jobs voting
We have enough infrastructure in place now to have the
pip-and-virtualenv free node type voting, while we progress towards
making this the default.

Change-Id: Ie2677364d8696d5c4f71eefe2054da1b39e8bb35
2020-04-17 10:44:56 +02:00
Ian Wienand
88124dffb9 Update Fedora to 31
Update the Fedora test platform to 31.  Since we have dropped the
pip-and-virtualenv element with this image, we do not need the
work-around for excluded packages any more.

Story: #2007386
Task: #39310

Change-Id: I713a3cc790039dee3d28d83198f5afff5bebc491
2020-04-17 14:50:51 +10:00
Zuul
21928eab52 Merge "Adds roles to install and run hashicorp packer" 2020-04-15 16:01:18 +00:00
Ian Wienand
5068744eff ensure-pip: Add role
This role is intended to ensure that the `pip:` module, and jobs that
may wish to use `pip` from the shell, have the necessary requirements.

It is intended as a partial replacement for the pip-and-virtualenv
element in diskimage-builder, which currently pre-installs pip on our
infra CI images during image build.

We wish to remain broady compatible with this element, but not
replicate some of the more problematic areas of its implementation.

By default, this installs the system packages for pip and setuptools
(the latter being a requirement of the Ansible pip module, which
imports it directly, despite pip itself not requiring it).

In this role, we ensure the libraries for the currently running
ansible_python_interpreter version are installed.

There is provision to provide a flag to install the packages directly
from upstream via get-pip.py, although this is not recommended.

Story: #2007386
Task: #39309

Change-Id: Iac2d518a66caf1b801273225f75a0a748412903c
2020-04-14 10:20:33 +10:00
vass
32a3ce3f5a Adds roles to install and run hashicorp packer
Change-Id: I82274ef59eda00a735579082bbd37b64c0599c7d
2020-04-13 16:44:32 +02:00
vass
da97adbfb7 Use ensure-* roles
The install-* roles in zuul-jobs have been renamed to ensure-*, follow
the rename.

Change-Id: I32984fcd408cf0cd3751c5c5960608bec6e202a2
2020-04-12 15:26:10 +02:00