2138 Commits

Author SHA1 Message Date
Sorin Sbarnea
0eaa5cf59a Upgrade ansible-lint to 5.0
- bumps ansible-lint to 5.0
- updates our custom rules to make them compatible with 5.0
- replace custom module mocking with native ansible-lint ones
- remove custom call of ansible-playbook --syntax-check as now this
  is done by ansible-lint
- assured molecule vars are hosted under a vars/ folder in order to
  avoid confusing linter detection.
- replaced custom rule for loop var names in role as now this this an
  optional core feature of the linter (see config)
- replaced custom rule no-same-owner with opt-in one (see config)

Change-Id: I233fae8c9036d295968a97ee80e07fde8846c633
2021-03-17 15:44:43 +00:00
Zuul
78f18243f1 Merge "cabal-test: add install_args and build_args role var" 2021-03-17 12:31:44 +00:00
Zuul
4b8795f185 Merge "ensure-helm: skip helm init for helm v3 and newer" 2021-03-17 12:26:25 +00:00
Tristan Cacqueray
7af05ebcdc bindep.txt: skip python-devel for el8 platform
The python-devel package is not available on this platform.

Change-Id: I966c313d6dd0282b495fdeae4e92bbbe9b4d8fe3
2021-03-11 17:43:59 +00:00
Sorin Sbarnea
59dad009e9 Bits to keep ansible-lint happy
- moved vars file under vars to avoid failure to load file (newer
  version raise error if you try to put vars outside vars/defaults as
  it will assume they are broken playbooks.
- added missing file-modes on few tasks

Change-Id: I7217469d089b655ee16cd038391cffa4197c8c83
Part-Of: https://review.opendev.org/c/zuul/zuul-jobs/+/773245
2021-03-10 18:04:08 +00:00
Benjamin Schanzel
dfbd78d40d ensure-helm: skip helm init for helm v3 and newer
The init command was removed from helm with v3, no initialization is
needed anymore. So only run the init task if the version is <= 2.

Change-Id: I59b0c6efa0a2e6b94267fd3bb7a9701bc4c51048
2021-03-10 11:03:18 +01:00
Sorin Sbarnea
ad3167bd56 Fixes all tasks should be named rule
This prepared for ansible-lint v5 which no longer has exceptions
for task naming, requiring all to be named.

Change-Id: I5e761d1e3836fa270d7afdcf01780320001f820d
Part-Of: #773245
2021-03-08 13:30:32 +00:00
Tristan Cacqueray
8bde6d3504 cabal-test: add install_args and build_args role var
This change adds new role vars to enable setting
custom cabal arguments.

Change-Id: I4e4174117608bb76ca6e497687753cfdbc3acc02
2021-03-04 07:51:13 +00:00
James E. Blair
a31d70e6df ensure-zookeeper: use a tmpfs
This is helpful in testing situations which can see high amounts
of write traffic.

Change-Id: I69ff8116369c4ad40e2dc0c20af22370f3ee9927
2021-02-19 10:51:57 -08:00
James E. Blair
9f0d22d678 GCS logs: more robust Credential class
Google keeps changing the with_scopes method signature.  We need to
override that method in order to keep our path attribute around when
it re-instantiates the object.  As long as we always call refresh
after the object is created, we can drop our custom constructor and
then change with_scopes to use generic arguments.  This should keep
up with any further minor API changes without needing further local
changes.

Also, a minor inconsistency in command line arguments (used only for
local manual testing) is corrected.

Change-Id: Id41dfc464eb86429771a78c1fefae006e0915cb9
2021-02-19 09:53:08 -08:00
James E. Blair
4170cedf60 Revert "Update upload-logs roles to support endpoint override"
This reverts commit 862ae3f5d6b1c0e94bb24cfebf0b6b71e680e878.

We did not consider the effoct on the quick-download link that
is generated in opendev:

http://paste.openstack.org/show/802839/

Change-Id: I9702f8f1c0155ee3b13c74baaf2c09db72e690fd
2021-02-19 09:08:24 -08:00
Zuul
0263bf88e6 Merge "Update upload-logs roles to support endpoint override" 2021-02-19 16:12:45 +00:00
Zuul
0ea462f725 Merge "ensure-zookeeper: add use_tls role var" 2021-02-19 00:22:37 +00:00
Zuul
463b7679cf Merge "ensure-openshift: remove unused role var" 2021-02-19 00:22:33 +00:00
okozachenko
862ae3f5d6 Update upload-logs roles to support endpoint override
Add zuul_log_storage_proxy_address variable attempting to replace
storage endpoint address with.
The usecase is when the storage proxy is positioned infront of storage
endpoint.

Change-Id: I353cd50b76683212b0319a1e28f34901267c08e4
2021-02-18 16:09:27 -08:00
dg012m
bb4ebf4720 Allow customization of helm charts repos
Added default variables to customize the repo name to download
the chart testing helm chart and config files

Change-Id: I37f8db66bdfef7d64e59a89ca8d99b12f95c1be2
2021-02-18 09:07:06 +00:00
Tristan Cacqueray
68409bc81f ensure-zookeeper: add use_tls role var
This change adds a new attribute to setup zookeeper TLS.

It also adds support for Debian-derived distros.

Change-Id: Ifb5fc51f3b66be0b2dd1b8003507e21d8afe16fc
2021-02-17 16:06:36 -08:00
Tristan Cacqueray
bd93b3d424 ensure-openshift: remove unused role var
This change removes an ununsed role var as a follow-up for
https://review.opendev.org/c/zuul/zuul-jobs/+/775511

Change-Id: I2d966d744caae744c76d5c8c6b2693eb07e444d1
2021-02-16 18:32:16 +00:00
Zuul
20cdac6e33 Merge "ensure-openshift: workaround missing ansible26 repository" 2021-02-16 18:25:15 +00:00
Tristan Cacqueray
55ca357e39 ensure-openshift: workaround missing ansible26 repository
This change shortcirtcuits openshift repository installation to avoid
pulling the ansible26 repository.

Change-Id: I912a03e6502627340d002b419514f1ebdffb61e5
2021-02-16 15:43:00 +00:00
Zuul
d054d7ecc8 Merge "Fix CentOS wheel mirror URL" 2021-02-15 23:47:11 +00:00
Sorin Sbarnea
762ce6e96e update-json-file: avoid failure when destination does not exists
It is perfectly valid to want to add a value to a file that
does not exist yet, even the path may be missing.

This fixes last night regression when installing docker no longer
creates the /etc/docker folder, causing our MTU update to fail.

Change-Id: I0f037d1d6664de3c3b777aaf6da9cd7c3e8bb15f
Reference: https://review.rdoproject.org/zuul/builds?job_name=tox-py36-ci-config&project=rdo-infra/ci-config
2021-02-12 13:26:49 +00:00
Jeremy Stanley
b1521dbdb5 Report initial node filesystem sizes/utilization
Change Iba195e7c5cec372c6ba4daf7059da5b6fb6740ec implemented
collection of output for `df -i` (inode counts) and `df -m`
(megabytes data) in validate-host, but did not add them to the
report file template. Correct this oversight so that the collected
information will be included in that file.

Change-Id: I8c2c4a90f18394a04fde84355a89a15bf5aa66b4
2021-02-07 18:25:32 +00:00
Jeremy Stanley
e7915b65b7 Collect initial node filesystem sizes/utilization
Include calls to `df -i` (inode counts) and `df -m` (megabytes data)
in validate-host, to aid in troubleshooting build failures where the
builds start out with too little free space. This way the initial
capacity and utilization of all available filesystems will be
recorded with other basic node diagnostic data.

Change-Id: Iba195e7c5cec372c6ba4daf7059da5b6fb6740ec
2021-02-06 20:27:01 +00:00
Zuul
8bdb2b538c Merge "bindep: remove set_fact usage when converting string to list" 2021-02-03 20:09:46 +00:00
Zuul
5bb91e867b Merge "Use urlencoded filenames in test fixtures" 2021-02-01 14:11:44 +00:00
Albin Vass
6f2ff3f1b8 bindep: remove set_fact usage when converting string to list
This can cause problems where set_fact isn't able to override the
variable when bindep is used with include_role and bindep_file is set as
a variable in the include_role task.

Change-Id: I3099f1802f2d30df1265a9abc11a53c923ed4832
2021-02-01 15:09:07 +01:00
Zuul
79c44ead3a Merge "Fix repo files for CentOS Stream" 2021-01-28 15:07:00 +00:00
Sorin Sbarnea
938da600b7 Add tox-py39 job
Change-Id: I46cac3396bd870eb1f54820921df09946184605a
2021-01-27 10:14:32 +00:00
Javier Pena
cfada06486 Fix repo files for CentOS Stream
Some repo file names and ids do not match the files provided by
CentOS, which can cause duplicated repos [1].

[1] - https://8a51f3027f5f50a14ba7-fcfc85786424b7f5a3fcadff9da35f94.ssl.cf5.rackcdn.com/770771/4/check/packstack-centos8s-integration-scenario002/9dc0bd2/logs/etc/yum.repos.d/index.html

Change-Id: Ib89ad9f79cd5317b3178301a47462277004fa81a
2021-01-22 17:46:26 +01:00
Zuul
7d0a646c9e Merge "Rename config repos file config for CentOS Stream" 2021-01-22 08:15:22 +00:00
Zuul
bf1eeb4e71 Merge "Temporarily stop running Gentoo base role tests" 2021-01-21 16:11:55 +00:00
Alfredo Moralejo
8beac88116 Rename config repos file config for CentOS Stream
Currently, config file is CentOS-8-stream.yaml but the value of
ansible_distribution_major_version fact in CentOS stream is 8, not
8-stream [1], so jobs running in centos-8-stream images are actually
using centos-8 repositories.

This patch is modyfing the list of config files to use ansible_lsb.id
fact wich is CentOSStream and renaming the file name.

[1] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_792/770771/2/check/packstack-centos8s-integration-scenario002/792342c/zuul-info/host-info.centos8s.yaml

Change-Id: I7199acdd45151d2553ef17588909615b5a7c5b61
2021-01-21 09:00:36 +00:00
Daniel Blixt
dfbc8e2ec8 Use urlencoded filenames in test fixtures
When checking out repo to Windows environment, some files in test
fixture file trees do not get checked out. By setting renaming those
files with an urlencoded name, and populating a temporary directory
with corresponding file structure and decoded filenames during test by
a fixtures script, this will not cause checkout problems for those who
 want to read the repo in Win environment

Change-Id: If354eaf3f0bf2e86ddc05e3750aa5f0067dd9e21
2021-01-20 08:37:26 +01:00
Gorka Eguileor
4284afc584 Fix CentOS wheel mirror URL
For CentOS we are currently using default values for "wheel_mirror" from
roles/configure-mirrors/defaults/main.yaml, which means that the URL
uses the full distribution version, which is wrong, we should be using
the major version instead.

As an example, centos-8 jobs are trying to use
  https://mirror.bhs1.ovh.opendev.org/wheel/centos-8.3-x86_64
When they should be using
  https://mirror.bhs1.ovh.opendev.org/wheel/centos-8-x86_64

Which makes some jobs fail to find alembic wheel files and then they
fail to build from source because the CI was expecting the wheel mirror
to have it.

This patch fixes this by creating a Centos specific "wheel_mirror"
variable.

Change-Id: I11199cf943a3070fd62bf486fe2b06381db4b04d
(cherry picked from commit c9cad38eebf1bdabb10b79699598d6ba0f098f54)
2021-01-19 14:13:04 +01:00
Zuul
57a2f59f5a Merge "Pass environment variables to 'tox envlist config' task" 2021-01-18 14:31:00 +00:00
Tobias Henkel
3cc366c9c6
Fail mirror-workspace-git-repos if checkout failed
If the checkout failed the mirror-workspace-git-repos must fail the
job. Otherwise this can lead to a broken gate because a different
state than the state that should be tested can be tested silently.

Change-Id: I98e948776ec4b749bc1475ee3a17e37007591118
2021-01-18 14:27:52 +01:00
Jeremy Stanley
30c2e573dc Temporarily stop running Gentoo base role tests
The zuul-jobs-test-base-roles-gentoo-17-0-systemd job isn't passing
at the moment because Gentoo now needs Python 3.8 to install
pip/wheel, but the images are still using 3.6. A change has been
merged to correct this, but OpenDev has been unable to build new
Gentoo images since May 2020, so that will need to be solved before
the job will work again. In the meantime, stop running it so our
changes aren't indefinitely blocked. We'll need to apply extra
scrutiny to (or simply hold) any Gentoo-related changes for now
until we're testing that again.

Change-Id: I276bc596c50e7185fae828afa79c73ca9d955ec5
2021-01-17 08:16:18 +01:00
Benedikt Loeffler
9eaab6fd74 Pass environment variables to 'tox envlist config' task
As of version 3.21.0, tox now throws an exception if an environment variable
used in config is missing. We now need to pass the variables also to the task
"Get tox envlist config".

Change-Id: I4073f55b6ba3456f3d2db40cafb692efbb478914
2021-01-14 15:25:31 +01:00
Zuul
71132b658c Merge "upload-artifactory: no_log upload task" 2021-01-14 08:51:01 +00:00
Zuul
6f6e244cf0 Merge "Enable installing nimble siblings" 2021-01-13 07:13:30 +00:00
Zuul
f388375572 Merge "Allow to retrieve releasenotes requirements from a dedicated place" 2021-01-13 01:36:09 +00:00
Zuul
de9111a9a3 Merge "Add configuration to make logs public" 2021-01-13 01:02:15 +00:00
Zuul
b3682d8fb3 Merge "Document Python siblings handling for tox role" 2021-01-13 00:05:34 +00:00
Zuul
e53340d70a Merge "Clarity tox_environment accepts dictionary not list" 2021-01-13 00:05:30 +00:00
Zuul
20271e1bb5 Merge "Better error output for update-test-platforms.py" 2021-01-11 12:17:56 +00:00
Vitaliy Lotorev
ecc11af6fe Document Python siblings handling for tox role
Brief overview how actually siblings are installed helps
proper usage of 'tox' role.

Change-Id: Id27c9c60a40e43c0e351849470ccb4d0d11739a3
2021-01-08 23:41:27 +03:00
Vitaliy Lotorev
4708a35220 Clarity tox_environment accepts dictionary not list
tox_environment variable is passed [1] to Ansible 'environment' property
which accepts dictionary not list.

[1] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/tasks/main.yaml

Change-Id: I1e7236540356b3c0da502bd86ef7c5877dd1005e
2021-01-08 23:41:24 +03:00
Hervé Beraud
6fed1caf0c Allow to retrieve releasenotes requirements from a dedicated place
These changes allow to store reno reqs in a dedicated place.

Indeed some projects just want to build release notes without other
documentation.

The releasenotes playbook will use the ensure-sphinx role so I think it
make sense to allow this specific use case (no doc dir available).

Change-Id: I1d244bb47fb6b0b35df130f34007ff51047cd2f1
2021-01-07 14:00:10 +01:00
Vitaliy Lotorev
ade85d3233 Use braces for 'file' var in download-logs.sh
Currenlty braces are printed in logs. Most probabrly braces were
meant for variable expansion.

Change-Id: Id5a8c89bb4118840f9938f01ab2f42eceece429d
2020-12-24 01:49:54 +03:00