1803 Commits

Author SHA1 Message Date
Zuul
4b8795f185 Merge "ensure-helm: skip helm init for helm v3 and newer" 2021-03-17 12:26:25 +00:00
Sorin Sbarnea
59dad009e9 Bits to keep ansible-lint happy
- moved vars file under vars to avoid failure to load file (newer
  version raise error if you try to put vars outside vars/defaults as
  it will assume they are broken playbooks.
- added missing file-modes on few tasks

Change-Id: I7217469d089b655ee16cd038391cffa4197c8c83
Part-Of: https://review.opendev.org/c/zuul/zuul-jobs/+/773245
2021-03-10 18:04:08 +00:00
Benjamin Schanzel
dfbd78d40d ensure-helm: skip helm init for helm v3 and newer
The init command was removed from helm with v3, no initialization is
needed anymore. So only run the init task if the version is <= 2.

Change-Id: I59b0c6efa0a2e6b94267fd3bb7a9701bc4c51048
2021-03-10 11:03:18 +01:00
Sorin Sbarnea
ad3167bd56 Fixes all tasks should be named rule
This prepared for ansible-lint v5 which no longer has exceptions
for task naming, requiring all to be named.

Change-Id: I5e761d1e3836fa270d7afdcf01780320001f820d
Part-Of: #773245
2021-03-08 13:30:32 +00:00
Tristan Cacqueray
8bde6d3504 cabal-test: add install_args and build_args role var
This change adds new role vars to enable setting
custom cabal arguments.

Change-Id: I4e4174117608bb76ca6e497687753cfdbc3acc02
2021-03-04 07:51:13 +00:00
James E. Blair
a31d70e6df ensure-zookeeper: use a tmpfs
This is helpful in testing situations which can see high amounts
of write traffic.

Change-Id: I69ff8116369c4ad40e2dc0c20af22370f3ee9927
2021-02-19 10:51:57 -08:00
James E. Blair
9f0d22d678 GCS logs: more robust Credential class
Google keeps changing the with_scopes method signature.  We need to
override that method in order to keep our path attribute around when
it re-instantiates the object.  As long as we always call refresh
after the object is created, we can drop our custom constructor and
then change with_scopes to use generic arguments.  This should keep
up with any further minor API changes without needing further local
changes.

Also, a minor inconsistency in command line arguments (used only for
local manual testing) is corrected.

Change-Id: Id41dfc464eb86429771a78c1fefae006e0915cb9
2021-02-19 09:53:08 -08:00
James E. Blair
4170cedf60 Revert "Update upload-logs roles to support endpoint override"
This reverts commit 862ae3f5d6b1c0e94bb24cfebf0b6b71e680e878.

We did not consider the effoct on the quick-download link that
is generated in opendev:

http://paste.openstack.org/show/802839/

Change-Id: I9702f8f1c0155ee3b13c74baaf2c09db72e690fd
2021-02-19 09:08:24 -08:00
Zuul
0263bf88e6 Merge "Update upload-logs roles to support endpoint override" 2021-02-19 16:12:45 +00:00
Zuul
0ea462f725 Merge "ensure-zookeeper: add use_tls role var" 2021-02-19 00:22:37 +00:00
Zuul
463b7679cf Merge "ensure-openshift: remove unused role var" 2021-02-19 00:22:33 +00:00
okozachenko
862ae3f5d6 Update upload-logs roles to support endpoint override
Add zuul_log_storage_proxy_address variable attempting to replace
storage endpoint address with.
The usecase is when the storage proxy is positioned infront of storage
endpoint.

Change-Id: I353cd50b76683212b0319a1e28f34901267c08e4
2021-02-18 16:09:27 -08:00
dg012m
bb4ebf4720 Allow customization of helm charts repos
Added default variables to customize the repo name to download
the chart testing helm chart and config files

Change-Id: I37f8db66bdfef7d64e59a89ca8d99b12f95c1be2
2021-02-18 09:07:06 +00:00
Tristan Cacqueray
68409bc81f ensure-zookeeper: add use_tls role var
This change adds a new attribute to setup zookeeper TLS.

It also adds support for Debian-derived distros.

Change-Id: Ifb5fc51f3b66be0b2dd1b8003507e21d8afe16fc
2021-02-17 16:06:36 -08:00
Tristan Cacqueray
bd93b3d424 ensure-openshift: remove unused role var
This change removes an ununsed role var as a follow-up for
https://review.opendev.org/c/zuul/zuul-jobs/+/775511

Change-Id: I2d966d744caae744c76d5c8c6b2693eb07e444d1
2021-02-16 18:32:16 +00:00
Zuul
20cdac6e33 Merge "ensure-openshift: workaround missing ansible26 repository" 2021-02-16 18:25:15 +00:00
Tristan Cacqueray
55ca357e39 ensure-openshift: workaround missing ansible26 repository
This change shortcirtcuits openshift repository installation to avoid
pulling the ansible26 repository.

Change-Id: I912a03e6502627340d002b419514f1ebdffb61e5
2021-02-16 15:43:00 +00:00
Zuul
d054d7ecc8 Merge "Fix CentOS wheel mirror URL" 2021-02-15 23:47:11 +00:00
Sorin Sbarnea
762ce6e96e update-json-file: avoid failure when destination does not exists
It is perfectly valid to want to add a value to a file that
does not exist yet, even the path may be missing.

This fixes last night regression when installing docker no longer
creates the /etc/docker folder, causing our MTU update to fail.

Change-Id: I0f037d1d6664de3c3b777aaf6da9cd7c3e8bb15f
Reference: https://review.rdoproject.org/zuul/builds?job_name=tox-py36-ci-config&project=rdo-infra/ci-config
2021-02-12 13:26:49 +00:00
Jeremy Stanley
b1521dbdb5 Report initial node filesystem sizes/utilization
Change Iba195e7c5cec372c6ba4daf7059da5b6fb6740ec implemented
collection of output for `df -i` (inode counts) and `df -m`
(megabytes data) in validate-host, but did not add them to the
report file template. Correct this oversight so that the collected
information will be included in that file.

Change-Id: I8c2c4a90f18394a04fde84355a89a15bf5aa66b4
2021-02-07 18:25:32 +00:00
Jeremy Stanley
e7915b65b7 Collect initial node filesystem sizes/utilization
Include calls to `df -i` (inode counts) and `df -m` (megabytes data)
in validate-host, to aid in troubleshooting build failures where the
builds start out with too little free space. This way the initial
capacity and utilization of all available filesystems will be
recorded with other basic node diagnostic data.

Change-Id: Iba195e7c5cec372c6ba4daf7059da5b6fb6740ec
2021-02-06 20:27:01 +00:00
Zuul
8bdb2b538c Merge "bindep: remove set_fact usage when converting string to list" 2021-02-03 20:09:46 +00:00
Zuul
5bb91e867b Merge "Use urlencoded filenames in test fixtures" 2021-02-01 14:11:44 +00:00
Albin Vass
6f2ff3f1b8 bindep: remove set_fact usage when converting string to list
This can cause problems where set_fact isn't able to override the
variable when bindep is used with include_role and bindep_file is set as
a variable in the include_role task.

Change-Id: I3099f1802f2d30df1265a9abc11a53c923ed4832
2021-02-01 15:09:07 +01:00
Javier Pena
cfada06486 Fix repo files for CentOS Stream
Some repo file names and ids do not match the files provided by
CentOS, which can cause duplicated repos [1].

[1] - https://8a51f3027f5f50a14ba7-fcfc85786424b7f5a3fcadff9da35f94.ssl.cf5.rackcdn.com/770771/4/check/packstack-centos8s-integration-scenario002/9dc0bd2/logs/etc/yum.repos.d/index.html

Change-Id: Ib89ad9f79cd5317b3178301a47462277004fa81a
2021-01-22 17:46:26 +01:00
Zuul
7d0a646c9e Merge "Rename config repos file config for CentOS Stream" 2021-01-22 08:15:22 +00:00
Alfredo Moralejo
8beac88116 Rename config repos file config for CentOS Stream
Currently, config file is CentOS-8-stream.yaml but the value of
ansible_distribution_major_version fact in CentOS stream is 8, not
8-stream [1], so jobs running in centos-8-stream images are actually
using centos-8 repositories.

This patch is modyfing the list of config files to use ansible_lsb.id
fact wich is CentOSStream and renaming the file name.

[1] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_792/770771/2/check/packstack-centos8s-integration-scenario002/792342c/zuul-info/host-info.centos8s.yaml

Change-Id: I7199acdd45151d2553ef17588909615b5a7c5b61
2021-01-21 09:00:36 +00:00
Daniel Blixt
dfbc8e2ec8 Use urlencoded filenames in test fixtures
When checking out repo to Windows environment, some files in test
fixture file trees do not get checked out. By setting renaming those
files with an urlencoded name, and populating a temporary directory
with corresponding file structure and decoded filenames during test by
a fixtures script, this will not cause checkout problems for those who
 want to read the repo in Win environment

Change-Id: If354eaf3f0bf2e86ddc05e3750aa5f0067dd9e21
2021-01-20 08:37:26 +01:00
Gorka Eguileor
4284afc584 Fix CentOS wheel mirror URL
For CentOS we are currently using default values for "wheel_mirror" from
roles/configure-mirrors/defaults/main.yaml, which means that the URL
uses the full distribution version, which is wrong, we should be using
the major version instead.

As an example, centos-8 jobs are trying to use
  https://mirror.bhs1.ovh.opendev.org/wheel/centos-8.3-x86_64
When they should be using
  https://mirror.bhs1.ovh.opendev.org/wheel/centos-8-x86_64

Which makes some jobs fail to find alembic wheel files and then they
fail to build from source because the CI was expecting the wheel mirror
to have it.

This patch fixes this by creating a Centos specific "wheel_mirror"
variable.

Change-Id: I11199cf943a3070fd62bf486fe2b06381db4b04d
(cherry picked from commit c9cad38eebf1bdabb10b79699598d6ba0f098f54)
2021-01-19 14:13:04 +01:00
Zuul
57a2f59f5a Merge "Pass environment variables to 'tox envlist config' task" 2021-01-18 14:31:00 +00:00
Tobias Henkel
3cc366c9c6
Fail mirror-workspace-git-repos if checkout failed
If the checkout failed the mirror-workspace-git-repos must fail the
job. Otherwise this can lead to a broken gate because a different
state than the state that should be tested can be tested silently.

Change-Id: I98e948776ec4b749bc1475ee3a17e37007591118
2021-01-18 14:27:52 +01:00
Benedikt Loeffler
9eaab6fd74 Pass environment variables to 'tox envlist config' task
As of version 3.21.0, tox now throws an exception if an environment variable
used in config is missing. We now need to pass the variables also to the task
"Get tox envlist config".

Change-Id: I4073f55b6ba3456f3d2db40cafb692efbb478914
2021-01-14 15:25:31 +01:00
Zuul
71132b658c Merge "upload-artifactory: no_log upload task" 2021-01-14 08:51:01 +00:00
Zuul
6f6e244cf0 Merge "Enable installing nimble siblings" 2021-01-13 07:13:30 +00:00
Zuul
f388375572 Merge "Allow to retrieve releasenotes requirements from a dedicated place" 2021-01-13 01:36:09 +00:00
Zuul
de9111a9a3 Merge "Add configuration to make logs public" 2021-01-13 01:02:15 +00:00
Vitaliy Lotorev
ecc11af6fe Document Python siblings handling for tox role
Brief overview how actually siblings are installed helps
proper usage of 'tox' role.

Change-Id: Id27c9c60a40e43c0e351849470ccb4d0d11739a3
2021-01-08 23:41:27 +03:00
Vitaliy Lotorev
4708a35220 Clarity tox_environment accepts dictionary not list
tox_environment variable is passed [1] to Ansible 'environment' property
which accepts dictionary not list.

[1] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/tasks/main.yaml

Change-Id: I1e7236540356b3c0da502bd86ef7c5877dd1005e
2021-01-08 23:41:24 +03:00
Hervé Beraud
6fed1caf0c Allow to retrieve releasenotes requirements from a dedicated place
These changes allow to store reno reqs in a dedicated place.

Indeed some projects just want to build release notes without other
documentation.

The releasenotes playbook will use the ensure-sphinx role so I think it
make sense to allow this specific use case (no doc dir available).

Change-Id: I1d244bb47fb6b0b35df130f34007ff51047cd2f1
2021-01-07 14:00:10 +01:00
Vitaliy Lotorev
ade85d3233 Use braces for 'file' var in download-logs.sh
Currenlty braces are printed in logs. Most probabrly braces were
meant for variable expansion.

Change-Id: Id5a8c89bb4118840f9938f01ab2f42eceece429d
2020-12-24 01:49:54 +03:00
Albin Vass
ca2936a855 upload-artifactory: no_log upload task
If the upload request fails artifactory can respond with the apiKey as part of
the body. Add a no_log to avoid leaking apiKeys.

Change-Id: I2e03c3e06754e650ca2987c6a0a7ae2b8ed86268
2020-12-21 18:33:35 +01:00
Albin Vass
5f9dcd987f Add configuration to make logs public
Change-Id: I5d196623df402bc4eb02bbb13e557da38a849018
2020-12-21 18:12:25 +01:00
Zuul
9721093097 Merge "Create tox_package_name for tox role" 2020-12-18 18:47:37 +00:00
5cfc37b300 Rename CentOS8 repo files to CentOS-Linux
With CentOS8.3 release repo files are now
renamed to CentOS-Linux*, update in zuul-jobs
too to sync, also update repo names as per
latest CentOS8 minor release.

The change is needed now as dib images are updated
to reflect new repo files post [1].

[1] https://review.opendev.org/c/openstack/diskimage-builder/+/765963

Change-Id: I7ed9bd582043717ae5bae303ea9b32db7f73008e
2020-12-18 14:23:08 +05:30
Paul Belanger
c6bf69e60b Create tox_package_name for tox role
This allows projects, that use tox but may not have a setup.cfg file
still use tox siblings. We do this to allow non-python project, to use
tox as an entry point for testing, and still have depends-on
requirements work in zuul.

Change-Id: I9b37117b27ff6b7e436d456b6cbae39ccb9b968c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2020-12-16 11:29:39 -05:00
manchandavishal
77833088f2 Add Firefox browser role for horizon nodejs job
This patch adds Firefox browser role in
``roles/nodejs-test-dependencies/tasks/main.yaml`` so that horizon
nodejs jobs can run with Firefox browser because as of now horizon
nodejs jobs fail with chromium-browser due to not able to access
some URL. So it is better to run these jobs with Firefox to unblock
the gate then we can investigate further how to fix it with chromium
browser.

Change-Id: I4063d88a7dfe30bef7cbbbfbbc1ac9790686ca8b
2020-12-14 04:43:28 +00:00
Albin Vass
49b06e31de Enable installing nimble siblings
Change-Id: I58f7b3038bd8239d5f04f66a18aad56ec77dad7d
2020-12-07 15:23:27 +01:00
Albin Vass
5b3a48e73b GZIPCompressedStream default read size to -1
To be compatible with the io.RawIOBase interface
the GZIPCompressedStream read method argument should be called
`size` and have a default value of -1 meaning until EOF.

See:
https://docs.python.org/3/library/io.html#io.RawIOBase

Change-Id: Ie8b4c77f6c730c91bb4d4997dcb7f9a9acde0f31
2020-12-02 20:59:55 +01:00
Paul Belanger
e9226bbaab Fix typo with container_images siblings logic
Change-Id: I4c54ccd47677fde30fc0621005f0ef9d51df1f66
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2020-11-25 14:30:48 -05:00
Jeremy Stanley
64d0d39d0b Use Python 3.x with launchpadlib
The add-launchpad-credentials role installs the distro's
python-launchpadlib package which, for the distros carrying that
package at all, sets it up for Python 2.7. At this point Ubuntu has
dropped the python-launchpadlib package entirely from their latest
LTS release and Debian will do the same in their next release, but
they have also carried a python3-launchpadlib package for years.
Switch to it so this will work on newer systems.

Change-Id: I92421cae32bf2da5851144337820685c79ec19ee
2020-11-24 01:29:36 +00:00