2157 Commits

Author SHA1 Message Date
Sorin Sbarnea
57415688ce Remove ansible-lint path exclusions
Fixes several rules with test-playbooks and remove path exclusions.

Since we introduced the progressive mode, exclusions are no longer
needed and not haivng them will assure newly added code follows
the same guidelines.

Change-Id: I0a72d34aff8cf23172e26c7f44e0f61571ec74b6
2021-04-27 08:22:32 +00:00
Clint Byrum
7eab57ab1e intercept-job -- self-service SSH access
This role is an attempt to allow self-service SSH access to nodes.

Change-Id: Icb6fb50b779c0bf2296e14436e4746355703f2ae
2021-04-23 15:14:06 +00:00
Tristan Cacqueray
6cd8980fc3 ensure-docker: prevent issue on centos-7 where the socket does not exists
This change mitigate an issue introduced by
https://review.opendev.org/c/zuul/zuul-jobs/+/787271

Fixes: https://bugs.launchpad.net/tripleo/+bug/1925372
Change-Id: I49f527b1c8a16e0b9f1718f23a960733003909ca
2021-04-21 17:12:09 +00:00
Gonéri Le Bouder
2bb9b4995a ensure-docker: ensure docker.socket is stopped
On Centos8, during the docker-ce installation, the docker.socket service
is start with a bogus state:

docker.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is not functional until restarted.

Later, when the `Assure docker service is running` task tries to start
the service, it fails with the following error:

dockerd[29743]: failed to load listeners: no sockets found via socket activation: make sure the service was started by systemd

Example:
https://0c7366f2ce9149f2de0c-399b55a396b5093070500a70ecbf09b9.ssl.cf1.rackcdn.com/410/c233496b96c70cfc6204e75d10116a96b08d4663/check/ansible-test-sanity-docker/787388f/ara-report/index.html

Another example: https://github.com/kata-containers/tests/issues/3103

Also: Remove use of kubectl --generator=run-pod/v1

This has been deprecated since 1.17 and removed since 1.20. run-pod wound
up being the only generator that did anything, so this parameter became a
no-op. This has to be squashed into this commit to unbreak the gate.

Change-Id: I666046fe2a3aa079643092c71573803851a67be2
2021-04-20 18:44:37 -05:00
Zuul
90c103eaad Merge "Add upload-logs-azure role" 2021-04-08 17:50:50 +00:00
James E. Blair
aec979387d Remove arm64 jobs (temporarily)
ARM64 nodes are currently unavailable, so temporarily remove these
jobs.  Unfortunately due to the auto-generated job listings, we need
to completely remove them, not just comment them out.

To be clear, we should add these back (but possibly in a dedicated
secondary pipeline) when we're able.

Change-Id: I462596dfb47ed88e2ba19e314c075227f6f912ac
2021-04-08 08:06:59 -07:00
Zuul
fbd26b177d Merge "ensure-zookeeper: add use_tmpfs parameter" 2021-04-06 13:47:21 +00:00
Zuul
c692c7f7df Merge "Document algorithm var for remove-build-sshkey" 2021-04-06 12:50:47 +00:00
Zuul
e29bcc6155 Merge "ensure-podman: Use official podman repos for ubuntu" 2021-04-06 09:17:02 +00:00
Paul Belanger
d2718417e2 ensure-podman: Use official podman repos for ubuntu
Reading the installation guide for podman, they reference opensuse.org
as the official package repos for ubuntu:

  https://podman.io/getting-started/installation

Using this repo allows us to pull in much newer version of podman on
ubuntu.  The current PPA package repo hasn't been updated since late
2019.

Change-Id: Ie34419184925a4bcf30422a782e6a238c11f2319
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2021-04-01 16:39:00 -04:00
Jeremy Stanley
016a20c0ff Document algorithm var for remove-build-sshkey
A recent hotfix added a fallback zuul_ssh_key_algorithm value for
the remove-build-sshkey role, reflect it in the role documentation.

Also remove some redundancies with default values specified in prose
for the add-build-sshkey role, when they were also indicated by the
:default: parameter.

Change-Id: If3f74256788283dedb17ad780da3ca4df4503cb4
2021-04-01 15:29:01 +00:00
James E. Blair
120a11ef20 Add upload-logs-azure role
Add support for uploading logs to Azure blob storage.

Change-Id: I0347977324b880123c6ed83ded3c39eb210612e2
2021-03-30 22:27:25 -07:00
Tristan Cacqueray
23cc3d4184 ensure-zookeeper: add use_tmpfs parameter
This change enables using the ensure-zookeeper role in environments
where mounting a filesystem is not allowed.

Change-Id: I1a05a895a45896eca91dccfcfc85aeb1a58689ff
2021-03-30 22:36:14 +00:00
Shturm, Svetlana
b5ffcea04d Fix undefined error for zuul_ssh_key_algorithm
Change-Id: I3466020028c01f0ada7965c6a1edf5a2730500b9
2021-03-29 11:54:33 -05:00
Zuul
20ceaad3c8 Merge "Create a template for ssh-key and size" 2021-03-29 16:10:25 +00:00
Zuul
6931e1cb9c Merge "Clone nvm from official source" 2021-03-22 02:17:49 +00:00
Monty Taylor
35b33341c4 Clone nvm from official source
nvm is now hosted at nvm-sh/nvm.

Change-Id: Idc00830e8404f556f8ebf38c93e67c2e574bf412
2021-03-20 10:10:14 -05:00
Jeremy Stanley
bac0326c00 Revert "Pin to npm4 until npm 5.6.0 comes out"
It looks like we never got around to cleaning this up after the npm
5.6.0 over 3 years ago. Now it seems to be causing the error
described in https://github.com/nodejs/help/issues/2874 so let's go
back to using the (presumed working) version of npm which the
node/nvm installation provides.

This reverts commit 0cd5075466ac77565f41074c28b05b0ad1459d86.

Change-Id: I0be3e8fd39845e82790b29d9f2c1e1179cb69e23
2021-03-20 13:30:34 +00:00
Sorin Sbarnea
0eaa5cf59a Upgrade ansible-lint to 5.0
- bumps ansible-lint to 5.0
- updates our custom rules to make them compatible with 5.0
- replace custom module mocking with native ansible-lint ones
- remove custom call of ansible-playbook --syntax-check as now this
  is done by ansible-lint
- assured molecule vars are hosted under a vars/ folder in order to
  avoid confusing linter detection.
- replaced custom rule for loop var names in role as now this this an
  optional core feature of the linter (see config)
- replaced custom rule no-same-owner with opt-in one (see config)

Change-Id: I233fae8c9036d295968a97ee80e07fde8846c633
2021-03-17 15:44:43 +00:00
GomathiselviS
e693c19d9a Create a template for ssh-key and size
Signed-off-by: GomathiselviS <gomathiselvi@gmail.com>
Change-Id: I0d34d3a0a40ea612dc06099b74d27216630f97c9
2021-03-17 09:54:45 -04:00
Zuul
78f18243f1 Merge "cabal-test: add install_args and build_args role var" 2021-03-17 12:31:44 +00:00
Zuul
4b8795f185 Merge "ensure-helm: skip helm init for helm v3 and newer" 2021-03-17 12:26:25 +00:00
Tristan Cacqueray
7af05ebcdc bindep.txt: skip python-devel for el8 platform
The python-devel package is not available on this platform.

Change-Id: I966c313d6dd0282b495fdeae4e92bbbe9b4d8fe3
2021-03-11 17:43:59 +00:00
Sorin Sbarnea
59dad009e9 Bits to keep ansible-lint happy
- moved vars file under vars to avoid failure to load file (newer
  version raise error if you try to put vars outside vars/defaults as
  it will assume they are broken playbooks.
- added missing file-modes on few tasks

Change-Id: I7217469d089b655ee16cd038391cffa4197c8c83
Part-Of: https://review.opendev.org/c/zuul/zuul-jobs/+/773245
2021-03-10 18:04:08 +00:00
Benjamin Schanzel
dfbd78d40d ensure-helm: skip helm init for helm v3 and newer
The init command was removed from helm with v3, no initialization is
needed anymore. So only run the init task if the version is <= 2.

Change-Id: I59b0c6efa0a2e6b94267fd3bb7a9701bc4c51048
2021-03-10 11:03:18 +01:00
Sorin Sbarnea
ad3167bd56 Fixes all tasks should be named rule
This prepared for ansible-lint v5 which no longer has exceptions
for task naming, requiring all to be named.

Change-Id: I5e761d1e3836fa270d7afdcf01780320001f820d
Part-Of: #773245
2021-03-08 13:30:32 +00:00
Tristan Cacqueray
8bde6d3504 cabal-test: add install_args and build_args role var
This change adds new role vars to enable setting
custom cabal arguments.

Change-Id: I4e4174117608bb76ca6e497687753cfdbc3acc02
2021-03-04 07:51:13 +00:00
James E. Blair
a31d70e6df ensure-zookeeper: use a tmpfs
This is helpful in testing situations which can see high amounts
of write traffic.

Change-Id: I69ff8116369c4ad40e2dc0c20af22370f3ee9927
2021-02-19 10:51:57 -08:00
James E. Blair
9f0d22d678 GCS logs: more robust Credential class
Google keeps changing the with_scopes method signature.  We need to
override that method in order to keep our path attribute around when
it re-instantiates the object.  As long as we always call refresh
after the object is created, we can drop our custom constructor and
then change with_scopes to use generic arguments.  This should keep
up with any further minor API changes without needing further local
changes.

Also, a minor inconsistency in command line arguments (used only for
local manual testing) is corrected.

Change-Id: Id41dfc464eb86429771a78c1fefae006e0915cb9
2021-02-19 09:53:08 -08:00
James E. Blair
4170cedf60 Revert "Update upload-logs roles to support endpoint override"
This reverts commit 862ae3f5d6b1c0e94bb24cfebf0b6b71e680e878.

We did not consider the effoct on the quick-download link that
is generated in opendev:

http://paste.openstack.org/show/802839/

Change-Id: I9702f8f1c0155ee3b13c74baaf2c09db72e690fd
2021-02-19 09:08:24 -08:00
Zuul
0263bf88e6 Merge "Update upload-logs roles to support endpoint override" 2021-02-19 16:12:45 +00:00
Zuul
0ea462f725 Merge "ensure-zookeeper: add use_tls role var" 2021-02-19 00:22:37 +00:00
Zuul
463b7679cf Merge "ensure-openshift: remove unused role var" 2021-02-19 00:22:33 +00:00
okozachenko
862ae3f5d6 Update upload-logs roles to support endpoint override
Add zuul_log_storage_proxy_address variable attempting to replace
storage endpoint address with.
The usecase is when the storage proxy is positioned infront of storage
endpoint.

Change-Id: I353cd50b76683212b0319a1e28f34901267c08e4
2021-02-18 16:09:27 -08:00
dg012m
bb4ebf4720 Allow customization of helm charts repos
Added default variables to customize the repo name to download
the chart testing helm chart and config files

Change-Id: I37f8db66bdfef7d64e59a89ca8d99b12f95c1be2
2021-02-18 09:07:06 +00:00
Tristan Cacqueray
68409bc81f ensure-zookeeper: add use_tls role var
This change adds a new attribute to setup zookeeper TLS.

It also adds support for Debian-derived distros.

Change-Id: Ifb5fc51f3b66be0b2dd1b8003507e21d8afe16fc
2021-02-17 16:06:36 -08:00
Tristan Cacqueray
bd93b3d424 ensure-openshift: remove unused role var
This change removes an ununsed role var as a follow-up for
https://review.opendev.org/c/zuul/zuul-jobs/+/775511

Change-Id: I2d966d744caae744c76d5c8c6b2693eb07e444d1
2021-02-16 18:32:16 +00:00
Zuul
20cdac6e33 Merge "ensure-openshift: workaround missing ansible26 repository" 2021-02-16 18:25:15 +00:00
Tristan Cacqueray
55ca357e39 ensure-openshift: workaround missing ansible26 repository
This change shortcirtcuits openshift repository installation to avoid
pulling the ansible26 repository.

Change-Id: I912a03e6502627340d002b419514f1ebdffb61e5
2021-02-16 15:43:00 +00:00
Zuul
d054d7ecc8 Merge "Fix CentOS wheel mirror URL" 2021-02-15 23:47:11 +00:00
Sorin Sbarnea
762ce6e96e update-json-file: avoid failure when destination does not exists
It is perfectly valid to want to add a value to a file that
does not exist yet, even the path may be missing.

This fixes last night regression when installing docker no longer
creates the /etc/docker folder, causing our MTU update to fail.

Change-Id: I0f037d1d6664de3c3b777aaf6da9cd7c3e8bb15f
Reference: https://review.rdoproject.org/zuul/builds?job_name=tox-py36-ci-config&project=rdo-infra/ci-config
2021-02-12 13:26:49 +00:00
Jeremy Stanley
b1521dbdb5 Report initial node filesystem sizes/utilization
Change Iba195e7c5cec372c6ba4daf7059da5b6fb6740ec implemented
collection of output for `df -i` (inode counts) and `df -m`
(megabytes data) in validate-host, but did not add them to the
report file template. Correct this oversight so that the collected
information will be included in that file.

Change-Id: I8c2c4a90f18394a04fde84355a89a15bf5aa66b4
2021-02-07 18:25:32 +00:00
Jeremy Stanley
e7915b65b7 Collect initial node filesystem sizes/utilization
Include calls to `df -i` (inode counts) and `df -m` (megabytes data)
in validate-host, to aid in troubleshooting build failures where the
builds start out with too little free space. This way the initial
capacity and utilization of all available filesystems will be
recorded with other basic node diagnostic data.

Change-Id: Iba195e7c5cec372c6ba4daf7059da5b6fb6740ec
2021-02-06 20:27:01 +00:00
Zuul
8bdb2b538c Merge "bindep: remove set_fact usage when converting string to list" 2021-02-03 20:09:46 +00:00
Zuul
5bb91e867b Merge "Use urlencoded filenames in test fixtures" 2021-02-01 14:11:44 +00:00
Albin Vass
6f2ff3f1b8 bindep: remove set_fact usage when converting string to list
This can cause problems where set_fact isn't able to override the
variable when bindep is used with include_role and bindep_file is set as
a variable in the include_role task.

Change-Id: I3099f1802f2d30df1265a9abc11a53c923ed4832
2021-02-01 15:09:07 +01:00
Zuul
79c44ead3a Merge "Fix repo files for CentOS Stream" 2021-01-28 15:07:00 +00:00
Sorin Sbarnea
938da600b7 Add tox-py39 job
Change-Id: I46cac3396bd870eb1f54820921df09946184605a
2021-01-27 10:14:32 +00:00
Javier Pena
cfada06486 Fix repo files for CentOS Stream
Some repo file names and ids do not match the files provided by
CentOS, which can cause duplicated repos [1].

[1] - https://8a51f3027f5f50a14ba7-fcfc85786424b7f5a3fcadff9da35f94.ssl.cf5.rackcdn.com/770771/4/check/packstack-centos8s-integration-scenario002/9dc0bd2/logs/etc/yum.repos.d/index.html

Change-Id: Ib89ad9f79cd5317b3178301a47462277004fa81a
2021-01-22 17:46:26 +01:00
Zuul
7d0a646c9e Merge "Rename config repos file config for CentOS Stream" 2021-01-22 08:15:22 +00:00