2373 Commits

Author SHA1 Message Date
Zuul
b70a6b14fe Merge "Handle no-sudo in stage-output" 2021-06-02 07:03:07 +00:00
Zuul
0427f2ab8d Merge "Add ensure-skopeo role" 2021-06-01 01:01:10 +00:00
Zuul
98384d489f Merge "Use openstacksdk 0.45.0 for python2.7" 2021-06-01 01:01:06 +00:00
James E. Blair
3c17cf9ccf Tidy up file matcher for bindep jobs
The bindep jobs have a file matcher on the general-roles-jobs.yaml
file, but that shouldn't be necessary since Zuul runs jobs if they
definitions change.  Remove the extraneous match so that we don't
run lots of unecessary bindep jobs.

Change-Id: Ic8b1e7bf4cccf555b3c47a7ac9fcb9aee422714b
2021-05-31 15:52:12 -07:00
James E. Blair
f0d7c980b5 Handle no-sudo in stage-output
The stage-output role had two assumptions:

1) The zuul_copy_output variable would always be defined
2) The role would be able to sudo on the remote node

To make it easier for users who want to use this in a base job,
remove both of those assumptions.

1) We now supply a default empty dict if the variable is not defined
so that the role does not fail with an error.

2) We check to see if we can sudo on the remote node, and if we can,
we do when copying files; otherwise we don't, and assume that the
user will only specify files they have access to (if they don't then
the copy will fail).

Change-Id: I5428c44adfafac4872342b59a92de311f41687b6
2021-05-31 15:47:57 -07:00
Rodion Gyrbu
ace6818e67 Bump golang version
Change-Id: I8dca48f45ee0191aae678c45d79c5841836d428e
2021-05-28 17:29:18 +03:00
Ian Wienand
3565eb4ab8 ensure-zookeeper: better match return code
Every time I open up the Zuul console for certain jobs it unrolls the
pre-playbook section in the console because it looks like this task
has a "FAILED" status.

pkill man page says that 0/1 are "process killed" and "no match"
respectively; set failed_when to >1 so these return codes don't cause
spurious failures.

Change-Id: I23112b1101c991c1714d69f7568f83c2dcd605dd
2021-05-28 14:05:52 +10:00
Guillaume Chauvel
911fe88a9f Fix buildset-registry test on focal
Fix ensure-kubernetes role for focal
As focal doesn't exist for project atomic ppa [1]
Install is performed from opensuse repository only

As cri-o package 1.15 for ubuntu focal doesn't exist, update to 1.16

[1] http://ppa.launchpad.net/projectatomic/ppa/ubuntu/dists/
[2] https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/

Change-Id: I7f02b7337a5f51a86be1a2332f2305b0ae722934
2021-05-25 16:38:00 +02:00
Guillaume Chauvel
b517747623 Add ensure-skopeo role
Role copied and modified from ensure-podman

As focal doesn't exist for project atomic ppa [1]
Install is performed from opensuse repository only

[1] http://ppa.launchpad.net/projectatomic/ppa/ubuntu/dists/

Change-Id: I72fc2e68768664b80c39bd47295330131337d8b5
2021-05-25 16:38:00 +02:00
Zuul
0fcc5ff5ac Merge "collect-container-logs: don't copy on failure" 2021-05-06 07:35:55 +00:00
Jeremy Stanley
795dad6444 Test Debian Buster and Bullseye
Debian Buster is the current stable, and Bullseye is the impending
stable (likely within the month). The parent of this change aims to
fix security mirror configuration for Bullseye, but to do that we
need to add testing for it. While we're at it, drop the Stretch
(oldoldstable) jobs to keep the job count from exploding too much.

Change-Id: I278246eff347dd4bdb3471a8db95d30fcba0f17a
2021-05-03 18:39:56 +00:00
Jeremy Stanley
750be2e2de Add new Debian security mirror suite pattern
Starting with Debian 11 (bullseye), security packages are in
bullseye-security as opposed to older releases like buster/updates.
List the last several stable releases in hopes nobody is trying to
use this role to configure platforms older than Debian 8 (jessie,
the current "oldoldstable").

A followup change demonstrates this works in the test-base-roles
job, but because the job matrices have to be updated in one fell
swoop, and many of those jobs won't work without this change already
merged (due to protected use in our base job), it's not tested
directly within this change.

Change-Id: I2d7712cbfd037a65b9025980a6c0cccd917f8947
2021-05-03 18:39:34 +00:00
Sorin Sbarnea
57415688ce Remove ansible-lint path exclusions
Fixes several rules with test-playbooks and remove path exclusions.

Since we introduced the progressive mode, exclusions are no longer
needed and not haivng them will assure newly added code follows
the same guidelines.

Change-Id: I0a72d34aff8cf23172e26c7f44e0f61571ec74b6
2021-04-27 08:22:32 +00:00
Clint Byrum
7eab57ab1e intercept-job -- self-service SSH access
This role is an attempt to allow self-service SSH access to nodes.

Change-Id: Icb6fb50b779c0bf2296e14436e4746355703f2ae
2021-04-23 15:14:06 +00:00
Tristan Cacqueray
6cd8980fc3 ensure-docker: prevent issue on centos-7 where the socket does not exists
This change mitigate an issue introduced by
https://review.opendev.org/c/zuul/zuul-jobs/+/787271

Fixes: https://bugs.launchpad.net/tripleo/+bug/1925372
Change-Id: I49f527b1c8a16e0b9f1718f23a960733003909ca
2021-04-21 17:12:09 +00:00
Gonéri Le Bouder
2bb9b4995a ensure-docker: ensure docker.socket is stopped
On Centos8, during the docker-ce installation, the docker.socket service
is start with a bogus state:

docker.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is not functional until restarted.

Later, when the `Assure docker service is running` task tries to start
the service, it fails with the following error:

dockerd[29743]: failed to load listeners: no sockets found via socket activation: make sure the service was started by systemd

Example:
https://0c7366f2ce9149f2de0c-399b55a396b5093070500a70ecbf09b9.ssl.cf1.rackcdn.com/410/c233496b96c70cfc6204e75d10116a96b08d4663/check/ansible-test-sanity-docker/787388f/ara-report/index.html

Another example: https://github.com/kata-containers/tests/issues/3103

Also: Remove use of kubectl --generator=run-pod/v1

This has been deprecated since 1.17 and removed since 1.20. run-pod wound
up being the only generator that did anything, so this parameter became a
no-op. This has to be squashed into this commit to unbreak the gate.

Change-Id: I666046fe2a3aa079643092c71573803851a67be2
2021-04-20 18:44:37 -05:00
Ian Wienand
9b17d6c9df collect-container-logs: don't copy on failure
After a docker failure on a recent job I noticed a single file called
"Got" in the container logs directory; turns out this had tried to
parse the error message "Got permission denied while trying to connect
to the Docker daemon socket ..." as containers.

We don't want this to fail the job hence the ignores here.  However,
we shouldn't try to collect any logs if listing the containers fails.
Move creating the directory first as that is safe.

Change-Id: I41db956964f695cfcc15e30cab8cd4f8c31d3706
2021-04-20 12:10:02 +10:00
Zuul
90c103eaad Merge "Add upload-logs-azure role" 2021-04-08 17:50:50 +00:00
James E. Blair
aec979387d Remove arm64 jobs (temporarily)
ARM64 nodes are currently unavailable, so temporarily remove these
jobs.  Unfortunately due to the auto-generated job listings, we need
to completely remove them, not just comment them out.

To be clear, we should add these back (but possibly in a dedicated
secondary pipeline) when we're able.

Change-Id: I462596dfb47ed88e2ba19e314c075227f6f912ac
2021-04-08 08:06:59 -07:00
Albin Vass
da8a64b638 Use openstacksdk 0.45.0 for python2.7
Change-Id: Ib5b2bbd148fcee73f0069288cd368759d7c10cfb
2021-04-06 16:53:30 +02:00
Zuul
fbd26b177d Merge "ensure-zookeeper: add use_tmpfs parameter" 2021-04-06 13:47:21 +00:00
Zuul
c692c7f7df Merge "Document algorithm var for remove-build-sshkey" 2021-04-06 12:50:47 +00:00
Zuul
e29bcc6155 Merge "ensure-podman: Use official podman repos for ubuntu" 2021-04-06 09:17:02 +00:00
Paul Belanger
d2718417e2 ensure-podman: Use official podman repos for ubuntu
Reading the installation guide for podman, they reference opensuse.org
as the official package repos for ubuntu:

  https://podman.io/getting-started/installation

Using this repo allows us to pull in much newer version of podman on
ubuntu.  The current PPA package repo hasn't been updated since late
2019.

Change-Id: Ie34419184925a4bcf30422a782e6a238c11f2319
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2021-04-01 16:39:00 -04:00
Jeremy Stanley
016a20c0ff Document algorithm var for remove-build-sshkey
A recent hotfix added a fallback zuul_ssh_key_algorithm value for
the remove-build-sshkey role, reflect it in the role documentation.

Also remove some redundancies with default values specified in prose
for the add-build-sshkey role, when they were also indicated by the
:default: parameter.

Change-Id: If3f74256788283dedb17ad780da3ca4df4503cb4
2021-04-01 15:29:01 +00:00
James E. Blair
120a11ef20 Add upload-logs-azure role
Add support for uploading logs to Azure blob storage.

Change-Id: I0347977324b880123c6ed83ded3c39eb210612e2
2021-03-30 22:27:25 -07:00
Tristan Cacqueray
23cc3d4184 ensure-zookeeper: add use_tmpfs parameter
This change enables using the ensure-zookeeper role in environments
where mounting a filesystem is not allowed.

Change-Id: I1a05a895a45896eca91dccfcfc85aeb1a58689ff
2021-03-30 22:36:14 +00:00
Shturm, Svetlana
b5ffcea04d Fix undefined error for zuul_ssh_key_algorithm
Change-Id: I3466020028c01f0ada7965c6a1edf5a2730500b9
2021-03-29 11:54:33 -05:00
Zuul
20ceaad3c8 Merge "Create a template for ssh-key and size" 2021-03-29 16:10:25 +00:00
Zuul
6931e1cb9c Merge "Clone nvm from official source" 2021-03-22 02:17:49 +00:00
Monty Taylor
35b33341c4 Clone nvm from official source
nvm is now hosted at nvm-sh/nvm.

Change-Id: Idc00830e8404f556f8ebf38c93e67c2e574bf412
2021-03-20 10:10:14 -05:00
Jeremy Stanley
bac0326c00 Revert "Pin to npm4 until npm 5.6.0 comes out"
It looks like we never got around to cleaning this up after the npm
5.6.0 over 3 years ago. Now it seems to be causing the error
described in https://github.com/nodejs/help/issues/2874 so let's go
back to using the (presumed working) version of npm which the
node/nvm installation provides.

This reverts commit 0cd5075466ac77565f41074c28b05b0ad1459d86.

Change-Id: I0be3e8fd39845e82790b29d9f2c1e1179cb69e23
2021-03-20 13:30:34 +00:00
Sorin Sbarnea
0eaa5cf59a Upgrade ansible-lint to 5.0
- bumps ansible-lint to 5.0
- updates our custom rules to make them compatible with 5.0
- replace custom module mocking with native ansible-lint ones
- remove custom call of ansible-playbook --syntax-check as now this
  is done by ansible-lint
- assured molecule vars are hosted under a vars/ folder in order to
  avoid confusing linter detection.
- replaced custom rule for loop var names in role as now this this an
  optional core feature of the linter (see config)
- replaced custom rule no-same-owner with opt-in one (see config)

Change-Id: I233fae8c9036d295968a97ee80e07fde8846c633
2021-03-17 15:44:43 +00:00
GomathiselviS
e693c19d9a Create a template for ssh-key and size
Signed-off-by: GomathiselviS <gomathiselvi@gmail.com>
Change-Id: I0d34d3a0a40ea612dc06099b74d27216630f97c9
2021-03-17 09:54:45 -04:00
Zuul
78f18243f1 Merge "cabal-test: add install_args and build_args role var" 2021-03-17 12:31:44 +00:00
Zuul
4b8795f185 Merge "ensure-helm: skip helm init for helm v3 and newer" 2021-03-17 12:26:25 +00:00
Tristan Cacqueray
7af05ebcdc bindep.txt: skip python-devel for el8 platform
The python-devel package is not available on this platform.

Change-Id: I966c313d6dd0282b495fdeae4e92bbbe9b4d8fe3
2021-03-11 17:43:59 +00:00
Sorin Sbarnea
59dad009e9 Bits to keep ansible-lint happy
- moved vars file under vars to avoid failure to load file (newer
  version raise error if you try to put vars outside vars/defaults as
  it will assume they are broken playbooks.
- added missing file-modes on few tasks

Change-Id: I7217469d089b655ee16cd038391cffa4197c8c83
Part-Of: https://review.opendev.org/c/zuul/zuul-jobs/+/773245
2021-03-10 18:04:08 +00:00
Benjamin Schanzel
dfbd78d40d ensure-helm: skip helm init for helm v3 and newer
The init command was removed from helm with v3, no initialization is
needed anymore. So only run the init task if the version is <= 2.

Change-Id: I59b0c6efa0a2e6b94267fd3bb7a9701bc4c51048
2021-03-10 11:03:18 +01:00
Sorin Sbarnea
ad3167bd56 Fixes all tasks should be named rule
This prepared for ansible-lint v5 which no longer has exceptions
for task naming, requiring all to be named.

Change-Id: I5e761d1e3836fa270d7afdcf01780320001f820d
Part-Of: #773245
2021-03-08 13:30:32 +00:00
Tristan Cacqueray
8bde6d3504 cabal-test: add install_args and build_args role var
This change adds new role vars to enable setting
custom cabal arguments.

Change-Id: I4e4174117608bb76ca6e497687753cfdbc3acc02
2021-03-04 07:51:13 +00:00
James E. Blair
a31d70e6df ensure-zookeeper: use a tmpfs
This is helpful in testing situations which can see high amounts
of write traffic.

Change-Id: I69ff8116369c4ad40e2dc0c20af22370f3ee9927
2021-02-19 10:51:57 -08:00
James E. Blair
9f0d22d678 GCS logs: more robust Credential class
Google keeps changing the with_scopes method signature.  We need to
override that method in order to keep our path attribute around when
it re-instantiates the object.  As long as we always call refresh
after the object is created, we can drop our custom constructor and
then change with_scopes to use generic arguments.  This should keep
up with any further minor API changes without needing further local
changes.

Also, a minor inconsistency in command line arguments (used only for
local manual testing) is corrected.

Change-Id: Id41dfc464eb86429771a78c1fefae006e0915cb9
2021-02-19 09:53:08 -08:00
James E. Blair
4170cedf60 Revert "Update upload-logs roles to support endpoint override"
This reverts commit 862ae3f5d6b1c0e94bb24cfebf0b6b71e680e878.

We did not consider the effoct on the quick-download link that
is generated in opendev:

http://paste.openstack.org/show/802839/

Change-Id: I9702f8f1c0155ee3b13c74baaf2c09db72e690fd
2021-02-19 09:08:24 -08:00
Zuul
0263bf88e6 Merge "Update upload-logs roles to support endpoint override" 2021-02-19 16:12:45 +00:00
Zuul
0ea462f725 Merge "ensure-zookeeper: add use_tls role var" 2021-02-19 00:22:37 +00:00
Zuul
463b7679cf Merge "ensure-openshift: remove unused role var" 2021-02-19 00:22:33 +00:00
okozachenko
862ae3f5d6 Update upload-logs roles to support endpoint override
Add zuul_log_storage_proxy_address variable attempting to replace
storage endpoint address with.
The usecase is when the storage proxy is positioned infront of storage
endpoint.

Change-Id: I353cd50b76683212b0319a1e28f34901267c08e4
2021-02-18 16:09:27 -08:00
dg012m
bb4ebf4720 Allow customization of helm charts repos
Added default variables to customize the repo name to download
the chart testing helm chart and config files

Change-Id: I37f8db66bdfef7d64e59a89ca8d99b12f95c1be2
2021-02-18 09:07:06 +00:00
Tristan Cacqueray
68409bc81f ensure-zookeeper: add use_tls role var
This change adds a new attribute to setup zookeeper TLS.

It also adds support for Debian-derived distros.

Change-Id: Ifb5fc51f3b66be0b2dd1b8003507e21d8afe16fc
2021-02-17 16:06:36 -08:00