1069 Commits

Author SHA1 Message Date
Mohammed Naser
9254fbb5ff ensure-tox: add prefer_python2
This patch enables the ability to set a value which prefers the
installation of tox within python2 over python3.

Change-Id: I29c9585aa9048c0e2855ec1eaf1f48041cfe46e2
2020-03-04 07:21:20 +00:00
Zuul
3cbce518ad Merge "install-javascript-packages: add tox_constraints_file" 2020-02-26 05:40:28 +00:00
Zuul
5ac43d06e1 Merge "Fix install-docker on centos-8" 2020-02-25 17:52:01 +00:00
Zuul
fe15bf4934 Merge "Fix cleanup of symlink fixtures" 2020-02-25 16:46:09 +00:00
Sorin Sbarnea
299a654806 Fix install-docker on centos-8
Implements workaround for the missing --nobest option in Ansible
package module in order to succeed installing docker-ce.

Change-Id: I96e133dc5c59b9c811ff69df7651adadfcdf3a97
2020-02-25 10:56:37 +00:00
Jeremy Stanley
f607fd9a24 Precreate AFS target directory in upload
This works around missing parent directories in AFS, since rsync
will create the target directory but not its parent directories. In
cases where the parent directory does not yet exist, it's nice to
create it automatically first, if we can.

Change-Id: Ia136ea3a416d6085d1f6e277b4c071573c5660e4
2020-02-24 19:40:04 +00:00
Zuul
f05f26b1ca Merge "Use CentOS8 dependencies repo to install openvswitch" 2020-02-24 18:40:56 +00:00
Zuul
59561f5a51 Merge "Remove deprecated upload-afs role" 2020-02-24 18:27:11 +00:00
Alfredo Moralejo
75d896ab97 Use CentOS8 dependencies repo to install openvswitch
Currently, centos8 jobs are using old RHEL repo. A repo with CentOS8
packages has been created by RDO, so let's switch to it.

Change-Id: I0df0950de5a42bb7a425609c3d5d065550d956f4
2020-02-24 18:08:14 +00:00
Monty Taylor
635be2a3ae Fix cleanup of symlink fixtures
These aren't all getting cleaned up, which winds up breaking
the second runs. Instead of doing addCleanup with a method that
does the loop again, which can fail in the middle and not
clean up subsequent files, add an individual cleanup when we
add the symlink. This results in all of the symlinks consistently
being cleaned.

Change-Id: Id5a5b09c830ad2ad3bb0b77fb9dbdc494c629824
2020-02-24 17:17:17 +00:00
Monty Taylor
2e33459b3e Add pause-buildset-registry role
This is a simple utility that provides the zuul_return functionality
for repos that need it encapsulating the existence of zuul_return.

For instance, system-config wanted to start doing this in a playbook
but linting the playbooks became super hard because of the new need
to have zuul installed for the zuul_return library. Just keep it here
in the family.

Change-Id: I6035009455f6434c2d6cfd976c7c6f414bdc2ae7
2020-02-24 17:17:12 +00:00
Monty Taylor
d36ba50765 Fix unittests for python2 and fedora
We need to pin enum34 on python2 because upstream broke it.

Also, we should not install zuul in python2, because it's not targetted
at python2. It's only here so that linters can deal with zuul_return
and other things from its library, so it should be fine to not install
it for python2 unittests.

GitPython was in here as a pin to the versions that support python3
for zuul. Since we're constraining zuul to only python3, we don't need
to mention GitPython at all.

Update the siblings unit test to use requests as the package we expect
to be already installed since we're no longer installing zuul for
python2.

Fedora 29 got archived upstream:

  http://mirror.sjc1.vexxhost.openstack.org/fedora/releases/29/README

So we need to switch to fedora 30.

Change-Id: I69e0b03f624ba45889916f89c0912df667aaf096
2020-02-24 10:50:13 -06:00
Ian Wienand
ec09127ca3 install-javascript-packages: add tox_constraints_file
I can't exactly say why, but the horizon postinstall in package.json
runs "tox -e npm" [1].  This then installs from requirements [2].

The requirements gate run a cross-horizon-npm job that ends up calling
this.  The short story is that if you modify upper-constraints.txt in
a requirements change, it is not picked up by this job.

Add tox_constraints_file to this job, so we can set the path to the
zuul checked-out version for the test.

[1] e515713084/package.json (L22)
[2] e515713084/tox.ini (L23)

Needed-By: https://review.opendev.org/709389
Change-Id: Ia7dd5af19b42c2ab53dfb70f92e184b0dc215d03
2020-02-24 19:24:15 +11:00
Sorin Sbarnea
22b0ed2c4d Enable testing of install-docker on multiple platforms
Change-Id: I6b9d4a51ed9966b8f39917783c8649fa3b4bd096
2020-02-21 18:55:10 +00:00
Zuul
2aa8ef9f41 Merge "Add ensure-java role" 2020-02-21 15:49:30 +00:00
Zuul
f4703021a4 Merge "Add ensure-bazelisk role" 2020-02-21 15:31:53 +00:00
James E. Blair
bfb3507c5b Add ensure-java role
Change-Id: Ibaf45bd09a47d55ce21af34763563c20f9396cd2
2020-02-21 06:55:47 -06:00
James E. Blair
d0591ad98f Add ensure-bazelisk role
This adds a role which installs bazelisk if it is not already
present.  It also adds a test job which exercises the role; the
job will be run on any changes to the new role.

Change-Id: I69b28b58338b46ec3fea696f9d97be4271ed6ae4
2020-02-21 06:55:44 -06:00
Matthew Thode
a5ace834b1
set jobs for installing openvswitch
On Gentoo this allows multiple packages to compile/install at once.

Change-Id: I23fd629a351b4c6b2acc325611ed47cc04bca404
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-02-19 13:33:48 -06:00
Zuul
84b7c745b2 Merge "Make ara-report role to zuul_return an artifact" 2020-02-17 17:27:55 +00:00
Zuul
1a84d2ff5a Merge "fetch-subunit-output: introduce zuul_use_fetch_output" 2020-02-17 16:42:46 +00:00
Zuul
fa6e25f730 Merge "fetch-javascript-tarball: introduce zuul_use_fetch_output" 2020-02-17 16:42:44 +00:00
Zuul
00dc242fc9 Merge "install-docker: option to install docker-compose" 2020-02-17 16:42:44 +00:00
Mohammed Naser
839a179048 install-docker: option to install docker-compose
The install-docker role currently takes care of setting up Docker
however it lacks the ability to install docker-compose at the moment
which is quite useful.

This adds a variable which defaults to false that allows you to use the
role to install docker-compose.  It was decided to be put within the
install-docker role because that role already manages the repository
configuration and therefore provide consistency with the repository that
is used to install docker-compose.

Change-Id: Ia6f4ed88b4baf53bd671c7a7b48a1564ff2c866d
2020-02-16 17:04:58 +00:00
Tristan Cacqueray
f8792a73d5 install-docker: enable setting docker userland proxy
This change enables docker user to disable the userland proxy.

Do not merge until the default is set to false (used for testing).

Change-Id: Ib30e409044ccc48b4c19beae36f1bcd453ebef8e
2020-02-14 23:08:45 +00:00
Tristan Cacqueray
211e92ec40 crio: add missing registries.conf
This change fixes recent cri-o test failure because of a missing
registries.conf file.

Change-Id: Ieb2efa0e097cc76dc9b67389e86c2646dfc0c00d
2020-02-14 22:22:18 +00:00
Zuul
cc4552bb04 Merge "fetch-output-openshift: initial role" 2020-02-07 16:01:35 +00:00
Zuul
cf898b26a1 Merge "Add event id to emit-job-header" 2020-02-07 13:17:33 +00:00
Simon Westphahl
dd0fa76cff Add event id to emit-job-header
This can aid in debugging a still running job.

Depends-On: https://review.opendev.org/#/c/706222/
Change-Id: Id6d52185b02447484bfc044b20f14088fbe26634
2020-02-06 13:43:06 +01:00
Zuul
d99dc2de67 Merge "Add CentOS8 High Availability repository" 2020-02-05 22:50:15 +00:00
Zuul
83627af4d5 Merge "Move index_links to correct task in generate_zuul_manifest" 2020-02-05 21:31:14 +00:00
James E. Blair
04daed7bc4 Move index_links to correct task in generate_zuul_manifest
This argument is intended for the generate role rather than the
artifact return.

Change-Id: Ie5af37e2f55faa758371327d508f0e255694edd7
2020-02-05 13:06:09 -08:00
Zuul
5a8b257115 Merge "Google logs: Link to index.html files" 2020-02-05 15:44:15 +00:00
Zuul
a568823027 Merge "Add index_links option to zuul manifest" 2020-02-05 15:40:09 +00:00
Zuul
965f68c5a1 Merge "Add upload-logs-gcs role" 2020-02-05 15:34:22 +00:00
James E. Blair
d34881b45a Google logs: Link to index.html files
When generating index.html files for uploading to Google Cloud
Storage, link directories to their index.html files because
(except in certain circumstances) Google does not automatically
serve index.html files at directory URLs.

Change-Id: Ie854de328057ed080c80df9328163da98014ea36
2020-02-05 15:21:37 +00:00
James E. Blair
2aa0dbcf29 Add index_links option to zuul manifest
This allows a deployer to indicate that they prefer the Zuul
dashboard to link to "dir/index.html" rather than just "/dir" when
displaying directory entries in the log browser.  This helps with
log storage systems which do not automatically serve index.html
files at directory urls.

Change-Id: I0444798c689667e51d8761a20b661f8a9b90323d
2020-02-05 07:19:20 -08:00
Alfredo Moralejo
5074bca828 Add CentOS8 High Availability repository
High Availability packages have been added as part of core CentOS
repositories in 8.1.

This patch add this repo and enables it as it provides corosync and
pacemaker based HA solution and some other packages required for it.

Change-Id: Idbddd81f251c1ade97892128e52f9214420bead7
2020-02-05 15:47:06 +01:00
Zuul
f2207abd50 Merge "Use unique loop vars to avoid conflicts" 2020-02-05 05:43:16 +00:00
Zuul
f759c1f92d Merge "upload-afs: rename to upload-afs-roots; add afs-upload-synchronize" 2020-02-04 21:56:05 +00:00
Clark Boylan
4ed66807a0 Use unique loop vars to avoid conflicts
We have to be careful about avoiding outer loop loop_var conflicts in
ansible. Because the zuul-jobs roles are meant to be reconsumed
elsewhere we should not use 'item' loopvars and instead set them to
something a bit more unique.

We use a zj_ prefix to try and be unique to this repo and document this
convention.

Change-Id: I20b9327a914890e9eafcb2b36dc8c23fb472bc8f
2020-02-04 12:23:36 -08:00
Mohammed Naser
45a3790d85 ensure-tox: save tox_executable fact
If we install our own local version of tox, we should set a fact
with the tox_executable so other tasks can leverage it.

Change-Id: If6895bbb898261e88c0e3083d21210209f79995f
2020-02-04 20:46:07 +01:00
James E. Blair
b8ce30f3cb Add upload-logs-gcs role
This uploads logs to Google Cloud Storage.  It is based on
upload-logs-swift.

Change-Id: I4d83c3f4eb7f186271302cbb6ebae6b1f280785b
2020-02-03 16:02:15 -08:00
Ian Wienand
8c44ef9385 Remove deprecated upload-afs role
See I493d8829e3fd98e84f1b8f0e776e7ba41abf16c2

Change-Id: I445b15a42e6c1f37edd12d7f43cfd5a1d055b450
Depends-On: https://review.opendev.org/705370
Depends-On: https://review.opendev.org/705371
Depends-On: https://review.opendev.org/705372
2020-02-04 07:41:59 +11:00
Ian Wienand
cd258e5a1d upload-afs: rename to upload-afs-roots; add afs-upload-synchronize
The upload-afs role is actually fairly specialised for uploading docs
with its root-marker scheme.  As we're moving more parts of our
infrastructure to publish via AFS I think it's worth making this quite
explicit by renaming the upload-afs role to upload-afs-roots.  This
should be transparent as we call the renamed role, and once callers
have been migrated we can use the usual deprecation process.

Add a partner role, upload-afs-synchronize, which is more of a
straight copy with options to synchronize.  This will be suitable for
uploading tarball and release artifacts.  This is similar in concept
to the promotion job used by zuul [1], but in this case will be
focused on the extant publishing jobs in project-config which don't
currently use the artifact-reporting/promote process.

[1] https://opendev.org/opendev/base-jobs/src/branch/master/playbooks/artifacts/promote.yaml#L39

Change-Id: I493d8829e3fd98e84f1b8f0e776e7ba41abf16c2
Story: #2006598
Task: #38600
2020-02-04 07:41:00 +11:00
Zuul
7a1684fffb Merge "Update upload-afs README" 2020-02-02 23:41:27 +00:00
Clark Boylan
e323dc117b Don't use item in collect container logs loop
The default loop variable 'item' cannot be nested. Because roles like
collect-container-logs may be called from higher level loops that use
'item' we override the loop control var to a name that should be unique.

This should probably be done to all of the other roles in zuul-jobs too.

Change-Id: I2f647596dd40c662aa7447e3d3c8844a77c6b109
2020-02-01 14:24:16 -08:00
Zuul
a7c7d4fa16 Merge "Add a markdownlint job and role" 2020-01-31 17:05:16 +00:00
Andreas Jaeger
1237417ea3 Update upload-afs README
Explain the intention of this role and the details of .root-marker,
some of it is copied verbatim from the infra-specs repository design
document.

Change-Id: I6d9124c7eafadc4b53626ff2fef6be6399f61947
2020-01-31 08:33:38 +01:00
Tristan Cacqueray
67e2abda5b fetch-output-openshift: initial role
This change adds a new fetch-output-openshift role to be able to
fetch-output from pod.

Change-Id: I45d5115a10254432805a02a77e0ebb35d7fd2dd4
2020-01-30 20:00:14 +00:00