The prepare_workspace_sync_required_projects_only variable allows
users to define which projects to sync to the node. This can prevent
syncing of unnecessary repositories. For some builds e.g. the
depends-on repositories dont need to be synced. The projects are
filtered based on the 'required' flag present in each zuul.project
entry and the required projects list also does not contain projects
which are present due to Depends-On or gate queue sequencing.
Having unnecessary repos in the workspace can for example also break
the analysis phase of bazel.
Change-Id: I3cc36cbfc60c81956caf5137da63973aeade4e21
Co-Authored-By: James E. Blair <jim@acmegating.com>
Co-Authored-By: Bernhard Berg <bernhard.berg@bearingpoint.com>
This is no longer present in Ansible 9.
Removing these upsets ansible-lint, so those errors are ignored.
The base roles job has bitrotted on centos-7 and bionic due to
a bad voluptuous release used in an stestr test. That is fixed in
this change as well.
Change-Id: I67886d5ad82ab590979f82bd102d6f974b9d4421
We can now run untrusted code on the executor, so we can re-enable
these tests.
This reverts commit 44d2187e7fcea0683c75db530458088b8371bb4e.
Change-Id: I65960af09a3cf3da1e780a96b6d13ddba3ce73d8
This test pins stestr (reason unclear) but does not pin python-subunit.
Python-subunit has made a release incompatible with that version of
stestr, so pin it.
Separately, it may be useful to see if we can unpin both.
Change-Id: Ia7cc45dc53ff0697e7ec84479c08c93e7d872a76
I74b9de7092f182c942a58ac7a46b9fbd791889de hit a common ansible gotcha
where it likes to strip the trailing newline after a {% endif %}.
This has resulted in invalid lines in our sources.list.
Unfortunately we miss this because it still exits with 0. Add a
simple test looking for warning output.
Change-Id: I46d393a5e67d10a52c4dcca803176ff368a4b5bd
The workspace setup role requires local code execution on the executor.
This is not allowed from an untrusted context so we disable it.
The previous assertions about the zuul-cloner setup depend on the
workspace setup running successfully. Disable those extra assertions.
subunit fetching role assertions grepped the html file for validity on
localhost. Disable this as well.
Change-Id: I7449749f50b6e4a34c4615b00836a7148e01c768
We should offer the option of https in addition to http in our
configure-mirrors role as users may want to consume mirrors using https.
This has become more viable in recent years with the releases of Debian
Buster and Ubuntu Bionic supporting it out of the box.
Change-Id: I747c1a379dfce9469e643d7fa199c8e8554f5289
Adds yamllint to the linters with a minimal configuration, some
rules are disabled to allow us to fix them in follow-ups, if
we agree on them.
Fixes invalid YAML file containing characters inside block.
Fixes few minor linting issues.
Change-Id: I936fe2c997597972d884c5fc62655d28e8aaf8c5
Call ensure-pip before installing the stest and subunit-output tools.
Use then ensure-pip virtualenv to create their environment.
Change-Id: I4c82867e29a8f3bdf902fac43b96cd6862fa9f48
Inventory hostnames like "abc/123" is valid in both ansible and zuul
but this role breaks since it uses it as part of a path. This sanitizes
the hostname from "abc/123" to "abc_123".
Change-Id: Ic89d595b6f004b5ca4805f1af8387e8ba56564aa
Stop compressing files that would otherwise be handled uncompressed in
intermediate log handling. The reason for this is final log/artifact
upload should decide if a file need to be compressed or not as some file
hosting services (like swift) may not return the expected data if we get
this wrong.
This was noticed when we used swift to store intermediate tar.gz
tarballs before final upload to permanent storage. When retrieving the
tar.gz files from swift we got them back as uncompressed .tar files
(because swift was being helpful) and then later when we try to
uncompress these files we break.
Instead we'll let the upload to swift (or other storage) decide if files
should be compressed further than their existing state.
This is the first change in the stack that updates this behavior for
fetch-subunit-output.
Change-Id: I5727b270d6d34c256fe78a443baa1ad6c3474108
Change new variable zuul_additional_subunit_dirs to
fetch_subunit_output_additional_dirs to name it after the role.
Change-Id: I7ac0b5d343701409a8741439334fe27f5374a258
In addition to the main subunit file from zuul_work_dir,
collect the subunit files from the elements (directories)
of the zuul_additional_subunit_dirs list.
The default behavior is unchanged.
While the documentation of this role states that zuul_work_dir
contains an absolute path, this is not always true.
So make sure to not make any assumption about zuul_work_dir
in order to not fail spectacularly as it happened with the
previous patch[0].
Add also some tests for the role: both the basic case
and with an additional test directory.
[0] https://review.opendev.org/673885
Change-Id: Iabf2e0cf6d86e36a174778367186bbd39a65c3dd
This is what this job should have been testing, as it's what we
recommend people use. We simply didn't notice that OpenDev never
finished the transition away from use-cached-repos (which isn't
even defined in this repo).
Change-Id: I9be05f9c035441c3a0f84206b25580263663d179
This more accurate reflects how this is run by opendev. Note this is not
strictly necessary to get in as the post-logs.yaml playbooks determine
where things are uploaded anyway. However, this may be slightly easier
for people to read in logs if things line up more closely to each other.
Depends-On: https://review.opendev.org/680476
Change-Id: Ied902ea2b8c7b965447e16f91400586ca1f708e6
For logs stored in object storage services we want to be able to shard
into different containers. This improves scalability of the object
stores. To do this consistently we prefix all log paths with the first 3
characters of the build uuid. This way we get container names like
log_123, log_abc, and so on for a total of 4096 containers.
This behavior is off by default because users of filesystems stores
without a zuul dashboard may still want periodic/ to be filed into a top
level dir for browseability.
Change-Id: I8e72a6e587edcbdf89b793cad2d7c96c535601e7
The fetch-zuul-cloner and use-cached-repos roles rely on an
unrelated repo appearing in required-projects. To allow us to
move this repo to the Zuul tenant, change that to
opendev/base-jobs, a repo which will appear in both the current
and future tenants.
Change-Id: I06703638b69d94531ad396224669980db9ababd4
These are jobs for the roles in this repo which are currently
tested by the "base-integration" job in openstack-zuul-jobs.
Move it into this repo and rename it "base-roles".
Change-Id: Ibd07964a08120e033bc5a2deca18649bd0e74a48
