Buildx image building (for multiarch support) requires us to set up CA
certificates that are trusted by the image for access to the
registr(y|ies). The buildx image switched to alpine at some point which
doesn't include the update-ca-certificates command we were depending on
for this by default. Install that package so that we can
update-ca-certificates and trust the registr(y|ies).
Note that there may be a better way via buildkitd.toml configuration
files that allow us to configure certs upfront and have the image set
them up for us. This is documented in a todo as a future action we can
investigate.
Change-Id: Ied0b8e81ed6a7a15ea2def26b85b933223cb42e6
If you need to run native arm64 builds, you can take advantage
of this change which will rely on the remote builders in order
to build things natively giving a significant speed up in
container build time.
Change-Id: I962bb2357a2c458d5e72b334b4fe36b55b034864
For symmetry and ease of transition between the docker specific
jobs/roles and generic container jobs/roles it is advantageous to have
the container upload role skip pushing artifacts to the final registry
location if we are relying on the intermediate registry instead.
Update the container upload role to skip pushing to the actual registry
if the promote var is set to intermediate registry. This allows us to
avoid reshuffling all of our jobs as we migrate between the two
implementations.
Change-Id: I3cae9e03517cb0a5ce8e9369bf43fd052cac97ff
The "docker build" path is missing the "--label" arguments which are
present in all other paths (docker buildx in both build-container-image
and build-docker-image roles). This means that images are being built
without expected metadata labels.
This change adds the extra arguments to match.
Change-Id: I607ae89cace87fe87b6dcfb95a1bae4fca516d3c
The container roles assume a full image url path and do not work with
shorted names like the docker roles do. The buildx path used the shorted
path when it should use the full path. Fix this.
Change-Id: Icdeee0ebb1c0d280968f425f8560cc5208ebd18d
When setting the containerfile variable, we need to be inside the
zk_images loop in order to set the correct default for each image.
Change-Id: I216ffd19dd797752b2e5ca2332e651b8e6ac8a3c
This uses the generic tag removal role added with
I7f2d9d00024e34451e2d20b2c2f8171ecd151943 to cleanup the promote tag
and any leaked tags.
Change-Id: I3f1b82d63874ee886048b9ccabe616a60dc09434
After recent conversations, we've come to the conclusion it will be
good to have two models of promotion
- using tags, where gate directly uploads to the final repository and
promote retags the image.
- from an intermediate-registry, where upload stores the built image
in an i-r and the promote step uploads to the final registry.
To facilitate this, we add a "promote_container_image_method" flag to
the promote roles.
The documentation is expanded to explain how all this is intended to
work together.
These roles haven't been publicised yet, but this should be a no-op as
it defaults to tags, which is the current operation.
c.f. Ia24bbd101e01ab371ceacfed006b5ff806418a97
Change-Id: I1c25f60f835b1cab983bcdd169eeffc0e250a56c
The multi-stage build/push was added with
I8036a9b4d4c515c20a05994741540b999e7cbcae (2020-05) which noted
When building multi-arch it's done in parallel which can result
in the same layer being pushed at the same time, which is bad for
the registries.
I believe this was probably actually a bug in zuul-registry, addressed
with Ibdf1ca554756af61247d705b2ea3cf85c39c2b83 (2022-02)
The way the registry was previously written if two concurrent
uploads of the same blob were happening one would fail to grab the
lock and then return early. The uploading client would then
immediately HEAD the blob and if it did so quickly enough would get
a short size or 404.
So this simplifies a couple of steps by returning the push to the
build step.
Change-Id: I41b8a495eb3097b358d7634b63b7eb9de6161059
This adds support for multiarch container image builds when using docker
as the container command to the container roleset.
Change-Id: I48bf2e34c258e54baf013d3c04c6d4baaacde04b
skopeo is used from the executor in the promote process, but isn't
required to be installed on nodes with ensure-skopeo. Clarify this
and the runtime requirements.
Change-Id: Ifaf4b788117986037b124972441b1884744374e5
This role uses skopeo to perform image operations.
Also update the container roles docs to add missing documentation
for the already existing upload-container-image role. Clarify
some ambiguity about the registry and repository attributes of
the container images data structure.
Change-Id: Ib66c85daf0edacf0dd797ab34b0d629f99c7111b
Co-Authored-By: James E. Blair <jim@acmegating.com>
These build tests are all running on Bionic for no particular reason
other than it was recent at the time; update them to Jammy.
Because Jammy appears to have a too old systemd to properly support
podman and non root continers [0] our image builds don't work by
default. We need to override the cgroup manager to cgroupfs which is
less than ideal but should be functional.
[0] https://github.com/containers/podman/issues/7650
Change-Id: I18c56aea776e9ef102fe9020db592fb04df372e2
This fixes a number of places where we do not have spaces between
filters. I think that this is a reasonable rule for readability (I
also think it probably was enforced, but maybe later versions got
better at detecting it?).
These are detected by a later version of Ansible lint; this change
should have no operational change to any roles but prepares us to
update in a follow-on change.
Change-Id: I07e1a109b87adce86f483d14d7e02fcecb8313d5
build-container-image may ends up pushing large files. Network is not
always as reliable as we want. This commit ensures we retry several
times in case of failure.
Change-Id: Ieaf92a8c3531e24ab5e41783b540de1df806c02f
Because buildset registries may be used by jobs that finish before other
jobs are finished using the buildset registry we must be careful not to
expose the registry credentials in the jobs that finish sooner.
Otherwise logs for the earlier job runs could potentially be used to
poison the registry for later jobs.
This is likely currently incomplete. Other Zuulians should look over it
carefully to ensure we're covering all the bases here.
The cases I've identified so far are:
* Setting facts that include passwords
* Reading and writing to files that include passwords (as content may be
logged)
* Calling modules with passwords passed as arguments (the module
invocation is logged)
I've also set no_log on zuul_return that passes up credentials because
while the logging for zuul_return is minimal today, I don't want to
count on it remaining that way.
We also use the yet to be merged secret_data attribute on zuul_return to
ensure that zuul_return itself does not expose anything unwanted.
Finally it would be great if others could check over the use of
buildset_registry variables to make sure there aren't any that got
missed. One thing I'm not sure of is whether or not when conditionals
get logged and if we need to be careful about their use too.
Temporarily remove some buildset-regitry jobs which are in a catch-22.
Change-Id: I2dea683e27f00b99a7766bf830981bf91b925265
This new role will be used to replace our upload-docker-image role in
the future.
Change-Id: I0e2b0cca6575255520aa6d4d48a12128ab5f46cc
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The buildx patch unfortunately changed the logic associated with
siblings to set up siblings in a loop one time, rather than to
do a loop of "set up siblings, build, cleanup siblings". This causes
builds to fail when they're using siblings with an error about
siblings dir not having been cleaned up.
Change-Id: I3c45bfa77ec9f2609689e04044c18f066adc9741
Move the setting of the containerfile into the loop (thus in build.yaml)
to follow the semantics from roles/build-container-image/common.rst for
container_filename and container_images.container_filename.
Co-Authored-By: Andreas Jaeger <aj@suse.com>
Change-Id: Id39f13cc666e9f99557447073cc0d362991f3506
This adds a custom ansible-lint rule at .rules/ZuulJobsNamespaceLoopVar.py
that enforces the loop var policy described at:
https://zuul-ci.org/docs/zuul-jobs/policy.html#ansible-loops-in-roles
It also updates existing roles to follow the policy.
Change-Id: I92b2ff56a1c2702542fc07b316f1809087a4c92f
Add support for "siblings:" to the build-container-image role. This
is a port of change I079d823e7194e15b1b496aea0f53f70f6b563f02 from the
build-docker-image role.
Note that we found a couple of issues with this role, fixed with
I9f3b0a1f71d20cf7511f224648dd2fa51a039015. However,
build-container-image has merged before this fix, so thess fixes now
follow this change. To avoid squashing this into something confusing
this update incorporates those fixes. However testing is left to the
follow-on change because the extant build-docker-image role would fail
the corrected testing. They should be merged roughly together.
One minor thing is the removal of an additional duplicated "tags:"
entry from the variable documentation.
Change-Id: I71e2f2ab29446bd113644ce8c75378b8d3ef8ba6