We try to be smart about when we attempt to create repos in quay. One
thing that was missing was checking if necessary info for the API
request is present. Skip attempting creating if not present.
Change-Id: I61e603c4d07280079e560215685bf09ebc0c4263
This updates the new ensure-quay-repo to run opportunistically if the
registry_type image flag is set to quay and the registry credentials
matching the container image has an api token defined. This will allow
us to include this role in base jobs and it will do what we need it to
do without impacting docker based images or quay managed images that
don't need automatic creation.
Change-Id: Ia419578bf0a27293757c5f723873e9930ee2c489
The container roles assume a full image url path and do not work with
shorted names like the docker roles do. The buildx path used the shorted
path when it should use the full path. Fix this.
Change-Id: Icdeee0ebb1c0d280968f425f8560cc5208ebd18d
This adds a new role that can be used to ensure a quay repo exists
before publishing to it. This is particularly useful for creating public
repos in quay as simply pushing to a repo with quay will create a
private repo by default.
Change-Id: I979f1b9b64f901bb8d54b8991bb9142b18b6330f
Virtualenv 20.22.0 dropped support for python<=3.6 (including 2.7). We
still want to run tests under 2.7. Work around this by capping off
virtualenv in tox's requires list.
Change-Id: I4868cd5a8a958f04ac782e0963c52a118b2f5ebf
When setting the containerfile variable, we need to be inside the
zk_images loop in order to set the correct default for each image.
Change-Id: I216ffd19dd797752b2e5ca2332e651b8e6ac8a3c
Just to be sure, add no_log here. Since the actual authenticated call
is no_log too; if this fails you at least know you didn't get the
credentials layout right which narrows down the problems the actual
call might be facing.
Change-Id: Ic7174c10f4e693f6a2c0554dc66ea22fd28d678f
This uses the generic tag removal role added with
I7f2d9d00024e34451e2d20b2c2f8171ecd151943 to cleanup the promote tag
and any leaked tags.
Change-Id: I3f1b82d63874ee886048b9ccabe616a60dc09434
This is a role to abstract removal of tags from registries, which is
an operation that practically has to be done via the registry API.
This implements removing tags from the quay and docker API's.
For the common case of working with a repository like
"quay.io/org/project" there is minimal configuration. However, if you
run a private repository, this is flexible with a few extra variables
to tell the role to use the quay API but your own URL.
By default it clears out old tags from the Zuul promote pipeline.
However if you set registry_tag_remove_tag it will only remove that
one tag.
This is inspired by the current work done in promote-docker-image
role.
Change-Id: I7f2d9d00024e34451e2d20b2c2f8171ecd151943
After recent conversations, we've come to the conclusion it will be
good to have two models of promotion
- using tags, where gate directly uploads to the final repository and
promote retags the image.
- from an intermediate-registry, where upload stores the built image
in an i-r and the promote step uploads to the final registry.
To facilitate this, we add a "promote_container_image_method" flag to
the promote roles.
The documentation is expanded to explain how all this is intended to
work together.
These roles haven't been publicised yet, but this should be a no-op as
it defaults to tags, which is the current operation.
c.f. Ia24bbd101e01ab371ceacfed006b5ff806418a97
Change-Id: I1c25f60f835b1cab983bcdd169eeffc0e250a56c
As noted inline, currently promote-image-container uses skopeo to
delete tags which is not the semantics we want. This results in the
whole image being removed.
For safety we remove this call; we have two solutions in follow-ons
(deleting tags directly from the registry with a generic tag, and
promoting images from the intermediate registry).
Change-Id: I4b257f593275413da9a50a0cc64e13638e7f94cb
We don't seem to have a policy file available in zuul-bwrap, but
we also don't really need it for this operation, so just tell skopeo
to skip it.
Also, fix the registry logout command.
Change-Id: I18ad59ffa8e8e80a720e798ab2bb81ea5d19619a
This ended up calling into push-to-intermediate-registry with both
docker_images *and* container_images variable set.
This hid from testing that push-to-intermeidate-registry was not
working with only the container_images variable set.
Split these calls up so we don't have both variables defined.
Change-Id: If84b039852f2afc4df66c98e64fcce6f30f51246
The multi-stage build/push was added with
I8036a9b4d4c515c20a05994741540b999e7cbcae (2020-05) which noted
When building multi-arch it's done in parallel which can result
in the same layer being pushed at the same time, which is bad for
the registries.
I believe this was probably actually a bug in zuul-registry, addressed
with Ibdf1ca554756af61247d705b2ea3cf85c39c2b83 (2022-02)
The way the registry was previously written if two concurrent
uploads of the same blob were happening one would fail to grab the
lock and then return early. The uploading client would then
immediately HEAD the blob and if it did so quickly enough would get
a short size or 404.
So this simplifies a couple of steps by returning the push to the
build step.
Change-Id: I41b8a495eb3097b358d7634b63b7eb9de6161059
This adds support for multiarch container image builds when using docker
as the container command to the container roleset.
Change-Id: I48bf2e34c258e54baf013d3c04c6d4baaacde04b
This corrects and simplifies the file matchers for these jobs at
the expense of possibly running some extra jobs in some cases.
The alternative is a lot of duplicated lists that will be hard
to maintain.
Change-Id: I97406cd0506a438e4b14bbbee3d1b61026f40cd7
skopeo is used from the executor in the promote process, but isn't
required to be installed on nodes with ensure-skopeo. Clarify this
and the runtime requirements.
Change-Id: Ifaf4b788117986037b124972441b1884744374e5