zuul/zuul-jobs
Code Issues Proposed changes
0f662a23d0
zuul-jobs/playbooks/container-image/README.rst
Clark Boylan 5994ce4049 Gracefully handle use of intermediate registry in container upload role 
For symmetry and ease of transition between the docker specific
jobs/roles and generic container jobs/roles it is advantageous to have
the container upload role skip pushing artifacts to the final registry
location if we are relying on the intermediate registry instead.

Update the container upload role to skip pushing to the actual registry
if the promote var is set to intermediate registry. This allows us to
avoid reshuffling all of our jobs as we migrate between the two
implementations.

Change-Id: I3cae9e03517cb0a5ce8e9369bf43fd052cac97ff
2023-05-09 16:18:42 -07:00

8.8 KiB
Raw Blame History

This is one of a collection of jobs which are designed to work together to build, upload, and promote container images in a gating context:

  • :zuulbuild-container-image: Build the images.
  • :zuulupload-container-image: Build and stage the images in a registry.
  • :zuulpromote-container-image: Promote previously uploaded images.

The jobs can work in multiple modes depending on your requirements. They all accept the same input data, principally a list of dictionaries representing the images to build. YAML anchors can be used to supply the same data to all three jobs.

Promotion via tags

The :zuulbuild-container-image job runs in the check pipeline to validate the change.

The :zuulupload-container-image job runs in the gate pipeline and builds and uploads the images to a remote registry, but only with a single temporary tag corresponding to the change ID. This is a speculative upload; the change is not "live" (the main tag is not updated) and other gate jobs may fail and the change may not merge, effectively invalidating the upload.

The :zuulpromote-container-image job runs in a post-merge promote pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a change with whatever tags are supplied by :zuulbuild-container-image.container_images.tags after the code has merged. It also cleans up and removes the change ID tag from the repository in the registry. If any changes fail to merge, this cleanup will not run and those tags will need to be deleted manually.

This advantage of this method is that it minimises the window in which the published image differs from the merged code. There are some caveats to be aware of. gate failures may mean that unused layers and tags are present in the remote repository, which need to be cleaned up. Removing registry tags is not a generic option; you will need to check the promote role documentation to ensure you are passing the right registry details so tags can be cleaned up.

In the tag and release pipelines there is no need for a speculative upload (the tagged/released change is committed code and has already passed gate tests). In this case, :zuulupload-container-image job is run with the flag upload_container_image_promote: false to directly build and push with the final tags.

Summary:

  • :zuulbuild-container-image in check
  • :zuulupload-container-image in gate
  • :zuulpromote-container-image in promote with promote_container_method: tag
  • :zuulupload-container-image with upload_container_image_promote: false in tag and release

Promotion via intermediate registry

The :zuulbuild-container-image runs in the check pipeline. It will build images then upload them to an intermediate registry.

The :zuulupload-container-image job runs in the gate. With this promotion method it will build and upload images to an intermediate registry. No images will be pushed to the upstream registry until promotion occurs.

The :zuulpromote-container-image job is designed to be used in a post-merge promote pipeline. It requires no nodes and run on the Zuul executor. It inspects the artifacts of the gate job to find the correct tags to pull from the intermediate registry. It then uploads this image from the intermediate registry to the remote registry with the final tags supplied by :zuulbuild-container-image.container_images.tags.

In the tag and release pipelines the :zuulupload-container-image job is run with the flag upload_container_image_promote: false to directly build and push with the final tags.

The advantages of this method is that no partial or unused images will ever be present in the final repository. Copying from the intermediate registry effectively caches the expensive build process. This means that although the window that the production tags are out-of-sync with the merged code is larger than when using speculative uploads, it is smaller than having to rebuild and upload the image. Copying is a generic operation, so it should work with any registry. The layer upload has more exposure to transient errors than the tag promotion step, so needs to be monitored more carefully. You also must manage an external intermediate registry to hold the image between upload and promote steps in this model.

Summary:

  • :zuulbuild-container-image in check
  • :zuulupload-container-image in gate. This must push to an intermediate registry.
  • :zuulpromote-container-image in promote with promote_container_method: intermediate-registry
  • :zuulupload-container-image with upload_container_image_promote: false in tag and release

Publish via full release

The :zuulbuild-container-image job runs in the check pipeline to validate the change.

The :zuulbuild-container-image job also runs in the gate pipeline to validate the change before merge.

Once the change has merged, :zuulupload-container-image job is run with the flag upload_container_image_promote: false to directly build and push with the final tags. This is also run in the tag and release piplines in the same way.

The advantage of this mode is that it requires no external dependencies or management of speculative uploads. The disadvantage is that it has the longest window where published image is out-of-sync with merged-code, as the post-merge release process must re-build the entire container and upload it.

  • :zuulbuild-container-image in check
  • :zuulbuild-container-image in gate
  • :zuulupload-container-image with upload_container_image_promote: false after code merge, and tag and release pipelines.

Job Variables

The default container filename name to use. Serves as the base for :zuulbuild-container-image.container_images.container_filename. This allows a global overriding of the container filename name, for example when building all images from different folders with similarily named containerfiles.

If omitted, the default depends on the container command used. Typically, this is Dockerfile for docker and Containerfile (with a fallback on Dockerfile) for podman.