From 2b7689ba0071882195ecefe578869291082c9fb9 Mon Sep 17 00:00:00 2001 From: "James E. Blair" Date: Mon, 21 Jan 2019 13:33:01 -0800 Subject: [PATCH] Add docker image build jobs These are generic docker image build and upload jobs which use the recently added roles of the same names. With the new pass-to-parent feature, child jobs can add their own credentials to these jobs. Change-Id: I669a9e6616a031837adab8c5c5806431fcca2150 --- playbooks/docker-image/README.rst | 80 ++++++++++++++++++++++++++ playbooks/docker-image/credentials.rst | 22 +++++++ playbooks/docker-image/pre.yaml | 3 + playbooks/docker-image/promote.yaml | 3 + playbooks/docker-image/run.yaml | 3 + playbooks/docker-image/upload.yaml | 3 + roles/build-docker-image/common.rst | 8 +-- zuul.yaml | 30 ++++++++++ 8 files changed, 148 insertions(+), 4 deletions(-) create mode 100644 playbooks/docker-image/README.rst create mode 100644 playbooks/docker-image/credentials.rst create mode 100644 playbooks/docker-image/pre.yaml create mode 100644 playbooks/docker-image/promote.yaml create mode 100644 playbooks/docker-image/run.yaml create mode 100644 playbooks/docker-image/upload.yaml diff --git a/playbooks/docker-image/README.rst b/playbooks/docker-image/README.rst new file mode 100644 index 000000000..91bffd43b --- /dev/null +++ b/playbooks/docker-image/README.rst @@ -0,0 +1,80 @@ +This is one of a collection of jobs which are designed to work +together to build, upload, and promote docker images in a gating +context: + + * :zuul:job:`build-docker-image`: Build the images. + * :zuul:job:`upload-docker-image`: Build and stage the images on dockerhub. + * :zuul:job:`promote-docker-image`: Promote previously uploaded images. + +The :zuul:job:`build-docker-image` job is designed to be used in +a `check` pipeline and simply builds the images to verify that +the build functions. + +The :zuul:job:`upload-docker-image` job builds and uploads the images +to Docker Hub, but only with a single tag corresponding to the +change ID. This job is designed in a `gate` pipeline so that the +build produced by the gate is staged and can later be promoted to +production if the change is successful. + +The :zuul:job:`promote-docker-image` job is designed to be used in a +`promote` pipeline. It requires no nodes and runs very quickly on the +Zuul executor. It simply re-tags a previously uploaded image for a +change with whatever tags are supplied by +:zuul:jobvar:`build-docker-image.docker_images.tags`. It also removes +the change ID tag from the repository in Docker Hub, and removes any +similar change ID tags more than 24 hours old. This keeps the +repository tidy in the case that gated changes fail to merge after +uploading their staged images. + +They all accept the same input data, principally a list of +dictionaries representing the images to build. YAML anchors_ can be +used to supply the same data to all three jobs. + +**Job Variables** + +.. zuul:jobvar:: zuul_work_dir + :default: {{ zuul.project.src_dir }} + + The project directory. Serves as the base for + :zuul:jobvar:`build-docker-image.docker_images.context`. + +.. zuul:jobvar:: docker_images + :type: list + + A list of images to build. Each item in the list should have: + + .. zuul:jobvar:: context + + The docker build context; this should be a directory underneath + :zuul:jobvar:`build-docker-image.zuul_work_dir`. + + .. zuul:jobvar:: repository + + The name of the target repository in dockerhub for the + image. Supply this even if the image is not going to be + uploaded (it will be tagged with this in the local + registry). + + .. zuul:jobvar:: path + + Optional: the directory that should be passed to docker build. + Useful for building images with a Dockerfile in the context + directory but a source repository elsewhere. + + .. zuul:jobvar:: build_args + :type: list + + Optional: a list of values to pass to the docker ``--build-arg`` + parameter. + + .. zuul:jobvar:: target + + Optional: the target for a multi-stage build. + + .. zuul:jobvar:: tags + :type: list + :default: ['latest'] + + A list of tags to be added to the image when promoted. + +.. _anchors: https://yaml.org/spec/1.2/spec.html#&%20anchor// diff --git a/playbooks/docker-image/credentials.rst b/playbooks/docker-image/credentials.rst new file mode 100644 index 000000000..4587dbf23 --- /dev/null +++ b/playbooks/docker-image/credentials.rst @@ -0,0 +1,22 @@ +.. zuul:jobvar:: docker_credentials + :type: dict + + This is expected to be a Zuul Secret with these keys: + + .. zuul:jobvar:: username + + The Docker Hub username. + + .. zuul:jobvar:: password + + The Docker Hub password. + + .. zuul:jobvar:: repository + + Optional; if supplied this is a regular expression which + restricts to what repositories the image may be uploaded. The + following example allows projects to upload images to + repositories within an organization based on their own names:: + + repository: "^myorgname/{{ zuul.project.short_name }}.*" + diff --git a/playbooks/docker-image/pre.yaml b/playbooks/docker-image/pre.yaml new file mode 100644 index 000000000..52f5b5b26 --- /dev/null +++ b/playbooks/docker-image/pre.yaml @@ -0,0 +1,3 @@ +- hosts: all + roles: + - install-docker diff --git a/playbooks/docker-image/promote.yaml b/playbooks/docker-image/promote.yaml new file mode 100644 index 000000000..8160bf438 --- /dev/null +++ b/playbooks/docker-image/promote.yaml @@ -0,0 +1,3 @@ +- hosts: localhost + roles: + - promote-docker-image diff --git a/playbooks/docker-image/run.yaml b/playbooks/docker-image/run.yaml new file mode 100644 index 000000000..d3525ad85 --- /dev/null +++ b/playbooks/docker-image/run.yaml @@ -0,0 +1,3 @@ +- hosts: all + roles: + - build-docker-image diff --git a/playbooks/docker-image/upload.yaml b/playbooks/docker-image/upload.yaml new file mode 100644 index 000000000..712f7261f --- /dev/null +++ b/playbooks/docker-image/upload.yaml @@ -0,0 +1,3 @@ +- hosts: all + roles: + - upload-docker-image diff --git a/roles/build-docker-image/common.rst b/roles/build-docker-image/common.rst index 45484a9aa..affb19fc4 100644 --- a/roles/build-docker-image/common.rst +++ b/roles/build-docker-image/common.rst @@ -20,8 +20,8 @@ production if the change is successful. The :zuul:role:`promote-docker-image` role is designed to be used in a `promote` pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a -change with whatever tags are supplied by the -:zuul:rolevar:`build-docker-image.docker_images.context`. It also +change with whatever tags are supplied by +:zuul:rolevar:`build-docker-image.docker_images.tags`. It also removes the change ID tag from the repository in Docker Hub, and removes any similar change ID tags more than 24 hours old. This keeps the repository tidy in the case that gated changes fail to merge after @@ -88,7 +88,7 @@ using this role. Useful for building images with a Dockerfile in the context directory but a source repository elsewhere. - .. zuul:jobvar:: build_args + .. zuul:rolevar:: build_args :type: list Optional: a list of values to pass to the docker ``--build-arg`` @@ -98,7 +98,7 @@ using this role. Optional: the target for a multi-stage build. - .. zuul:jobvar:: tags + .. zuul:rolevar:: tags :type: list :default: ['latest'] diff --git a/zuul.yaml b/zuul.yaml index 761951e68..2d1aa57e6 100644 --- a/zuul.yaml +++ b/zuul.yaml @@ -20,6 +20,36 @@ pre-run: playbooks/unittests/pre.yaml post-run: playbooks/unittests/post.yaml +- job: + name: build-docker-image + description: | + Build a docker image. + + .. include:: ../../playbooks/docker-image/README.rst + pre-run: playbooks/docker-image/pre.yaml + run: playbooks/docker-image/run.yaml + +- job: + name: upload-docker-image + parent: build-docker-image + description: | + Build and upload a docker image. + + .. include:: ../../playbooks/docker-image/README.rst + .. include:: ../../playbooks/docker-image/credentials.rst + post-run: playbooks/zuul/docker-image/upload.yaml + +- job: + name: promote-docker-image + description: | + Retag a previously-uploaded docker image. + + .. include:: ../../playbooks/docker-image/README.rst + .. include:: ../../playbooks/docker-image/credentials.rst + run: playbooks/zuul/build-image/promote.yaml + nodeset: + nodes: [] + - job: name: tox parent: unittests