Browse Source

Add docker image promotion roles

This adds three roles which can be used to build a docker image
promotion system.

Change-Id: Iefd9278cdb90bbbaab93a4d23c055e9289fde5ba
changes/78/631078/1
James E. Blair 6 months ago
parent
commit
8640466183

+ 3
- 0
roles/build-docker-image/README.rst View File

@@ -0,0 +1,3 @@
1
+Build one or more docker images.
2
+
3
+.. include:: ../../roles/build-docker-image/common.rst

+ 98
- 0
roles/build-docker-image/common.rst View File

@@ -0,0 +1,98 @@
1
+This is one of a collection of roles which are designed to work
2
+together to build, upload, and promote docker images in a gating
3
+context:
4
+
5
+* :zuul:role:`build-docker-image`: Build the images.
6
+* :zuul:role:`upload-docker-image`: Stage the images on dockerhub.
7
+* :zuul:role:`promote-docker-image`: Promote previously uploaded images.
8
+
9
+The :zuul:role:`build-docker-image` role is designed to be used in
10
+`check` and `gate` pipelines and simply builds the images.  It can be
11
+used to verify that the build functions, or it can be followed by the
12
+use of subsequent roles to upload the images to Docker Hub.
13
+
14
+The :zuul:role:`upload-docker-image` role uploads the images to Docker
15
+Hub, but only with a single tag corresponding to the change ID.  This
16
+role is designed to be used in a job in a `gate` pipeline so that the
17
+build produced by the gate is staged and can later be promoted to
18
+production if the change is successful.
19
+
20
+The :zuul:role:`promote-docker-image` role is designed to be used in a
21
+`promote` pipeline.  It requires no nodes and runs very quickly on the
22
+Zuul executor.  It simply re-tags a previously uploaded image for a
23
+change with whatever tags are supplied by the
24
+:zuul:rolevar:`build-docker-image.docker_images.context`.  It also
25
+removes the change ID tag from the repository in Docker Hub, and
26
+removes any similar change ID tags more than 24 hours old.  This keeps
27
+the repository tidy in the case that gated changes fail to merge after
28
+uploading their staged images.
29
+
30
+They all accept the same input data, principally a list of
31
+dictionaries representing the images to build.  YAML anchors_ can be
32
+used to supply the same data to all three jobs.
33
+
34
+Use the :zuul:role:`install-docker` role to install Docker before
35
+using this role.
36
+
37
+**Role Variables**
38
+
39
+.. zuul:rolevar:: zuul_work_dir
40
+   :default: {{ zuul.project.src_dir }}
41
+
42
+   The project directory.  Serves as the base for
43
+   :zuul:rolevar:`build-docker-image.docker_images.context`.
44
+
45
+.. zuul:rolevar:: credentials
46
+   :type: dict
47
+
48
+   This is only required for the upload and promote roles.  This is
49
+   expected to be a Zuul Secret with two keys:
50
+
51
+   .. zuul:rolevar:: username
52
+
53
+      The Docker Hub username.
54
+
55
+   .. zuul:rolevar:: username
56
+
57
+      The Docker Hub password
58
+
59
+.. zuul:rolevar:: docker_images
60
+   :type: list
61
+
62
+   A list of images to build.  Each item in the list should have:
63
+
64
+   .. zuul:rolevar:: context
65
+
66
+      The docker build context; this should be a directory underneath
67
+      :zuul:rolevar:`build-docker-image.zuul_work_dir`.
68
+
69
+   .. zuul:rolevar:: repository
70
+
71
+      The name of the target repository in dockerhub for the
72
+      image.  Supply this even if the image is not going to be
73
+      uploaded (it will be tagged with this in the local
74
+      registry).
75
+
76
+   .. zuul:rolevar:: path
77
+
78
+      Optional: the directory that should be passed to docker build.
79
+      Useful for building images with a Dockerfile in the context
80
+      directory but a source repository elsewhere.
81
+
82
+   .. zuul:jobvar:: build_args
83
+      :type: list
84
+
85
+      Optional: a list of values to pass to the docker ``--build-arg``
86
+      parameter.
87
+
88
+   .. zuul:rolevar:: target
89
+
90
+      Optional: the target for a multi-stage build.
91
+
92
+   .. zuul:jobvar:: tags
93
+      :type: list
94
+      :default: ['latest']
95
+
96
+      A list of tags to be added to the image when promoted.
97
+
98
+.. _anchors: https://yaml.org/spec/1.2/spec.html#&%20anchor//

+ 1
- 0
roles/build-docker-image/defaults/main.yaml View File

@@ -0,0 +1 @@
1
+zuul_work_dir: "{{ zuul.project.src_dir }}"

+ 13
- 0
roles/build-docker-image/tasks/main.yaml View File

@@ -0,0 +1,13 @@
1
+- name: Build a docker image
2
+  command: >-
3
+    docker build {{ item.path | default('.') }} -f Dockerfile
4
+    {% if target | default(false) -%}
5
+      --target {{ target }}
6
+    {% endif -%}
7
+    {% for build_arg in item.build_args | default([]) -%}
8
+      --build-arg {{ build_arg }}
9
+    {% endfor -%}
10
+    --tag {{ item.repository }}:change_{{ zuul.change }}
11
+  args:
12
+    chdir: "{{ zuul_work_dir }}/{{ item.context }}"
13
+  loop: "{{ images }}"

+ 3
- 0
roles/promote-docker-image/README.rst View File

@@ -0,0 +1,3 @@
1
+Promote one or more previously uploaded docker images.
2
+
3
+.. include:: ../../roles/build-docker-image/common.rst

+ 1
- 0
roles/promote-docker-image/defaults/main.yaml View File

@@ -0,0 +1 @@
1
+zuul_work_dir: "{{ zuul.project.src_dir }}"

+ 20
- 0
roles/promote-docker-image/tasks/main.yaml View File

@@ -0,0 +1,20 @@
1
+# This is used by the delete tasks
2
+- name: Get dockerhub JWT token
3
+  no_log: true
4
+  uri:
5
+    url: "https://hub.docker.com/v2/users/login/"
6
+    body_format: json
7
+    body:
8
+      username: "{{ credentials.username }}"
9
+      password: "{{ credentials.password }}"
10
+  register: jwt_token
11
+- name: Promote image
12
+  loop: "{{ images }}"
13
+  loop_control:
14
+    loop_var: image
15
+  include_tasks: promote-retag.yaml
16
+- name: Delete obsolete tags
17
+  loop: "{{ images }}"
18
+  loop_control:
19
+    loop_var: image
20
+  include_tasks: promote-cleanup.yaml

+ 20
- 0
roles/promote-docker-image/tasks/promote-cleanup.yaml View File

@@ -0,0 +1,20 @@
1
+- name: List tags
2
+  uri:
3
+    url: "https://hub.docker.com/v2/repositories/{{ image.repository }}/tags?page_size=1000"
4
+    status_code: 200
5
+  register: tags
6
+- name: Set cutoff timestamp to 24 hours ago
7
+  command: "python3 -c \"import datetime; print((datetime.datetime.utcnow()-datetime.timedelta(days=1)).strftime('%Y-%m-%dT%H:%M:%fZ'))\""
8
+  register: cutoff
9
+- name: Delete all change tags older than the cutoff
10
+  no_log: true
11
+  loop: "{{ tags.json.results }}"
12
+  loop_control:
13
+    loop_var: docker_tag
14
+  when: docker_tag.last_updated < cutoff.stdout and docker_tag.name.startswith('change_')
15
+  uri:
16
+    url: "https://hub.docker.com/v2/repositories/{{ image.repository }}/tags/{{ docker_tag.name }}/"
17
+    method: DELETE
18
+    status_code: 204
19
+    headers:
20
+      Authorization: "JWT {{ jwt_token.json.token }}"

+ 39
- 0
roles/promote-docker-image/tasks/promote-retag.yaml View File

@@ -0,0 +1,39 @@
1
+- name: Get dockerhub token
2
+  no_log: true
3
+  uri:
4
+    url: "https://auth.docker.io/token?service=registry.docker.io&scope=repository:{{ image.repository }}:pull,push"
5
+    user: "{{ credentials.username }}"
6
+    password: "{{ credentials.password }}"
7
+    force_basic_auth: true
8
+  register: token
9
+- name: Get manifest
10
+  no_log: true
11
+  uri:
12
+    url: "https://registry.hub.docker.com/v2/{{ image.repository }}/manifests/change_{{ zuul.change }}"
13
+    status_code: 200
14
+    headers:
15
+      Accept: "application/vnd.docker.distribution.manifestv2+json"
16
+      Authorization: "Bearer {{ token.json.token }}"
17
+    return_content: true
18
+  register: manifest
19
+- name: "Put manifest"
20
+  no_log: true
21
+  loop: "{{ image.tags | default(['latest']) }}"
22
+  loop_control:
23
+    loop_var: new_tag
24
+  uri:
25
+    url: "https://registry.hub.docker.com/v2/{{ image.repository }}/manifests/{{ new_tag }}"
26
+    method: PUT
27
+    status_code: 201
28
+    body: "{{ manifest.content | string }}"
29
+    headers:
30
+      Content-Type: "application/vnd.docker.distribution.manifestv2+json"
31
+      Authorization: "Bearer {{ token.json.token }}"
32
+- name: Delete the current change tag
33
+  no_log: true
34
+  uri:
35
+    url: "https://hub.docker.com/v2/repositories/{{ image.repository }}/tags/change_{{ zuul.change }}/"
36
+    method: DELETE
37
+    status_code: 204
38
+    headers:
39
+      Authorization: "JWT {{ jwt_token.json.token }}"

+ 3
- 0
roles/upload-docker-image/README.rst View File

@@ -0,0 +1,3 @@
1
+Upload one or more docker images.
2
+
3
+.. include:: ../../roles/build-docker-image/common.rst

+ 1
- 0
roles/upload-docker-image/defaults/main.yaml View File

@@ -0,0 +1 @@
1
+zuul_work_dir: "{{ zuul.project.src_dir }}"

+ 6
- 0
roles/upload-docker-image/tasks/main.yaml View File

@@ -0,0 +1,6 @@
1
+- name: Log in to dockerhub
2
+  command: "docker login -u {{ credentials.username }} -p {{ credentials.password }}"
3
+  no_log: true
4
+- name: Upload to dockerhub
5
+  command: "docker push {{ item.repository }}:change_{{ zuul.change }}"
6
+  loop: "{{ images }}"

Loading…
Cancel
Save