Browse Source

Add spec for publishing build artifacts on server

This change adds the spec file which described how official
build artifacts will be published on the world-facing server.

Change-Id: I0704b3ecbd1f34138edc7c3a5efe5cd888821fe5
Signed-off-by: Jason McKenna <jason.mckenna@windriver.com>
Authored-By: Jason McKenna <jason.mckenna@windriver.com>
Co-Authored-By: Scott Little <scott.little@windriver.com>
Story: 2003906
Task: 27974
changes/87/618587/4
Jason McKenna 6 months ago
parent
commit
1bd070a63d
1 changed files with 307 additions and 0 deletions
  1. 307
    0
      specs/2019.03/approved/mirror_2003906_published_builds.rst

+ 307
- 0
specs/2019.03/approved/mirror_2003906_published_builds.rst View File

@@ -0,0 +1,307 @@
1
+..
2
+  This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+  License. http://creativecommons.org/licenses/by/3.0/legalcode
4
+
5
+..
6
+
7
+===================================================
8
+StarlingX: Host Build Artifacts on StarlingX Mirror
9
+===================================================
10
+
11
+https://storyboard.openstack.org/#!/story/2003906
12
+
13
+An external mirror hosting build artifacts allows for quicker ramp-up
14
+of community members, and provides for reference artifacts.
15
+
16
+This story will implement the building of artifacts on a server and
17
+the population of those artifacts to the existing community-accessible
18
+mirror.
19
+
20
+
21
+Problem description
22
+===================
23
+
24
+A new community member who wishes to evaluate StarlingX currently needs
25
+to download the source code, set up a build environment, and build all
26
+the StarlingX components.  They will then generate their own ISO.  This
27
+process can take a significant amount of time, and lead to a poor initial
28
+impression of StarlingX.
29
+
30
+The initial implementation of hosted builds would have a server:
31
+* Build all RPMs used for the build
32
+* Build an installer image based on the produced RPMs
33
+* Build a reference ISO based on the produced RPMs
34
+* Publish RPMs, installer, and reference ISO to the existing mirror
35
+
36
+The initial implementation would provide artifacts for the r/2018.10
37
+and the master branch of StarlingX.  Future work would build and host
38
+artifacts for evolving branches at an interval to be determined.
39
+
40
+
41
+Use Cases
42
+=========
43
+
44
+Evaluators of the StarlingX project want to use a prebuilt ISO image to
45
+install and test StarlingX.
46
+
47
+Developers want to compare their local changes to a package with a
48
+reference image containing an unmodified version of the package
49
+
50
+Producers of an ISO want to use the prebuilt installer image rather than
51
+one they produce themselves.
52
+
53
+
54
+Proposed change
55
+===============
56
+
57
+Provision a server such that it is able to build the StarlingX
58
+project.  Allow for this build to produce artifacts based on either
59
+the master branch, release branches (e.g. r/2018.10), or feature branches
60
+(e.g. f/somefeature).  Building and hosting of other builds (branches or
61
+tags) should be accomodated.  Provide a mechanism for the build to publish its
62
+artifacts (RPMs, ISO, installer) to the existing mirror.  Initial
63
+implementation will use a build server at CENGN.
64
+
65
+Periodic building of the master branch, and publishing of the resulting
66
+artifacts via the StarlingX mirror, will be performed.  Other branches
67
+(e.g. r/2018.10) will also be built and published, but only as required.
68
+
69
+The publication path for each build shall be unique and will include branch
70
+and UTC time stamp.  An alias for the latest build of the branch shall be
71
+provided.
72
+
73
+Initial rate of builds is 1 build of the master branch per day.  This
74
+cadence may be adjusted in the future.  Non-master branch builds shall
75
+be performed on an on-demand basis as determined by the build team.
76
+Non-master branch builds include release builds, and milestone builds.
77
+Milestone builds are to be built periodically, at a cadence to be determined.
78
+
79
+The retention period of master branch build shall initially be 14 days.
80
+The retention period may be revisited and adjusted at a later time.
81
+The retention time for release branch builds will be much greater then
82
+master branch builds, possibly indefinitely.  Milestone builds shall
83
+be retained for at least one release period past their introduction.
84
+
85
+Hosted artifacts proposed directory structure:
86
+
87
+Build artifacts hosted by the mirror shall be available in following
88
+directory structure:
89
+
90
+http://mirror.starlingx.cengn.ca/mirror/starlingx/<path>
91
+
92
+<path> = <release-identifier>/<distro>/<build-date>/outputs/<output-artifact-type>
93
+         <release-identifier>/<distro>/<build-date>/inputs/<input-artifact-type>
94
+
95
+<release-identifier> = 'master'
96
+                       'feature/'<feature-branch-name>
97
+                       'release/'<release-branch-name>
98
+                       'milestone/'<milestone-tag>
99
+                       ...
100
+                       'r2018.10' (legacy symlink to release/2018.10)
101
+
102
+<distro> = 'centos'
103
+
104
+<feature-branch-name> = 'centos76'
105
+                        ...
106
+
107
+<release-branch-name> = '2018.10'
108
+                        ...
109
+
110
+<build-date> = $(date +%Y%m%dT%H%M%SZ)   # time stamp of start of 'repo sync'
111
+               <latest_build>   # a symlink to the last successful build date
112
+
113
+<input-artifact-type> = RPMS/
114
+                        SRPMS/
115
+                        downloads/
116
+
117
+<output-artifact-type> = RPMS/<build-phase>/
118
+                         SRPMS/<build-phase>/
119
+                         iso/
120
+                         installer/
121
+                         wheels/
122
+                         helm-charts/
123
+                         CONTEXT
124
+                         <build-result>
125
+                         logs/
126
+
127
+<build-phase> = std
128
+                rt
129
+                installer
130
+                container
131
+                ...
132
+
133
+<build-result> = 'success' | 'fail'
134
+
135
+Non-hosted artifacts:
136
+
137
+Other artifacts produced by the build may be hosted outside of the
138
+mirror.  In particular, container images shall be hosted on DockerHub.
139
+
140
+The DockerHub container image path shall be
141
+
142
+<image-name>=<org>/<image>:<tag>
143
+
144
+<org>=starlingx
145
+
146
+<image>=stx-<component>
147
+
148
+<tag>=<git-tag-or-branch>-<os>-<openstack-release>[-<qualifier>]
149
+
150
+<os>=centos | ubuntu | clear-linux
151
+
152
+<openstack-release>=pike | queens | rocky ...
153
+
154
+<component>=aodh | ceilometer| cinder | glance | gnocchi | heat | horizon |
155
+            ironic | keystone | libvirt | magnum | murano | neutron |
156
+            nova-api-proxy | nova | panko ...
157
+
158
+<qualifier>=<timestamp> | latest | stable
159
+
160
+<git-tag-or-branch>=dev | r-2018.10.0 | m-2019.05b1 | ...
161
+    (This naming convention is for reference only, and has not been finalized)
162
+
163
+
164
+
165
+Alternatives
166
+============
167
+
168
+The existing mechanism of requiring all persons to perform their own
169
+builds is an alternative.
170
+
171
+Providing a prebuilt version of StarlingX via a container (rather than
172
+as build artifacts) has not been considered, as it would still depend
173
+on an official build of StarlingX artifacts to install in said
174
+container.
175
+
176
+
177
+Data model impact
178
+=================
179
+
180
+None
181
+
182
+
183
+REST API impact
184
+===============
185
+
186
+None
187
+
188
+
189
+Security impact
190
+===============
191
+
192
+Server administration:
193
+The build server is to be administered by a small group of core
194
+developers from the existing StarlingX Build team.  The server
195
+should be kept up to date with security related OS updates.  The
196
+address and access mechanism of the server are not to be published
197
+to those outside the StarlingX Build team or StarlingX Security
198
+team.
199
+
200
+Build initiation:
201
+There shall be a mechanism for triggering builds.  This mechanism
202
+shall support self-initiated builds (periodic, or based on
203
+observed external git commits, etc).  The mechanism shall also
204
+support manually-initiated builds.  The mechanism for manually
205
+initiated builds shall initially be through the administration
206
+interface (discussed above).  Providing other mechanisms to
207
+initialize builds (for example, providing a web interface through
208
+Jenkins, or a similar mechanism) is beyond the scope of this
209
+document, however security concerns must be taken into account
210
+if providing an additional interface.
211
+
212
+
213
+Other end user impact
214
+=====================
215
+
216
+End user will only see a change in StarlingX deployment if they
217
+choose to make use of the pre-published artifacts.  The automated
218
+use of pre-published artifacts (example: having build scripts
219
+use the pre-built installer by default) are potential future
220
+improvements outside the scope of this document.
221
+
222
+
223
+Performance Impact
224
+==================
225
+
226
+No impact to running systems.  Time to obtain and evaluation
227
+copy of StarlingX will be drastically reduced.
228
+
229
+
230
+Other Deployer Impact
231
+=====================
232
+
233
+None
234
+
235
+
236
+Developer impact
237
+=================
238
+
239
+Reference RPMs, Installer image, and ISO can be used to compare code
240
+in development.
241
+
242
+
243
+Upgrade impact
244
+===============
245
+
246
+None
247
+
248
+
249
+Implementation
250
+==============
251
+
252
+
253
+Assignee(s)
254
+===========
255
+
256
+Primary assignee:
257
+  Scott Little <slittle1>
258
+
259
+Other contributors:
260
+  Jason McKenna <jmckenna>
261
+  Don Penney <dpenney>
262
+  Al Bailey <albailey>
263
+
264
+
265
+Repos Impacted
266
+==============
267
+
268
+stx-tools - scripts to assist in publication of build outputs
269
+
270
+
271
+Work Items
272
+===========
273
+
274
+* Setup base metal (CENGN) Server to perform builds
275
+* Create mechanism for server to perform periodic build of master branch
276
+* Create mechanism for server to perform build of designated release branch
277
+* Create script or mechanism to publish builds to existing mirror
278
+
279
+
280
+Dependencies
281
+============
282
+
283
+None
284
+
285
+
286
+Testing
287
+=======
288
+
289
+Basic installation and load sanity testing of produced iso.
290
+
291
+
292
+Documentation Impact
293
+====================
294
+
295
+Documentation for the artifacts will be posted on the StarlingX wiki
296
+
297
+
298
+References
299
+==========
300
+
301
+https://www.cengn.ca/
302
+http://mirror.starlingx.cengn.ca/mirror/
303
+
304
+
305
+History
306
+=======
307
+

Loading…
Cancel
Save