zuul/zuul-website
Code Issues Proposed changes

Add a Blog system to the zuul website

Browse Source 
This is a rough first draft at simple blog system. I think this will
work for maybe up to 5 posts but then we'll need to add on pagination.
I'm open to help and ideas :)

Make each blog entry an "<article>".  Add 3em of margin to the top
of articles so that there is more visual separation for blog
entries.

Import a reprint of https://opensource.com/article/20/2/zuul in
order to better test out our blog formatting options, and
potentially to serve as a starter article while we work on drafting
an updated 10-year birthday article.

A child of this change will serve to preview the look of multiple
articles.

Change-Id: Ibf496b6b56e8fa57bb114a7fabcb9639fdf8daef
Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
Co-Authored-By: James E. Blair <jim@acmegating.com>
This commit is contained in:
Clark Boylan 2022-05-06 08:52:21 -07:00 committed by Jeremy Stanley
parent 442732a7ea
commit 04d5e9550b
8 changed files with 349 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
www/assets/css/main.css
View File

@ -1467,6 +1467,10 @@ h2 {
font-size: 1.75em; font-size: 1.75em;
letter-spacing: -0.025em; } letter-spacing: -0.025em; }
article.blog {
margin-top: 3em;
}
h3 { h3 {
font-size: 1.2em; font-size: 1.2em;
letter-spacing: -0.025em; } letter-spacing: -0.025em; }

336
www/blog/index.html Normal file
View File

@ -0,0 +1,336 @@
<!DOCTYPE HTML>
<!--
Arcana by HTML5 UP
html5up.net | @ajlkn
Free for personal and commercial use under the CCA 3.0 license (html5up.net/license)
-->
<html>
<head>
<title>Zuul Project Gating Blog</title>
<meta charset="utf-8" />
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/site.webmanifest">
<link rel="mask-icon" href="/safari-pinned-tab.svg" color="#071d49">
<meta name="msapplication-TileColor" content="#00aba9">
<meta name="theme-color" content="#ffffff">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<!--[if lte IE 8]><script src="../assets/js/ie/html5shiv.js"></script><![endif]-->
<link rel="stylesheet" href="../assets/css/main.css" />
<!--[if lte IE 8]><link rel="stylesheet" href="../assets/css/ie8.css" /><![endif]-->
<!--[if lte IE 9]><link rel="stylesheet" href="../assets/css/ie9.css" /><![endif]-->
</head>
<body>
<div id="page-wrapper">
<!-- Header -->
<div id="header">
<!-- Logo -->
<h1>
<a href="/"><img id="logo" src="../images/logo.svg" alt="Zuul"/></a>
</h1>
<!-- Nav -->
<nav id="nav">
<ul>
<li><a href="../index.html">Home</a></li>
<li><a href="../users.html">Users</a></li>
<li class="current"><a href=".">Blog</a></li>
<li>
<a href="../docs/">Documentation</a>
<ul>
<li><a href="../docs/zuul/">Zuul</a></li>
<li><a href="../docs/nodepool">Nodepool</a></li>
<li><a href="../docs/zuul-jobs">Zuul Jobs</a></li>
<li><a href="../docs/zuul-base-jobs">Base Jobs</a></li>
<li><a href="../docs/zuul-sphinx/">Zuul Sphinx</a></li>
<li><a href="../docs/zuul-client/">Zuul Client</a></li>
<li><a href="../docs/zuul-operator/">Zuul Operator</a></li>
<li><a href="../docs/faq.html">Zuul FAQ</a></li>
</ul>
</li>
<li><a href="../community.html">Community</a></li>
<li><a href="../get-started.html">Get Started</a></li>
</ul>
</nav>
</div>
<section class="wrapper style1">
<div class="container">
<div id="content">
<header class="major">
<h2>Zuul Blog</h2>
</header>
<article class="blog">
<h2>
<a id="20200207a">Introducing Zuul for improved CI/CD</a>
</h2>
<h3>
A quick history of how and why Zuul is replacing Jenkins in CI
testing in the OpenStack community.
</h3>
<h4>Authored by Jeremy Stanley, February 7, 2020</h4>
<h5>
(This article originally ran <a
href="https://opensource.com/article/20/2/zuul">on opensource.com</a>
and is reprinted here with permission of the author under the <a
href="http://creativecommons.org/licenses/by-sa/4.0/">Creative
Commons Attribution-Share Alike 4.0 International License</a>.)
</h5>
<p>
<a href="https://jenkins.io/">Jenkins</a> is a marvelous piece of
software. As an execution and automation engine, it's one of the best
you're going to find. Jenkins serves as a key component in countless
continuous integration (CI) systems, and this is a testament to the
value of what its community has built over the years. But that's what
it is&mdash;a component. Jenkins is not a CI system itself; it just
runs things for you. It does that really well and has a variety of
built-ins and a vibrant ecosystem of plugins to help you tell it what
to run, when, and where.
</p>
<p>
CI is, at the most fundamental level, about integrating the work of
multiple software development streams into a coherent whole with as
much frequency and as little friction as possible. Jenkins, on its
own, doesn't know about your source code or how to merge it together,
nor does it know how to give constructive feedback to you and your
colleagues. You can, of course, glue it together with other software
that can perform these activities, and this is how many CI systems
incorporate Jenkins.
</p>
<p>
It's what we did for OpenStack, too, at least at first.
</p>
<h3>If it's not tested, it's broken</h3>
<p>
In 2010, an open source community of projects called <a
href="https://www.openstack.org/">OpenStack</a> was forming. Some of
the developers brought in to assist with the collaboration
infrastructure also worked on a free database project called <a
href="https://en.wikipedia.org/wiki/Drizzle_(database_server)"
>Drizzle</a>, and a key philosophy within that community was the idea
"if it's not tested, it's broken." So OpenStack, on day one, required
all proposed changes of its software to be reviewed and tested for
regressions before they could be approved to merge into the trunk of
any source code repositories. To do this, Hudson (which later forked
to form the Jenkins project) was configured to run tests exercising
every change.
</p>
<p>
A plugin was installed to interface with the <a
href="https://www.gerritcodereview.com/">Gerrit</a> code review
system, automatically triggering jobs when new changes were proposed
and reporting back with review comments indicating whether they
succeeded or failed. This may sound rudimentary by today's standards,
but at the time, it was a revolutionary advancement for an open
source collaboration. No developer on OpenStack was special in the
eyes of CI, and everyone's changes had to pass this growing battery
of tests before they could merge&mdash;a concept the project called
"project gating."
</p>
<p>
There was, however, an emerging flaw with this gating idea: To
guarantee two unrelated changes didn't alter a piece of software in
functionally incompatible ways, they had to be tested one at a time
in sequence before they could merge. OpenStack was complicated to
install and test, even back then, and quickly grew in popularity. The
rising volume of developer contributions coupled with increasing test
coverage meant that, during busy periods, there was simply not enough
time to test every change that passed review. Some longer-running
jobs took nearly an hour to complete, so the upper bound for what
could get through the gate was roughly two dozen changes in a day.
The resulting merge backlog showed a new solution was required.
</p>
<h3>Enter Zuul</h3>
<p>
During an OpenStack CI meeting in May 2012, one of the CI team
members, James Blair, <a
href="http://eavesdrop.openstack.org/irclogs/%23openstack-meeting/%23openstack-meeting.2012-05-22.log.html#t2012-05-22T19:42:27"
>announced</a> that he'd "been working on speculative execution of
Jenkins jobs." <strong>Speculative execution</strong> is an
optimization most commonly found in the pipelines of modern
microprocessors. Much like the analogy with processor hardware, the
theory was that by optimistically predicting positive gating results
for changes recently approved but that had not yet completed their
tests, subsequently approved changes could be tested concurrently and
then conditionally merged as long as their predecessors also passed
tests and merged. James said he had a name for this intelligent
scheduler: <a href="https://zuul-ci.org/">Zuul</a>.
</p>
<p>
Within this time frame, challenges from trying to perform better
revision control for Jenkins' XML job configuration led to the
creation of the human-readable YAML-based <a
href="https://jenkins-job-builder.readthedocs.io/">Jenkins Job
Builder</a> templating engine. Limited success with the JClouds
plugin for Jenkins and cumbersome attempts to use jobs for refreshing
cloud images of single-use Jenkins slaves ended with the creation of
the <a href="https://zuul-ci.org/docs/nodepool/">Nodepool</a>
service. Limited log-storage capabilities resulted in the team adding
separate external solutions for organizing, serving, and indexing job
logs and assuming maintainership of an abandoned secure copy protocol
(SCP) plugin replacing the less-secure FTP option that Jenkins
provided out of the box. The OpenStack infrastructure team was slowly
building a fleet of services and utilities around Jenkins but began
to bump up against a performance limitation.
</p>
<h3>Multiplying Jenkins</h3>
<p>
By mid-2013, Nodepool was constantly recycling as many as 100 virtual
machines registered with Jenkins as slaves, but this was no longer
enough to keep up with the growing workload. Thread contention for
global locks in Jenkins thwarted all attempts to push past this
threshold, no matter how much processor power and memory was thrown
at the master server. The project had offers to donate additional
capacity for Jenkins slaves to help relieve the frequent job backlog,
but this would require an additional Jenkins master. The efficient
division of work between multiple masters needed a new channel of
communication for dispatch and coordination of jobs. Zuul's
maintainers identified the <a href="http://gearman.org/">Gearman</a>
job server protocol as an ideal fit, so they outfitted Zuul with a
new <a
href="https://docs.opendev.org/opendev/gear/#server-example"
>geard</a> service and extended Jenkins with a custom Gearman client
plugin.
</p>
<p>
Now that jobs were spread across a growing assembly of Jenkins
masters, there was no longer any single dashboard with a complete
view of job activity and results. In order to facilitate this new
multi-master world, Zuul grew its own status API and WebUI, as well
as a feature to emit metrics through the <a
href="https://github.com/statsd/statsd">StatsD</a> protocol. Over the
next few years, Zuul steadily subsumed more of the CI features its
users relied on, while Jenkins' place in the system waned
accordingly, and it was becoming a liability. OpenStack made an early
choice to standardize on the Python programming language; this was
reflected in Zuul's development, yet Jenkins and its plugins were
implemented in Java. Zuul's configuration was maintained in the same
YAML serialization format that OpenStack used to template its own
Jenkins jobs, while Jenkins kept everything in baroque XML. These
differences complicated ongoing maintenance and led to an
unnecessarily steep learning curve for new administrators from
related communities that had started trying to run Zuuls.
</p>
<p>
The time was right for another revolution.
</p>
<h3>The rise of Ansible</h3>
<p>
In early 2016, Zuul's maintainers embarked on an ambitious year-long
overhaul of their growing fleet of services with the goal of
eliminating Jenkins from the overall system design. By this time,
Jenkins was serving only as a conduit for running jobs consisting
mostly of shell scripts on slave nodes over SSH, providing real-time
streaming of job output and copying resulting artifacts to
longer-term storage. <a href="https://www.ansible.com/">Ansible</a>
was found to be a great fit for that first need; purpose-built to run
commands remotely over SSH, it was written in Python, just like Zuul,
and also used YAML to define its tasks. It even had built-in modules
for features the team had previously implemented as bespoke Jenkins
plugins. Ansible provided true multi-node support right out of the
box, so the same playbooks could be used for both simulating and
performing complex production deployments. An ever-expanding
ecosystem of third-party modules filled in any gaps, in much the same
way as the Jenkins community's plugins had before.
</p>
<p>
A new Zuul executor service filled the prior role of the Jenkins
master: it acted on pending requests in the scheduler's geard,
dispatched them via Ansible to ephemeral servers managed by Nodepool,
then collected results and artifacts for publication. It also exposed
in-progress build output over the classic <a
href="https://tools.ietf.org/html/rfc742">RFC 742 Name/Finger
protocol</a>, streamed in real time from an extension of Ansible's
command output module. Once it was no longer necessary to limit jobs
to what Jenkins' parser could comprehend, Zuul was free to grow new
features like distributed in-repository job definitions, shareable
between projects with inheritance and secure handling of secrets, as
well as the ability to test-drive proposed changes for the jobs
themselves. Jenkins served its purpose admirably, but at least for
Zuul, its usefulness was finally at an end.
</p>
<h3>Testing the future</h3>
<p>
Zuul's community likes to say that it "tests the future" through its
novel application of speculative execution. Gone are the harrowing
days of wondering whether the improvement you want to make to an
existing job will render it non-functional once it's applied in
production. Overloaded review teams for a massive central job
repository are a thing of the past. Jobs are treated as a part of the
software and shipped right alongside the rest of the source code,
taking advantage of Zuul's other features like cross-repository
dependencies so that your change to part of a job in one project can
be exercised with a proposed job change in another project. It will
even comment on your job changes, highlighting specific lines with
syntax problems as if it were another code reviewer giving you
advice.
</p>
<p>
These were features Zuul only dreamed of before, but which required
freedom from Jenkins so that it could take job parsing into its own
hands. This is the future of CI, and Zuul's users are living it.
</p>
<p>
As of early 2019, the OpenStack Foundation recognized Zuul as an
independent, openly governed project with its own identity and
flourishing community. If you're into open source CI, consider taking
a look. Development on the next evolution of Zuul is always underway,
and you're welcome to help. Find out more on <a
href="https://zuul-ci.org/">Zuul's website</a>.
</p>
</article>
</div>
</div>
</section>
<!-- Footer -->
<div id="footer">
<div class="container">
<div class="row">
<section class="3u 6u(narrower) 12u$(mobilep)">
<h3>Get Zuul</h3>
<ul class="links">
<li><a href="https://opendev.org/zuul/zuul/">Source</a></li>
<li><a href="https://pypi.org/project/zuul/">Install</a></li>
<li><a href="https://zuul-ci.org/docs/">Documentation</a></li>
<li><a href="https://zuul-ci.org/docs/faq.html">FAQ</a></li>
</ul>
</section>
<section class="3u 6u$(narrower) 12u$(mobilep)">
<h3>Get Involved</h3>
<ul class="links">
<li><a href="http://lists.zuul-ci.org/cgi-bin/mailman/listinfo">Mailing lists</a></li>
<li><a href="https://matrix.to/#/#zuul:opendev.org">Matrix Chat</a></li>
<li><a href="https://storyboard.openstack.org/#!/project/679">Bugs</a></li>
<li><a href="https://review.opendev.org/#/q/status:open+project:zuul/zuul">Code Review</a></li>
</ul>
</section>
</div>
</div>
<!-- Copyright -->
<div class="copyright">
<ul class="menu">
<li>&copy; Zuul Contributors. Licensed under Creative Commons Attribution 3.0</li><li>Zuul is collaboratively developed under the Apache 2 license and managed by the Open Infrastructure Foundation.</li><li>Design based on Arcana from: <a href="https://htmlup.net">HTML5 UP</a></li><li><a href="https://opendev.org/zuul/zuul-website/">Browse page source</a></li>
</ul>
</div>
</div>
</div>
<!-- Scripts -->
<script src="../assets/js/jquery.min.js"></script>
<script src="../assets/js/jquery.dropotron.min.js"></script>
<script src="../assets/js/skel.min.js"></script>
<script src="../assets/js/util.js"></script>
<!--[if lte IE 8]><script src="../assets/js/ie/respond.min.js"></script><![endif]-->
<script src="../assets/js/main.js"></script>
</body>
</html>

1
www/community.html
View File

@ -37,6 +37,7 @@
<ul> <ul>
<li><a href="index.html">Home</a></li> <li><a href="index.html">Home</a></li>
<li><a href="users.html">Users</a></li> <li><a href="users.html">Users</a></li>
<li><a href="blog/">Blog</a></li>
<li> <li>
<a href="docs/">Documentation</a> <a href="docs/">Documentation</a>
<ul> <ul>

1
www/docs/index.html
View File

@ -37,6 +37,7 @@
<ul> <ul>
<li><a href="../index.html">Home</a></li> <li><a href="../index.html">Home</a></li>
<li><a href="../users.html">Users</a></li> <li><a href="../users.html">Users</a></li>
<li><a href="../blog">Blog</a></li>
<li class="current"> <li class="current">
<a href=".">Documentation</a> <a href=".">Documentation</a>
<ul> <ul>

1
www/get-started.html
View File

@ -50,6 +50,7 @@
<ul> <ul>
<li><a href="index.html">Home</a></li> <li><a href="index.html">Home</a></li>
<li><a href="users.html">Users</a></li> <li><a href="users.html">Users</a></li>
<li><a href="blog/">Blog</a></li>
<li> <li>
<a href="docs/">Documentation</a> <a href="docs/">Documentation</a>
<ul> <ul>

1
www/index.html
View File

@ -50,6 +50,7 @@
<ul> <ul>
<li class="current"><a href="index.html">Home</a></li> <li class="current"><a href="index.html">Home</a></li>
<li><a href="users.html">Users</a></li> <li><a href="users.html">Users</a></li>
<li><a href="blog/">Blog</a></li>
<li> <li>
<a href="docs/">Documentation</a> <a href="docs/">Documentation</a>
<ul> <ul>

4
www/sitemap.xml
View File

@ -16,6 +16,10 @@
<loc>https://zuul-ci.org/users.html</loc> <loc>https://zuul-ci.org/users.html</loc>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url>
<loc>https://zuul-ci.org/blog/</loc>
<changefreq>daily</changefreq>
</url>
<url> <url>
<loc>https://zuul-ci.org/docs/index.html</loc> <loc>https://zuul-ci.org/docs/index.html</loc>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>

1
www/users.html
View File

@ -37,6 +37,7 @@
<ul> <ul>
<li><a href="index.html">Home</a></li> <li><a href="index.html">Home</a></li>
<li class="current"><a href="users.html">Users</a></li> <li class="current"><a href="users.html">Users</a></li>
<li><a href="blog/">Blog</a></li>
<li> <li>
<a href="docs/">Documentation</a> <a href="docs/">Documentation</a>
<ul> <ul>