zuul/zuul
Code Issues Proposed changes

Reorganize docs into user/admin guide

Browse Source 
Refresh the user and admin guide for v3 changes, and reorganize into
a narrative structure which makes more sense for v3.

Change-Id: I4ac3b18d5ed33b0fea4e2ef0318b19bfc3447ccc
This commit is contained in:
James E. Blair
2017-06-20 00:00:37 -07:00
parent 1de8d40f38
commit eff5a9d8d7
32 changed files with 1518 additions and 2381 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
doc/source/admin-config.rst
View File

@@ -1,24 +0,0 @@
:title: Administrative Configuration
.. _admin-config:
.. _drivers:
Administrative Configuration
============================
TODO:
* zuul.conf
* connections
* main.yaml
* tenants
* trusted/untrusted
* drivers
* gerrit
* trigger config
* reporter config
* github
* trigger config
* reporter config

0
doc/source/client.rst → doc/source/admin/client.rst
View File

312
doc/source/admin/components.rst Normal file
View File

@@ -0,0 +1,312 @@
:title: Components
.. _components:
Components
==========
Zuul is a distributed system consisting of several components, each of
which is described below. All Zuul processes read the
**/etc/zuul/zuul.conf** file (an alternate location may be supplied on
the command line) which uses an INI file syntax. Each component may
have its own configuration file, though you may find it simpler to use
the same file for all components.
A minimal Zuul system may consist of a *scheduler* and *executor* both
running on the same host. Larger installations should consider
running multiple executors, each on a dedicated host, and running
mergers on dedicated hosts as well.
Common
------
The following applies to all Zuul components.
Configuration
~~~~~~~~~~~~~
The following sections of **zuul.conf** are used by all Zuul components:
gearman
"""""""
Client connection information for gearman.
**server**
Hostname or IP address of the Gearman server.
``server=gearman.example.com`` (required)
**port**
Port on which the Gearman server is listening.
``port=4730`` (optional)
**ssl_ca**
Optional: An openssl file containing a set of concatenated
“certification authority” certificates in PEM formet.
**ssl_cert**
Optional: An openssl file containing the client public certificate in
PEM format.
**ssl_key**
Optional: An openssl file containing the client private key in PEM format.
Scheduler
---------
The scheduler is the primary component of Zuul. The scheduler is not
a scalable component; one, and only one, scheduler must be running at
all times for Zuul to be operational. It receives events from any
connections to remote systems which have been configured, enqueues
items into pipelines, distributes jobs to executors, and reports
results.
Configuration
~~~~~~~~~~~~~
The following sections of **zuul.conf** are used by the scheduler:
gearman_server
""""""""""""""
The builtin gearman server. Zuul can fork a gearman process from itself rather
than connecting to an external one.
**start**
Whether to start the internal Gearman server (default: False).
``start=true``
**listen_address**
IP address or domain name on which to listen (default: all addresses).
``listen_address=127.0.0.1``
**log_config**
Path to log config file for internal Gearman server.
``log_config=/etc/zuul/gearman-logging.yaml``
**ssl_ca**
Optional: An openssl file containing a set of concatenated “certification authority” certificates
in PEM formet.
**ssl_cert**
Optional: An openssl file containing the server public certificate in PEM format.
**ssl_key**
Optional: An openssl file containing the server private key in PEM format.
webapp
""""""
**listen_address**
IP address or domain name on which to listen (default: 0.0.0.0).
``listen_address=127.0.0.1``
**port**
Port on which the webapp is listening (default: 8001).
``port=8008``
.. TODO: move this to webapp (currently in 'zuul')
**status_expiry**
Zuul will cache the status.json file for this many seconds. This is an
optional value and ``1`` is used by default.
``status_expiry=1``
scheduler
"""""""""
.. TODO: rename this to 'scheduler' (currently 'zuul') and update to match these docs
**tenant_config**
Path to tenant config file.
``layout_config=/etc/zuul/tenant.yaml``
**log_config**
Path to log config file.
``log_config=/etc/zuul/scheduler-logging.yaml``
**pidfile**
Path to PID lock file.
``pidfile=/var/run/zuul/scheduler.pid``
**state_dir**
Path to directory that Zuul should save state to.
``state_dir=/var/lib/zuul``
Operation
~~~~~~~~~
To start the scheduler, run ``zuul-scheduler``. To stop it, kill the
PID which was saved in the pidfile specified in the configuration.
Most of Zuul's configuration is automatically updated as changes to
the repositories which contain it are merged. However, Zuul must be
explicitly notified of changes to the tenant config file, since it is
not read from a git repository. To do so, send the scheduler PID
(saved in the pidfile specified in the configuration) a SIGHUP signal.
Merger
------
Mergers are an optional Zuul service; they are not required for Zuul
to operate, but some high volume sites may benefit from running them.
Zuul performs quite a lot of git operations in the course of its work.
Each change that is to be tested must be speculatively merged with the
current state of its target branch to ensure that it can merge, and to
ensure that the tests that Zuul perform accurately represent the
outcome of merging the change. Because Zuul's configuration is stored
in the git repos it interacts with, and is dynamically evaluated, Zuul
often needs to perform a speculative merge in order to determine
whether it needs to perform any further actions.
All of these git operations add up, and while Zuul executors can also
perform them, large numbers may impact their ability to run jobs.
Therefore, administrators may wish to run standalone mergers in order
to reduce the load on executors.
Configuration
~~~~~~~~~~~~~
The following section of **zuul.conf** is used by the merger:
merger
""""""
**git_dir**
Directory that Zuul should clone local git repositories to.
``git_dir=/var/lib/zuul/git``
**git_user_email**
Optional: Value to pass to `git config user.email`.
``git_user_email=zuul@example.com``
**git_user_name**
Optional: Value to pass to `git config user.name`.
``git_user_name=zuul``
**log_config**
Path to log config file for the merger process.
``log_config=/etc/zuul/logging.yaml``
**pidfile**
Path to PID lock file for the merger process.
``pidfile=/var/run/zuul-merger/merger.pid``
Operation
~~~~~~~~~
To start the merger, run ``zuul-merger``. To stop it, kill the
PID which was saved in the pidfile specified in the configuration.
Executor
--------
Executors are responsible for running jobs. At the start of each job,
an executor prepares an environment in which to run Ansible which
contains all of the git repositories specified by the job with all
dependent changes merged into their appropriate branches. The branch
corresponding to the proposed change will be checked out (in all
projects, if it exists). Any roles specified by the job will also be
present (also with dependent changes merged, if appropriate) and added
to the Ansible role path. The executor also prepares an Ansible
inventory file with all of the nodes requested by the job.
The executor also contains a merger. This is used by the executor to
prepare the git repositories used by jobs, but is also available to
perform any tasks normally performed by standalone mergers. Because
the executor performs both roles, small Zuul installations may not
need to run standalone mergers.
Trusted and Untrusted Playbooks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The executor runs playbooks in one of two execution contexts depending
on whether the project containing the playbook is a *config project*
or an *untrusted project*. If the playbook is in a *config project*,
the executor runs the playbook in the *trusted* execution context,
otherwise, it is run in the *untrusted* execution context.
Both execution contexts use `bubblewrap`_ to create a namespace to
ensure that playbook executions are isolated and are unable to access
files outside of a restricted environment. The administrator may
configure additional local directories on the executor to be made
available to the restricted environment.
The *trusted* execution context has access to all Ansible features,
including the ability to load custom Ansible modules. Needless to
say, extra scrutiny should be given to code that runs in a trusted
context as it could be used to compromise other jobs running on the
executor, or the executor itself, especially if the administrator has
granted additional access through bubblewrap, or a method of escaping
the restricted environment created by bubblewrap is found.
Playbooks run in the *untrusted* execution context are not permitted
to load additional Ansible modules or access files outside of the
restricted environment prepared for them by the executor. In addition
to the bubblewrap environment applied to both execution contexts, in
the *untrusted* context some standard Ansible modules are replaced
with versions which prohibit some actions, including attempts to
access files outside of the restricted execution context. These
redundant protections are made as part of a defense-in-depth strategy.
.. _bubblewrap: https://github.com/projectatomic/bubblewrap
Configuration
~~~~~~~~~~~~~
The following sections of **zuul.conf** are used by the executor:
executor
""""""""
**finger_port**
Port to use for finger log streamer.
``finger_port=79``
**git_dir**
Directory that Zuul should clone local git repositories to.
``git_dir=/var/lib/zuul/git``
**log_config**
Path to log config file for the executor process.
``log_config=/etc/zuul/logging.yaml``
**private_key_file**
SSH private key file to be used when logging into worker nodes.
``private_key_file=~/.ssh/id_rsa``
**user**
User ID for the zuul-executor process. In normal operation as a daemon,
the executor should be started as the ``root`` user, but it will drop
privileges to this user during startup.
``user=zuul``
merger
""""""
**git_user_email**
Optional: Value to pass to `git config user.email`.
``git_user_email=zuul@example.com``
**git_user_name**
Optional: Value to pass to `git config user.name`.
``git_user_name=zuul``
Operation
~~~~~~~~~
To start the executor, run ``zuul-executor``.
There are several commands which can be run to control the executor's
behavior once it is running.
To stop the executor immediately, aborting all jobs (they may be
relaunched according to their retry policy), run ``zuul-executor
stop``.
To request that the executor stop executing new jobs and exit when all
currently running jobs have completed, run ``zuul-executor graceful``.
To enable or disable running Ansible in verbose mode (with the '-vvv'
argument to ansible-playbook) run ``zuul-executor verbose`` and
``zuul-executor unverbose``.

59
doc/source/admin/connections.rst Normal file
View File

@@ -0,0 +1,59 @@
:title: Connection Configuration
.. _connection-config:
Connection Configuration
========================
Most of Zuul's configuration is contained in the git repositories upon
which Zuul operates, however, some configuration outside of git
repositories is still required to bootstrap the system. This includes
information on connections between Zuul and other systems, as well as
identifying the projects Zuul uses.
.. _connections:
Connections
-----------
In order to interact with external systems, Zuul must have a
*connection* to that system configured. Zuul includes a number of
drivers, each of which implements the functionality necessary to
connect to a system. Each connection in Zuul is associated with a
driver.
To configure a connection in Zuul, select a unique name for the
connection and add a section to **zuul.conf** with the form
"[connection NAME]". For example, a connection to a gerrit server may
appear as::
[connection mygerritserver]
driver=gerrit
server=review.example.com
.. _drivers:
Drivers
-------
Drivers may support any of the following functions:
* Sources -- hosts git repositories for projects. Zuul can clone git
repos for projects and fetch refs.
* Triggers -- emits events to which Zuul may respond. Triggers are
configured in pipelines to cause changes or other refs to be
enqueued.
* Reporters -- outputs information when a pipeline is finished
processing an item.
Zuul includes the following drivers:
.. toctree::
:maxdepth: 2
drivers/gerrit
drivers/github
drivers/smtp
drivers/sql
drivers/timer
drivers/zuul

168
doc/source/admin/drivers/gerrit.rst Normal file
View File

@@ -0,0 +1,168 @@
:title: Gerrit Driver
Gerrit
======
`Gerrit`_ is a code review system. The Gerrit driver supports
sources, triggers, and reporters.
.. _Gerrit: https://www.gerritcodereview.com/
Zuul will need access to a Gerrit user.
Create an SSH keypair for Zuul to use if there isn't one already, and
create a Gerrit user with that key::
cat ~/id_rsa.pub | ssh -p29418 review.example.com gerrit create-account --ssh-key - --full-name Zuul zuul
Give that user whatever permissions will be needed on the projects you
want Zuul to report on. For instance, you may want to grant
``Verified +/-1`` and ``Submit`` to the user. Additional categories
or values may be added to Gerrit. Zuul is very flexible and can take
advantage of those.
Connection Configuration
------------------------
The supported options in zuul.conf connections are:
**driver=gerrit**
**server**
FQDN of Gerrit server.
``server=review.example.com``
**canonical_hostname**
The canonical hostname associated with the git repos on the Gerrit
server. Defaults to the value of **server**. This is used to
identify repos from this connection by name and in preparing repos
on the filesystem for use by jobs.
``canonical_hostname=git.example.com``
**port**
Optional: Gerrit server port.
``port=29418``
**baseurl**
Optional: path to Gerrit web interface. Defaults to ``https://<value
of server>/``. ``baseurl=https://review.example.com/review_site/``
**user**
User name to use when logging into above server via ssh.
``user=zuul``
**sshkey**
Path to SSH key to use when logging into above server.
``sshkey=/home/zuul/.ssh/id_rsa``
**keepalive**
Optional: Keepalive timeout, 0 means no keepalive.
``keepalive=60``
Trigger Configuration
---------------------
Zuul works with standard versions of Gerrit by invoking the ``gerrit
stream-events`` command over an SSH connection. It also reports back
to Gerrit using SSH.
If using Gerrit 2.7 or later, make sure the user is a member of a group
that is granted the ``Stream Events`` permission, otherwise it will not
be able to invoke the ``gerrit stream-events`` command over SSH.
The supported pipeline trigger options are:
**event**
The event name from gerrit. Examples: ``patchset-created``,
``comment-added``, ``ref-updated``. This field is treated as a
regular expression.
**branch**
The branch associated with the event. Example: ``master``. This
field is treated as a regular expression, and multiple branches may
be listed.
**ref**
On ref-updated events, the branch parameter is not used, instead the
ref is provided. Currently Gerrit has the somewhat idiosyncratic
behavior of specifying bare refs for branch names (e.g., ``master``),
but full ref names for other kinds of refs (e.g., ``refs/tags/foo``).
Zuul matches what you put here exactly against what Gerrit
provides. This field is treated as a regular expression, and
multiple refs may be listed.
**ignore-deletes**
When a branch is deleted, a ref-updated event is emitted with a newrev
of all zeros specified. The ``ignore-deletes`` field is a boolean value
that describes whether or not these newrevs trigger ref-updated events.
The default is True, which will not trigger ref-updated events.
**approval**
This is only used for ``comment-added`` events. It only matches if
the event has a matching approval associated with it. Example:
``code-review: 2`` matches a ``+2`` vote on the code review category.
Multiple approvals may be listed.
**email**
This is used for any event. It takes a regex applied on the performer
email, i.e. Gerrit account email address. If you want to specify
several email filters, you must use a YAML list. Make sure to use non
greedy matchers and to escapes dots!
Example: ``email: ^.*?@example\.org$``.
**email_filter** (deprecated)
A deprecated alternate spelling of *email*. Only one of *email* or
*email_filter* should be used.
**username**
This is used for any event. It takes a regex applied on the performer
username, i.e. Gerrit account name. If you want to specify several
username filters, you must use a YAML list. Make sure to use non greedy
matchers and to escapes dots!
Example: ``username: ^jenkins$``.
**username_filter** (deprecated)
A deprecated alternate spelling of *username*. Only one of *username* or
*username_filter* should be used.
**comment**
This is only used for ``comment-added`` events. It accepts a list of
regexes that are searched for in the comment string. If any of these
regexes matches a portion of the comment string the trigger is
matched. ``comment: retrigger`` will match when comments
containing 'retrigger' somewhere in the comment text are added to a
change.
**comment_filter** (deprecated)
A deprecated alternate spelling of *comment*. Only one of *comment* or
*comment_filter* should be used.
*require-approval*
This may be used for any event. It requires that a certain kind
of approval be present for the current patchset of the change (the
approval could be added by the event in question). It follows the
same syntax as the :ref:`"approval" pipeline requirement
<pipeline-require-approval>`. For each specified criteria there must
exist a matching approval.
*reject-approval*
This takes a list of approvals in the same format as
*require-approval* but will fail to enter the pipeline if there is
a matching approval.
Reporter Configuration
----------------------
Zuul works with standard versions of Gerrit by invoking the
``gerrit`` command over an SSH connection. It reports back to
Gerrit using SSH.
The dictionary passed to the Gerrit reporter is used for ``gerrit
review`` arguments, with the boolean value of ``true`` simply
indicating that the argument should be present without following it
with a value. For example, ``verified: 1`` becomes ``gerrit review
--verified 1`` and ``submit: true`` becomes ``gerrit review
--submit``.
A :ref:`connection<connections>` that uses the gerrit driver must be
supplied to the trigger.

162
doc/source/admin/drivers/github.rst Normal file
View File

@@ -0,0 +1,162 @@
:title: GitHub Driver
GitHub
======
The GitHub driver supports sources, triggers, and reporters. It can
interact with the public GitHub service as well as site-local
installations of GitHub enterprise.
.. TODO: make this section more user friendly
Configure GitHub `webhook events
<https://developer.github.com/webhooks/creating/>`_.
Set *Payload URL* to
``http://<zuul-hostname>/connection/<connection-name>/payload``.
Set *Content Type* to ``application/json``.
Select *Events* you are interested in. See below for the supported events.
Connection Configuration
------------------------
The supported options in zuul.conf connections are:
**driver=github**
**api_token**
API token for accessing GitHub.
See `Creating an access token for command-line use
<https://help.github.com/articles/creating-an-access-token-for-command-line-use/>`_.
**webhook_token**
Optional: Token for validating the webhook event payloads.
If not specified, payloads are not validated.
See `Securing your webhooks
<https://developer.github.com/webhooks/securing/>`_.
**sshkey**
Path to SSH key to use when cloning github repositories.
``sshkey=/home/zuul/.ssh/id_rsa``
**git_host**
Optional: Hostname of the github install (such as a GitHub Enterprise)
If not specified, defaults to ``github.com``
``git_host=github.myenterprise.com``
Trigger Configuration
---------------------
GitHub webhook events can be configured as triggers.
A connection name with the github driver can take multiple events with the
following options.
**event**
The event from github. Supported events are ``pull_request``,
``pull_request_review``, and ``push``.
A ``pull_request`` event will
have associated action(s) to trigger from. The supported actions are:
*opened* - pull request opened
*changed* - pull request synchronized
*closed* - pull request closed
*reopened* - pull request reopened
*comment* - comment added on pull request
*labeled* - label added on pull request
*unlabeled* - label removed from pull request
*status* - status set on commit
A ``pull_request_review`` event will
have associated action(s) to trigger from. The supported actions are:
*submitted* - pull request review added
*dismissed* - pull request review removed
**branch**
The branch associated with the event. Example: ``master``. This
field is treated as a regular expression, and multiple branches may
be listed. Used for ``pull_request`` and ``pull_request_review`` events.
**comment**
This is only used for ``pull_request`` ``comment`` actions. It accepts a
list of regexes that are searched for in the comment string. If any of these
regexes matches a portion of the comment string the trigger is matched.
``comment: retrigger`` will match when comments containing 'retrigger'
somewhere in the comment text are added to a pull request.
**label**
This is only used for ``labeled`` and ``unlabeled`` ``pull_request`` actions.
It accepts a list of strings each of which matches the label name in the
event literally. ``label: recheck`` will match a ``labeled`` action when
pull request is labeled with a ``recheck`` label. ``label: 'do not test'``
will match a ``unlabeled`` action when a label with name ``do not test`` is
removed from the pull request.
**state**
This is only used for ``pull_request_review`` events. It accepts a list of
strings each of which is matched to the review state, which can be one of
``approved``, ``comment``, or ``request_changes``.
**status**
This is used for ``pull-request`` and ``status`` actions. It accepts a
list of strings each of which matches the user setting the status, the
status context, and the status itself in the format of
``user:context:status``. For example,
``zuul_github_ci_bot:check_pipeline:success``.
**ref**
This is only used for ``push`` events. This field is treated as a regular
expression and multiple refs may be listed. GitHub always sends full ref
name, eg. ``refs/tags/bar`` and this string is matched against the regexp.
Reporter Configuration
----------------------
Zuul reports back to GitHub via GitHub API. Available reports include a PR
comment containing the build results, a commit status on start, success and
failure, an issue label addition/removal on the PR, and a merge of the PR
itself. Status name, description, and context is taken from the pipeline.
A :ref:`connection<connections>` that uses the github driver must be
supplied to the reporter. It has the following options:
**status**
String value (``pending``, ``success``, ``failure``) that the reporter should
set as the commit status on github.
``status: 'success'``
**status-url**
String value for a link url to set in the github status. Defaults to the zuul
server status_url, or the empty string if that is unset.
**comment**
Boolean value (``true`` or ``false``) that determines if the reporter should
add a comment to the pipeline status to the github pull request. Defaults
to ``true``. Only used for Pull Request based events.
``comment: false``
**merge**
Boolean value (``true`` or ``false``) that determines if the reporter should
merge the pull reqeust. Defaults to ``false``. Only used for Pull Request based
events.
``merge=true``
**label**
List of strings each representing an exact label name which should be added
to the pull request by reporter. Only used for Pull Request based events.
``label: 'test successful'``
**unlabel**
List of strings each representing an exact label name which should be removed
from the pull request by reporter. Only used for Pull Request based events.
``unlabel: 'test failed'``

53
doc/source/admin/drivers/smtp.rst Normal file
View File

@@ -0,0 +1,53 @@
:title: SMTP Driver
SMTP
====
The SMTP driver supports reporters only. It is used to send email
when items report.
Connection Configuration
------------------------
**driver=smtp**
**server**
SMTP server hostname or address to use.
``server=localhost``
**port**
Optional: SMTP server port.
``port=25``
**default_from**
Who the email should appear to be sent from when emailing the report.
This can be overridden by individual pipelines.
``default_from=zuul@example.com``
**default_to**
Who the report should be emailed to by default.
This can be overridden by individual pipelines.
``default_to=you@example.com``
Reporter Configuration
----------------------
A simple email reporter is also available.
A :ref:`connection<connections>` that uses the smtp driver must be supplied to the
reporter. The connection also may specify a default *To* or *From*
address.
Each pipeline can overwrite the ``subject`` or the ``to`` or ``from`` address by
providing alternatives as arguments to the reporter. For example, ::
- pipeline:
name: post-merge
success:
outgoing_smtp:
to: you@example.com
failure:
internal_smtp:
to: you@example.com
from: alternative@example.com
subject: Change {change} failed

44
doc/source/admin/drivers/sql.rst Normal file
View File

@@ -0,0 +1,44 @@
:title: SQL Driver
SQL
===
The SQL driver supports reporters only. Only one connection per
database is permitted. The connection options are:
**driver=sql**
**dburi**
Database connection information in the form of a URI understood by
sqlalchemy. eg http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html#database-urls
``dburi=mysql://user:pass@localhost/db``
Reporter Configuration
----------------------
This reporter is used to store results in a database.
A :ref:`connection<connections>` that uses the sql driver must be
supplied to the reporter.
zuul.conf contains the database connection and credentials. To store different
reports in different databases you'll need to create a new connection per
database.
The SQL reporter does nothing on "start" or "merge-failure"; it only
acts on "success" or "failure" reporting stages.
**score**
A score to store for the result of the build. eg: -1 might indicate a failed
build.
For example ::
- pipeline:
name: post-merge
success:
mydb_conn:
score: 1
failure:
mydb_conn:
score: -1

24
doc/source/admin/drivers/timer.rst Normal file
View File

@@ -0,0 +1,24 @@
:title: Timer Driver
Timer
=====
The timer driver supports triggers only. It is used for configuring
pipelines so that jobs run at scheduled times. No connection
configuration is required.
Trgger Configuration
--------------------
Timers don't require a special connection or driver. Instead they can
simply be used by listing **timer** as the trigger.
This trigger will run based on a cron-style time specification.
It will enqueue an event into its pipeline for every project
defined in the configuration. Any job associated with the
pipeline will run in response to that event.
**time**
The time specification in cron syntax. Only the 5 part syntax is
supported, not the symbolic names. Example: ``0 0 * * *`` runs
at midnight.

40
doc/source/admin/drivers/zuul.rst Normal file
View File

@@ -0,0 +1,40 @@
:title: Zuul Driver
Zuul
====
The Zuul driver supports triggers only. It is used for triggering
pipelines based on internal Zuul events.
Trigger Configuration
---------------------
Zuul events don't require a special connection or driver. Instead they
can simply be used by listing **zuul** as the trigger.
**event**
The event name. Currently supported:
*project-change-merged* when Zuul merges a change to a project,
it generates this event for every open change in the project.
*parent-change-enqueued* when Zuul enqueues a change into any
pipeline, it generates this event for every child of that
change.
**pipeline**
Only available for ``parent-change-enqueued`` events. This is the
name of the pipeline in which the parent change was enqueued.
*require-approval*
This may be used for any event. It requires that a certain kind
of approval be present for the current patchset of the change (the
approval could be added by the event in question). It follows the
same syntax as the :ref:`"approval" pipeline requirement
<pipeline-require-approval>`. For each specified criteria there must
exist a matching approval.
*reject-approval*
This takes a list of approvals in the same format as
*require-approval* but will fail to enter the pipeline if there is
a matching approval.

20
doc/source/admin/index.rst Normal file
View File

@@ -0,0 +1,20 @@
Administrator's Guide
=====================
This guide is intended for administrators of Zuul systems. It covers
installation, operation, and the portion of Zuul configuration that
occurs outside of the projects upon which Zuul operates. Advanced
users may be interested in some of the concepts described here, as
well as understanding what features the underlying configuration
provides to in-project configuration.
.. toctree::
:maxdepth: 2
quick-start
installation
components
connections
tenants
monitoring
client

69
doc/source/admin/installation.rst Normal file
View File

@@ -0,0 +1,69 @@
Installation
============
Install Zuul
------------
To install a Zuul release from PyPI, run::
pip install zuul
Or from a git checkout, run::
pip install .
That will also install Zuul's python dependencies. To minimize
interaction with other python packages installed on a system, you may
wish to install Zuul within a Python virtualenv.
Zuul has several system-level dependencies as well. You can find a
list of operating system packages in `bindep.txt` in Zuul's source
directory.
External Dependencies
---------------------
Zuul interacts with several other systems described below.
Gearman
~~~~~~~
Gearman is a job distribution system that Zuul uses to communicate
with its distributed components. The Zuul scheduler distributes work
to Zuul mergers and executors use Gearman. You may supply your own
gearman server, but the Zuul scheduler includes a built-in server
which is recommended. Ensure that all Zuul hosts can communicate with
the gearman server.
Zuul distributes secrets to executors via gearman, so be sure to
secure it with TLS and certificate authentication. Obtain (or
generate) a certificate for both the server and the clients (they may
use the same certificate or have individual certificates). They must
be signed by a CA, but it can be your own CA.
Nodepool
~~~~~~~~
In order to run all but the simplest jobs, Zuul uses a companion
program, Nodepool, to supply the nodes (whether dynamic cloud
instances or static hardware) used by jobs. Before starting Zuul,
ensure you have Nodepool installed and any images you require built.
Zuul only makes one requirement of these nodes: that it be able to log
in given a username and ssh private key.
.. TODO: SpamapS any zookeeper config recommendations?
Nodepool uses Zookeeper to communicate internally among its
components, and also to communicate with Zuul. You can run a simple
single-node Zookeeper instance, or a multi-node cluster. Ensure that
The host running the Zuul scheduler has access to the cluster.
Ansible
~~~~~~~
Zuul uses Ansible to run jobs. Each version of Zuul is designed to
work with a specific, contemporary version of Ansible. Zuul specifies
that version of Ansible in its python package metadata, and normally
the correct version will be installed automatically with Zuul.
Because of the close integration of Zuul and Ansible, attempting to
use other versions of Ansible with Zuul is not recommended.

37
doc/source/statsd.rst → doc/source/admin/monitoring.rst
View File

@@ -1,15 +1,17 @@
:title: Statsd reporting
:title: Monitoring
Monitoring
==========
Statsd reporting
================
----------------
Zuul comes with support for the statsd protocol, when enabled and configured
(see below), the Zuul scheduler will emit raw metrics to a statsd receiver
which let you in turn generate nice graphics. An example is OpenStack Zuul
status page: http://status.openstack.org/zuul/
which let you in turn generate nice graphics.
Configuration
-------------
~~~~~~~~~~~~~
Statsd support uses the statsd python module. Note that Zuul will start without
the statsd python module, so an existing Zuul installation may be missing it.
@@ -27,17 +29,16 @@ Your init script most probably loads a configuration file named
STATSD_PORT=8125
Metrics
-------
~~~~~~~
The metrics are emitted by the Zuul scheduler (`zuul/scheduler.py`):
**gerrit.event.<type> (counters)**
Gerrit emits different kind of message over its `stream-events` interface. As
a convenience, Zuul emits metrics to statsd which save you from having to use
a different daemon to measure Gerrit events.
The Gerrit events have different types defined by Gerrit itself, Zuul will
relay any type of event reusing the name defined by Gerrit. Some of the
events emitted are:
Gerrit emits different kind of message over its `stream-events`
interface. Zuul will report counters for each type of event it
receives from Gerrit.
Some of the events emitted are:
* patchset-created
* draft-published
@@ -52,18 +53,6 @@ The metrics are emitted by the Zuul scheduler (`zuul/scheduler.py`):
Refer to your Gerrit installation documentation for an exhaustive list of
Gerrit event types.
**zuul.node_type.**
Holds metrics specifc to build nodes per label. The hierarchy is:
#. **<build node label>** each of the labels associated to a build in
Jenkins. It contains:
#. **job.<jobname>** subtree detailing per job statistics:
#. **wait_time** counter and timer of the wait time, with the
difference of the job start time and the execute time, in
milliseconds.
**zuul.pipeline.**
Holds metrics specific to jobs. The hierarchy is:

120
doc/source/admin/quick-start.rst Normal file
View File

@@ -0,0 +1,120 @@
Quick Start Guide
=================
This provides a very simple overview of Zuul. It is recommended to
read the following sections for more details.
Install Zuul
------------
You can get zuul from pypi via::
pip install zuul
Zuul Components
---------------
Zuul provides the following components:
- **zuul-scheduler**: The main Zuul process. Handles receiving
events, executing jobs, collecting results and posting reports.
Coordinates the work of the other components.
- **zuul-merger**: Scale-out component that performs git merge
operations. Zuul performs a large number of git operations in
the course of its work. Adding merger processes can help speed
Zuul's processing. This component is optional (zero or more of
these can be run).
- **zuul-executor**: Scale-out component for executing jobs. At
least one of these is required. Depending on system
configuration, you can expect a single executor to handle up to
about 100 simultaneous jobs. Can handle the functions of a
merger if dedicated mergers are not provided. One or more of
these must be run.
- **gearman**: optional builtin gearman daemon provided by zuul-scheduler
External components:
- **gearman**: A gearman daemon if the built-in daemon is not used.
- **zookeeper**: A zookeeper cluster (or single host) for
communicating with Nodepool.
- **nodepool**: Provides nodes for Zuul to use when executing jobs.
Zuul Setup
----------
At minimum you need to provide **zuul.conf** and **main.yaml** placed
in **/etc/zuul/**. The following example uses the builtin gearman
service in Zuul, and a connection to Gerrit.
**zuul.conf**::
[zuul]
tenant_config=/etc/zuul/main.yaml
[gearman_server]
start=true
[gearman]
server=127.0.0.1
[connection gerrit]
driver=gerrit
server=git.example.com
port=29418
baseurl=https://git.example.com/gerrit/
user=zuul
sshkey=/home/zuul/.ssh/id_rsa
See :ref:`components` and :ref:`connections` for more details.
The following tells Zuul to read its configuration from and operate on
the *example-project* project:
**main.yaml**::
- tenant:
name: example-tenant
source:
gerrit:
untrusted-projects:
- example-project
Starting Zuul
-------------
You can run any zuul process with the **-d** option to make it not
daemonize. It's a good idea at first to confirm there's no issues with
your configuration.
To start, simply run::
zuul-scheduler
Once run you should have two zuul-scheduler processes (if using the
built-in gearman server, or one process otherwise).
Before Zuul can run any jobs, it needs to load its configuration, most
of which is in the git repositories that Zuul operates on. Start an
executor to allow zuul to do that::
zuul-executor
Zuul should now be able to read its configuration from the configured
repo and process any jobs defined therein.
Troubleshooting
---------------
You can use telnet to connect to gearman to check which Zuul
components are online::
telnet <gearman_ip> 4730
Useful commands are **workers** and **status** which you can run by just
typing those commands once connected to gearman.

95
doc/source/admin/tenants.rst Normal file
View File

@@ -0,0 +1,95 @@
:title: Tenant Configuration
.. _tenant-config:
Tenant Configuration
====================
After *zuul.conf* is configured, Zuul component servers will be able
to start, but a tenant configuration is required in order for Zuul to
perform any actions. The tenant configuration file specifies upon
which projects Zuul should operate. These repositories are
grouped into tenants. The configuration of each tenant is separate
from the rest (no pipelines, jobs, etc are shared between them).
A project may appear in more than one tenant; this may be useful if
you wish to use common job definitions across multiple tenants.
The tenant configuration file is specified by the *tenant_config*
setting in the *scheduler* section of *zuul.yaml*. It is a YAML file
which, like other Zuul configuration files, is a list of configuration
objects, though only one type of object is supported, *tenant*.
Tenant
------
A tenant is a collection of projects which share a Zuul
configuration. An example tenant definition is::
- tenant:
name: my-tenant
source:
gerrit:
config-projects:
- common-config
- shared-jobs:
include: jobs
untrusted-projects:
- project1
- project2
The following attributes are supported:
**name** (required)
The name of the tenant. This may appear in URLs, paths, and
monitoring fields, and so should be restricted to URL friendly
characters (ASCII letters, numbers, hyphen and underscore) and you
should avoid changing it unless necessary.
**source** (required)
A dictionary of sources to consult for projects. A tenant may
contain projects from multiple sources; each of those sources must
be listed here, along with the projects it supports. The name of a
:ref:`connection<connections>` is used as the dictionary key
(e.g. `gerrit` in the example above), and the value is a further
dictionary containing the keys below.
**config-projects**
A list of projects to be treated as config projects in this
tenant. The jobs in a config project are trusted, which means
they run with extra privileges, do not have their configuration
dynamically loaded for proposed changes, and zuul.yaml files are
only searched for in the master branch.
**untrusted-projects**
A list of projects to be treated as untrusted in this tenant. An
untrusted project is the typical project operated on by Zuul.
Their jobs run in a more restrictive environment, they may not
define pipelines, their configuration dynamically changes in
response to proposed changes, Zuul will read configuration files
in all of their branches.
Each of the projects listed may be either a simple string value, or
it may be a dictionary with the following keys:
**include**
Normally Zuul will load all of the configuration classes
appropriate for the type of project (config or untrusted) in
question. However, if you only want to load some items, the
*include* attribute can be used to specify that *only* the
specified classes should be loaded. Supplied as a string, or a
list of strings.
**exclude**
A list of configuration classes that should not be loaded.
The order of the projects listed in a tenant is important. A job
which is defined in one project may not be redefined in another
project; therefore, once a job appears in one project, a project
listed later will be unable to define a job with that name.
Further, some aspects of project configuration (such as the merge
mode) may only be set on the first appearance of a project
definition.
Zuul loads the configuration from all *config-projects* in the order
listed, followed by all *trusted-projects* in order.

110
doc/source/cloner.rst
View File

@@ -1,110 +0,0 @@
:title: Zuul Cloner
Zuul Cloner
===========
Zuul includes a simple command line client that may be used to clone
repositories with Zuul references applied.
Configuration
-------------
Clone map
'''''''''
By default, Zuul cloner will clone the project under ``basepath`` which
would create sub directories whenever a project name contains slashes. Since
you might want to finely tweak the final destination, a clone map lets you
change the destination on a per project basis. The configuration is done using
a YAML file passed with ``-m``.
With a project hierarchy such as::
project
thirdparty/plugins/plugin1
You might want to get ``project`` straight in the base path, the clone map would be::
clonemap:
- name: 'project'
dest: '.'
Then to strip out ``thirdparty`` such that the plugins land under the
``/plugins`` directory of the basepath, you can use regex and capturing
groups::
clonemap:
- name: 'project'
dest: '.'
- name: 'thirdparty/(plugins/.*)'
dest: '\1'
The resulting workspace will contains::
project -> ./
thirdparty/plugins/plugin1 -> ./plugins/plugin1
Zuul parameters
'''''''''''''''
The Zuul cloner reuses Zuul parameters such as ZUUL_BRANCH, ZUUL_REF or
ZUUL_PROJECT. It will attempt to load them from the environment variables or
you can pass them as parameters (in which case it will override the
environment variable if it is set). The matching command line parameters use
the ``zuul`` prefix hence ZUUL_REF can be passed to the cloner using
``--zuul-ref``.
Usage
-----
The general options that apply are:
.. program-output:: zuul-cloner --help
Ref lookup order
''''''''''''''''
The Zuul cloner will attempt to lookup references in this order:
1) Zuul reference for the indicated branch
2) Zuul reference for the master branch
3) The tip of the indicated branch
4) The tip of the master branch
The "indicated branch" is one of the following:
A) The project-specific override branch (from project_branches arg)
B) The user specified branch (from the branch arg)
C) ZUUL_BRANCH (from the zuul_branch arg)
Clone order
-----------
When cloning repositories, the destination folder should not exist or
``git clone`` will complain. This happens whenever cloning a sub project
before its parent project. For example::
zuul-cloner project/plugins/plugin1 project
Will create the directory ``project`` when cloning the plugin. The
cloner processes the clones in the order supplied, so you should swap the
projects::
zuul-cloner project project/plugins/plugin1
Cached repositories
-------------------
The ``--cache-dir`` option can be used to reduce network traffic by
cloning from a local repository which may not be up to date.
If the ``--cache-dir`` option is supplied, zuul-cloner will start by
cloning any projects it processes from those found in that directory.
The URL of origin remote of the resulting clone will be reset to use
the ``git_base_url`` and then the remote will be updated so that the
repository has all the information in the upstream repository.
The default for ``--cache-dir`` is taken from the environment variable
``ZUUL_CACHE_DIR``. A value provided explicitly on the command line
overrides the environment variable setting.

38
doc/source/configuration.rst
View File

@@ -1,38 +0,0 @@
:title: Configuration
.. _config:
Configuration
=============
Zuul uses a very flexible configuration system. Some initial
configuration must be supplied by the system administrator. From that
beginning, Zuul builds its configuration by dynamically evaluating the
contents of every git repository indicated by the configuration.
Zuul configuration is focused primarily around the concept of a
*Pipeline*. Through a pipeline, a Zuul user expresses a connection
between events, projects, jobs, and reports. The general flow of
operations is as follows:
A *Connection* to a remote system generates an event; if it matches a
*Trigger* on a *Pipeline*, then an *Item* is enqueued into the
*Pipeline*. An *Item* is some kind of git reference, such as a branch
tip, a proposed change, or a pull request, and is associated with a
*Project*. Zuul launches the *Jobs* specified for that *Project* in
that *Pipeline*, and when they complete, Zuul reports the results as
specified by the pipeline's *Reporter*.
TODO: flow diagram
The following sections describe how to configure Zuul.
:ref:`admin-config` covers areas of interest primarily to system
administrators. :ref:`project-config` covers the portion of Zuul
configuration of interest to most users -- that which is included
directly in the git repositories operated on by Zuul.
.. toctree::
:maxdepth: 2
admin-config
project-config

128
doc/source/connections.rst
View File

@@ -1,128 +0,0 @@
:title: Connections
.. _connections:
Connections
===========
zuul coordinates talking to multiple different systems via the concept
of connections. A connection is listed in the :ref:`zuulconf` file and is
then referred to from the :ref:`layoutyaml`. This makes it possible to
receive events from gerrit via one connection and post results from another
connection that may report back as a different user.
Gerrit
------
Create a connection with gerrit.
**driver=gerrit**
**server**
FQDN of Gerrit server.
``server=review.example.com``
**canonical_hostname**
The canonical hostname associated with the git repos on the Gerrit
server. Defaults to the value of **server**. This is used to
identify repos from this connection by name and in preparing repos
on the filesystem for use by jobs.
``canonical_hostname=git.example.com``
**port**
Optional: Gerrit server port.
``port=29418``
**baseurl**
Optional: path to Gerrit web interface. Defaults to ``https://<value
of server>/``. ``baseurl=https://review.example.com/review_site/``
**user**
User name to use when logging into above server via ssh.
``user=zuul``
**sshkey**
Path to SSH key to use when logging into above server.
``sshkey=/home/zuul/.ssh/id_rsa``
**keepalive**
Optional: Keepalive timeout, 0 means no keepalive.
``keepalive=60``
Gerrit Configuration
~~~~~~~~~~~~~~~~~~~~
Zuul will need access to a Gerrit user.
Create an SSH keypair for Zuul to use if there isn't one already, and
create a Gerrit user with that key::
cat ~/id_rsa.pub | ssh -p29418 gerrit.example.com gerrit create-account --ssh-key - --full-name Jenkins jenkins
Give that user whatever permissions will be needed on the projects you
want Zuul to gate. For instance, you may want to grant ``Verified
+/-1`` and ``Submit`` to the user. Additional categories or values may
be added to Gerrit. Zuul is very flexible and can take advantage of
those.
GitHub
------
Create a connection with GitHub.
**driver=github**
**api_token**
API token for accessing GitHub.
See `Creating an access token for command-line use
<https://help.github.com/articles/creating-an-access-token-for-command-line-use/>`_.
**webhook_token**
Optional: Token for validating the webhook event payloads.
If not specified, payloads are not validated.
See `Securing your webhooks
<https://developer.github.com/webhooks/securing/>`_.
**sshkey**
Path to SSH key to use when cloning github repositories.
``sshkey=/home/zuul/.ssh/id_rsa``
**git_host**
Optional: Hostname of the github install (such as a GitHub Enterprise)
If not specified, defaults to ``github.com``
``git_host=github.myenterprise.com``
SMTP
----
**driver=smtp**
**server**
SMTP server hostname or address to use.
``server=localhost``
**port**
Optional: SMTP server port.
``port=25``
**default_from**
Who the email should appear to be sent from when emailing the report.
This can be overridden by individual pipelines.
``default_from=zuul@example.com``
**default_to**
Who the report should be emailed to by default.
This can be overridden by individual pipelines.
``default_to=you@example.com``
SQL
----
Only one connection per a database is permitted.
**driver=sql**
**dburi**
Database connection information in the form of a URI understood by
sqlalchemy. eg http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html#database-urls
``dburi=mysql://user:pass@localhost/db``

8
doc/source/encryption.rst
View File

@@ -1,8 +0,0 @@
:title: Encryption
.. _encryption:
Encryption
==========
TODO

317
doc/source/executors.rst
View File

@@ -1,317 +0,0 @@
:title: Executors
.. _Gearman: http://gearman.org/
.. _`Gearman Plugin`:
https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin
.. _`Turbo-Hipster`:
https://git.openstack.org/cgit/openstack/turbo-hipster/
.. _`Turbo-Hipster Documentation`:
http://turbo-hipster.rtfd.org/
.. _executors:
Executors
=========
Zuul has a modular architecture for executing jobs. Currently, the
only supported module interfaces with Gearman_. This design allows
any system to run jobs for Zuul simply by interfacing with a Gearman
server. The recommended way of integrating a new job-runner with Zuul
is via this method.
If Gearman is unsuitable, Zuul may be extended with a new executor
module. Zuul makes very few assumptions about the interface to a
executor -- if it can trigger jobs, cancel them, and receive success
or failure reports, it should be able to be used with Zuul. Patches
to this effect are welcome.
Zuul Parameters
---------------
Zuul will pass some parameters with every job it executes. These are
for workers to be able to get the repositories into the state they are
intended to be tested in. Builds can be triggered either by an action
on a change or by a reference update. Both events share a common set
of parameters and more specific parameters as follows:
Common parameters
~~~~~~~~~~~~~~~~~
**ZUUL_UUID**
Zuul provided key to link builds with Gerrit events.
**ZUUL_REF**
Zuul provided ref that includes commit(s) to build.
**ZUUL_COMMIT**
The commit SHA1 at the head of ZUUL_REF.
**ZUUL_PROJECT**
The project that triggered this build.
**ZUUL_PIPELINE**
The Zuul pipeline that is building this job.
**ZUUL_URL**
The URL for the zuul server as configured in zuul.conf.
A test runner may use this URL as the basis for fetching
git commits.
**BASE_LOG_PATH**
zuul suggests a path to store and address logs. This is deterministic
and hence useful for where you want workers to upload to a specific
destination or need them to have a specific final URL you can link to
in advanced. For changes it is:
"last-two-digits-of-change/change-number/patchset-number".
For reference updates it is: "first-two-digits-of-newrev/newrev"
**LOG_PATH**
zuul also suggests a unique path for logs to the worker. This is
"BASE_LOG_PATH/pipeline-name/job-name/uuid"
**ZUUL_VOTING**
Whether Zuul considers this job voting or not. Note that if Zuul is
reconfigured during the run, the voting status of a job may change
and this value will be out of date. Values are '1' if voting, '0'
otherwise.
Change related parameters
~~~~~~~~~~~~~~~~~~~~~~~~~
The following additional parameters will only be provided for builds
associated with changes (i.e., in response to patchset-created or
comment-added events):
**ZUUL_BRANCH**
The target branch for the change that triggered this build.
**ZUUL_CHANGE**
The Gerrit change ID for the change that triggered this build.
**ZUUL_CHANGES**
A caret character separated list of the changes upon which this build
is dependent upon in the form of a colon character separated list
consisting of project name, target branch, and revision ref.
**ZUUL_CHANGE_IDS**
All of the Gerrit change IDs that are included in this build (useful
when the DependentPipelineManager combines changes for testing).
**ZUUL_PATCHSET**
The Gerrit patchset number for the change that triggered this build.
Reference updated parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following additional parameters will only be provided for
post-merge (ref-updated) builds:
**ZUUL_OLDREV**
The SHA1 of the old revision at this ref (recall the ref name is
in ZUUL_REF).
**ZUUL_NEWREV**
The SHA1 of the new revision at this ref (recall the ref name is
in ZUUL_REF).
**ZUUL_SHORT_OLDREV**
The shortened (7 character) SHA1 of the old revision.
**ZUUL_SHORT_NEWREV**
The shortened (7 character) SHA1 of the new revision.
Unset revisions default to 00000000000000000000000000000000.
Examples:
When a reference is created::
ZUUL_OLDREV=00000000000000000000000000000000
ZUUL_NEWREV=123456789abcdef123456789abcdef12
ZUUL_SHORT_OLDREV=0000000
ZUUL_SHORT_NEWREV=1234567
When a reference is deleted::
ZUUL_OLDREV=123456789abcdef123456789abcdef12
ZUUL_NEWREV=00000000000000000000000000000000
ZUUL_SHORT_OLDREV=1234567
ZUUL_SHORT_NEWREV=0000000
And finally a reference being altered::
ZUUL_OLDREV=123456789abcdef123456789abcdef12
ZUUL_NEWREV=abcdef123456789abcdef123456789ab
ZUUL_SHORT_OLDREV=1234567
ZUUL_SHORT_NEWREV=abcdef1
Your jobs can check whether the parameters are ``000000`` to act
differently on each kind of event.
Gearman
-------
Gearman_ is a general-purpose protocol for distributing jobs to any
number of workers. Zuul works with Gearman by sending specific
information with job requests to Gearman, and expects certain
information to be returned on completion. This protocol is described
in `Zuul-Gearman Protocol`_.
In order for Zuul to run any jobs, you will need a running Gearman
server. Zuul includes a Gearman server, and it is recommended that it
be used as it supports the following features needed by Zuul:
* Canceling jobs in the queue (admin protocol command "cancel job").
* Strict FIFO queue operation (gearmand's round-robin mode may be
sufficient, but is untested).
To enable the built-in server, see the ``gearman_server`` section of
``zuul.conf``. Be sure that the host allows connections from Zuul and
any workers (e.g., Jenkins masters) on TCP port 4730, and nowhere else
(as the Gearman protocol does not include any provision for
authentication).
Gearman Jenkins Plugin
~~~~~~~~~~~~~~~~~~~~~~
The `Gearman Jenkins Plugin`_ makes it easy to use Jenkins with Zuul
by providing an interface between Jenkins and Gearman. In this
configuration, Zuul asks Gearman to run jobs, and Gearman can then
distribute those jobs to any number of Jenkins systems (including
multiple Jenkins masters).
The `Gearman Plugin`_ can be installed in Jenkins in order to
facilitate Jenkins running jobs for Zuul. Install the plugin and
configure it with the hostname or IP address of your Gearman server
and the port on which it is listening (4730 by default). It will
automatically register all known Jenkins jobs as functions that Zuul
can invoke via Gearman.
Any number of masters can be configured in this way, and Gearman will
distribute jobs to all of them as appropriate.
No special Jenkins job configuration is needed to support triggering
by Zuul.
The Gearman Plugin will ensure the `Zuul Parameters`_ are supplied as
Jenkins build parameters, so they will be available for use in the job
configuration as well as to the running job as environment variables.
Jenkins git plugin configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In order to test the correct build, configure the Jenkins Git SCM
plugin as follows::
Source Code Management:
Git
Repositories:
Repository URL: <your Gerrit or Zuul repository URL>
Advanced:
Refspec: ${ZUUL_REF}
Branches to build:
Branch Specifier: ${ZUUL_COMMIT}
Advanced:
Clean after checkout: True
That should be sufficient for a job that only builds a single project.
If you have multiple interrelated projects (i.e., they share a Zuul
Change Queue) that are built together, you may be able to configure
the Git plugin to prepare them, or you may chose to use a shell script
instead. As an example, the OpenStack project uses the following
script to prepare the workspace for its integration testing:
https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate-wrap.sh
Turbo Hipster Worker
~~~~~~~~~~~~~~~~~~~~
As an alternative to Jenkins, `Turbo-Hipster`_ is a small python
project designed specifically as a zuul job worker which can be
registered with gearman as a job runner. Please see the
`Turbo-Hipster Documentation`_ for details on how to set it up.
Zuul-Gearman Protocol
~~~~~~~~~~~~~~~~~~~~~
This section is only relevant if you intend to implement a new kind of
worker that runs jobs for Zuul via Gearman. If you just want to use
Jenkins, see `Gearman Jenkins Plugin`_ instead.
The Zuul protocol as used with Gearman is as follows:
Starting Builds
^^^^^^^^^^^^^^^
To start a build, Zuul invokes a Gearman function with the following
format:
build:FUNCTION_NAME
where **FUNCTION_NAME** is the name of the job that should be run. If
the job should run on a specific node (or class of node), Zuul will
instead invoke:
build:FUNCTION_NAME:NODE_NAME
where **NODE_NAME** is the name or class of node on which the job
should be run.
Zuul sends the ZUUL_* parameters described in `Zuul Parameters`_
encoded in JSON format as the argument included with the
SUBMIT_JOB_UNIQ request to Gearman. A unique ID (equal to the
ZUUL_UUID parameter) is also supplied to Gearman, and is accessible as
an added Gearman parameter with GRAB_JOB_UNIQ.
When a Gearman worker starts running a job for Zuul, it should
immediately send a WORK_DATA packet with the following information
encoded in JSON format:
**name**
The name of the job.
**number**
The build number (unique to this job).
**manager**
A unique identifier associated with the Gearman worker that can
abort this build. See `Stopping Builds`_ for more information.
**url** (optional)
The URL with the status or results of the build. Will be used in
the status page and the final report.
To help with debugging builds a worker may send back some optional
metadata:
**worker_name** (optional)
The unique name of the worker.
**worker_hostname** (optional)
The hostname of the worker.
It should then immediately send a WORK_STATUS packet with a value of 0
percent complete. It may then optionally send subsequent WORK_STATUS
packets with updated completion values.
When the build is complete, it should send a final WORK_DATA packet
with the following in JSON format:
**result**
Either the string 'SUCCESS' if the job succeeded, or any other value
that describes the result if the job failed.
Finally, it should send either a WORK_FAIL or WORK_COMPLETE packet as
appropriate. A WORK_EXCEPTION packet will be interpreted as a
WORK_FAIL, but the exception will be logged in Zuul's error log.
Stopping Builds
^^^^^^^^^^^^^^^
If Zuul needs to abort a build already in progress, it will invoke the
following function through Gearman:
stop:MANAGER_NAME
Where **MANAGER_NAME** is the name of the manager worker supplied in
the initial WORK_DATA packet when the job started. This is used to
direct the stop: function invocation to the correct Gearman worker
that is capable of stopping that particular job. The argument to the
function should be the following encoded in JSON format:
**name**
The job name of the build to stop.
**number**
The build number of the build to stop.
The original job is expected to complete with a WORK_DATA and
WORK_FAIL packet as described in `Starting Builds`_.

29
doc/source/index.rst
View File

@@ -1,31 +1,24 @@
Zuul - A Project Gating System
==============================
Zuul is a program that is used to gate the source code repository of a
project so that changes are only merged if they pass tests.
Zuul is a program that drives continuous integration, delivery, and
deployment systems with a focus on project gating and interrelated
projects.
The main component of Zuul is the scheduler. It receives events
related to proposed changes, triggers tests based on those events, and
reports back.
Zuul's documentation is organized in three guides based on audience.
If Zuul is being used to gate or drive automation around your project,
read the :doc:`user/index` to find out how to configure Zuul. If you
are installing or operating a Zuul system, you will also find the
:doc:`admin/index` useful. If you want help make Zuul itself better,
take a look at the :doc:`developer/index`.
Contents:
.. toctree::
:maxdepth: 2
quick-start
gating
configuration
encryption
connections
triggers
reporters
zuul
merger
cloner
executors
statsd
client
user/index
admin/index
developer/index
Indices and tables

74
doc/source/merger.rst
View File

@@ -1,74 +0,0 @@
:title: Merger
Merger
======
The Zuul Merger is a separate component which communicates with the
main Zuul server via Gearman. Its purpose is to speculatively merge
the changes for Zuul in preparation for testing. The resulting git
commits also must be served to the test workers, and the server(s)
running the Zuul Merger are expected to do this as well. Because both
of these tasks are resource intensive, any number of Zuul Mergers can
be run in parallel on distinct hosts.
Configuration
~~~~~~~~~~~~~
The Zuul Merger can read the same zuul.conf file as the main Zuul
server and requires the ``gearman``, ``gerrit``, ``merger``, and
``zuul`` sections (indicated fields only). Be sure the zuul_url is
set appropriately on each host that runs a zuul-merger.
Zuul References
~~~~~~~~~~~~~~~
As the DependentPipelineManager may combine several changes together
for testing when performing speculative execution, determining exactly
how the workspace should be set up when running a Job can be complex.
To alleviate this problem, Zuul performs merges itself, merging or
cherry-picking changes as required and identifies the result with a
Git reference of the form ``refs/zuul/<branch>/Z<random sha1>``.
Preparing the workspace is then a simple matter of fetching that ref
and checking it out. The parameters that provide this information are
described in :ref:`executors`.
These references need to be made available via a Git repository that
is available to workers (such as Jenkins). This is accomplished by
serving Zuul's Git repositories directly.
Serving Zuul Git Repos
~~~~~~~~~~~~~~~~~~~~~~
Zuul maintains its own copies of any needed Git repositories in the
directory specified by ``git_dir`` in the ``merger`` section of
zuul.conf (by default, /var/lib/zuul/git). To directly serve Zuul's
Git repositories in order to provide Zuul refs for workers, you can
configure Apache to do so using the following directives::
SetEnv GIT_PROJECT_ROOT /var/lib/zuul/git
SetEnv GIT_HTTP_EXPORT_ALL
AliasMatch ^/p/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/lib/zuul/git/$1
AliasMatch ^/p/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/lib/zuul/git/$1
ScriptAlias /p/ /usr/lib/git-core/git-http-backend/
Note that Zuul's Git repositories are not bare, which means they have
a working tree, and are not suitable for public consumption (for
instance, a clone will produce a repository in an unpredictable state
depending on what the state of Zuul's repository is when the clone
happens). They are, however, suitable for automated systems that
respond to Zuul triggers.
Clearing old references
~~~~~~~~~~~~~~~~~~~~~~~
The references created under refs/zuul are not garbage collected. Since
git fetch send them all to Gerrit to sync the repositories, the time
spent on merge will slightly grow overtime and start being noticeable.
To clean them you can use the ``tools/zuul-clear-refs.py`` script on
each repositories. It will delete Zuul references that point to commits
for which the commit date is older than a given amount of days (default
360)::
./tools/zuul-clear-refs.py /path/to/zuul/git/repo

162
doc/source/quick-start.rst
View File

@@ -1,162 +0,0 @@
Quick Start Guide
=================
System Requirements
-------------------
For most deployments zuul only needs 1-2GB. OpenStack uses a 30GB setup.
Install Zuul
------------
You can get zuul from pypi via::
pip install zuul
Zuul Components
---------------
Zuul provides the following components:
- **zuul-server**: scheduler daemon which communicates with Gerrit and
Gearman. Handles receiving events, executing jobs, collecting results
and postingreports.
- **zuul-merger**: speculative-merger which communicates with Gearman.
Prepares Git repositories for jobs to test against. This additionally
requires a web server hosting the Git repositories which can be cloned
by the jobs.
- **zuul-cloner**: client side script used to setup job workspace. It is
used to clone the repositories prepared by the zuul-merger described
previously.
- **gearmand**: optional builtin gearman daemon provided by zuul-server
External components:
- Jenkins Gearman plugin: Used by Jenkins to connect to Gearman
Zuul Communication
------------------
All the Zuul components communicate with each other using Gearman. As well as
the following communication channels:
zuul-server:
- Gerrit
- Gearman Daemon
zuul-merger:
- Gerrit
- Gearman Daemon
zuul-cloner:
- http hosted zuul-merger git repos
Jenkins:
- Gearman Daemon via Jenkins Gearman Plugin
Zuul Setup
----------
At minimum we need to provide **zuul.conf** and **layout.yaml** and placed
in /etc/zuul/ directory. You will also need a zuul user and ssh key for the
zuul user in Gerrit. The following example uses the builtin gearmand service
in zuul.
**zuul.conf**::
[zuul]
layout_config=/etc/zuul/layout.yaml
[merger]
git_dir=/git
zuul_url=http://zuul.example.com/p
[gearman_server]
start=true
[gearman]
server=127.0.0.1
[connection gerrit]
driver=gerrit
server=git.example.com
port=29418
baseurl=https://git.example.com/gerrit/
user=zuul
sshkey=/home/zuul/.ssh/id_rsa
See :doc:`zuul` for more details.
The following sets up a basic timer triggered job using zuul.
**layout.yaml**::
pipelines:
- name: periodic
source: gerrit
manager: IndependentPipelineManager
trigger:
timer:
- time: '0 * * * *'
projects:
- name: aproject
periodic:
- aproject-periodic-build
Starting Zuul
-------------
You can run zuul-server with the **-d** option to make it not daemonize. It's
a good idea at first to confirm there's no issues with your configuration.
Simply run::
zuul-server
Once run you should have 2 zuul-server processes::
zuul 12102 1 0 Jan21 ? 00:15:45 /home/zuul/zuulvenv/bin/python /home/zuul/zuulvenv/bin/zuul-server -d
zuul 12107 12102 0 Jan21 ? 00:00:01 /home/zuul/zuulvenv/bin/python /home/zuul/zuulvenv/bin/zuul-server -d
Note: In this example zuul was installed in a virtualenv.
The 2nd zuul-server process is gearmand running if you are using the builtin
gearmand server, otherwise there will only be 1 process.
Zuul won't actually process your Job queue however unless you also have a
zuul-merger process running.
Simply run::
zuul-merger
Zuul should now be able to process your periodic job as configured above once
the Jenkins side of things is configured.
Jenkins Setup
-------------
Install the Jenkins Gearman Plugin via Jenkins Plugin management interface.
Then naviage to **Manage > Configuration > Gearman** and setup the Jenkins
server hostname/ip and port to connect to gearman.
At this point gearman should be running your Jenkins jobs.
Troubleshooting
---------------
Checking Gearman function registration (jobs). You can use telnet to connect
to gearman to check that Jenkins is registering your configured jobs in
gearman::
telnet <gearman_ip> 4730
Useful commands are **workers** and **status** which you can run by just
typing those commands once connected to gearman. Every job in your Jenkins
master must appear when you run **workers** for Zuul to be able to run jobs
against your Jenkins instance.

143
doc/source/reporters.rst
View File

@@ -1,143 +0,0 @@
:title: Reporters
Reporters
=========
Zuul can communicate results and progress back to configurable
protocols. For example, after succeeding in a build a pipeline can be
configured to post a positive review back to Gerrit.
There are three stages when a report can be handled. That is on:
Start, Success or Failure. Each stage can have multiple reports.
For example, you can set verified on Gerrit and send an email.
Gerrit
------
Zuul works with standard versions of Gerrit by invoking the
``gerrit`` command over an SSH connection. It reports back to
Gerrit using SSH.
The dictionary passed to the Gerrit reporter is used for ``gerrit
review`` arguments, with the boolean value of ``true`` simply
indicating that the argument should be present without following it
with a value. For example, ``verified: 1`` becomes ``gerrit review
--verified 1`` and ``submit: true`` becomes ``gerrit review
--submit``.
A :ref:`connection` that uses the gerrit driver must be supplied to the
trigger.
GitHub
------
Zuul reports back to GitHub via GitHub API. Available reports include a PR
comment containing the build results, a commit status on start, success and
failure, an issue label addition/removal on the PR, and a merge of the PR
itself. Status name, description, and context is taken from the pipeline.
A :ref:`connection` that uses the github driver must be supplied to the
reporter. It has the following options:
**status**
String value (``pending``, ``success``, ``failure``) that the reporter should
set as the commit status on github.
``status: 'success'``
**status-url**
String value for a link url to set in the github status. Defaults to the zuul
server status_url, or the empty string if that is unset.
**comment**
Boolean value (``true`` or ``false``) that determines if the reporter should
add a comment to the pipeline status to the github pull request. Defaults
to ``true``. Only used for Pull Request based events.
``comment: false``
**merge**
Boolean value (``true`` or ``false``) that determines if the reporter should
merge the pull reqeust. Defaults to ``false``. Only used for Pull Request based
events.
``merge=true``
**label**
List of strings each representing an exact label name which should be added
to the pull request by reporter. Only used for Pull Request based events.
``label: 'test successful'``
**unlabel**
List of strings each representing an exact label name which should be removed
from the pull request by reporter. Only used for Pull Request based events.
``unlabel: 'test failed'``
SMTP
----
A simple email reporter is also available.
A :ref:`connection` that uses the smtp driver must be supplied to the
reporter.
SMTP Configuration
~~~~~~~~~~~~~~~~~~
zuul.conf contains the SMTP server and default to/from as described
in :ref:`zuulconf`.
Each pipeline can overwrite the ``subject`` or the ``to`` or ``from`` address by
providing alternatives as arguments to the reporter. For example, ::
pipelines:
- name: post-merge
manager: IndependentPipelineManager
source: my_gerrit
trigger:
my_gerrit:
- event: change-merged
success:
outgoing_smtp:
to: you@example.com
failure:
internal_smtp:
to: you@example.com
from: alternative@example.com
subject: Change {change} failed
SQL
---
This reporter is used to store results in a database.
A :ref:`connection` that uses the sql driver must be supplied to the
reporter.
SQL Configuration
~~~~~~~~~~~~~~~~~
zuul.conf contains the database connection and credentials. To store different
reports in different databases you'll need to create a new connection per
database.
The sql reporter is used to store the results from individual builds rather
than the change. As such the sql reporter does nothing on "start" or
"merge-failure".
**score**
A score to store for the result of the build. eg: -1 might indicate a failed
build similar to the vote posted back via the gerrit reporter.
For example ::
pipelines:
- name: post-merge
manager: IndependentPipelineManager
source: my_gerrit
trigger:
my_gerrit:
- event: change-merged
success:
mydb_conn:
score: 1
failure:
mydb_conn:
score: -1

247
doc/source/triggers.rst
View File

@@ -1,247 +0,0 @@
:title: Triggers
Triggers
========
The process of merging a change starts with proposing a change to be
merged. Zuul supports Gerrit and GitHub as triggering systems.
Zuul's design is modular, so alternate triggering and reporting
systems can be supported.
Gerrit
------
Zuul works with standard versions of Gerrit by invoking the ``gerrit
stream-events`` command over an SSH connection. It also reports back
to Gerrit using SSH.
If using Gerrit 2.7 or later, make sure the user is a member of a group
that is granted the ``Stream Events`` permission, otherwise it will not
be able to invoke the ``gerrit stream-events`` command over SSH.
A connection name with the gerrit driver can take multiple events with
the following options.
**event**
The event name from gerrit. Examples: ``patchset-created``,
``comment-added``, ``ref-updated``. This field is treated as a
regular expression.
**branch**
The branch associated with the event. Example: ``master``. This
field is treated as a regular expression, and multiple branches may
be listed.
**ref**
On ref-updated events, the branch parameter is not used, instead the
ref is provided. Currently Gerrit has the somewhat idiosyncratic
behavior of specifying bare refs for branch names (e.g., ``master``),
but full ref names for other kinds of refs (e.g., ``refs/tags/foo``).
Zuul matches what you put here exactly against what Gerrit
provides. This field is treated as a regular expression, and
multiple refs may be listed.
**ignore-deletes**
When a branch is deleted, a ref-updated event is emitted with a newrev
of all zeros specified. The ``ignore-deletes`` field is a boolean value
that describes whether or not these newrevs trigger ref-updated events.
The default is True, which will not trigger ref-updated events.
**approval**
This is only used for ``comment-added`` events. It only matches if
the event has a matching approval associated with it. Example:
``code-review: 2`` matches a ``+2`` vote on the code review category.
Multiple approvals may be listed.
**email**
This is used for any event. It takes a regex applied on the performer
email, i.e. Gerrit account email address. If you want to specify
several email filters, you must use a YAML list. Make sure to use non
greedy matchers and to escapes dots!
Example: ``email: ^.*?@example\.org$``.
**email_filter** (deprecated)
A deprecated alternate spelling of *email*. Only one of *email* or
*email_filter* should be used.
**username**
This is used for any event. It takes a regex applied on the performer
username, i.e. Gerrit account name. If you want to specify several
username filters, you must use a YAML list. Make sure to use non greedy
matchers and to escapes dots!
Example: ``username: ^jenkins$``.
**username_filter** (deprecated)
A deprecated alternate spelling of *username*. Only one of *username* or
*username_filter* should be used.
**comment**
This is only used for ``comment-added`` events. It accepts a list of
regexes that are searched for in the comment string. If any of these
regexes matches a portion of the comment string the trigger is
matched. ``comment: retrigger`` will match when comments
containing 'retrigger' somewhere in the comment text are added to a
change.
**comment_filter** (deprecated)
A deprecated alternate spelling of *comment*. Only one of *comment* or
*comment_filter* should be used.
*require-approval*
This may be used for any event. It requires that a certain kind
of approval be present for the current patchset of the change (the
approval could be added by the event in question). It follows the
same syntax as the :ref:`"approval" pipeline requirement
<pipeline-require-approval>`. For each specified criteria there must
exist a matching approval.
*reject-approval*
This takes a list of approvals in the same format as
*require-approval* but will fail to enter the pipeline if there is
a matching approval.
GitHub
------
Github webhook events can be configured as triggers.
A connection name with the github driver can take multiple events with the
following options.
**event**
The event from github. Supported events are ``pull_request``,
``pull_request_review``, and ``push``.
A ``pull_request`` event will
have associated action(s) to trigger from. The supported actions are:
*opened* - pull request opened
*changed* - pull request synchronized
*closed* - pull request closed
*reopened* - pull request reopened
*comment* - comment added on pull request
*labeled* - label added on pull request
*unlabeled* - label removed from pull request
*review* - review added on pull request
*push* - head reference updated (pushed to branch)
*status* - status set on commit
A ``pull_request_review`` event will
have associated action(s) to trigger from. The supported actions are:
*submitted* - pull request review added
*dismissed* - pull request review removed
**branch**
The branch associated with the event. Example: ``master``. This
field is treated as a regular expression, and multiple branches may
be listed. Used for ``pull_request`` and ``pull_request_review`` events.
**comment**
This is only used for ``pull_request`` ``comment`` actions. It accepts a
list of regexes that are searched for in the comment string. If any of these
regexes matches a portion of the comment string the trigger is matched.
``comment: retrigger`` will match when comments containing 'retrigger'
somewhere in the comment text are added to a pull request.
**label**
This is only used for ``labeled`` and ``unlabeled`` ``pull_request`` actions.
It accepts a list of strings each of which matches the label name in the
event literally. ``label: recheck`` will match a ``labeled`` action when
pull request is labeled with a ``recheck`` label. ``label: 'do not test'``
will match a ``unlabeled`` action when a label with name ``do not test`` is
removed from the pull request.
**state**
This is only used for ``pull_request_review`` events. It accepts a list of
strings each of which is matched to the review state, which can be one of
``approved``, ``comment``, or ``request_changes``.
**status**
This is only used for ``status`` actions. It accepts a list of strings each of
which matches the user setting the status, the status context, and the status
itself in the format of ``user:context:status``. For example,
``zuul_github_ci_bot:check_pipeline:success``.
**ref**
This is only used for ``push`` events. This field is treated as a regular
expression and multiple refs may be listed. Github always sends full ref
name, eg. ``refs/tags/bar`` and this string is matched against the regexp.
GitHub Configuration
~~~~~~~~~~~~~~~~~~~~
Configure GitHub `webhook events
<https://developer.github.com/webhooks/creating/>`_.
Set *Payload URL* to
``http://<zuul-hostname>/connection/<connection-name>/payload``.
Set *Content Type* to ``application/json``.
Select *Events* you are interested in. See above for the supported events.
Timer
-----
A simple timer trigger is available as well. It supports triggering
jobs in a pipeline based on cron-style time instructions.
Timers don't require a special connection or driver. Instead they can
be used by listing **timer** as the trigger.
This trigger will run based on a cron-style time specification.
It will enqueue an event into its pipeline for every project
defined in the configuration. Any job associated with the
pipeline will run in response to that event.
**time**
The time specification in cron syntax. Only the 5 part syntax is
supported, not the symbolic names. Example: ``0 0 * * *`` runs
at midnight.
Zuul
----
The Zuul trigger generates events based on internal actions in Zuul.
Multiple events may be listed.
Zuul events don't require a special connection or driver. Instead they
can be used by listing **zuul** as the trigger.
**event**
The event name. Currently supported:
*project-change-merged* when Zuul merges a change to a project,
it generates this event for every open change in the project.
*parent-change-enqueued* when Zuul enqueues a change into any
pipeline, it generates this event for every child of that
change.
**pipeline**
Only available for ``parent-change-enqueued`` events. This is the
name of the pipeline in which the parent change was enqueued.
*require-approval*
This may be used for any event. It requires that a certain kind
of approval be present for the current patchset of the change (the
approval could be added by the event in question). It follows the
same syntax as the :ref:`"approval" pipeline requirement
<pipeline-require-approval>`. For each specified criteria there must
exist a matching approval.
*reject-approval*
This takes a list of approvals in the same format as
*require-approval* but will fail to enter the pipeline if there is
a matching approval.

85
doc/source/user/concepts.rst Normal file
View File

@@ -0,0 +1,85 @@
:title: Zuul Concepts
Zuul Concepts
=============
Zuul's configuration is organized around the concept of a *pipeline*.
In Zuul, a pipeline encompasses a workflow process which can be
applied to one or more projects. For instance, a "check" pipeline
might describe the actions which should cause newly proposed changes
to projects to be tested. A "gate" pipeline might implement
:ref:`project_gating` to automate merging changes to projects only if
their tests pass. A "post" pipeline might update published
documentation for a project when changes land.
The names "check", "gate", and "post" are arbitrary -- these are not
concepts built into Zuul, but rather they are just a few common
examples of workflow processes that can be described to Zuul and
implemented as pipelines.
Once a pipeline has been defined, any number of projects may be
associated with it, each one specifying what jobs should be run for
that project in a given pipeline.
Pipelines have associated *triggers* which are descriptions of events
which should cause something to be enqueued into a pipeline. For
example, when a patchset is uploaded to Gerrit, a Gerrit
*patchset-created* event is emitted. A pipeline configured to trigger
on *patchset-created* events would then enqueue the associated change
when Zuul receives that event. If there are jobs associated with that
project and pipeline, they will be run. In addition to Gerrit, other
triggers are available, including GitHub, timer, and zuul. See
:ref:`drivers` for a full list of available triggers.
Once all of the jobs for an item in a pipeline have been run, the
pipeline's *reporters* are responsible for reporting the results of
all of the jobs. Continuing the example from earlier, if a pipeline
were configured with a Gerrit reporter, it would leave a review
comment on the change and set any approval votes that are configured.
There are several reporting phases available; each of which may be
configured with any number of reporters. See :ref:`drivers` for a
full list of available reporters.
The items enqueued into a pipeline are each associated with a git ref.
That ref may point to a proposed change, or it may be the tip of a
branch or a tag. The triggering event determines the ref, and whether
it represents a proposed or merged commit. Zuul prepares the ref for
an item before running jobs. In the case of a proposed change, that
means speculatively merging the change into its target branch. This
means that any jobs that operate on the change will run with the git
repo in the state it will be in after the change merges (which may be
substantially different than the git repo state of the change itself
since the repo may have merged other changes since the change was
originally authored). Items in a pipeline may depend on other items,
and if they do, all of their dependent changes will be included in the
git repo state that Zuul prepares. For more detail on this process,
see :ref:`project_gating` and :ref:`dependencies`.
The configuration for nearly everything described above is held in
files inside of the git repos upon which Zuul operates. Zuul's
configuration is global, but distributed. Jobs which are defined in
one project might be used in another project while pipelines are
available to all projects. When Zuul starts, it reads its
configuration from all of the projects it knows about, and when
changes to its configuration are proposed, those changes may take
effect temporarily as part of the proposed change, or immediately
after the change merges, depending on the type of project in which the
change appears.
Jobs specify the type and quantity of nodes which they require.
Before executing each job, Zuul will contact it's companion program,
Nodepool, to supply them. Nodepool may be configured to supply static
nodes or contact cloud providers to create or delete nodes as
necessary. The types of nodes available to Zuul are determined by the
Nodepool administrator.
The executable contents of jobs themselves are Ansible playbooks.
Ansible's support for orchestrating tasks on remote nodes is
particularly suited to Zuul's support for multi-node testing. Ansible
is also easy to use for simple tasks (such as executing a shell
script) or sophisticated deployment scenarios. When Zuul runs
Ansible, it attempts to do so in a manner most similar to the way that
Ansible might be used to orchestrate remote systems. Ansible itself
is run on the executor and acts remotely upon the test nodes supplied
to a job. This facilitates continuous delivery by making it possible
to use the same Ansible playbooks in testing and production.

35
doc/source/project-config.rst → doc/source/user/config.rst
View File

@@ -33,7 +33,7 @@ Configuration Loading
---------------------
When Zuul starts, it examines all of the git repositories which are
specified by the system administrator in :ref:`admin-config` and searches
specified by the system administrator in :ref:`tenant-config` and searches
for files in the root of each repository. In the case of a
*config-project*, Zuul looks for a file named `zuul.yaml`. In the
case of an *untrusted-project*, Zuul looks first for `zuul.yaml` and
@@ -111,7 +111,7 @@ success, the pipeline reports back to Gerrit with a *Verified* vote of
my_gerrit
verified: -1
See TODO for more annotated examples of common pipeline configurations.
.. TODO: See TODO for more annotated examples of common pipeline configurations.
The attributes available on a pipeline are as follows (all are
optional unless otherwise specified):
@@ -123,6 +123,8 @@ optional unless otherwise specified):
**manager** (required)
There are currently two schemes for managing pipelines:
.. _independent_pipeline_manager:
*independent*
Every event in this pipeline should be treated as independent of
other events in the pipeline. This is appropriate when the order of
@@ -138,6 +140,8 @@ optional unless otherwise specified):
pipeline. In that case, the changes have already merged, so the
results can not affect any other events in the pipeline.
.. _dependent_pipeline_manager:
*dependent*
The dependent pipeline manager is designed for gating. It ensures
that every change is tested exactly as it is going to be merged
@@ -190,7 +194,7 @@ optional unless otherwise specified):
Triggers are loaded from their connection name. The driver type of
the connection will dictate which options are available.
See :doc:`triggers`.
See :ref:`drivers`.
**require**
If this section is present, it established pre-requisites for any
@@ -199,7 +203,7 @@ optional unless otherwise specified):
the conditions specified here must be met or the item will not be
enqueued.
.. TODO this section is in flux in v3 _pipeline-require-approval:
.. _pipeline-require-approval:
**approval**
This requires that a certain kind of approval be present for the
@@ -281,6 +285,13 @@ optional unless otherwise specified):
this to ``true``. This option is ignored by dependent pipelines.
The default is: ``false``.
**precedence**
Indicates how the build scheduler should prioritize jobs for
different pipelines. Each pipeline may have one precedence, jobs
for pipelines with a higher precedence will be run before ones with
lower. The value should be one of ``high``, ``normal``, or ``low``.
Default: ``normal``.
The following options configure *reporters*. Reporters are
complementary to triggers; where a trigger is an event on a connection
which causes Zuul to enqueue an item, a reporter is the action
@@ -315,6 +326,11 @@ which implements it. See :ref:`drivers` for more information.
These reporters describe what Zuul should do when a pipeline is
disabled. See ``disable-after-consecutive-failures``.
The following options can be used to alter Zuul's behavior to mitigate
situations in which jobs are failing frequently (perhaps due to a
problem with an external dependency, or unusually high
non-deterministic test failures).
**disable-after-consecutive-failures**
If set, a pipeline can enter a ''disabled'' state if too many changes
in a row fail. When this value is exceeded the pipeline will stop
@@ -322,13 +338,6 @@ which implements it. See :ref:`drivers` for more information.
reporters and instead only report to the ``disabled`` reporters.
(No ``start`` reports are made when a pipeline is disabled).
**precedence**
Indicates how the build scheduler should prioritize jobs for
different pipelines. Each pipeline may have one precedence, jobs
for pipelines with a higher precedence will be run before ones with
lower. The value should be one of ``high``, ``normal``, or ``low``.
Default: ``normal``.
**window**
Dependent pipeline managers only. Zuul can rate limit dependent
pipelines in a manner similar to TCP flow control. Jobs are only
@@ -515,7 +524,7 @@ unless otherwise specified:
variants used in constructing the frozen job, with no duplication.
Default: none.
** branches **
**branches**
A regular expression (or list of regular expressions) which describe
on what branches a job should run (or in the case of variants: to
alter the behavior of a job for a certain branch).
@@ -691,6 +700,8 @@ unless otherwise specified:
be installed (and therefore referenced from Ansible), the `name`
attribute may be used to specify an alternate.
.. note:: galaxy roles are not yet implemented
**galaxy**
The name of the role in Ansible Galaxy. If this attribute is
supplied, Zuul will search Ansible Galaxy for a role by this name

46
doc/source/user/encryption.rst Normal file
View File

@@ -0,0 +1,46 @@
:title: Encryption
.. _encryption:
Encryption
==========
Zuul supports storing encrypted data directly in the git repositories
of projects it operates on. If you have a job which requires private
information in order to run (e.g., credentials to interact with a
third-party service) those credentials can be stored along with the
job definition.
Each project in Zuul has its own automatically generated RSA keypair
which can be used by anyone to encrypt a secret and only Zuul is able
to decrypt it. Zuul serves each project's public key using its
build-in webserver. They can be fetched at the path
``/keys/<source>/<project>.pub`` where ``<project>`` is the name of a
project and ``<source>`` is the name of that project's connection in
the main Zuul configuration file.
Zuul currently supports one encryption scheme, PKCS#1 with OAEP, which
can not store secrets longer than the key length, 4096 bits. The
padding used by this scheme ensures that someone examining the
encrypted data can not determine the length of the plaintext version
of the data, except to know that it is not longer than 4096 bits.
In the config files themselves, Zuul uses an extensible method of
specifying the encryption scheme used for a secret so that other
schemes may be added later. To specify a secret, use the
``!encrypted/pkcs1-oaep`` YAML tag along with the base64 encoded
value. For example::
- secret:
name: test_secret
data:
password: !encrypted/pkcs1-oaep |
BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71YUsi1wGZZ
...
Zuul provides a standalone script to make encrypting values easy; it
can be found at `tools/encrypt_secret.py` in the Zuul source
directory.
.. program-output:: python3 ../../tools/encrypt_secret.py --help

175
doc/source/gating.rst → doc/source/user/gating.rst
View File

@@ -1,16 +1,18 @@
:title: Project Gating
.. _project_gating:
Project Gating
==============
Traditionally, many software development projects merge changes from
developers into the repository, and then identify regressions
resulting from those changes (perhaps by running a test suite with a
continuous integration system such as Jenkins), followed by more
patches to fix those bugs. When the mainline of development is
broken, it can be very frustrating for developers and can cause lost
productivity, particularly so when the number of contributors or
contributions is large.
continuous integration system), followed by more patches to fix those
bugs. When the mainline of development is broken, it can be very
frustrating for developers and can cause lost productivity,
particularly so when the number of contributors or contributions is
large.
The process of gating attempts to prevent changes that introduce
regressions from being merged. This keeps the mainline of development
@@ -25,9 +27,6 @@ is not the best use of a developer's time. Zuul can help automate
this process, with a particular emphasis on ensuring large numbers of
changes are tested correctly.
Zuul was designed to handle the workflow of the OpenStack project, but
can be used with any project.
Testing in parallel
-------------------
@@ -42,18 +41,17 @@ projects, it may take hours to test changes, and it is easy for
developers to create changes at a rate faster than they can be tested
and merged.
Zuul's DependentPipelineManager allows for parallel execution of test
jobs for gating while ensuring changes are tested correctly, exactly
as if they had been tested one at a time. It does this by performing
speculative execution of test jobs; it assumes that all jobs will
succeed and tests them in parallel accordingly. If they do succeed,
they can all be merged. However, if one fails, then changes that were
expecting it to succeed are re-tested without the failed change. In
the best case, as many changes as execution contexts are available may
be tested in parallel and merged at once. In the worst case, changes
are tested one at a time (as each subsequent change fails, changes
behind it start again). In practice, the OpenStack project observes
something closer to the best case.
Zuul's :ref:`dependent pipeline manager<dependent_pipeline_manager>`
allows for parallel execution of test jobs for gating while ensuring
changes are tested correctly, exactly as if they had been tested one
at a time. It does this by performing speculative execution of test
jobs; it assumes that all jobs will succeed and tests them in parallel
accordingly. If they do succeed, they can all be merged. However, if
one fails, then changes that were expecting it to succeed are
re-tested without the failed change. In the best case, as many
changes as execution contexts are available may be tested in parallel
and merged at once. In the worst case, changes are tested one at a
time (as each subsequent change fails, changes behind it start again).
For example, if a core developer approves five changes in rapid
succession::
@@ -220,80 +218,34 @@ changes entering the gate are going to be tested with the version of
other projects currently enqueued in the gate (since they will
eventually be merged and might introduce breaking features).
Such relationships can be defined in Zuul configuration by registering
a job in a DependentPipeline of several projects. Whenever a change
enters such a pipeline, it will create references for the other
projects as well. As an example, given a main project ``acme`` and a
plugin ``plugin`` you can define a job ``acme-tests`` which should be
run for both projects:
Such relationships can be defined in Zuul configuration by placing
projects in a shared queue within a dependent pipeline. Whenever
changes for any project enter a pipeline with such a shared queue,
they are tested together, such that the commits for the changes ahead
in the queue are automatically present in the jobs for the changes
behind them. See :ref:`project` for more details.
.. code-block:: yaml
A given dependent pipeline may have as many shared change queues as
necessary, so groups of related projects may share a change queue
without interfering with unrelated projects. Independent pipelines do
not use shared change queues, however, they may still be used to test
changes across projects using cross-project dependencies.
pipelines:
- name: gate
manager: DependentPipelineManager
.. _dependencies:
projects::
- name: acme
gate:
- acme-tests
- name: plugin
gate:
- acme-tests # Register job again
Cross-Project Dependencies
--------------------------
Whenever a change enters the ``gate`` pipeline queue, Zuul creates a reference
for it. For each subsequent change, an additional reference is created for the
changes ahead in the queue. As a result, you will always be able to fetch the
future state of your project dependencies for each change in the queue.
Zuul permits users to specify dependencies across projects. Using a
special footer in Git commit messages, users may specify that a change
depends on another change in any repository known to Zuul.
Based on the pipeline and project definitions above, three changes are
inserted in the ``gate`` pipeline with the associated references:
Zuul's cross-project dependencies behave like a directed acyclic graph
(DAG), like git itself, to indicate a one-way dependency relationship
between changes in different git repositories. Change A may depend on
B, but B may not depend on A.
======== ======= ====== =========
Change Project Branch Zuul Ref.
======== ======= ====== =========
Change 1 acme master master/Z1
Change 2 plugin stable stable/Z2
Change 3 plugin master master/Z3
======== ======= ====== =========
Since the changes enter a DependentPipelineManager pipeline, Zuul creates
additional references:
====== ======= ========= =============================
Change Project Zuul Ref. Description
====== ======= ========= =============================
1 acme master/Z1 acme master + change 1
------ ------- --------- -----------------------------
2 acme master/Z2 acme master + change 1
2 plugin stable/Z2 plugin stable + change 2
------ ------- --------- -----------------------------
3 acme master/Z3 acme master + change 1
3 plugin stable/Z3 plugin stable + change 2
3 plugin master/Z3 plugin master + change 3
====== ======= ========= =============================
In order to test change 3, you would clone both repositories and simply
fetch the Z3 reference for each combination of project/branch you are
interested in testing. For example, you could fetch ``acme`` with
master/Z3 and ``plugin`` with master/Z3 and thus have ``acme`` with
change 1 applied as the expected state for when Change 3 would merge.
When your job fetches several repositories without changes ahead in the
queue, they may not have a Z reference in which case you can just check
out the branch.
Cross Repository Dependencies
-----------------------------
Zuul permits users to specify dependencies across repositories. Using
a special header in Git commit messages, Users may specify that a
change depends on another change in any repository known to Zuul.
Zuul's cross-repository dependencies (CRD) behave like a directed
acyclic graph (DAG), like git itself, to indicate a one-way dependency
relationship between changes in different git repositories. Change A
may depend on B, but B may not depend on A.
.. TODO: update for v3 crd syntax
To use them, include ``Depends-On: <gerrit-change-id>`` in the footer of
a commit message. Use the full Change-ID ('I' + 40 characters).
@@ -302,10 +254,10 @@ a commit message. Use the full Change-ID ('I' + 40 characters).
Dependent Pipeline
~~~~~~~~~~~~~~~~~~
When Zuul sees CRD changes, it serializes them in the usual manner when
enqueuing them into a pipeline. This means that if change A depends on
B, then when they are added to a dependent pipeline, B will appear first
and A will follow:
When Zuul sees changes with cross-project dependencies, it serializes
them in the usual manner when enqueuing them into a pipeline. This
means that if change A depends on B, then when they are added to a
dependent pipeline, B will appear first and A will follow:
.. blockdiag::
:align: center
@@ -333,25 +285,26 @@ it will not be possible for A to merge until B does.
.. note::
If changes with CRD do not share a change queue then Zuul is unable
to enqueue them together, and the first will be required to merge
before the second is enqueued.
If changes with cross-project dependencies do not share a change
queue then Zuul is unable to enqueue them together, and the first
will be required to merge before the second is enqueued.
Independent Pipeline
~~~~~~~~~~~~~~~~~~~~
When changes are enqueued into an independent pipeline, all of the
related dependencies (both normal git-dependencies that come from parent
commits as well as CRD changes) appear in a dependency graph, as in a
dependent pipeline. This means that even in an independent pipeline,
your change will be tested with its dependencies. So changes that were
previously unable to be fully tested until a related change landed in a
different repository may now be tested together from the start.
related dependencies (both normal git-dependencies that come from
parent commits as well as cross-project dependencies) appear in a
dependency graph, as in a dependent pipeline. This means that even in
an independent pipeline, your change will be tested with its
dependencies. Changes that were previously unable to be fully tested
until a related change landed in a different repository may now be
tested together from the start.
All of the changes are still independent (so you will note that the
whole pipeline does not share a graph as in a dependent pipeline), but
for each change tested, all of its dependencies are visually connected
to it, and they are used to construct the git references that Zuul uses
All of the changes are still independent (you will note that the whole
pipeline does not share a graph as in a dependent pipeline), but for
each change tested, all of its dependencies are visually connected to
it, and they are used to construct the git repositories that Zuul uses
when testing.
When looking at this graph on the status page, you will note that the
@@ -383,6 +336,8 @@ will show up as a grey dot when used as a dependency, but separately and
additionally will appear as its own red or green dot for its test.
.. TODO: relevant for v3?
Multiple Changes
~~~~~~~~~~~~~~~~
@@ -462,10 +417,12 @@ the commit message footer.
B, C <- A
}
.. TODO: update for v3
Cycles
~~~~~~
If a cycle is created by use of CRD, Zuul will abort its work very
early. There will be no message in Gerrit and no changes that are part
of the cycle will be enqueued into any pipeline. This is to protect
Zuul from infinite loops.
If a cycle is created by use of cross-project dependencies, Zuul will
abort its work very early. There will be no message in Gerrit and no
changes that are part of the cycle will be enqueued into any pipeline.
This is to protect Zuul from infinite loops.

18
doc/source/user/index.rst Normal file
View File

@@ -0,0 +1,18 @@
User's Guide
============
This guide is for all users of Zuul. If you work on a project where
Zuul is used to drive automation (whether that's testing proposed
changes, building artifacts, or deploying builds), this guide will
help you understand the concepts that underly Zuul, and how to
configure it to meet your needs.
.. toctree::
:maxdepth: 2
concepts
gating
config
jobs
encryption

90
doc/source/user/jobs.rst Normal file
View File

@@ -0,0 +1,90 @@
:title: Job Content
Job Content
===========
Zuul jobs are implemneted as Ansible playbooks. Zuul prepares the
repositories used for a job, installs any required Ansible roles, and
then executes the job's playbooks. Any setup or artifact collection
required is the responsibility of the job itself. While this flexible
arrangement allows for almost any kind of job to be run by Zuul,
batteries are included. Zuul has a standard library of jobs upon
which to build.
Working Directory
-----------------
Before starting each job, the Zuul executor creates a directory to
hold all of the content related to the job. This includes some
directories which are used by Zuul to configure and run Ansible and
may not be accessible, as well as a directory tree, under ``work/``,
that is readable and writable by the job. The hierarchy is:
**work/**
The working directory of the job.
**work/src/**
Contains the prepared git repositories for the job.
**work/logs/**
Where the Ansible log for the job is written; your job
may place other logs here as well.
Git Repositories
----------------
The git repositories in ``work/src`` contain the repositories for all
of the projects specified in the ``required-projects`` section of the
job, plus the project associated with the queue item if it isn't
already in that list. In the case of a proposed change, that change
and all of the changes ahead of it in the pipeline queue will already
be merged into their respective repositories and target branches. The
change's project will have the change's branch checked out, as will
all of the other projects, if that branch exists (otherwise, a
fallback or default branch will be used). If your job needs to
operate on multiple branches, simply checkout the appropriate branches
of these git repos to ensure that the job results reflect the proposed
future state that Zuul is testing, and all dependencies are present.
Do not use any git remotes; the local repositories are guaranteed to
be up to date.
Note that these git repositories are located on the executor; in order
to be useful to most kinds of jobs, they will need to be present on
the test nodes. The ``base`` job in the standard library contains a
pre-playbook which copies the repositories to all of the job's nodes.
It is recommended to always inherit from this base job to ensure that
behavior.
.. TODO: link to base job documentation and/or document src (and logs?) directory
Zuul Variables
--------------
Zuul supplies not only the variables specified by the job definition
to Ansible, but also some variables from the executor itself. They
are:
**zuul.executor.hostname**
The hostname of the executor.
**zuul.executor.src_root**
The path to the source directory.
**zuul.executor.log_root**
The path to the logs directory.
SSH Keys
--------
Zuul starts each job with an SSH agent running and the key used to
access the job's nodes added to that agent. Generally you won't need
to be aware of this since Ansible will use this when performing any
tasks on remote nodes. However, under some circumstances you may want
to interact with the agent. For example, you may wish to add a key
provided as a secret to the job in order to access a specific host, or
you may want to, in a pre-playbook, replace the key used to log into
the assigned nodes in order to further protect it from being abused by
untrusted job content.
.. TODO: describe standard lib and link to published docs for it.

967
doc/source/zuul.rst
View File

@@ -1,967 +0,0 @@
:title: Zuul
Zuul
====
Configuration
-------------
Zuul has three configuration files:
**zuul.conf**
Connection information for Gerrit and Gearman, locations of the
other config files. (required)
**layout.yaml**
Project and pipeline configuration -- what Zuul does. (required)
**logging.conf**
Python logging config. (optional)
Examples of each of the three files can be found in the etc/ directory
of the source distribution.
.. _zuulconf:
zuul.conf
~~~~~~~~~
Zuul will look for ``/etc/zuul/zuul.conf`` or ``~/zuul.conf`` to
bootstrap its configuration. Alternately, you may specify ``-c
/path/to/zuul.conf`` on the command line.
Gerrit and Gearman connection information are each described in a
section of zuul.conf. The location of the other two configuration
files (as well as the location of the PID file when running Zuul as a
server) are specified in a third section.
The three sections of this config and their options are documented below.
You can also find an example zuul.conf file in the git
`repository
<https://git.openstack.org/cgit/openstack-infra/zuul/tree/etc/zuul.conf-sample>`_
gearman
"""""""
Client connection information for gearman. If using Zuul's builtin gearmand
server just set **server** to 127.0.0.1.
**server**
Hostname or IP address of the Gearman server.
``server=gearman.example.com`` (required)
**port**
Port on which the Gearman server is listening.
``port=4730`` (optional)
**ssl_ca**
Optional: An openssl file containing a set of concatenated “certification authority” certificates
in PEM formet.
**ssl_cert**
Optional: An openssl file containing the client public certificate in PEM format.
**ssl_key**
Optional: An openssl file containing the client private key in PEM format.
gearman_server
""""""""""""""
The builtin gearman server. Zuul can fork a gearman process from itself rather
than connecting to an external one.
**start**
Whether to start the internal Gearman server (default: False).
``start=true``
**listen_address**
IP address or domain name on which to listen (default: all addresses).
``listen_address=127.0.0.1``
**log_config**
Path to log config file for internal Gearman server.
``log_config=/etc/zuul/gearman-logging.yaml``
**ssl_ca**
Optional: An openssl file containing a set of concatenated “certification authority” certificates
in PEM formet.
**ssl_cert**
Optional: An openssl file containing the server public certificate in PEM format.
**ssl_key**
Optional: An openssl file containing the server private key in PEM format.
webapp
""""""
**listen_address**
IP address or domain name on which to listen (default: 0.0.0.0).
``listen_address=127.0.0.1``
**port**
Port on which the webapp is listening (default: 8001).
``port=8008``
zuul
""""
Zuul's main configuration section. At minimum zuul must be able to find
layout.yaml to be useful.
.. note:: Must be provided when running zuul-server
.. _layout_config:
**layout_config**
Path to layout config file. Used by zuul-server only.
``layout_config=/etc/zuul/layout.yaml``
**log_config**
Path to log config file. Used by zuul-server only.
``log_config=/etc/zuul/logging.yaml``
**pidfile**
Path to PID lock file. Used by zuul-server only.
``pidfile=/var/run/zuul/zuul.pid``
**state_dir**
Path to directory that Zuul should save state to. Used by all Zuul
commands.
``state_dir=/var/lib/zuul``
**jobroot_dir**
Path to directory that Zuul should store temporary job files.
``jobroot_dir=/tmp``
**report_times**
Boolean value (``true`` or ``false``) that determines if Zuul should
include elapsed times for each job in the textual report. Used by
zuul-server only.
``report_times=true``
**status_url**
URL that will be posted in Zuul comments made to Gerrit changes when
starting jobs for a change. Used by zuul-server only.
``status_url=https://zuul.example.com/status``
**status_expiry**
Zuul will cache the status.json file for this many seconds. This is an
optional value and ``1`` is used by default.
``status_expiry=1``
merger
""""""
The zuul-merger process configuration. Detailed documentation on this process
can be found on the :doc:`merger` page.
.. note:: Must be provided when running zuul-merger. Both services may share the
same configuration (and even host) or otherwise have an individual
zuul.conf.
**git_dir**
Directory that Zuul should clone local git repositories to.
``git_dir=/var/lib/zuul/git``
**git_user_email**
Optional: Value to pass to `git config user.email`.
``git_user_email=zuul@example.com``
**git_user_name**
Optional: Value to pass to `git config user.name`.
``git_user_name=zuul``
**zuul_url**
URL of this merger's git repos, accessible to test workers. Usually
"http://zuul.example.com/p" or "http://zuul-merger01.example.com/p"
depending on whether the merger is co-located with the Zuul server.
**log_config**
Path to log config file for the merger process.
``log_config=/etc/zuul/logging.yaml``
**pidfile**
Path to PID lock file for the merger process.
``pidfile=/var/run/zuul-merger/merger.pid``
executor
""""""""
The zuul-executor process configuration.
**finger_port**
Port to use for finger log streamer.
``finger_port=79``
**git_dir**
Directory that Zuul should clone local git repositories to.
``git_dir=/var/lib/zuul/git``
**log_config**
Path to log config file for the executor process.
``log_config=/etc/zuul/logging.yaml``
**private_key_file**
SSH private key file to be used when logging into worker nodes.
``private_key_file=~/.ssh/id_rsa``
**user**
User ID for the zuul-executor process. In normal operation as a daemon,
the executor should be started as the ``root`` user, but it will drop
privileges to this user during startup.
``user=zuul``
.. _connection:
connection ArbitraryName
""""""""""""""""""""""""
A connection can be listed with any arbitrary name. The required
parameters are specified in the :ref:`connections` documentation
depending on what driver you are using.
.. _layoutyaml:
layout.yaml
~~~~~~~~~~~
This is the main configuration file for Zuul, where all of the pipelines
and projects are defined, what tests should be run, and what actions
Zuul should perform. There are three sections: pipelines, jobs, and
projects.
Pipelines
"""""""""
Zuul can have any number of independent pipelines. Whenever a matching
Gerrit event is found for a pipeline, that event is added to the
pipeline, and the jobs specified for that pipeline are run. When all
jobs specified for the pipeline that were triggered by an event are
completed, Zuul reports back to Gerrit the results.
There are no pre-defined pipelines in Zuul, rather you can define
whatever pipelines you need in the layout file. This is a very flexible
system that can accommodate many kinds of workflows.
Here is a quick example of a pipeline definition followed by an
explanation of each of the parameters::
- name: check
manager: IndependentPipelineManager
source: my_gerrit
trigger:
my_gerrit:
- event: patchset-created
success:
my_gerrit:
verified: 1
failure:
my_gerrit
verified: -1
**name**
This is used later in the project definition to indicate what jobs
should be run for events in the pipeline.
**description**
This is an optional field that may be used to provide a textual
description of the pipeline.
**source**
A required field that specifies a connection that provides access to
the change objects that this pipeline operates on. The name of the
connection as per the zuul.conf should be specified. The driver used
for the connection named will be the source. Currently only ``gerrit``
drivers are supported.
**success-message**
An optional field that supplies the introductory text in message
reported back to Gerrit when all the voting builds are successful.
Defaults to "Build successful."
**failure-message**
An optional field that supplies the introductory text in message
reported back to Gerrit when at least one voting build fails.
Defaults to "Build failed."
**merge-failure-message**
An optional field that supplies the introductory text in message
reported back to Gerrit when a change fails to merge with the
current state of the repository.
Defaults to "Merge failed."
**footer-message**
An optional field to supply additional information after test results.
Useful for adding information about the CI system such as debugging
and contact details.
**manager**
There are currently two schemes for managing pipelines:
*IndependentPipelineManager*
Every event in this pipeline should be treated as independent of
other events in the pipeline. This is appropriate when the order of
events in the pipeline doesn't matter because the results of the
actions this pipeline performs can not affect other events in the
pipeline. For example, when a change is first uploaded for review,
you may want to run tests on that change to provide early feedback
to reviewers. At the end of the tests, the change is not going to
be merged, so it is safe to run these tests in parallel without
regard to any other changes in the pipeline. They are independent.
Another type of pipeline that is independent is a post-merge
pipeline. In that case, the changes have already merged, so the
results can not affect any other events in the pipeline.
*DependentPipelineManager*
The dependent pipeline manager is designed for gating. It ensures
that every change is tested exactly as it is going to be merged
into the repository. An ideal gating system would test one change
at a time, applied to the tip of the repository, and only if that
change passed tests would it be merged. Then the next change in
line would be tested the same way. In order to achieve parallel
testing of changes, the dependent pipeline manager performs
speculative execution on changes. It orders changes based on
their entry into the pipeline. It begins testing all changes in
parallel, assuming that each change ahead in the pipeline will pass
its tests. If they all succeed, all the changes can be tested and
merged in parallel. If a change near the front of the pipeline
fails its tests, each change behind it ignores whatever tests have
been completed and are tested again without the change in front.
This way gate tests may run in parallel but still be tested
correctly, exactly as they will appear in the repository when
merged.
One important characteristic of the DependentPipelineManager is that
it analyzes the jobs that are triggered by different projects, and
if those projects have jobs in common, it treats those projects as
related, and they share a single virtual queue of changes. Thus,
if there is a job that performs integration testing on two
projects, those two projects will automatically share a virtual
change queue. If a third project does not invoke that job, it
will be part of a separate virtual change queue, and changes to
it will not depend on changes to the first two jobs.
For more detail on the theory and operation of Zuul's
DependentPipelineManager, see: :doc:`gating`.
**trigger**
At least one trigger source must be supplied for each pipeline.
Triggers are not exclusive -- matching events may be placed in
multiple pipelines, and they will behave independently in each of
the pipelines they match.
Triggers are loaded from their connection name. The driver type of
the connection will dictate which options are available.
See :doc:`triggers`.
**require**
If this section is present, it established pre-requisites for any
kind of item entering the Pipeline. Regardless of how the item is
to be enqueued (via any trigger or automatic dependency resolution),
the conditions specified here must be met or the item will not be
enqueued.
.. _pipeline-require-approval:
**approval**
This requires that a certain kind of approval be present for the
current patchset of the change (the approval could be added by the
event in question). It takes several sub-parameters, all of which
are optional and are combined together so that there must be an
approval matching all specified requirements.
*username*
If present, an approval from this username is required. It is
treated as a regular expression.
*email*
If present, an approval with this email address is required. It
is treated as a regular expression.
*email-filter* (deprecated)
A deprecated alternate spelling of *email*. Only one of *email* or
*email_filter* should be used.
*older-than*
If present, the approval must be older than this amount of time
to match. Provide a time interval as a number with a suffix of
"w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
(seconds). Example ``48h`` or ``2d``.
*newer-than*
If present, the approval must be newer than this amount of time
to match. Same format as "older-than".
Any other field is interpreted as a review category and value
pair. For example ``verified: 1`` would require that the approval
be for a +1 vote in the "Verified" column. The value may either
be a single value or a list: ``verified: [1, 2]`` would match
either a +1 or +2 vote.
**open**
A boolean value (``true`` or ``false``) that indicates whether the change
must be open or closed in order to be enqueued.
**current-patchset**
A boolean value (``true`` or ``false``) that indicates whether the change
must be the current patchset in order to be enqueued.
**status**
A string value that corresponds with the status of the change
reported by the trigger. For example, when using the Gerrit
trigger, status values such as ``NEW`` or ``MERGED`` may be useful.
**reject**
If this section is present, it establishes pre-requisites that can
block an item from being enqueued. It can be considered a negative
version of **require**.
**approval**
This takes a list of approvals. If an approval matches the provided
criteria the change can not be entered into the pipeline. It follows
the same syntax as the :ref:`"require approval" pipeline above
<pipeline-require-approval>`.
Example to reject a change with any negative vote::
reject:
approval:
- code-review: [-1, -2]
**dequeue-on-new-patchset**
Normally, if a new patchset is uploaded to a change that is in a
pipeline, the existing entry in the pipeline will be removed (with
jobs canceled and any dependent changes that can no longer merge as
well. To suppress this behavior (and allow jobs to continue
running), set this to ``false``. Default: ``true``.
**ignore-dependencies**
In any kind of pipeline (dependent or independent), Zuul will
attempt to enqueue all dependencies ahead of the current change so
that they are tested together (independent pipelines report the
results of each change regardless of the results of changes ahead).
To ignore dependencies completely in an independent pipeline, set
this to ``true``. This option is ignored by dependent pipelines.
The default is: ``false``.
**success**
Describes where Zuul should report to if all the jobs complete
successfully.
This section is optional; if it is omitted, Zuul will run jobs and
do nothing on success; it will not even report a message to Gerrit.
If the section is present, the listed reporter plugins will be
asked to report on the jobs.
The reporters are listed by their connection name. The options
available depend on the driver for the supplied connection.
See :doc:`reporters` for more details.
**failure**
Uses the same syntax as **success**, but describes what Zuul should
do if at least one job fails.
**merge-failure**
Uses the same syntax as **success**, but describes what Zuul should
do if it is unable to merge in the patchset. If no merge-failure
reporters are listed then the ``failure`` reporters will be used to
notify of unsuccessful merges.
**start**
Uses the same syntax as **success**, but describes what Zuul should
do when a change is added to the pipeline manager. This can be used,
for example, to reset the value of the Verified review category.
**disabled**
Uses the same syntax as **success**, but describes what Zuul should
do when a pipeline is disabled.
See ``disable-after-consecutive-failures``.
**disable-after-consecutive-failures**
If set, a pipeline can enter a ''disabled'' state if too many changes
in a row fail. When this value is exceeded the pipeline will stop
reporting to any of the ``success``, ``failure`` or ``merge-failure``
reporters and instead only report to the ``disabled`` reporters.
(No ``start`` reports are made when a pipeline is disabled).
**precedence**
Indicates how the build scheduler should prioritize jobs for
different pipelines. Each pipeline may have one precedence, jobs
for pipelines with a higher precedence will be run before ones with
lower. The value should be one of ``high``, ``normal``, or ``low``.
Default: ``normal``.
**window**
DependentPipelineManagers only. Zuul can rate limit
DependentPipelineManagers in a manner similar to TCP flow control.
Jobs are only started for changes in the queue if they sit in the
actionable window for the pipeline. The initial length of this window
is configurable with this value. The value given should be a positive
integer value. A value of ``0`` disables rate limiting on the
DependentPipelineManager.
Default: ``20``.
**window-floor**
DependentPipelineManagers only. This is the minimum value for the
window described above. Should be a positive non zero integer value.
Default: ``3``.
**window-increase-type**
DependentPipelineManagers only. This value describes how the window
should grow when changes are successfully merged by zuul. A value of
``linear`` indicates that ``window-increase-factor`` should be added
to the previous window value. A value of ``exponential`` indicates
that ``window-increase-factor`` should be multiplied against the
previous window value and the result will become the window size.
Default: ``linear``.
**window-increase-factor**
DependentPipelineManagers only. The value to be added or multiplied
against the previous window value to determine the new window after
successful change merges.
Default: ``1``.
**window-decrease-type**
DependentPipelineManagers only. This value describes how the window
should shrink when changes are not able to be merged by Zuul. A value
of ``linear`` indicates that ``window-decrease-factor`` should be
subtracted from the previous window value. A value of ``exponential``
indicates that ``window-decrease-factor`` should be divided against
the previous window value and the result will become the window size.
Default: ``exponential``.
**window-decrease-factor**
DependentPipelineManagers only. The value to be subtracted or divided
against the previous window value to determine the new window after
unsuccessful change merges.
Default: ``2``.
Some example pipeline configurations are included in the sample layout
file. The first is called a *check* pipeline::
- name: check
manager: IndependentPipelineManager
trigger:
my_gerrit:
- event: patchset-created
success:
my_gerrit:
verified: 1
failure:
my_gerrit:
verified: -1
This will trigger jobs each time a new patchset (or change) is
uploaded to Gerrit, and report +/-1 values to Gerrit in the
``verified`` review category. ::
- name: gate
manager: DependentPipelineManager
trigger:
my_gerrit:
- event: comment-added
approval:
- approved: 1
success:
my_gerrit:
verified: 2
submit: true
failure:
my_gerrit:
verified: -2
This will trigger jobs whenever a reviewer leaves a vote of ``1`` in the
``approved`` review category in Gerrit (a non-standard category).
Changes will be tested in such a way as to guarantee that they will be
merged exactly as tested, though that will happen in parallel by
creating a virtual queue of dependent changes and performing
speculative execution of jobs. ::
- name: post
manager: IndependentPipelineManager
trigger:
my_gerrit:
- event: ref-updated
ref: ^(?!refs/).*$
This will trigger jobs whenever a change is merged to a named branch
(e.g., ``master``). No output will be reported to Gerrit. This is
useful for side effects such as creating per-commit tarballs. ::
- name: silent
manager: IndependentPipelineManager
trigger:
my_gerrit:
- event: patchset-created
This also triggers jobs when changes are uploaded to Gerrit, but no
results are reported to Gerrit. This is useful for jobs that are in
development and not yet ready to be presented to developers. ::
pipelines:
- name: post-merge
manager: IndependentPipelineManager
trigger:
my_gerrit:
- event: change-merged
success:
my_gerrit:
force-message: True
failure:
my_gerrit:
force-message: True
The ``change-merged`` events happen when a change has been merged in the git
repository. The change is thus closed and Gerrit will not accept modifications
to the review scoring such as ``code-review`` or ``verified``. By using the
``force-message: True`` parameter, Zuul will pass ``--force-message`` to the
``gerrit review`` command, thus making sure the message is actually
sent back to Gerrit regardless of approval scores.
That kind of pipeline is nice to run regression or performance tests.
.. note::
The ``change-merged`` event does not include the commit sha1 which can be
hazardous, it would let you report back to Gerrit though. If you were to
build a tarball for a specific commit, you should consider instead using
the ``ref-updated`` event which does include the commit sha1 (but lacks the
Gerrit change number).
.. _jobs:
Jobs
""""
The jobs section is optional, and can be used to set attributes of
jobs that are independent of their association with a project. For
example, if a job should return a customized message on failure, that
may be specified here. Otherwise, Zuul does not need to be told about
each job as it builds a list from the project specification.
**name**
The name of the job. This field is treated as a regular expression
and will be applied to each job that matches.
**queue-name (optional)**
Zuul will automatically combine projects that share a job into
shared change queues for dependent pipeline managers. In order to
report statistics about these queues, it is convenient for them to
have names. Zuul can automatically name change queues, however
these can grow quite long and are prone to changing as projects in
the queue change. If you assign a queue-name to a job, Zuul will
use that as the name for the shared change queue that contains that
job instead of the automatically generated one. It is an error for
a shared change queue to have more than one job with a queue-name if
they are not the same.
**failure-message (optional)**
The message that should be reported to Gerrit if the job fails.
**success-message (optional)**
The message that should be reported to Gerrit if the job fails.
**failure-pattern (optional)**
The URL that should be reported to Gerrit if the job fails.
Defaults to the build URL or the url_pattern configured in
zuul.conf. May be supplied as a string pattern with substitutions
as described in url_pattern in :ref:`zuulconf`.
**success-pattern (optional)**
The URL that should be reported to Gerrit if the job succeeds.
Defaults to the build URL or the url_pattern configured in
zuul.conf. May be supplied as a string pattern with substitutions
as described in url_pattern in :ref:`zuulconf`.
**hold-following-changes (optional)**
This is a boolean that indicates that changes that follow this
change in a dependent change pipeline should wait until this job
succeeds before executing. If this is applied to a very short job
that can predict whether longer jobs will fail early, this can be
used to reduce the number of jobs that Zuul will execute and
ultimately have to cancel. In that case, a small amount of
parallelization of jobs is traded for more efficient use of testing
resources. On the other hand, to apply this to a long running job
would largely defeat the parallelization of dependent change testing
that is the main feature of Zuul. Default: ``false``.
**semaphore (optional)**
This is a string that names a semaphore that should be observed by this
job. The semaphore defines how many jobs which reference that semaphore
can be enqueued at a time. This applies across all pipelines in the same
tenant. The max value of the semaphore can be specified in the config
repositories and defaults to 1.
**branch (optional)**
This job should only be run on matching branches. This field is
treated as a regular expression and multiple branches may be
listed.
**files (optional)**
This job should only be run if at least one of the files involved in
the change (added, deleted, or modified) matches at least one of the
file patterns listed here. This field is treated as a regular
expression and multiple expressions may be listed.
**skip-if (optional)**
This job should not be run if all the patterns specified by the
optional fields listed below match on their targets. When multiple
sets of parameters are provided, this job will be skipped if any set
matches. For example: ::
jobs:
- name: check-tempest-dsvm-neutron
skip-if:
- project: ^openstack/neutron$
branch: ^stable/juno$
all-files-match-any:
- ^neutron/tests/.*$
- ^tools/.*$
- all-files-match-any:
- ^doc/.*$
- ^.*\.rst$
With this configuration, the job would be skipped for a neutron
patchset for the stable/juno branch provided that every file in the
change matched at least one of the specified file regexes. The job
will also be skipped for any patchset that modified only the doc
tree or rst files.
*project* (optional)
The regular expression to match against the project of the change.
*branch* (optional)
The regular expression to match against the branch or ref of the
change.
*all-files-match-any* (optional)
A list of regular expressions intended to match the files involved
in the change. This parameter will be considered matching a
change only if all files in a change match at least one of these
expressions.
The pattern for '/COMMIT_MSG' is always matched on and does not
have to be included. Exception is merge commits (without modified
files), in this case '/COMMIT_MSG' is not matched, and job is not
skipped. In case of merge commits it's assumed that list of modified
files isn't predictible and CI should be run.
**voting (optional)**
Boolean value (``true`` or ``false``) that indicates whatever
a job is voting or not. Default: ``true``.
**attempts (optional)**
Number of attempts zuul will execute a job. Once reached, zuul will report
RETRY_LIMIT as the job result.
Defaults to 3.
**tags (optional)**
A list of arbitrary strings which will be associated with the job.
Here is an example of setting the failure message for jobs that check
whether a change merges cleanly::
- name: ^.*-merge$
failure-message: This change or one of its cross-repo dependencies
was unable to be automatically merged with the current state of
its repository. Please rebase the change and upload a new
patchset.
Projects
""""""""
The projects section indicates what jobs should be run in each pipeline
for events associated with each project. It contains a list of
projects. Here is an example::
- name: example/project
check:
- project-merge:
- project-unittest
- project-pep8
- project-pyflakes
gate:
- project-merge:
- project-unittest
- project-pep8
- project-pyflakes
post:
- project-publish
**name**
The name of the project (as known by Gerrit).
**merge-mode (optional)**
An optional value that indicates what strategy should be used to
merge changes to this project. Supported values are:
** merge-resolve **
Equivalent to 'git merge -s resolve'. This corresponds closely to
what Gerrit performs (using JGit) for a project if the "Merge if
necessary" merge mode is selected and "Automatically resolve
conflicts" is checked. This is the default.
** merge **
Equivalent to 'git merge'.
** cherry-pick **
Equivalent to 'git cherry-pick'.
This is followed by a section for each of the pipelines defined above.
Pipelines may be omitted if no jobs should run for this project in a
given pipeline. Within the pipeline section, the jobs that should be
executed are listed. If a job is entered as a dictionary key, then
jobs contained within that key are only executed if the key job
succeeds. In the above example, project-unittest, project-pep8, and
project-pyflakes are only executed if project-merge succeeds.
Furthermore, project-finaltest is executed only if project-unittest,
project-pep8 and project-pyflakes all succeed. This can help avoid
running unnecessary jobs while maximizing parallelism. It is also
useful when distributing results between jobs.
The special job named ``noop`` is internal to Zuul and will always
return ``SUCCESS`` immediately. This can be useful if you require
that all changes be processed by a pipeline but a project has no jobs
that can be run on it.
.. seealso:: The OpenStack Zuul configuration for a comprehensive example: https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml
Project Templates
"""""""""""""""""
Whenever you have lot of similar projects (such as plugins for a project) you
will most probably want to use the same pipeline configurations. The
project templates let you define pipelines and job name templates to trigger.
One can then just apply the template on its project which make it easier to
update several similar projects. As an example::
project-templates:
# Name of the template
- name: plugin-triggering
# Definition of pipelines just like for a `project`
check:
- '{jobprefix}-merge':
- '{jobprefix}-pep8'
- '{jobprefix}-pyflakes'
gate:
- '{jobprefix}-merge':
- '{jobprefix}-unittest'
- '{jobprefix}-pep8'
- '{jobprefix}-pyflakes'
In your projects definition, you will then apply the template using the template
key::
projects:
- name: plugin/foobar
template:
- name: plugin-triggering
jobprefix: plugin-foobar
You can pass several parameters to a template. A ``parameter`` value
will be used for expansion of ``{parameter}`` in the template
strings. The parameter ``name`` will be automatically provided and
will contain the short name of the project, that is the portion of the
project name after the last ``/`` character.
Multiple templates can be combined in a project, and the jobs from all
of those templates will be added to the project. Individual jobs may
also be added::
projects:
- name: plugin/foobar
template:
- name: plugin-triggering
jobprefix: plugin-foobar
- name: plugin-extras
jobprefix: plugin-foobar
check:
- foobar-extra-special-job
Individual jobs may optionally be added to pipelines (e.g. check,
gate, et cetera) for a project, in addition to those provided by
templates.
The order of the jobs listed in the project (which only affects the
order of jobs listed on the report) will be the jobs from each
template in the order listed, followed by any jobs individually listed
for the project.
Note that if multiple templates are used for a project and one
template specifies a job that is also specified in another template,
or specified in the project itself, the configuration defined by
either the last template or the project itself will take priority.
Semaphores
""""""""""
When using semaphores the maximum value of each one can be specified in their
respective config repositories. Unspecified semaphores default to 1::
- semaphore:
name: semaphore-foo
max: 5
- semaphore:
name: semaphore-bar
max: 3
logging.conf
~~~~~~~~~~~~
This file is optional. If provided, it should be a standard
:mod:`logging.config` module configuration file. If not present, Zuul will
output all log messages of DEBUG level or higher to the console.
Starting Zuul
-------------
To start Zuul, run **zuul-server**::
usage: zuul-server [-h] [-c CONFIG] [-l LAYOUT] [-d] [-t] [--version]
Project gating system.
optional arguments:
-h, --help show this help message and exit
-c CONFIG specify the config file
-l LAYOUT specify the layout file
-d do not run as a daemon
-t validate layout file syntax
--version show zuul version
You may want to use the ``-d`` argument while you are initially setting
up Zuul so you can detect any configuration errors quickly. Under
normal operation, omit ``-d`` and let Zuul run as a daemon.
If you send signal 1 (SIGHUP) to the zuul-server process, Zuul will
stop executing new jobs, wait until all executing jobs are finished,
reload its layout.yaml, and resume. Changes to any connections or
the PID file will be ignored until Zuul is restarted.
If you send a SIGUSR1 to the zuul-server process, Zuul will stop
executing new jobs, wait until all executing jobs are finished,
then exit. While waiting to exit Zuul will queue Gerrit events and
save these events prior to exiting. When Zuul starts again it will
read these saved events and act on them.
If you need to abort Zuul and intend to manually requeue changes for
jobs which were running in its pipelines, prior to terminating you can
use the zuul-changes.py tool script to simplify the process. For
example, this would give you a list of zuul-enqueue commands to requeue
changes for the gate and check pipelines respectively::
./tools/zuul-changes.py http://zuul.openstack.org/ gate
./tools/zuul-changes.py http://zuul.openstack.org/ check
If you send a SIGUSR2 to the zuul-server process, or the forked process
that runs the Gearman daemon, Zuul will dump a stack trace for each
running thread into its debug log. It is written under the log bucket
``zuul.stack_dump``. This is useful for tracking down deadlock or
otherwise slow threads.
When `yappi <https://code.google.com/p/yappi/>`_ (Yet Another Python
Profiler) is available, additional functions' and threads' stats are
emitted as well. The first SIGUSR2 will enable yappi, on the second
SIGUSR2 it dumps the information collected, resets all yappi state and
stops profiling. This is to minimize the impact of yappi on a running
system.