Add configuration documentation
This adds the start of v3 configuration documentation. Change-Id: I3cc2f69c61795ef82d0e1c71d639e46fd1d5a7d1 Story: 2000790 Task: 3314
This commit is contained in:
parent
6bc671a4ec
commit
1de8d40f38
|
@ -0,0 +1,24 @@
|
|||
:title: Administrative Configuration
|
||||
|
||||
.. _admin-config:
|
||||
.. _drivers:
|
||||
|
||||
Administrative Configuration
|
||||
============================
|
||||
|
||||
TODO:
|
||||
|
||||
* zuul.conf
|
||||
* connections
|
||||
|
||||
* main.yaml
|
||||
* tenants
|
||||
* trusted/untrusted
|
||||
|
||||
* drivers
|
||||
* gerrit
|
||||
* trigger config
|
||||
* reporter config
|
||||
* github
|
||||
* trigger config
|
||||
* reporter config
|
|
@ -0,0 +1,38 @@
|
|||
:title: Configuration
|
||||
|
||||
.. _config:
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
Zuul uses a very flexible configuration system. Some initial
|
||||
configuration must be supplied by the system administrator. From that
|
||||
beginning, Zuul builds its configuration by dynamically evaluating the
|
||||
contents of every git repository indicated by the configuration.
|
||||
|
||||
Zuul configuration is focused primarily around the concept of a
|
||||
*Pipeline*. Through a pipeline, a Zuul user expresses a connection
|
||||
between events, projects, jobs, and reports. The general flow of
|
||||
operations is as follows:
|
||||
|
||||
A *Connection* to a remote system generates an event; if it matches a
|
||||
*Trigger* on a *Pipeline*, then an *Item* is enqueued into the
|
||||
*Pipeline*. An *Item* is some kind of git reference, such as a branch
|
||||
tip, a proposed change, or a pull request, and is associated with a
|
||||
*Project*. Zuul launches the *Jobs* specified for that *Project* in
|
||||
that *Pipeline*, and when they complete, Zuul reports the results as
|
||||
specified by the pipeline's *Reporter*.
|
||||
|
||||
TODO: flow diagram
|
||||
|
||||
The following sections describe how to configure Zuul.
|
||||
:ref:`admin-config` covers areas of interest primarily to system
|
||||
administrators. :ref:`project-config` covers the portion of Zuul
|
||||
configuration of interest to most users -- that which is included
|
||||
directly in the git repositories operated on by Zuul.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
admin-config
|
||||
project-config
|
|
@ -0,0 +1,8 @@
|
|||
:title: Encryption
|
||||
|
||||
.. _encryption:
|
||||
|
||||
Encryption
|
||||
==========
|
||||
|
||||
TODO
|
|
@ -15,6 +15,8 @@ Contents:
|
|||
|
||||
quick-start
|
||||
gating
|
||||
configuration
|
||||
encryption
|
||||
connections
|
||||
triggers
|
||||
reporters
|
||||
|
|
|
@ -0,0 +1,941 @@
|
|||
:title: Project Configuration
|
||||
|
||||
.. _project-config:
|
||||
|
||||
Project Configuration
|
||||
=====================
|
||||
|
||||
The following sections describe the main part of Zuul's configuration.
|
||||
All of what follows is found within files inside of the repositories
|
||||
that Zuul manages.
|
||||
|
||||
Security Contexts
|
||||
-----------------
|
||||
|
||||
When a system administrator configures Zuul to operate on a project,
|
||||
they specify one of two security contexts for that project. A
|
||||
*config-project* is one which is primarily tasked with holding
|
||||
configuration information and job content for Zuul. Jobs which are
|
||||
defined in a *config-project* are run with elevated privileges, and
|
||||
all Zuul configuration items are available for use. It is expected
|
||||
that changes to *config-projects* will undergo careful scrutiny before
|
||||
being merged.
|
||||
|
||||
An *untrusted-project* is a project whose primary focus is not to
|
||||
operate Zuul, but rather it is one of the projects being tested or
|
||||
deployed. The Zuul configuration language available to these projects
|
||||
is somewhat restricted (as detailed in individual section below), and
|
||||
jobs defined in these projects run in a restricted execution
|
||||
environment since they may be operating on changes which have not yet
|
||||
undergone review.
|
||||
|
||||
Configuration Loading
|
||||
---------------------
|
||||
|
||||
When Zuul starts, it examines all of the git repositories which are
|
||||
specified by the system administrator in :ref:`admin-config` and searches
|
||||
for files in the root of each repository. In the case of a
|
||||
*config-project*, Zuul looks for a file named `zuul.yaml`. In the
|
||||
case of an *untrusted-project*, Zuul looks first for `zuul.yaml` and
|
||||
if that is not found, `.zuul.yaml` (with a leading dot). In the case
|
||||
of an *untrusted-project*, the configuration from every branch is
|
||||
included, however, in the case of a *config-project*, only the
|
||||
`master` branch is examined.
|
||||
|
||||
When a change is proposed to one of these files in an
|
||||
*untrusted-project*, the configuration proposed in the change is
|
||||
merged into the running configuration so that any changes to Zuul's
|
||||
configuration are self-testing as part of that change. If there is a
|
||||
configuration error, no jobs will be run and the error will be
|
||||
reported by any applicable pipelines. In the case of a change to a
|
||||
*config-project*, the new configuration is parsed and examined for
|
||||
errors, but the new configuration is not used in testing the change.
|
||||
This is because configuration in *config-projects* is able to access
|
||||
elevated privileges and should always be reviewed before being merged.
|
||||
|
||||
As soon as a change containing a Zuul configuration change merges to
|
||||
any Zuul-managed repository, the new configuration takes effect
|
||||
immediately.
|
||||
|
||||
Configuration Items
|
||||
-------------------
|
||||
|
||||
The `zuul.yaml` and `.zuul.yaml` configuration files are
|
||||
YAML-formatted and are structured as a series of items, each of which
|
||||
is described below.
|
||||
|
||||
.. _pipeline:
|
||||
|
||||
Pipeline
|
||||
~~~~~~~~
|
||||
|
||||
A pipeline describes a workflow operation in Zuul. It associates jobs
|
||||
for a given project with triggering and reporting events.
|
||||
|
||||
Its flexible configuration allows for characterizing any number of
|
||||
workflows, and by specifying each as a named configuration, makes it
|
||||
easy to apply similar workflow operations to projects or groups of
|
||||
projects.
|
||||
|
||||
By way of example, one of the primary uses of Zuul is to perform
|
||||
project gating. To do so, one can create a *gate* pipeline which
|
||||
tells Zuul that when a certain event (such as approval by a code
|
||||
reviewer) occurs, the corresponding change or pull request should be
|
||||
enqueued into the pipeline. When that happens, the jobs which have
|
||||
been configured to run for that project in the *gate* pipeline are
|
||||
run, and when they complete, the pipeline reports the results to the
|
||||
user.
|
||||
|
||||
Pipeline configuration items may only appear in *config-projects*.
|
||||
|
||||
Generally, a Zuul administrator would define a small number of
|
||||
pipelines which represent the workflow processes used in their
|
||||
environment. Each project can then be added to the available
|
||||
pipelines as appropriate.
|
||||
|
||||
Here is an example *check* pipeline, which runs whenever a new
|
||||
patchset is created in Gerrit. If the associated jobs all report
|
||||
success, the pipeline reports back to Gerrit with a *Verified* vote of
|
||||
+1, or if at least one of them fails, a -1::
|
||||
|
||||
- pipeline:
|
||||
name: check
|
||||
manager: independent
|
||||
trigger:
|
||||
my_gerrit:
|
||||
- event: patchset-created
|
||||
success:
|
||||
my_gerrit:
|
||||
verified: 1
|
||||
failure:
|
||||
my_gerrit
|
||||
verified: -1
|
||||
|
||||
See TODO for more annotated examples of common pipeline configurations.
|
||||
|
||||
The attributes available on a pipeline are as follows (all are
|
||||
optional unless otherwise specified):
|
||||
|
||||
**name** (required)
|
||||
This is used later in the project definition to indicate what jobs
|
||||
should be run for events in the pipeline.
|
||||
|
||||
**manager** (required)
|
||||
There are currently two schemes for managing pipelines:
|
||||
|
||||
*independent*
|
||||
Every event in this pipeline should be treated as independent of
|
||||
other events in the pipeline. This is appropriate when the order of
|
||||
events in the pipeline doesn't matter because the results of the
|
||||
actions this pipeline performs can not affect other events in the
|
||||
pipeline. For example, when a change is first uploaded for review,
|
||||
you may want to run tests on that change to provide early feedback
|
||||
to reviewers. At the end of the tests, the change is not going to
|
||||
be merged, so it is safe to run these tests in parallel without
|
||||
regard to any other changes in the pipeline. They are independent.
|
||||
|
||||
Another type of pipeline that is independent is a post-merge
|
||||
pipeline. In that case, the changes have already merged, so the
|
||||
results can not affect any other events in the pipeline.
|
||||
|
||||
*dependent*
|
||||
The dependent pipeline manager is designed for gating. It ensures
|
||||
that every change is tested exactly as it is going to be merged
|
||||
into the repository. An ideal gating system would test one change
|
||||
at a time, applied to the tip of the repository, and only if that
|
||||
change passed tests would it be merged. Then the next change in
|
||||
line would be tested the same way. In order to achieve parallel
|
||||
testing of changes, the dependent pipeline manager performs
|
||||
speculative execution on changes. It orders changes based on
|
||||
their entry into the pipeline. It begins testing all changes in
|
||||
parallel, assuming that each change ahead in the pipeline will pass
|
||||
its tests. If they all succeed, all the changes can be tested and
|
||||
merged in parallel. If a change near the front of the pipeline
|
||||
fails its tests, each change behind it ignores whatever tests have
|
||||
been completed and are tested again without the change in front.
|
||||
This way gate tests may run in parallel but still be tested
|
||||
correctly, exactly as they will appear in the repository when
|
||||
merged.
|
||||
|
||||
For more detail on the theory and operation of Zuul's dependent
|
||||
pipeline manager, see: :doc:`gating`.
|
||||
|
||||
**description**
|
||||
This field may be used to provide a textual description of the
|
||||
pipeline. It may appear in the status page or in documentation.
|
||||
|
||||
**success-message**
|
||||
The introductory text in reports when all the voting jobs are
|
||||
successful. Defaults to "Build successful."
|
||||
|
||||
**failure-message**
|
||||
The introductory text in reports when at least one voting job fails.
|
||||
Defaults to "Build failed."
|
||||
|
||||
**merge-failure-message**
|
||||
The introductory text in the message reported when a change fails to
|
||||
merge with the current state of the repository. Defaults to "Merge
|
||||
failed."
|
||||
|
||||
**footer-message**
|
||||
Supplies additional information after test results. Useful for
|
||||
adding information about the CI system such as debugging and contact
|
||||
details.
|
||||
|
||||
**trigger**
|
||||
At least one trigger source must be supplied for each pipeline.
|
||||
Triggers are not exclusive -- matching events may be placed in
|
||||
multiple pipelines, and they will behave independently in each of
|
||||
the pipelines they match.
|
||||
|
||||
Triggers are loaded from their connection name. The driver type of
|
||||
the connection will dictate which options are available.
|
||||
See :doc:`triggers`.
|
||||
|
||||
**require**
|
||||
If this section is present, it established pre-requisites for any
|
||||
kind of item entering the Pipeline. Regardless of how the item is
|
||||
to be enqueued (via any trigger or automatic dependency resolution),
|
||||
the conditions specified here must be met or the item will not be
|
||||
enqueued.
|
||||
|
||||
.. TODO this section is in flux in v3 _pipeline-require-approval:
|
||||
|
||||
**approval**
|
||||
This requires that a certain kind of approval be present for the
|
||||
current patchset of the change (the approval could be added by the
|
||||
event in question). It takes several sub-parameters, all of which
|
||||
are optional and are combined together so that there must be an
|
||||
approval matching all specified requirements.
|
||||
|
||||
*username*
|
||||
If present, an approval from this username is required. It is
|
||||
treated as a regular expression.
|
||||
|
||||
*email*
|
||||
If present, an approval with this email address is required. It
|
||||
is treated as a regular expression.
|
||||
|
||||
*email-filter* (deprecated)
|
||||
A deprecated alternate spelling of *email*. Only one of *email* or
|
||||
*email_filter* should be used.
|
||||
|
||||
*older-than*
|
||||
If present, the approval must be older than this amount of time
|
||||
to match. Provide a time interval as a number with a suffix of
|
||||
"w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
|
||||
(seconds). Example ``48h`` or ``2d``.
|
||||
|
||||
*newer-than*
|
||||
If present, the approval must be newer than this amount of time
|
||||
to match. Same format as "older-than".
|
||||
|
||||
Any other field is interpreted as a review category and value
|
||||
pair. For example ``verified: 1`` would require that the approval
|
||||
be for a +1 vote in the "Verified" column. The value may either
|
||||
be a single value or a list: ``verified: [1, 2]`` would match
|
||||
either a +1 or +2 vote.
|
||||
|
||||
**open**
|
||||
A boolean value (``true`` or ``false``) that indicates whether the change
|
||||
must be open or closed in order to be enqueued.
|
||||
|
||||
**current-patchset**
|
||||
A boolean value (``true`` or ``false``) that indicates whether the change
|
||||
must be the current patchset in order to be enqueued.
|
||||
|
||||
**status**
|
||||
A string value that corresponds with the status of the change
|
||||
reported by the trigger.
|
||||
|
||||
**reject**
|
||||
If this section is present, it establishes pre-requisites that can
|
||||
block an item from being enqueued. It can be considered a negative
|
||||
version of **require**.
|
||||
|
||||
**approval**
|
||||
This takes a list of approvals. If an approval matches the provided
|
||||
criteria the change can not be entered into the pipeline. It follows
|
||||
the same syntax as the :ref:`"require approval" pipeline above
|
||||
<pipeline-require-approval>`.
|
||||
|
||||
Example to reject a change with any negative vote::
|
||||
|
||||
reject:
|
||||
approval:
|
||||
- code-review: [-1, -2]
|
||||
|
||||
**dequeue-on-new-patchset**
|
||||
Normally, if a new patchset is uploaded to a change that is in a
|
||||
pipeline, the existing entry in the pipeline will be removed (with
|
||||
jobs canceled and any dependent changes that can no longer merge as
|
||||
well. To suppress this behavior (and allow jobs to continue
|
||||
running), set this to ``false``. Default: ``true``.
|
||||
|
||||
**ignore-dependencies**
|
||||
In any kind of pipeline (dependent or independent), Zuul will
|
||||
attempt to enqueue all dependencies ahead of the current change so
|
||||
that they are tested together (independent pipelines report the
|
||||
results of each change regardless of the results of changes ahead).
|
||||
To ignore dependencies completely in an independent pipeline, set
|
||||
this to ``true``. This option is ignored by dependent pipelines.
|
||||
The default is: ``false``.
|
||||
|
||||
The following options configure *reporters*. Reporters are
|
||||
complementary to triggers; where a trigger is an event on a connection
|
||||
which causes Zuul to enqueue an item, a reporter is the action
|
||||
performed on a connection when an item is dequeued after its jobs
|
||||
complete. The actual syntax for a reporter is defined by the driver
|
||||
which implements it. See :ref:`drivers` for more information.
|
||||
|
||||
**success**
|
||||
Describes where Zuul should report to if all the jobs complete
|
||||
successfully. This section is optional; if it is omitted, Zuul will
|
||||
run jobs and do nothing on success -- it will not report at all. If
|
||||
the section is present, the listed reporters will be asked to report
|
||||
on the jobs. The reporters are listed by their connection name. The
|
||||
options available depend on the driver for the supplied connection.
|
||||
|
||||
**failure**
|
||||
These reporters describe what Zuul should do if at least one job
|
||||
fails.
|
||||
|
||||
**merge-failure**
|
||||
These reporters describe what Zuul should do if it is unable to
|
||||
merge in the patchset. If no merge-failure reporters are listed then
|
||||
the ``failure`` reporters will be used to notify of unsuccessful
|
||||
merges.
|
||||
|
||||
**start**
|
||||
These reporters describe what Zuul should do when a change is added
|
||||
to the pipeline. This can be used, for example, to reset a
|
||||
previously reported result.
|
||||
|
||||
**disabled**
|
||||
These reporters describe what Zuul should do when a pipeline is
|
||||
disabled. See ``disable-after-consecutive-failures``.
|
||||
|
||||
**disable-after-consecutive-failures**
|
||||
If set, a pipeline can enter a ''disabled'' state if too many changes
|
||||
in a row fail. When this value is exceeded the pipeline will stop
|
||||
reporting to any of the ``success``, ``failure`` or ``merge-failure``
|
||||
reporters and instead only report to the ``disabled`` reporters.
|
||||
(No ``start`` reports are made when a pipeline is disabled).
|
||||
|
||||
**precedence**
|
||||
Indicates how the build scheduler should prioritize jobs for
|
||||
different pipelines. Each pipeline may have one precedence, jobs
|
||||
for pipelines with a higher precedence will be run before ones with
|
||||
lower. The value should be one of ``high``, ``normal``, or ``low``.
|
||||
Default: ``normal``.
|
||||
|
||||
**window**
|
||||
Dependent pipeline managers only. Zuul can rate limit dependent
|
||||
pipelines in a manner similar to TCP flow control. Jobs are only
|
||||
started for items in the queue if they are within the actionable
|
||||
window for the pipeline. The initial length of this window is
|
||||
configurable with this value. The value given should be a positive
|
||||
integer value. A value of ``0`` disables rate limiting on the
|
||||
DependentPipelineManager. Default: ``20``.
|
||||
|
||||
**window-floor**
|
||||
Dependent pipeline managers only. This is the minimum value for the
|
||||
window described above. Should be a positive non zero integer value.
|
||||
Default: ``3``.
|
||||
|
||||
**window-increase-type**
|
||||
Dependent pipeline managers only. This value describes how the window
|
||||
should grow when changes are successfully merged by zuul. A value of
|
||||
``linear`` indicates that ``window-increase-factor`` should be added
|
||||
to the previous window value. A value of ``exponential`` indicates
|
||||
that ``window-increase-factor`` should be multiplied against the
|
||||
previous window value and the result will become the window size.
|
||||
Default: ``linear``.
|
||||
|
||||
**window-increase-factor**
|
||||
Dependent pipeline managers only. The value to be added or multiplied
|
||||
against the previous window value to determine the new window after
|
||||
successful change merges.
|
||||
Default: ``1``.
|
||||
|
||||
**window-decrease-type**
|
||||
Dependent pipeline managers only. This value describes how the window
|
||||
should shrink when changes are not able to be merged by Zuul. A value
|
||||
of ``linear`` indicates that ``window-decrease-factor`` should be
|
||||
subtracted from the previous window value. A value of ``exponential``
|
||||
indicates that ``window-decrease-factor`` should be divided against
|
||||
the previous window value and the result will become the window size.
|
||||
Default: ``exponential``.
|
||||
|
||||
**window-decrease-factor**
|
||||
Dependent pipline managers only. The value to be subtracted or divided
|
||||
against the previous window value to determine the new window after
|
||||
unsuccessful change merges.
|
||||
Default: ``2``.
|
||||
|
||||
|
||||
.. _job:
|
||||
|
||||
Job
|
||||
~~~
|
||||
|
||||
A job is a unit of work performed by Zuul on an item enqueued into a
|
||||
pipeline. Items may run any number of jobs (which may depend on each
|
||||
other). Each job is an invocation of an Ansible playbook with a
|
||||
specific inventory of hosts. The actual tasks that are run by the job
|
||||
appear in the playbook for that job while the attributes that appear in the
|
||||
Zuul configuration specify information about when, where, and how the
|
||||
job should be run.
|
||||
|
||||
Jobs in Zuul support inheritance. Any job may specify a single parent
|
||||
job, and any attributes not set on the child job are collected from
|
||||
the parent job. In this way, a configuration structure may be built
|
||||
starting with very basic jobs which describe characteristics that all
|
||||
jobs on the system should have, progressing through stages of
|
||||
specialization before arriving at a particular job. A job may inherit
|
||||
from any other job in any project (however, if the other job is marked
|
||||
as `final`, some attributes may not be overidden).
|
||||
|
||||
Jobs also support a concept called variance. The first time a job
|
||||
definition appears is called the reference definition of the job.
|
||||
Subsequent job definitions with the same name are called variants.
|
||||
These may have different selection criteria which indicate to Zuul
|
||||
that, for instance, the job should behave differently on a different
|
||||
git branch. Unlike inheritance, all job variants must be defined in
|
||||
the same project.
|
||||
|
||||
When Zuul decides to run a job, it performs a process known as
|
||||
freezing the job. Because any number of job variants may be
|
||||
applicable, Zuul collects all of the matching variants and applies
|
||||
them in the order they appeared in the configuration. The resulting
|
||||
frozen job is built from attributes gathered from all of the
|
||||
matching variants. In this way, exactly what is run is dependent on
|
||||
the pipeline, project, branch, and content of the item.
|
||||
|
||||
In addition to the job's main playbook, each job may specify one or
|
||||
more pre- and post-playbooks. These are run, in order, before and
|
||||
after (respectively) the main playbook. They may be used to set up
|
||||
and tear down resources needed by the main playbook. When combined
|
||||
with inheritance, they provide powerful tools for job construction. A
|
||||
job only has a single main playbook, and when inheriting from a
|
||||
parent, the child's main playbook overrides (or replaces) the
|
||||
parent's. However, the pre- and post-playbooks are appended and
|
||||
prepended in a nesting fashion. So if a parent job and child job both
|
||||
specified pre and post playbooks, the sequence of playbooks run would
|
||||
be:
|
||||
|
||||
* parent pre-run playbook
|
||||
* child pre-run playbook
|
||||
* child playbook
|
||||
* child post-run playbook
|
||||
* parent post-run playbook
|
||||
|
||||
Further inheritance would nest even deeper.
|
||||
|
||||
Here is an example of two job definitions::
|
||||
|
||||
- job:
|
||||
name: base
|
||||
pre-run: copy-git-repos
|
||||
post-run: copy-logs
|
||||
|
||||
- job:
|
||||
name: run-tests
|
||||
parent: base
|
||||
nodes:
|
||||
- name: test-node
|
||||
image: fedora
|
||||
|
||||
The following attributes are available on a job; all are optional
|
||||
unless otherwise specified:
|
||||
|
||||
**name** (required)
|
||||
The name of the job. By default, Zuul looks for a playbook with
|
||||
this name to use as the main playbook for the job. This name is
|
||||
also referenced later in a project pipeline configuration.
|
||||
|
||||
**parent**
|
||||
Specifies a job to inherit from. The parent job can be defined in
|
||||
this or any other project. Any attributes not specified on a job
|
||||
will be collected from its parent.
|
||||
|
||||
**description**
|
||||
A textual description of the job. Not currently used directly by
|
||||
Zuul, but it is used by the zuul-sphinx extension to Sphinx to
|
||||
auto-document Zuul jobs (in which case it is interpreted as
|
||||
ReStructuredText.
|
||||
|
||||
**success-message**
|
||||
Normally when a job succeeds, the string "SUCCESS" is reported as
|
||||
the result for the job. If set, this option may be used to supply a
|
||||
different string. Default: "SUCCESS".
|
||||
|
||||
**failure-message**
|
||||
Normally when a job fails, the string "FAILURE" is reported as
|
||||
the result for the job. If set, this option may be used to supply a
|
||||
different string. Default: "FAILURE".
|
||||
|
||||
**success-url**
|
||||
When a job succeeds, this URL is reported along with the result.
|
||||
Default: none.
|
||||
|
||||
**failure-url**
|
||||
When a job fails, this URL is reported along with the result.
|
||||
Default: none.
|
||||
|
||||
**hold-following-changes**
|
||||
In a dependent pipeline, this option may be used to indicate that no
|
||||
jobs should start on any items which depend on the current item
|
||||
until this job has completed successfully. This may be used to
|
||||
conserve build resources, at the expense of inhibiting the
|
||||
parallelization which speeds the processing of items in a dependent
|
||||
pipeline. A boolean value, default: false.
|
||||
|
||||
**voting**
|
||||
Indicates whether the result of this job should be used in
|
||||
determining the overall result of the item. A boolean value,
|
||||
default: true.
|
||||
|
||||
**semaphore**
|
||||
The name of a :ref:`semaphore` which should be acquired and released
|
||||
when the job begins and ends. If the semaphore is at maximum
|
||||
capacity, then Zuul will wait until it can be acquired before
|
||||
starting the job. Default: none.
|
||||
|
||||
**tags**
|
||||
Metadata about this job. Tags are units of information attached to
|
||||
the job; they do not affect Zuul's behavior, but they can be used
|
||||
within the job to characterize the job. For example, a job which
|
||||
tests a certain subsystem could be tagged with the name of that
|
||||
subsystem, and if the job's results are reported into a database,
|
||||
then the results of all jobs affecting that subsystem could be
|
||||
queried. This attribute is specified as a list of strings, and when
|
||||
inheriting jobs or applying variants, tags accumulate in a set, so
|
||||
the result is always a set of all the tags from all the jobs and
|
||||
variants used in constructing the frozen job, with no duplication.
|
||||
Default: none.
|
||||
|
||||
** branches **
|
||||
A regular expression (or list of regular expressions) which describe
|
||||
on what branches a job should run (or in the case of variants: to
|
||||
alter the behavior of a job for a certain branch).
|
||||
|
||||
If there is no job definition for a given job which matches the
|
||||
branch of an item, then that job is not run for the item.
|
||||
Otherwise, all of the job variants which match that branch (and any
|
||||
other selection criteria) are used when freezing the job.
|
||||
|
||||
This example illustrates a job called *run-tests* which uses a
|
||||
nodeset based on the current release of an operating system to
|
||||
perform its tests, except when testing changes to the stable/2.0
|
||||
branch, in which case it uses an older release::
|
||||
|
||||
- job:
|
||||
name: run-tests
|
||||
nodes: current-release
|
||||
|
||||
- job:
|
||||
name: run-tests
|
||||
branch: stable/2.0
|
||||
nodes: old-release
|
||||
|
||||
In some cases, Zuul uses an implied value for the branch specifier
|
||||
if none is supplied:
|
||||
|
||||
* For a job definition in a *config-project*, no implied branch
|
||||
specifier is used. If no branch specifier appears, the job
|
||||
applies to all branches.
|
||||
|
||||
* In the case of an *untrusted-project*, no implied branch specifier
|
||||
is applied to the reference definition of a job. That is to say,
|
||||
that if the first appearance of the job definition appears without
|
||||
a branch specifier, then it will apply to all branches. Note that
|
||||
when collecting its configuration, Zuul reads the `master` branch
|
||||
of a given project first, then other branches in alphabetical
|
||||
order.
|
||||
|
||||
* Any further job variants other than the reference definition in an
|
||||
*untrusted-project* will, if they do not have a branch specifier,
|
||||
will have an implied branch specifier for the current branch
|
||||
applied.
|
||||
|
||||
This allows for the very simple and expected workflow where if a
|
||||
project defines a job on the master branch with no branch specifier,
|
||||
and then creates a new branch based on master, any changes to that
|
||||
job definition within the new branch only affect that branch.
|
||||
|
||||
**files**
|
||||
This attribute indicates that the job should only run on changes
|
||||
where the specified files are modified. This is a regular
|
||||
expression or list of regular expressions. Default: none.
|
||||
|
||||
**irrelevant-files**
|
||||
This is a negative complement of `files`. It indicates that the job
|
||||
should run unless *all* of the files changed match this list. In
|
||||
other words, if the regular expression `docs/.*` is supplied, then
|
||||
this job will not run if the only files changed are in the docs
|
||||
directory. A regular expression or list of regular expressions.
|
||||
Default: none.
|
||||
|
||||
**auth**
|
||||
Authentication information to be made available to the job. This is
|
||||
a dictionary with two potential keys:
|
||||
|
||||
**inherit**
|
||||
A boolean indicating that the authentication information referenced
|
||||
by this job should be able to be inherited by child jobs. Normally
|
||||
when a job inherits from another job, the auth section is not
|
||||
included. This permits jobs to inherit the same basic structure and
|
||||
playbook, but ensures that secret information is unable to be
|
||||
exposed by a child job which may alter the job's behavior. If it is
|
||||
safe for the contents of the authentication section to be used by
|
||||
child jobs, set this to ``true``. Default: ``false``.
|
||||
|
||||
**secrets**
|
||||
A list of secrets which may be used by the job. A :ref:`secret` is
|
||||
a named collection of private information defined separately in the
|
||||
configuration. The secrets that appear here must be defined in the
|
||||
same project as this job definition.
|
||||
|
||||
In the future, other types of authentication information may be
|
||||
added.
|
||||
|
||||
**nodes**
|
||||
A list of nodes which should be supplied to the job. This parameter
|
||||
may be supplied either as a string, in which case it references a
|
||||
:ref:`nodeset` definition which appears elsewhere in the
|
||||
configuration, or a list, in which case it is interpreted in the
|
||||
same way as a Nodeset definition (in essence, it is an anonymous
|
||||
Node definition unique to this job). See the :ref:`nodeset`
|
||||
reference for the syntax to use in that case.
|
||||
|
||||
If a job has an empty or no node definition, it will still run and
|
||||
may be able to perform actions on the Zuul executor.
|
||||
|
||||
**override-branch**
|
||||
When Zuul runs jobs for a proposed change, it normally checks out
|
||||
the branch associated with that change on every project present in
|
||||
the job. If jobs are running on a ref (such as a branch tip or
|
||||
tag), then that ref is normally checked out. This attribute is used
|
||||
to override that behavior and indicate that this job should,
|
||||
regardless of the branch for the queue item, use the indicated
|
||||
branch instead. This can be used, for example, to run a previous
|
||||
version of the software (from a stable maintenance branch) under
|
||||
test even if the change being tested applies to a different branch
|
||||
(this is only likely to be useful if there is some cross-branch
|
||||
interaction with some component of the system being tested). See
|
||||
also the project-specific **override-branch** attribute under
|
||||
**required-projects** to apply this behavior to a subset of a job's
|
||||
projects.
|
||||
|
||||
**timeout**
|
||||
The time in minutes that the job should be allowed to run before it
|
||||
is automatically aborted and failure is reported. If no timeout is
|
||||
supplied, the job may run indefinitely. Supplying a timeout is
|
||||
highly recommended.
|
||||
|
||||
**attempts**
|
||||
When Zuul encounters an error running a job's pre-run playbook, Zuul
|
||||
will stop and restart the job. Errors during the main or
|
||||
post-run -playbook phase of a job are not affected by this parameter
|
||||
(they are reported immediately). This parameter controls the number
|
||||
of attempts to make before an error is reported. Default: 3.
|
||||
|
||||
**pre-run**
|
||||
The name of a playbook or list of playbooks to run before the main
|
||||
body of a job. The playbook is expected to reside in the
|
||||
`playbooks/` directory of the project where the job is defined.
|
||||
|
||||
When a job inherits from a parent, the child's pre-run playbooks are
|
||||
run after the parent's. See :ref:`job` for more information.
|
||||
|
||||
**post-run**
|
||||
The name of a playbook or list of playbooks to run after the main
|
||||
body of a job. The playbook is expected to reside in the
|
||||
`playbooks/` directory of the project where the job is defined.
|
||||
|
||||
When a job inherits from a parent, the child's post-run playbooks
|
||||
are run before the parent's. See :ref:`job` for more information.
|
||||
|
||||
**run**
|
||||
The name of the main playbook for this job. This parameter is not
|
||||
normally necessary, as it defaults to the name of the job. However,
|
||||
if a playbook with a different name is needed, it can be specified
|
||||
here. The playbook is expected to reside in the `playbooks/`
|
||||
directory of the project where the job is defined. When a child
|
||||
inherits from a parent, a playbook with the name of the child job is
|
||||
implicitly searched first, before falling back on the playbook used
|
||||
by the parent job (unless the child job specifies a ``run``
|
||||
attribute, in which case that value is used). Default: the name of
|
||||
the job.
|
||||
|
||||
**roles**
|
||||
A list of Ansible roles to prepare for the job. Because a job runs
|
||||
an Ansible playbook, any roles which are used by the job must be
|
||||
prepared and installed by Zuul before the job begins. This value is
|
||||
a list of dictionaries, each of which indicates one of two types of
|
||||
roles: a Galaxy role, which is simply a role that is installed from
|
||||
Ansible Galaxy, or a Zuul role, which is a role provided by a
|
||||
project managed by Zuul. Zuul roles are able to benefit from
|
||||
speculative merging and cross-project dependencies when used by jobs
|
||||
in untrusted projects.
|
||||
|
||||
A project which supplies a role may be structured in one of two
|
||||
configurations: a bare role (in which the role exists at the root of
|
||||
the project), or a contained role (in which the role exists within
|
||||
the `roles/` directory of the project, perhaps along with other
|
||||
roles). In the case of a contained role, the `roles/` directory of
|
||||
the project is added to the role search path. In the case of a bare
|
||||
role, the project itself is added to the role search path. In case
|
||||
the name of the project is not the name under which the role should
|
||||
be installed (and therefore referenced from Ansible), the `name`
|
||||
attribute may be used to specify an alternate.
|
||||
|
||||
**galaxy**
|
||||
The name of the role in Ansible Galaxy. If this attribute is
|
||||
supplied, Zuul will search Ansible Galaxy for a role by this name
|
||||
and install it. Mutually exclusive with ``zuul``; either
|
||||
``galaxy`` or ``zuul`` must be supplied.
|
||||
|
||||
**zuul**
|
||||
The name of a Zuul project which supplies the role. Mutually
|
||||
exclusive with ``galaxy``; either ``galaxy`` or ``zuul`` must be
|
||||
supplied.
|
||||
|
||||
**name**
|
||||
The installation name of the role. In the case of a bare role,
|
||||
the role will be made available under this name. Ignored in the
|
||||
case of a contained role.
|
||||
|
||||
**required-projects**
|
||||
A list of other projects which are used by this job. Any Zuul
|
||||
projects specified here will also be checked out by Zuul into the
|
||||
working directory for the job. Speculative merging and cross-repo
|
||||
dependencies will be honored.
|
||||
|
||||
The format for this attribute is either a list of strings or
|
||||
dictionaries. Strings are interpreted as project names,
|
||||
dictionaries may have the following attributes:
|
||||
|
||||
**name**
|
||||
The name of the required project.
|
||||
|
||||
**override-branch**
|
||||
When Zuul runs jobs for a proposed change, it normally checks out
|
||||
the branch associated with that change on every project present in
|
||||
the job. If jobs are running on a ref (such as a branch tip or
|
||||
tag), then that ref is normally checked out. This attribute is
|
||||
used to override that behavior and indicate that this job should,
|
||||
regardless of the branch for the queue item, use the indicated
|
||||
branch instead, for only this project. See also the
|
||||
**override-branch** attribute of jobs to apply the same behavior
|
||||
to all projects in a job.
|
||||
|
||||
**vars**
|
||||
|
||||
A dictionary of variables to supply to Ansible. When inheriting from
|
||||
a job (or creating a variant of a job) vars are merged with previous
|
||||
definitions. This means a variable definition with the same name will
|
||||
override a previously defined variable, but new variable names will be
|
||||
added to the set of defined variables.
|
||||
|
||||
**dependencies**
|
||||
A list of other jobs upon which this job depends. Zuul will not
|
||||
start executing this job until all of its dependencies have
|
||||
completed successfully, and if one or more of them fail, this job
|
||||
will not be run.
|
||||
|
||||
**allowed-projects**
|
||||
A list of Zuul projects which may use this job. By default, a job
|
||||
may be used by any other project known to Zuul, however, some jobs
|
||||
use resources or perform actions which are not appropriate for other
|
||||
projects. In these cases, a list of projects which are allowed to
|
||||
use this job may be supplied. If this list is not empty, then it
|
||||
must be an exhaustive list of all projects permitted to use the job.
|
||||
The current project (where the job is defined) is not automatically
|
||||
included, so if it should be able to run this job, then it must be
|
||||
explicitly listed. Default: the empty list (all projects may use
|
||||
the job).
|
||||
|
||||
|
||||
.. _project:
|
||||
|
||||
Project
|
||||
~~~~~~~
|
||||
|
||||
A project corresponds to a source code repository with which Zuul is
|
||||
configured to interact. The main responsibility of the `Project`
|
||||
configuration item is to specify which jobs should run in which
|
||||
pipelines for a given project. Within each `Project` definition, a
|
||||
section for each `Pipeline` may appear. This project-pipeline
|
||||
definition is what determines how a project participates in a
|
||||
pipeline.
|
||||
|
||||
Consider the following `Project` definition::
|
||||
|
||||
- project:
|
||||
name: yoyodyne
|
||||
check:
|
||||
jobs:
|
||||
- check-syntax
|
||||
- unit-tests
|
||||
gate:
|
||||
queue: integrated
|
||||
jobs:
|
||||
- unit-tests
|
||||
- integration-tests
|
||||
|
||||
The project has two project-pipeline stanzas, one for the `check`
|
||||
pipeline, and one for `gate`. Each specifies which jobs shuld run
|
||||
when a change for that project enteres the respective pipeline -- when
|
||||
a change enters `check`, the `check-syntax` and `unit-test` jobs are
|
||||
run.
|
||||
|
||||
Pipelines which use the dependent pipeline manager (e.g., the `gate`
|
||||
example shown earlier) maintain separate queues for groups of
|
||||
projects. When Zuul serializes a set of changes which represent
|
||||
future potential project states, it must know about all of the
|
||||
projects within Zuul which may have an effect on the outcome of the
|
||||
jobs it runs. If project *A* uses project *B* as a library, then Zuul
|
||||
must be told about that relationship so that it knows to serialize
|
||||
changes to A and B together, so that it does not merge a change to B
|
||||
while it is testing a change to A.
|
||||
|
||||
Zuul could simply assume that all projects are related, or even infer
|
||||
relationships by which projects a job indicates it uses, however, in a
|
||||
large system that would become unwieldy very quickly, and
|
||||
unnecessarily delay changes to unrelated projects. To allow for
|
||||
flexibility in the construction of groups of related projects, the
|
||||
change queues used by dependent pipeline managers are specified
|
||||
manually. To group two or more related projects into a shared queue
|
||||
for a dependent pipeline, set the ``queue`` parameter to the same
|
||||
value for those projects.
|
||||
|
||||
The `gate` project-pipeline definition above specifies that this
|
||||
project participates in the `integrated` shared queue for that
|
||||
pipeline.
|
||||
|
||||
In addition to a project-pipeline definition for one or more
|
||||
`Pipelines`, the following attributes may appear in a Project:
|
||||
|
||||
**name** (required)
|
||||
The name of the project. If Zuul is configured with two or more
|
||||
unique projects with the same name, the canonical hostname for the
|
||||
project should be included (e.g., `git.example.com/foo`).
|
||||
|
||||
**templates**
|
||||
A list of :ref:`project-template` references; the project-pipeline
|
||||
definitions of each Project Template will be applied to this
|
||||
project. If more than one template includes jobs for a given
|
||||
pipeline, they will be combined, as will any jobs specified in
|
||||
project-pipeline definitions on the project itself.
|
||||
|
||||
.. _project-template:
|
||||
|
||||
Project Template
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
A Project Template defines one or more project-pipeline definitions
|
||||
which can be re-used by multiple projects.
|
||||
|
||||
A Project Template uses the same syntax as a :ref:`project`
|
||||
definition, however, in the case of a template, the ``name`` attribute
|
||||
does not refer to the name of a project, but rather names the template
|
||||
so that it can be referenced in a `Project` definition.
|
||||
|
||||
.. _secret:
|
||||
|
||||
Secret
|
||||
~~~~~~
|
||||
|
||||
A Secret is a collection of private data for use by one or more jobs.
|
||||
In order to maintain the security of the data, the values are usually
|
||||
encrypted, however, data which are not sensitive may be provided
|
||||
unencrypted as well for convenience.
|
||||
|
||||
A Secret may only be used by jobs defined within the same project. To
|
||||
use a secret, a :ref:`job` must specify the secret within its `auth`
|
||||
section. To protect against jobs in other repositories declaring a
|
||||
job with a secret as a parent and then exposing that secret, jobs
|
||||
which inherit from a job with secrets will not inherit the secrets
|
||||
themselves. To alter that behavior, see the `inherit` job attribute.
|
||||
Further, jobs which do not permit children to inherit secrets (the
|
||||
default) are also automatically marked `final`, meaning that their
|
||||
execution related attributes may not be changed in a project-pipeline
|
||||
stanza. This is to protect against a job with secrets defined in one
|
||||
project being used by another project in a way which might expose the
|
||||
secrets. If a job with secrets is unsafe to be used by other
|
||||
projects, the `allowed-projects` job attribute can be used to restrict
|
||||
the projects which can invoke that job. Finally, pipelines which are
|
||||
used to execute proposed but unreviewed changes can set the
|
||||
`allow-secrets` attribute to indicate that they should not supply
|
||||
secrets at all in order to protect against someone proposing a change
|
||||
which exposes a secret.
|
||||
|
||||
The following attributes are required:
|
||||
|
||||
**name** (required)
|
||||
The name of the secret, used in a :ref:`Job` definition to request
|
||||
the secret.
|
||||
|
||||
**data** (required)
|
||||
A dictionary which will be added to the Ansible variables available
|
||||
to the job. The values can either be plain text strings, or
|
||||
encrypted values. See :ref:`encryption` for more information.
|
||||
|
||||
.. _nodeset:
|
||||
|
||||
Nodeset
|
||||
~~~~~~~
|
||||
|
||||
A Nodeset is a named collection of nodes for use by a job. Jobs may
|
||||
specify what nodes they require individually, however, by defining
|
||||
groups of node types once and referring to them by name, job
|
||||
configuration may be simplified.
|
||||
|
||||
A Nodeset requires two attributes:
|
||||
|
||||
**name** (required)
|
||||
The name of the Nodeset, to be referenced by a :ref:`job`.
|
||||
|
||||
**nodes** (required)
|
||||
A list of node definitions, each of which has the following format:
|
||||
|
||||
**name** (required)
|
||||
The name of the node. This will appear in the Ansible inventory
|
||||
for the job.
|
||||
|
||||
**label** (required)
|
||||
The Nodepool label for the node. Zuul will request a node with
|
||||
this label.
|
||||
|
||||
.. _semaphore:
|
||||
|
||||
Semaphore
|
||||
~~~~~~~~~
|
||||
|
||||
Semaphores can be used to restrict the number of certain jobs which
|
||||
are running at the same time. This may be useful for jobs which
|
||||
access shared or limited resources. A semaphore has a value which
|
||||
represents the maximum number of jobs which use that semaphore at the
|
||||
same time.
|
||||
|
||||
Semaphores are never subject to dynamic reconfiguration. If the value
|
||||
of a semaphore is changed, it will take effect only when the change
|
||||
where it is updated is merged. An example follows::
|
||||
|
||||
- semaphore:
|
||||
name: semaphore-foo
|
||||
max: 5
|
||||
- semaphore:
|
||||
name: semaphore-bar
|
||||
max: 3
|
||||
|
||||
The following attributes are available:
|
||||
|
||||
**name** (required)
|
||||
The name of the semaphore, referenced by jobs.
|
||||
|
||||
**max**
|
||||
The maximum number of running jobs which can use this semaphore.
|
||||
Defaults to 1.
|
Loading…
Reference in New Issue