The Zuul admin can configure authenticators with an optional
"max_validity_time" field, which is the maximum age in seconds
for a valid authentication token. By default there is no
maximum age set for tokens, except the one deduced from
the token's "exp" claim.
If "max_validity" is set, tokens without an "iat" claim will
be rejected.
This is meant as an extra security to avoid accidentally issueing
very long lived tokens through the CLI.
The "skew" field can be used to mitigate clocks discrepancies
between Zuul and a JWT emitter.
Change-Id: I9351ca016b60050b5f3b3950b840d5f719e919ce
Record the number of attempts zuul has made to run this job in the job
inventory. This will help expose reliability information in job logs and
in job log indexers. We want to try and expose job reliability as much
as possible and this is one way to do that.
Change-Id: I2f7c31ce510f59bc569c5db89ce6626d9e3ef436
This adds the ability to specify a different ansible install path when
running zuul-manage-ansible. Venv location list is also reversed so that
we prefer the venvs at the user specified location if present.
Finally docs are updated to more clearly specify there are two options
for managing ansible installations and why you should prefer
zuul-manage-ansible.
Change-Id: If84a92bc542e24eedac09f3dc9fd2954db43734c
We have noticed that some quickstart jobs fail due to timeouts waiting
to connect to mariadb. On further investigation I've found that this
slowness is related to bootstrapping of TZINFO tables on initial startup
[0][1].
The workaround for this problem that I've chosen is to disable this
TZINFO generation on startup. Other options include using mariadb 10.4.7
(10.4.8 introduced the issue upstream), switching to mysql or percona,
or increasing out timeout to cover the longer startup time (probably
want to set the timeout around 5 minutes).
[0] https://github.com/docker-library/mariadb/issues/261
[1] https://github.com/docker-library/mariadb/issues/262
Change-Id: Ib27681c1e5ac25a0a5f75a6b1bfb24029d0be3c8
I explicitly maintain a <= 1 year expiration on my public OpenPGP
key and bump it periodically. As a result, the convenience export we
publish should also be refreshed with some regularity so that an old
expired version is not served in error. Update my key here with a
more recent export.
Change-Id: I41b1292dfd2c8743e0cedfb5df1239201b318395
In some cases, especially on systems under heavy load, it is
helpful to start executors in paused mode. Preventing them
to start accepting new jobs right away until such executors
are unpaused manually allows to test new features, configuration
or with analysing production problems.
Change-Id: I64c39e3b58c802577201280c855fdf7f13cc7538
Updates the environment variable processing to only affect variables
prefixed with ZUUL_.
Adds a test showing the os.environ with % in it.
This reverts commit b3929b5633.
Change-Id: Ic6c3dd0327ef70dc1375486827e4503a4cea9bfc
In prod for OpenDev we're seeing things like this:
http://paste.openstack.org/show/785704/
which lead us to believe this is somehow connected.
This reverts commit f2229705f3.
Change-Id: I0b73b71f72483e6c6e511411c3c59729761cec9b
Mirror the GitHub reporter's "comment" option to allow for supressing
review messages when reporting to Gerrit. This is needed for use with
the checks api since the purpose is to avoid leaving job results in
comments. This removes the ability for 'comment' to be a review label,
which seems unlikely, but plausible, so a release note is added.
Rename the "checks_api" Gerrit reporter option to "checks-api" for
hyphen consistency with the rest of the configuration. This is an
experimental feature and not generally useful yet, so no release note
is included. This change supports both forms and updates the
documentation; a later change will remove "checks_api".
While adding documentation for the "comment" option, the description
of the Gerrit reporter is updated.
A small refactoring of the Gerrit reporter is undertaken to make
dealing with additional non-label options easier.
Change-Id: I44c0ba93601f3be3eb0f094bd782cecc4a0372f7
Add the correct libre2 package name for Debian buster, and also
update the quickstart playbook and documentation to deal with
the change in default rsa key encoding format from newer
versions of ssh-keygen.
Change-Id: I6ada88cd896d844c1171f7bcaf4691dea023d51f
This will allow users to set environment variables with sensitive
strings like passwords, but keep a single config file. This comes
in handy when using Kubernetes in particular, as it wants to
handle sensitive data and templated config files in a very different
manner.
Change-Id: I38f6c4da82e1647ad197908f19ea6df23e04fc32
The nodepool "python-path" config variable makes it's way through from
the node arguments and ends up as the "ansible_python_interpreter"
variable for the inventory when running the job.
Notably, Python 3 only distributions require this to be set to
/usr/bin/python3 to avoid what can often be confusing red-herring
errors (e.g. things like dnf packages incorrectly appearing to be
missing on Fedora, for example [1]).
Upstream is aware of this often confusing behaviour and has made an
"ansible_python_interpreter" value of "auto" to, essentially, "do the
right thing" [2] and choose the right python for the target
environment. This is available in Ansible >=2.8 and will become
default in 2.12.
This allows, and defaults to, an interpreter value of "auto" when
running with Ansible >=2.8. On the supported prior Ansible releases,
"auto" will be translated into "/usr/bin/python2" to maintain
backwards compatability. Of course a node explicity setting
"python-path" already will override this.
Nodepool is updated to set this by default with
I02a1a618c8806b150049e91b644ec3c0cb826ba4.
I think this is much more user friendly as it puts the work of
figuring out what platform has what interpreter into Ansible. It
alleviates the need for admins to know anything at all about
"python-path" for node configurations unless they are actually doing
something out of the ordinary like using a virtualenv. At the moment,
if you put a modern Python-3 only distro into nodepool, Zuul always
does the wrong thing by selecting /usr/bin/python2; you are left to
debug the failures and need to know to go and manually update the
python-path to Python 3.
Documentation is updated. Detailed discussion is moved into the
executor section; the README is simplified a bit to avoid confusion.
A release note is added.
A test-case is added. Note that it is also self-testing in that jobs
using Ansible 2.8 use the updated value
(c.f. I7cdcfc760975871f7fa9949da1015d7cec92ee67)
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1696404
[2] https://docs.ansible.com/ansible/2.8/reference_appendices/interpreter_discovery.html
Change-Id: I2b3bc6d4f873b7d653cfaccd1598464583c561e7
This adds max_hold_expiration and default_hold_expiration as
scheduler options.
max_hold_expiration sets the absolute maximum age, in seconds,
a node placed in the hold state will remain available. This
defaults to 0, which means there is no maximum.
default_hold_expiration sets the default value used if no value
is supplied. This defaults to max_hold_expiration.
Change-Id: Ia483ac664e0a2adcec9efb29d3d701f6d315ef3b
Without this requirement it was possible to run jobs via
a recheck on closed pull requests. Recommended pipelines
should not allow that.
Change-Id: I9693d9ab906024137d48b4d473d1292a383d9e65
This change implements event handling for pull-request.tags.added.
Tags can be used as trigger event filter or required metadata.
Change-Id: I128bbef34245932e3bbee1f848ad1c484d3ccae3
The current location of the reference pipelines is within the
quick-start repo, which causes all sorts of configuration errors
to be logged (but since Zuul is so tolerant of them, the job
doesn't actually fail).
Move them outside of one of the demo project directories into their
own.
Change-Id: I9cb1ddd803d938fc154a32308cae99dbab9392e2
This adds initial support for the Gerrit checks plugin.
Development of that plugin is still in progress, and hopefully it
(and our support for it) will change over time. Because we expect
to change how we interact with it in the near future, this is
documented as experimental support for now. A release note is
intentionally omitted -- that's more appropriate when we remove
the 'experimental' label.
Change-Id: Ida0cdef682ca2ce117617eacfb67f371426a3131
This facilitates integration with the gerrit checks API (and may
prove useful for other similar APIs). It will allow us to report
that a change has no jobs in a particular pipeline. A Zuul
pipeline will correspond to a Gerrit check, which means we can
update the status for that check from "SCHEDULED" to "NOT_RELEVANT"
if we determine that no jobs should run for the change. This
closes out the status of the check in Gerrit when a project is
configured to participate in a check/pipeline but no jobs are
actually configured.
Test coverage for this will be added in change
Ida0cdef682ca2ce117617eacfb67f371426a3131.
Change-Id: Ide2a332b294d7efe23601d80eeb92b5af1d4c21b
This facilitates integration with the gerrit checks API (and may
prove useful for other similar APIs). It will allow us to report
that a change has been enqueued in a particular pipeline. A Zuul
pipeline will correspond to a Gerrit check, which means we can
update the status for that check from "NOT_STARTED" to "SCHEDULED"
when it enters the pipeline. This is important for our check
polling loop, and it will cause that check to stop appearing in
the list of pending checks.
Test coverage for this is added in change
Ida0cdef682ca2ce117617eacfb67f371426a3131.
Change-Id: I9ec329b446fa51e0911d4d9ff67eea7ddd55ab5d
New command for the zuul CLI client to retrieve autohold details.
Currently, the only new information included is the 'current_count'
field, but this will later be extended to include held nodes.
Change-Id: Ieae2aea73123b5467d825d4738be07481bb15348
Storing autohold requests in ZooKeeper, rather than in-memory,
allows us to remember requests across restarts, and is a necessity
for future work to scale out the scheduler.
Future changes to build on this will allow us to store held node
information with the change for easy node identification, and to
delete any held nodes for a request using the zuul CLI.
A new 'zuul autohold-delete' command is added since hold requests
are no longer automatically deleted.
This makes the autohold API:
zuul autohold: Create a new hold request
zuul autohold-list: List current hold requests
zuul autohold-delete: Delete a hold request
Change-Id: I6130175d1dc7d6c8ce8667f9b14ae9377737d280
This expands the discussion of executor-only jobs with some additional
notes.
Additionally a unit test is added to explicitly test executor-only
(i.e. blank nodeset) jobs.
Change-Id: I8fd2f932290e49da5a3605737e8940425cd092f4
This change adds a zuul.conf option to disable the global merge
jobs from running on an executor node.
Change-Id: Icd6374a6c97a404662b39de9df54f4b7c5ab36aa
As it has been done for the Pagure driver, include
a reference pipelines section in the Gerrit driver
documentation. It helps the reader to better
understand the big picture of the Gerrit's pipelines
configuration but also provide an almost ready to use
pipelines definition file.
Change-Id: I5234476f7393ba938e8365b8c908a2ff6f9e8c76
This was missing and a user following this doc ran into exceptions
caused by git errors which happened because the git user was not
configured.
Change-Id: Iab50d58de8e798e63e8ed0459a903ebe60cecf5d
We need the js tools to be present so that zuul from scratch install
docs produce a working javascript build. Add a step to that process to
run our helper script for js toolchain install.
Note this adds support for opensuse to that script as opensuse is a
valid from scratch install target.
Change-Id: I4c3584957f7a4afaeb0315a57d56f93e915eace1
An executor is accepting up to twice as many starting builds as defined
by the load_multiplier option. On system with high CPU/vCPU count an
executor may accept too many starting builds. This can be overwritten
using a new max_starting_builds option.
Change-Id: Ic7c121e795e4e3cecec25b2b06dd1a26aa798439
Matthieu Huin