The Zuul admin can configure authenticators with an optional
"max_validity_time" field, which is the maximum age in seconds
for a valid authentication token. By default there is no
maximum age set for tokens, except the one deduced from
the token's "exp" claim.
If "max_validity" is set, tokens without an "iat" claim will
be rejected.
This is meant as an extra security to avoid accidentally issueing
very long lived tokens through the CLI.
The "skew" field can be used to mitigate clocks discrepancies
between Zuul and a JWT emitter.
Change-Id: I9351ca016b60050b5f3b3950b840d5f719e919ce
The patchset or ref, pipeline and project should be enough to trigger an
enqueue. The trigger argument is not validated or used anymore when
enqueueing via RPC.
Change-Id: I9166e6d44291070f01baca9238f04feedcee7f5b
Return the hold request IDs returned from the zookeeper API as
a sorted list so that they will appear in sorted order in the
zuul CLI output.
Change-Id: I3a3d738ac2bebb8b446cb0710bf9f5452c232372
Prior to this change we looked for the current change/PR's url in any
other change/PR's message body. This meant any cross referencing of urls
would create further lookups to determine if there was a real dependency
there. Restrict this a bit more to require the Depends-On string too
when searching to limit the number of spidering queries that must be
done.
This is particularly useful for the github driver because queries are
expensive there and may be rate limited.
Change-Id: Ie49fe1a72dc844b14003d942684fd3d2a9478d21
Record the number of attempts zuul has made to run this job in the job
inventory. This will help expose reliability information in job logs and
in job log indexers. We want to try and expose job reliability as much
as possible and this is one way to do that.
Change-Id: I2f7c31ce510f59bc569c5db89ce6626d9e3ef436
We see cases where the github driver uses the search API to learn more
about an PR but ends up return information about other projects not in
Zuuls configuration. In some cases, this results in zuul getting
rate-limited by github, which can block event processing.
Change-Id: Ie09eee17e1adf87c212da240eeaa8c098d48a1d6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
If change B has a job J1 which requires something provided by a
dependent change A, and there is another job J2 on change B which
has a dependency on J1, then if the providing job on change A fails,
it is possible for change B to get stuck in the queue waiting for
J2 to start.
This is because when Zuul detects the requirement failure of J1,
it sets the result of J1 immediately, but did not call the 'setResult'
method which is responsible for marking child jobs as skipped. This
change adds that call which will then change the result of J2 to
skipped when J1 fails the requirement check.
This change also updates a test fixture to describe this case. It
is covered by test_provides_requires_check_old_failure. Several
other test cases are updated merely to accomodate the new job since
they share the fixture.
Change-Id: Ic541cb3ff6f811ca3a5a543c04321714ba965d24
Story: 2006988
Task: 37752
In some cases, Zuul records warning messages to accompany the
build report. These typically constitute decisions Zuul has made
which affect execution (or non-execution) of jobs, so log them
at info level so that admins have at least as much context as
end-users.
Change-Id: I73a0f6d7878873c259207a75d8b8ba1374ddc7b7
Having the zuul event id available in the database and also in the build
and buildset detail page makes debugging a lot easier.
Change-Id: Ia1e4aaf50fb28bb27cbcfcfc3b5a92bba88fc85e
The build result in case it should be retried is None. This was showing
up as metric name:
zuul.tenant.*.pipeline.*.project.*.*.*.job.<jobname>.None
This is now fixed to have a better metric name:
zuul.tenant.*.pipeline.*.project.*.*.*.job.<jobname>.RETRY
Change-Id: I4ff8f1cf2e02091f275acb9fbcb628bae014d0aa
When the queue for a project is changed during reconfiguration the
dependent pipeline manager was using the queue of the item
ahead in the old queue.
This lead to items ending up in the wrong queues and caused the
following exception in the run handler:
2019-11-28 13:33:47,313 zuul.Scheduler ERROR Exception in run handler:
Traceback (most recent call last):
File "/tmp/zuul/zuul/scheduler.py", line 1145, in run
while (pipeline.manager.processQueue() and
File "/tmp/zuul/zuul/manager/__init__.py", line 914, in processQueue
item, nnfi)
File "/tmp/zuul/zuul/manager/__init__.py", line 898, in _processOneItem
priority = item.getNodePriority()
File "/tmp/zuul/zuul/model.py", line 2684, in getNodePriority
return self.pipeline.manager.getNodePriority(self)
File "/tmp/zuul/zuul/manager/dependent.py", line 101, in getNodePriority
return items.index(item)
ValueError: <QueueItem 0x7f02583af4e0 for <Change 0x7f02583a9cf8 org/project2 2,1> in gate> is not in list
The issue is fixed, by ignoring the given existing queue in the
dependent pipeline manager, since it's always possible to get the
correct queue from the pipeline itself.
Change-Id: Ia5b1b58377e4420b9ab1440c0b9f67cb15967263
When trying to trace logs of builds it is often useful to search for log
messages via the event id of a specific build.
The event id is printed in (nearly) all log messages but is not provided
by the MQTT reporter, so one has to look it up first based on the build
id. To circumvent this extra step and make searching the logs more
straight forward, this patch makes sure the event id is provided in the
JSON message by the MQTT reporter.
Change-Id: I908dd7eca250825eed97bf8261fd33b69cc5f543
Threshold_reached will be None (in the API) if no minimal score
to merge PR is configured. So the value needs to be handled
as True because the threshold is by default reached in that case.
Change-Id: I103c9f601e5362f451647fb3597fa790df481dd6
This change prevents an exception from happening in the zuul_stream
module when the msg result is a list instead of a string.
This is the case for the package module trying to install missing rpm
on ansible 2.8 with python3.
Change-Id: I242711ebc0b2dcd1262de38e76efc48554361bd1
Story: 2006830
This adds the ability to specify a different ansible install path when
running zuul-manage-ansible. Venv location list is also reversed so that
we prefer the venvs at the user specified location if present.
Finally docs are updated to more clearly specify there are two options
for managing ansible installations and why you should prefer
zuul-manage-ansible.
Change-Id: If84a92bc542e24eedac09f3dc9fd2954db43734c
We have noticed that some quickstart jobs fail due to timeouts waiting
to connect to mariadb. On further investigation I've found that this
slowness is related to bootstrapping of TZINFO tables on initial startup
[0][1].
The workaround for this problem that I've chosen is to disable this
TZINFO generation on startup. Other options include using mariadb 10.4.7
(10.4.8 introduced the issue upstream), switching to mysql or percona,
or increasing out timeout to cover the longer startup time (probably
want to set the timeout around 5 minutes).
[0] https://github.com/docker-library/mariadb/issues/261
[1] https://github.com/docker-library/mariadb/issues/262
Change-Id: Ib27681c1e5ac25a0a5f75a6b1bfb24029d0be3c8
This code had a bug -- it didn't build the full path.
This code was not tested.
These two things are related.
Change-Id: I7881fb30017cedc12435e0fcbfda321bdf20d611
This change prevent issue when the tree component is not able
to render a child attached to multiple parents.
Change-Id: Iec0de04bc91deb8642afa3a7d0ba023b3eadf8b4
The pre.yaml playbook for the zuul-stream-functional tests copies
the Ansible inventory.yaml file from the executor to the test
"controller" node. The controller then runs the specified version
of Ansible 2.x against the other nodes. This fails because the
executor version of inventory.yaml contains "auto" for the Ansible
python interpreter which is valid under the version of Ansible used
on the executor, but on the controller node, which runs the older
versions of Ansible, this is *not* a valid value. Thus it fails.
This change forces the executor to use the version of Ansible being
tested on the controller so that the inventory.yaml will be correct.
Also, Ansible 2.8 now throws a FileNotFoundError exception instead
of OSError when a referenced file is not found.
Change-Id: Ibd31f1161df0076ed7498fd1d7b1ae76c802c6e4
Mocked issue search was broken in multiple ways:
- tokenize() was wrongly splitting search modifieres (e.g. type:pr)
into tokens (e.g. [type, pr])
- code for removing search modifiers (type, is, in) from terms was
never reached and also used set(...) in a wrong way
- set intersection of search terms and body doesn't make any sense since
this will almost always have ANY overlap
Simply extracting the URLs from the query and checking for in PR body
should make the mock work for most of the tests.
Change-Id: I9f896af85e21770bba80857511aae8505f3e5b84
Otherwise messages from zuul and gerrit loggers are printed
twice: from the logger and from root logger.
Change-Id: I377758d4b70e250f8b2ba281cd0dabcf12555b62
I explicitly maintain a <= 1 year expiration on my public OpenPGP
key and bump it periodically. As a result, the convenience export we
publish should also be refreshed with some regularity so that an old
expired version is not served in error. Update my key here with a
more recent export.
Change-Id: I41b1292dfd2c8743e0cedfb5df1239201b318395
An authenticated user can query this endpoint to get an authorization
tree, letting her know which actions are available to her. This is
useful for frontends.
Change-Id: Ibda4eabe496f2c37a17a8ce2a2acfcf3e4cb97e3
In some cases, especially on systems under heavy load, it is
helpful to start executors in paused mode. Preventing them
to start accepting new jobs right away until such executors
are unpaused manually allows to test new features, configuration
or with analysing production problems.
Change-Id: I64c39e3b58c802577201280c855fdf7f13cc7538
Updates the environment variable processing to only affect variables
prefixed with ZUUL_.
Adds a test showing the os.environ with % in it.
This reverts commit b3929b5633.
Change-Id: Ic6c3dd0327ef70dc1375486827e4503a4cea9bfc
In prod for OpenDev we're seeing things like this:
http://paste.openstack.org/show/785704/
which lead us to believe this is somehow connected.
This reverts commit f2229705f3.
Change-Id: I0b73b71f72483e6c6e511411c3c59729761cec9b
Thanks to:
- https://pagure.io/pagure/pull-request/4625 event branch creation
- https://pagure.io/pagure/pull-request/4626 oldrev for branch update
This patch:
- Handles git.branch.creation and git.branch.deletion events.
- Adds support for old_commit event attribute.
It also removes unexisting stop_commit event atribute handling,
it is only end_commit.
Change-Id: I10423b654df48436bb0ad11c2043c2d67cac7780
Since https://pagure.io/pagure/issue/4400 Pagure sends an event
when a git repo is tagged. This patch brings the support
for the event.
Change-Id: I9e09cd6dd409f9442763a7d765e2ce261bb42843
Matthieu Huin