For very large projects, when Gerrit is under load, it can take
longer than 60 seconds for the submit action to be processed and the
associated commit to replicate to its local mirror, causing Zuul to
think the change did not merge and resetting any remaining changes
in that dependent queue. Increase this to 5 minutes, since the
impact from these "phantom gate resets" is more significant than any
potential increase in merge wait time.
Change-Id: If57d5ed2720707c1ec3a3477dd229f042789fa1f
Adds missing periods in item descriptions to bring in line with
the rest of the documentation. Also fixes some typos and styling.
Change-Id: I11ef0bf3f89ecf5f09bb5781cced9b655c71bc65
Zuul creates fake Build objects for jobs that it skips due to
merge conflicts and other such situations. If Zuul decides to
cancel such a build, we should handle it gracefully instead of
throwing an exception.
Also log received gearman job handles to better correlate builds
with gearman debug logs.
Change-Id: I1cd14c8249ac4c68e797430b2b4941f2eeee2acf
We don't really offer a facility to match on non-review category
approvals (like SUBM) so ignore those when matching events to
triggers. Otherwise, matching logic that expects only review
categories will be fooled by submit or other similar approval
types.
Fixes-bug: 1284842
Fixes-bug: 1285210
Change-Id: I68b58f611df8fc2b9b2753d1579eae2e6898acc9
To keep Python 3.x compatibility, use six.moves.urllib.parse
to replace urlparse.
Add six as a dependency in test_requirements.txt
Change-Id: Ia6cd042970c2d4e5d418a39e25bab32b4d93c544
If Zuul is disconnected from gearman, have Zuul retry the job.
Also, report slightly more useful status words for various kinds
of failures when able.
Change-Id: I2d0f2986199516f6251bb122ab619ac56a87ee8e
Gear 0.5.0 added statsd support for the gear server. Have Zuul
use it based on the existence of the same env vars Zuul uses
to configure its own use of statsd.
Change-Id: I09ea2541d1f2eff5f4fd2e0c78da2acfdc1776ab
When jobs share gates, Zuul has a nice feature which is to creates
reference of the Change in every involved repositories. That was sadly
undocumented.
Reference:
James E. Blair on openstack-infra mailing list:
http://lists.openstack.org/pipermail/openstack-infra/2014-January/000605.html
Change-Id: Ia3bd640f6a90e12192af1686547a5f4146192dbe
* requirements.txt: The latest release of the gear library fixes an
enqueuing/function registration order problem exposed by the recent
zuul merger fan-out implementation. Insist upon it.
Change-Id: Id2275477b12ce62fc485e5769029fe107a82b52d
The "building_jobs" dictionary on pipeline managers is no longer
necessary because the object data model is sufficient to find all
of the running builds. Remove the unnecessary second list.
Some of the protections that checked that builds were in this
dictionary are also no longer needed as it is not possible to
receive a result for a build that the running Zuul did not
request since the change to use Gearman.
Change-Id: I10de7edcf8356885f5f2d2f45d7014582bcbc3cf
Some git merge commands fails for me with an AssertionError coming from
GitPython:
AssertionError: len([
"5a0a437ee9a6fd4b30484dfdc2455c83ea00a058\t\t'refs/changes/80/111180/1'
of ssh://gerrit.wikimedia.org:29418/mediawiki/core\n"]
)
!= len([
'Total 10 (delta 9), reused 10 (delta 9)',
' * branch refs/changes/80/111180/1 -> FETCH_HEAD'
])
Zuul.Merger explicitly log merge exceptions at debug level, though we
should only load GitCommandError exceptions and properly log other
exceptions (such as AssertionError).
Change-Id: Idca682c0920b9e6eac3f12cc05634dfe304a0305
Connect it to Zuul via Gearman. Any number of mergers may be
deployed.
Directly find the pipeline for a build when processing a result,
so that the procedure is roughly the same for build and merge
results.
The timer trigger currently requires the gerrit trigger also be
configured. Make that explicit inside of the timer trigger so
that the scheduler API interaction with triggers is cleaner.
Change-Id: I69498813764753c97c426e42d17596c2ef1d87cf
Instead of processing all of the queues each time any event happens,
first process all of the events (which are really queued up just so
that they can modify the pipeline data safely within the main run loop)
and then run the queue processor (which will then launch or cancel jobs
based on the new state of the queues).
This adds a lock to the run processor which is only needed so that
the test suite can safely inspect the state of the scheduler to
determine whether it is idle.
The result queue is made somewhat more generic in preparation for
handling merge result events.
Change-Id: I6bbb52f77b070df04a110a9d61b426265b1e89cc
In order to split out the merger as a new component, remove any
direct Zuul model object manipulation, instead passing in a
lists of dictionaries for merge instructions. Change the merger
algorithm so that it is compatible with this new method and makes
no assumptions about whether it has merged any changes previously.
There is likely to be a minor performance impact as some information
which was previously kept in-memory will now be fetched from the git
index.
The merger is also no-longer pre-populated with clones of git repos
at startup. Some tests were adjusted to accomodate this.
Change-Id: I3df86ae36b4969d568d1fb03df1e6569553d1226
To help with debugging, allow workers to send back information about
itself such as hostname and ips. This will allow for an RPC/Client
call to get the information on running jobs.
Change-Id: I03b553293923cd65b3a9651a8877aa23688d2909
* tools/trigger-job.py: Add a --url option which passes the required
ZUUL_URL parameter with a sane default for OpenStack Infra's
environment.
Change-Id: I2d8aa73b11b0069e31ffa599c1c9d9059919fb7a
When developping or wondering why Zuul is stuck/having a problem, I
found it very useful to know which tasks it already started. This patch
adds a logger to the Server() class and log informational message before
most components are started.
Change-Id: I19d0d439c50b42ad6217d92654b9d5e2558d9a7f
This reverts commit 87650fa736.
The problem of distributing the load of serving Zuul Git refs will
be addressed by separating the merger component so that it can be
scaled out. Remove this feature which is hopefully new enough that
no one is using it.
Change-Id: Id2be95c85f5c3464a66537ae3095024a964ee1c0
This is in preparation to move to separate merger process, which
will not have access to the trigger. Similar functionality can
be obtained using the new replication feature, and docs have been
updated to that effect.
Change-Id: I9397f7eb8466af464c8e7adb02d0a8d3eff04f9f
argparse does not support optional subparser, hence the `zuul` client
command would always require to be passed a sub command. That prevented
`zuul --version` from showing up with a 'too few arguments' error.
I found out add_argument() has a action='version' which would print the
content of the 'version' argument and exit.
Change-Id: I553d3b2ab7aa04865e44559135adc5a7640307ed
In order to render queue window information nicely in status pages pass
the window for each queue in the JSON status blob. Also, annotate each
change in a queue with an active flag indicating whether or not it is
currently within the window.
Change-Id: Iccba9fb307f98a4fc145402ecd0a38f3d75b3253
This feature allows Zuul to consider existing (or new) approval
votes associated with a change when determining whether an event
matches. For example, it can be used to require that a Verified
vote of a certain age be present before a change is enqueued in
a pipeline.
Change-Id: I81344713d71b345b08576334568b9c49c810c7e9
Jenkins
Jeremy Stanley
Łukasz Jernaś
James E. Blair
tanlin
Antoine Musso
Joshua Hesketh
Akihiro Motoki
Clark Boylan