* zuul/trigger/gerrit.py(Gerrit.isMerged): Make sure to return False
if we fell through all the conditionals, rather than defaulting to
None. This should have no current impact since all comparisons to
the returned value elsewhere are boolean and not "is None" or
similar.
Change-Id: I05360833f1e5b35fbfb1894dadcb46e82ff78a11
It's possible for a test to complete before python gets around to
actually starting the webapp run thread. Make sure that the
object referenced in the stop method is always instantiated to
avoid "AttributeError: 'WebApp' object has no attribute 'server'"
errors.
Also, mark the webapp thread as daemon on general principle.
Change-Id: I9b4c0c20a98d5429c299748178e31baf8b6912d9
Canceling a build is naturally subject to race conditions as the
build is started (since canceling a build is different depending
on whether or not it has started). This has been handled by
checking whether the build has started, and if so canceling the
build, if not, removing it from the queue, and if that fails,
checking to see if the build just started and if so, canceling
that.
But even that is still racy because it's possible for the build
to have started but for zuul to not have received the gearman
packet indicating that it had. To handle that, simply don't check
whether the build has started for the third attempt.
(The reason we even check at all before the first attempt is that
canceling a running build in jenkins is somewhat expensive (it
involves iterating over all the builds) so it's better to avoid
that if we think it won't work.)
Also, add an extra check in the unit test suite when deciding
whether the system has settled. This should deal with the case
that a trigger_event -> job transition is happening during the
haveAllBuildsReported check (which only checks jobs).
Change-Id: I60018a5215e7d8230bdf6ef67ec7bc9c719fc286
In particular, shut it down after all its clients have been shut
down. In some cases, it was shut down early enough that the RPC
listener was able to attempt to reconnect to it and trigger a
rare condition in gear that would cause the reconnection thread
to hang.
Change-Id: I1dc5d77484eabf9fa43f16ae8295ddb4df4af056
Add logging to the zuul merger process that includes change numbers so
that threads of execution can be easily tracked from the zuul server log
files to the zuul merger log files.
Change-Id: Ic591f3941d6fe84874e16c8309a101a2918979a2
The init script for the zuul-merger file does not look for the
zuul-merger PIDFILE in the same default location that the zuul-merger
uses. Make them both look at /var/run/zuul-merger/zuul-merger.pid.
Change-Id: Ia5829ef3887a5651d23a80dbd00e76972c7cf6a7
Zuul send different sets of parameters depending on the event that
triggered a build, split the parameters in sub sub sections for some
clarity.
Added examples of ref-updated parameters.
Sub-sub section for the jenkins configuration.
Change-Id: I1edcd5c3a101e2afe0025ad4cd3e511582998999
* zuul/trigger/gerrit.py(Gerrit._isMerged): Only consider MERGED
status to indicate a change is successfully merged. SUBMITTED can
exist on changes which Gerrit was unable to merge due to nonexistent
parent/dependent commits, and possibly for other reasons too.
Change-Id: I673be5010031f8bc26ab31db9afcc7c28854db85
For very large projects, when Gerrit is under load, it can take
longer than 60 seconds for the submit action to be processed and the
associated commit to replicate to its local mirror, causing Zuul to
think the change did not merge and resetting any remaining changes
in that dependent queue. Increase this to 5 minutes, since the
impact from these "phantom gate resets" is more significant than any
potential increase in merge wait time.
Change-Id: If57d5ed2720707c1ec3a3477dd229f042789fa1f
Currently there is no way to add extra information to the end of a
report sent back by zuul. This adds a 'footer-message' field to the
pipelines allowing operators to leave extra information at the end
of a build result.
This is useful for leaving CI contact and debugging information.
Change-Id: Ieae62e845915fa3997353b6b425e215966a9338c
In a dependent pipeline with changes to multiple branches of
a single project, ZUUL_COMMIT would sometimes point to a commit
further back in the series rather than the commit that should
be tested (ie, not the commit pointed to by ZUUL_REF). It's
possible this situation could appear in other circumstances as
well.
The error was the re-use of a variable when creating Zuul refs
for the entire series. The variable should have been reserved
to indicate the final commit but instead could end up containing
the most recent commit of another branch or even project.
A test case is updated to detect both the condition where the
ZUUL_COMMIT for two different changes are the same, as well as
when ZUUL_REF does not point to ZUUL_COMMIT.
Change-Id: I9074746070edeebdff9f2ef3c1bb00526c21cfee
Adds missing periods in item descriptions to bring in line with
the rest of the documentation. Also fixes some typos and styling.
Change-Id: I11ef0bf3f89ecf5f09bb5781cced9b655c71bc65
Zuul creates fake Build objects for jobs that it skips due to
merge conflicts and other such situations. If Zuul decides to
cancel such a build, we should handle it gracefully instead of
throwing an exception.
Also log received gearman job handles to better correlate builds
with gearman debug logs.
Change-Id: I1cd14c8249ac4c68e797430b2b4941f2eeee2acf
We don't really offer a facility to match on non-review category
approvals (like SUBM) so ignore those when matching events to
triggers. Otherwise, matching logic that expects only review
categories will be fooled by submit or other similar approval
types.
Fixes-bug: 1284842
Fixes-bug: 1285210
Change-Id: I68b58f611df8fc2b9b2753d1579eae2e6898acc9
To keep Python 3.x compatibility, use six.moves.urllib.parse
to replace urlparse.
Add six as a dependency in test_requirements.txt
Change-Id: Ia6cd042970c2d4e5d418a39e25bab32b4d93c544
If Zuul is disconnected from gearman, have Zuul retry the job.
Also, report slightly more useful status words for various kinds
of failures when able.
Change-Id: I2d0f2986199516f6251bb122ab619ac56a87ee8e
Gear 0.5.0 added statsd support for the gear server. Have Zuul
use it based on the existence of the same env vars Zuul uses
to configure its own use of statsd.
Change-Id: I09ea2541d1f2eff5f4fd2e0c78da2acfdc1776ab
When jobs share gates, Zuul has a nice feature which is to creates
reference of the Change in every involved repositories. That was sadly
undocumented.
Reference:
James E. Blair on openstack-infra mailing list:
http://lists.openstack.org/pipermail/openstack-infra/2014-January/000605.html
Change-Id: Ia3bd640f6a90e12192af1686547a5f4146192dbe
* requirements.txt: The latest release of the gear library fixes an
enqueuing/function registration order problem exposed by the recent
zuul merger fan-out implementation. Insist upon it.
Change-Id: Id2275477b12ce62fc485e5769029fe107a82b52d
The "building_jobs" dictionary on pipeline managers is no longer
necessary because the object data model is sufficient to find all
of the running builds. Remove the unnecessary second list.
Some of the protections that checked that builds were in this
dictionary are also no longer needed as it is not possible to
receive a result for a build that the running Zuul did not
request since the change to use Gearman.
Change-Id: I10de7edcf8356885f5f2d2f45d7014582bcbc3cf
Some git merge commands fails for me with an AssertionError coming from
GitPython:
AssertionError: len([
"5a0a437ee9a6fd4b30484dfdc2455c83ea00a058\t\t'refs/changes/80/111180/1'
of ssh://gerrit.wikimedia.org:29418/mediawiki/core\n"]
)
!= len([
'Total 10 (delta 9), reused 10 (delta 9)',
' * branch refs/changes/80/111180/1 -> FETCH_HEAD'
])
Zuul.Merger explicitly log merge exceptions at debug level, though we
should only load GitCommandError exceptions and properly log other
exceptions (such as AssertionError).
Change-Id: Idca682c0920b9e6eac3f12cc05634dfe304a0305
Connect it to Zuul via Gearman. Any number of mergers may be
deployed.
Directly find the pipeline for a build when processing a result,
so that the procedure is roughly the same for build and merge
results.
The timer trigger currently requires the gerrit trigger also be
configured. Make that explicit inside of the timer trigger so
that the scheduler API interaction with triggers is cleaner.
Change-Id: I69498813764753c97c426e42d17596c2ef1d87cf
Instead of processing all of the queues each time any event happens,
first process all of the events (which are really queued up just so
that they can modify the pipeline data safely within the main run loop)
and then run the queue processor (which will then launch or cancel jobs
based on the new state of the queues).
This adds a lock to the run processor which is only needed so that
the test suite can safely inspect the state of the scheduler to
determine whether it is idle.
The result queue is made somewhat more generic in preparation for
handling merge result events.
Change-Id: I6bbb52f77b070df04a110a9d61b426265b1e89cc
In order to split out the merger as a new component, remove any
direct Zuul model object manipulation, instead passing in a
lists of dictionaries for merge instructions. Change the merger
algorithm so that it is compatible with this new method and makes
no assumptions about whether it has merged any changes previously.
There is likely to be a minor performance impact as some information
which was previously kept in-memory will now be fetched from the git
index.
The merger is also no-longer pre-populated with clones of git repos
at startup. Some tests were adjusted to accomodate this.
Change-Id: I3df86ae36b4969d568d1fb03df1e6569553d1226
To help with debugging, allow workers to send back information about
itself such as hostname and ips. This will allow for an RPC/Client
call to get the information on running jobs.
Change-Id: I03b553293923cd65b3a9651a8877aa23688d2909
Jenkins
Jeremy Stanley
James E. Blair
Clark Boylan
Antoine Musso
Joshua Hesketh
Łukasz Jernaś
tanlin