When the gearman server was added, the exit handler was not updated
correctly. It should tell the scheduler to exit, wait for the
scheduler to empty its pipelines, and then kill the gearman server
and exit.
Change-Id: Ie0532c2ea058ed56217e41641f8eec45080a9470
Instead of "resetting" the local repo (git remote update,
git checkout master, git reset --hard, git clean -xfdq) before
merging each change, just fetch the remote ref for the branch
and check that out (as a detached head). Or, if we are merging
a change that depends on another change in the queue, just check
that change out.
Change-Id: I0a9b839a0c75c04eca7393d7bb58cf89448b6494
The current behavior is that for every event, run
'git remote origin update', which is quite a bit of overhead and
doesn't match what the comments say should be happening. The goal
is to ensure that when new tags arrive, we have them locally in
our repo. It's also not a bad idea for us to keep up with remote
branch movements as well.
This updates the event pre-processor to fetch the ref for each
ref-updated event as they are processed. This is much faster than
the git remote update that was happening before. It also adds
a git remote update to the repo initialization step so that when
Zuul starts, it will pick up any remote changes since it last ran.
Change-Id: I671bb43eddf41c7403de53bb4a223762101adc3c
We can more closely approximate Gerrit's behavior by using the
'resolve' git merge strategy. Make that the default, and leave
the previous behavior ('git merge') as an option. Also, finish
and correct the partially implemented plumbing for other merge
strategies (including cherry-pick).
(Note the previous unfinished implementation attempted to mimic
Gerrit's option names; the new implementation does not, but rather
documents the alignment. It's not a perfect translation anyway,
and this gives us more room to support other strategies not
currently supported by Gerrit).
Change-Id: Ie1ce4fde5980adf99bba69a5aa1d4e81026db676
We have a graph on our status page showing all of the jobs Zuul
launched, but it's built from more than 1000 graphite keys which
is a little inefficient. Add a key for convenience that rolls
up all of the job completions in a pipeline, so that such a graph
can be built with only about 10 keys.
Change-Id: Ie6dbcca68c8a118653effe90952c7921a9de9ad1
If a job is complete with no build result, it has failed to
run to completion. In this case, discard the previous build
and launch a replacement (in the next run of the queue processor).
Change-Id: Ib8fc245a5becb1e7deb13f1ea0721fdb6ceb9f6f
This change causes the gearman launcher to completely ignore builds
after the stop job request has been sent. This should prevent
them from updating build status with confusing results. It will
also help us avoid restarting canceled builds in a subsequent
change.
Change-Id: Id31bcbfb6f24a7ec9f5f0a776d7d2c30f36685b4
Several assignments in updateChange would actually just keep
appending data causing immensely large data structures (which
are later traversed putting Zuul into a significant busy loop).
Make sure that data are replaced instead of augmented.
Change-Id: I8c6528adbbe24d30f8d5bb8b55bb731fefd9941a
* doc/source/zuul.rst: Document SIGUSR2 behavior.
* zuul/cmd/server.py: When SIGUSR2 is received log stack traces for all
active running threads. This is useful for debugging deadlock
situations. Note that this makes use of sys._current_frames which may
not play nice with all implementations of Python.
*tests/test_stack_dump.py: Test the stack dump signal handler with a new
test file, class, and test method.
* requirements.txt: Add argparse to requirements list so that py26 tests
can pass when zuul.cmd.server is imported.
Change-Id: I8ad8155b16f324e832c191f0a619ff89ef804a87
If job_name_in_report is false, the 'name' variable may be
undefined. This makes sure it is always defined.
Change-Id: Ie544ccdf1661e08e2aa4c8055999f16e20d7584b
Add the ability for Zuul to accept inputs from multiple trigger
sources simultaneously.
Pipelines are associated with exactly one trigger, which must now
be named in the configuration file.
Co-Authored-By: Monty Taylor <mordred@inaugust.com>
Change-Id: Ief2b31a7b8d85d30817f2747c1e2635f71ea24b9
The multi-branch test occasionally picked the wrong change's commit
to inspect, resulting in a flakey test. Try to pick the right one
instead.
Change-Id: I1cefa1b96cac9e5a8fdfe0f4de4704202dd6d072
For every job completed, record the result of that job separately
to statsd. For successful and failed jobs, record the runtimes
of the jobs separately by result (others are not interesting).
Also, substitute '_' for '.' in job names in statsd keys.
This is backwards-incompatible with current statsd keys.
Change-Id: I7b6152bcc7ea5ce6e37bf90ed41aee89baa29309
Pass a reference to the current job about to be run to the custom
parameter function.
Remove the un-needed ZUUL_SHORT_* parameters.
Change-Id: I39538b3815ce89fae0b59c21c5cff588509cfe4e
Add an option to the syntax validator to test that job
referenced in the layout are defined in a file. Creating the
file with the list of jobs is an exercise for user.
Change-Id: Iceb74440cb004e9ebe6fc08a4eedf7715de2d485
Now that live reconfiguration is an option, we sometimes load the
Zuul config before the remote repo is created. To handle that,
gracefully handle that case when the Repo object is created by the
Merger. Keep track of whether the initial clone has happened, and
check for that before every local Repo operation. If we get an
event that involves a repo before it exists (unlikely!) that will
still error and raise an exception (that should be caught higher
up in the stack).
Add a test for this case.
In the test suite, when adding a fake change to the upstream repo,
always reset it so that HEAD is master so that new repos clone
from a consistent state.
Also remove an errant assertEmptyQueues because it's handled by
assertFinalState.
Change-Id: Ic6eec83e3faa2a15be4b23d4cfcfbddcac82983c
We don't actually know how long it will take for a change to report
if a change ahead of it has failed (because this may need to be
restarted). We could do a better job of estimating a minimum if
we had historic run-times of jobs internally, but we don't right
now, so at least let's say it's unknown for now.
Change-Id: I7a3e4d07a02893122564817f8119cb54632464f4
With this change, we now should properly gate on Sphinx errors /
warnings.
Change-Id: I7e789ad83e81286575c74e1e5256da20a717e982
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Metajobs were being applied in dict key order, which meant that
if more that one metajob matched a job, the actual attributes
applied were non-deterministic. This was compounded by the fact
that attributes of each metajob were being strictly copied to
the real job, so the attributes of the second metajob would always
completely replace the first.
Instead, keep metajobs in config file order, and only copy attributes
that are non-null. Boolean attributes are still last-wins, and
so must be set explicitly by each matching metajob.
Change-Id: Ie255658719d5ded1663c3513dae1fc297ce357c4
The jobs were starting in the correct order, but because they all
ran in parallel, they had a race to finish, and the build history
is built from finished jobs, which is what the test checked.
Instead, only allow one job at a time to run.
Change-Id: Ie09369d7c27b286c2339c9cc40658f8d5f629488
test_build_configuration_multi_branch hit a nondeterministic
failure in Jenkins and it's not clear why.
Change-Id: I2d5d5e2ed6174cf8f3a67f4dbb42ecbf4b038856
When checking for a new version of a change, compare patchsets
as integers.
Change-Id: Ic5ceae9e89cd28a9f391896bcdbf68313d70adc4
Reviewed-on: https://review.openstack.org/36734
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
We get this a lot. It's not helpful.
Change-Id: Icf0191ded019c705a42446269d647cb52843ef58
Reviewed-on: https://review.openstack.org/36733
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
James E. Blair
Clark Boylan
Jenkins
Joshua Hesketh
Jeremy Stanley
Paul Belanger