When a new patchset is created for a change that is in a pipeline,
cancel running builds and dequeue that change (and possibly
dependent changes that can no longer merge).
Make this optional (and document the option).
Fixes bug 1022643.
Change-Id: I8f591956cf86645443e4b6075b8cdfc95a939e4f
Reviewed-on: https://review.openstack.org/20948
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: Iae253977eafaff1a7cbdcaf1b873388487c4e36e
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/21730
Reviewed-by: Elizabeth Krumbach <lyz@princessleia.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Change-Id: I6dcd87a59d836dc22b67a2bee7c764e0b603a809
Reviewed-on: https://review.openstack.org/21543
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
A working example that can be customized for site use.
Change-Id: I6e6fdf245a55a767aa7e86486b6c47dae221f47b
Reviewed-on: https://review.openstack.org/19989
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The ref objects were missing a python __repr__, ended up with a not so
useful trace in the debug.log. This creates a meaningful representation
for human consumption:
<Ref 0x7f51f848ffd0 creates refs/tags/v2.0.0 on 4eb656fe>
<Ref 0x7f51f8491990 refs/tags/v2.0.0 updated 4eb656f..2260abb>
<Ref 0x7f51f848f610 deletes refs/tags/v2.0.0 from 2260abb>
(sha1 above have been made shorter than the actual output)
Change-Id: If3b60494f1984a04e9ffada38f7e3b99f2a8ba0e
Reviewed-on: https://review.openstack.org/19925
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
* Number of changes processed.
* Time each change spent in pipeline.
* Current number of changes in pipeline.
All of those per-pipeline, and per-pipeline, per-project.
Change-Id: I609ff33ab7c26fc43d323f8a0bbb61f93382af90
Reviewed-on: https://review.openstack.org/19764
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I7e3bd8eed585133ee217a1e462b9833df494be21
Reviewed-on: https://review.openstack.org/19718
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The documentation files often have minor typo or badly formatted
commands which makes sphinx emits a warning. To make sure we always
catch them via the Jenkins build, this patch makes sphinx to convert
warning to errors thus aborting the build, simply add to pass the -W
option to sphinx-build.
This patch also fix some warnings:
* There is no source/_static dir so disable html_static_path
* In launchers.rst, File reference name should be before the title
Test plan:
$ make clean; make html
rm -rf build/*
sphinx-build -b html -d build/doctrees -W source build/html
Making output directory...
Running Sphinx v1.1.3
loading pickled environment... not yet created
loading intersphinx inventory from
http://docs.python.org/2.7/objects.inv...
building [html]: targets for 5 source files that are out of date
updating environment: 5 added, 0 changed, 0 removed
reading sources... [100%] zuul
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] zuul
writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded.
Build finished. The HTML pages are in build/html.
$
Change-Id: Id907cc2c9aeccd077593b2fa1b78b220a159ed71
Reviewed-on: https://review.openstack.org/16316
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
By setting parameters on a pipeline, you can change the default
"Build successful/failed" introductory messages left on Gerrit.
Change-Id: Ie05c8c87a47994faced4ee67fda4ab4e64ea0d20
Reviewed-on: https://review.openstack.org/19677
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Based on voluptuous library.
Basic validation should catch typos, missing or extra attributes.
Can be expanded to do more serious validation (ie, specifying
a comment in a trigger should require the event be comment-added).
Adds a command line option to validate a named layout file and
exit.
(Also add dist/ to .gitignore.)
Change-Id: Ia864ebde1765141d4d1a52bc77033689b6210e81
Reviewed-on: https://review.openstack.org/19443
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Instead, use queue task_done calls to indicate that the scheduler
has finished processing events. This lets the tests know when
the queues are both empty and all requests have been handled.
Add a lock around reporting complete events in fake jenkins jobs
so that waitUntilSettled can be assured that no new events will
arrive.
Directly report LOST builds when a job doesn't exist, rather
than spawning a new thread (which was only done to work around
the lock).
Change-Id: I32ad46648c82d7458fb5be779c62ac5b57857674
Reviewed-on: https://review.openstack.org/19330
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Zuul already treats LOST jobs as failures in much of the code. Do the
same in didAnyJobFail so that changes behind a change with a LOST job
get their jobs cancelled.
Change-Id: I1eb819a6cfb3b3ee0a2c131417b00dab1d83f29d
Reviewed-on: https://review.openstack.org/19270
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
These occur at critical points in the code and should be protected
with exception handlers.
Make sure that a build actually has a start time before calculating
its duration.
Change-Id: I1ff1260da0b7b5095c6b075bd0d78c7a1490686a
Reviewed-on: https://review.openstack.org/19023
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When a build is reported as lost, it has no start_time, so the
code that calculated the run-time for statsd threw an exception.
Correct that by only calculating the run time for jobs that have
really completed.
Add debug lines to onBuild* that would have helped diagnose this.
Keep track of the throwaway threads that are used to report LOST
builds so that the test suite can more correctly know when the
system is settled.
Add a test case for a single non-existent post job.
Add STATSD_* env variables in tox to make sure that the statsd
code paths are executed by tests.
Change-Id: I339443cbad7b0858d70e9180d63dfb701b8eaee9
Reviewed-on: https://review.openstack.org/18792
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
The filtering regexp should not allow greedy matchers (such as .*) but
use non greedy ones (such as .*?). One want to escape dots as well.
Additionally, whenever one want to add several emails, he should use a
YAML list.
Change-Id: I9deea0893151358ddde9b1ec5a40c322c45d7ff2
Reviewed-on: https://review.openstack.org/18350
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Also, minor optimization to email filter matching.
Change-Id: Iedea0fc14b10a8f68ec52e38ceab4b2fb407e364
Reviewed-on: https://review.openstack.org/18515
Reviewed-by: Antoine Musso <hashar@free.fr>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Reports statistics on job runtime, number/rate of jobs run, and
number/rate of individual gerrit events to statsd.
Change-Id: I0db9352610b4c268c95da98fd282d7f463945f90
Reviewed-on: https://review.openstack.org/18628
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
So that status pages can easily link to gitweb urls of ref-update
triggered changes.
Change-Id: Id9b708a473d92c30727f66578a0174f18c657723
Reviewed-on: https://review.openstack.org/18602
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Can be used for nifty ajax-style status pages.
Add optional description field to pipeline.
Change-Id: If5db3f6945f65f038833cbf9c783de5ffef63b49
Reviewed-on: https://review.openstack.org/18579
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add the option to specify the URL pattern left in Gerrit, per-job.
Example: a job that builds documentation and uploads it to a staging
site can leave the URL of the staging site iff the job succeeds.
Change-Id: I34841e61e9116fd8d4de7ac09d8f9cfe36fe78ec
Reviewed-on: https://review.openstack.org/18264
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Some repositories might not have a master branch or could use a
different default HEAD. A git reset was made to remotes/master which
would produce a stacktrace.
Fix https://bugs.launchpad.net/zuul/+bug/1083154
Change-Id: Ie9390c0c5dd707d77fbfa8cff2078a9b31fd31ae
Reviewed-on: https://review.openstack.org/17239
Reviewed-by: Antoine Musso <hashar@free.fr>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
On some setup we might restrict a pipeline to trusted users. The new
email_filter will matches the email coming from the event account
attribute.
In Gerrit, each event hold the account in a different field
name so I have simply added a map to find out the correct field.
email_filter works just like comment_filter, fields are ORed and are
considered to be regex.
Change-Id: I775f67b48d5f162106c024f94fe498a049b3fe94
Reviewed-on: https://review.openstack.org/17609
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
Change-Id: I840e1d1fb66d22d14deb4e3307962c7b7f85c63a
Reviewed-on: https://review.openstack.org/18194
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The gerrit trigger took an sshkey parameter, but the merger,
which uses GitPython, did not honor it. Add a GIT_SSH wrapper
for the merger to use to support arbitrary ssh keys for remote
git operation.
Change-Id: I43f0830905ee7103a2acbfa4b4f90959254f60b2
Reviewed-on: https://review.openstack.org/17269
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
We're starting to use this elsewhere in OpenStack... avoids having to
do pyflakes workaround tricks.
Change-Id: Ic767677e87f30794d49baf100e3ded7555272df6
Reviewed-on: https://review.openstack.org/16851
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
If we put the code for zuul-server into the zuul module, then autodoc
will pick it up for documentation.
Change-Id: Idf2db82f706243c6e69984170b4783f1eece2237
Reviewed-on: https://review.openstack.org/16850
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Our code was expecting the Gerrit web interface to be available at the
root of the Gerrit server which is not always the case since people
could publish the Gerrit web interface under a subdirectory. In such
configuration, the existing assumption prevents Zuul from getting
references information.
This patch adds a new optional configuration variable 'baseurl' to the
'gerrit' section. It lets one override the default:
[gerrit]
server=gerrit.wikimedia.org
baseurl=https://gerrit.wikimedia.org/r
Updated Gerrit._getInfoRefs() to use 'baseurl' instead of 'server' when
creating the url to fetch references from. This is still falling back to
the old default whenever gerrit.baseurl is not set.
Change-Id: Ia9910acf46fe82c425911784412356598b3c7eac
Reviewed-on: https://review.openstack.org/16884
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
This document the required permissions of the Zuul Jenkins user.
Change-Id: Id977e3813f943b032c7628d8cdc663b3bdbb688d
Reviewed-on: https://review.openstack.org/16528
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
After successfully adding openstack versioning to jenkins-job-builder
this add the same support for zuul.
Change-Id: Ia5baab2b0d9392c1b3c70bf890eaf7c6a2ea5c29
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/16219
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
If a large change is fetched, it may arrive in only a pack file,
which tickles the invalid cache bug in git-python.
Fixes bug 1078946.
Change-Id: Ib912967e37c6aed906d86dc35a4b4a65e4f9c3bd
Reviewed-on: https://review.openstack.org/16199
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Change-Id: If1c43a2965c30e56b09a6cf85bcf8959c67d62c6
Reviewed-on: https://review.openstack.org/15927
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
GitPython when fetching refs may improperly read in git progress info
and interpret it as fetch info. When this happens an AssertionError is
thrown. However, since fetching seems to cache the ref locally
subsequent fetches work without running into the progress info problem.
So fetch again if an AssertionError is caught.
Upstream bugfix
https://github.com/gitpython-developers/GitPython/pull/42
Change-Id: I1768b836aade2853d4dd3b2f4cd1fce7ec035524
Reviewed-on: https://review.openstack.org/15219
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Reacting on `change-merged` event definitely requires --force-message to
be passed to `gerrit review` or Gerrit will complain the change is
closed whenever it is also passed --verify X or --code-review Y.
Added a simple example to setup a post-merge independant pipeline.
Left a note about ``change-merged`` lacking the commit sha1. One should
instead use the ``ref-updated`` when the commit sha1 is needed.
Close #1067083
Change-Id: Ic402a0048651d10ac619010fdfd58725374368cd
Reviewed-on: https://review.openstack.org/14476
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: Ieda682233843b4500e8d59d91ac24b36dfd172dd
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/14433
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The addition of ZUUL_COMMIT (commit 81515ad, change I88c38a28)
deprecated the UUID parameter in favor of ZUUL_UUID but did not honor
the new parameter.
This patch add ZUUL_UUID lookup in jenkins_endpoint() and
findBuildInQueue() with a fallback on UUID. Probably only needed on
jenkins_endpoint() when a Jenkins job lack the UUID parameter, but I
think we might as well add it in findBuildInQueue() just to be safe.
Also update the documentation to uses ZUUL_ prefixed variables where
relevant and added a note about GERRIT_PROJECT and UUID being
deprecated.
Change-Id: Ie0426f388cbd3a373af2110b7460aca6ff4f1751
Reviewed-on: https://review.openstack.org/14327
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
We always assume being able to get the references from a specific URL
although that might not be available (for example if Apache is not
properly setup). Logging an error with the URL we attempted to reach and
raising the urllib2 exception will give an obvious hint to the
administrator.
Change-Id: I0465f8ec0749f6f16d64ddb22dd2ad93dbc42c82
Reviewed-on: https://review.openstack.org/13815
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
I hope.
Change-Id: I813fdeb9ecd2e6f64871bc9499b0d0919a9cea83
Reviewed-on: https://review.openstack.org/14171
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
To facilitate using zuul with just the jenkins git plugin, add
the ZUUL_COMMIT parameter. Also, always include ZUUL_REF, whether
the job is pre- or post-commit. If it's pre, it will be a
refs/zuul ref, if it's post, it will be refs/tags or a branch
name.
Change-Id: I88c38a28dcd552b2540095d36caacd10acf167b8
Reviewed-on: https://review.openstack.org/13934
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Waiting until settled was not considering jobs in the jenkins queue.
This corrects that, as well as adding a little more debugging info
for the problematic job that was tickling this.
Change-Id: I1f4c76ff3e4adc830f47156d0740384815d5a51b
Reviewed-on: https://review.openstack.org/14106
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: If46832e6c1d5538fad88c9503c248372cb5b9416
Reviewed-on: https://review.openstack.org/14096
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I6cdd20e6238fafdca626eb2531a3a78c985bb680
Reviewed-on: https://review.openstack.org/13974
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
In the case of ref deletion, the null sha is not very
unique, and could be present in another project. Make
sure to compare the project, ref, and sha to determine
if two ref-updated events are equal.
Change-Id: I6bd4a02f21bd31b91ef369783641ded014f7717c
Reviewed-on: https://review.openstack.org/13839
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When commenting on Gerrit that jobs are beginning include a link to the
zuul status page if one is configured.
Change-Id: Iba5f315f4baa006939dbdcb07b0758e570242ca8
Reviewed-on: https://review.openstack.org/13858
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
I have been playing with zuul on port 8001 using curl:
curl http://127.0.0.1:8001/jenkins_endpoint
Had to `strace -f` the process to find out what was going wrong. Logging
the exception will simply make it easier to track possible issues coming
from the Jenkins Notification plugin..
Change-Id: I461146d0a6e7d63a0d8bedc3d4a6ad19e22548d6
Reviewed-on: https://review.openstack.org/13817
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fixed all pep8 errors.
E125 is ignored because of false alarms.
Change-Id: I4da60409e0095c0896230cd01bda548ed2e3f741
Reviewed-on: https://review.openstack.org/13740
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
To make port setting of gerrit server in zuul.conf really work.
Change-Id: I6f44f21256391fd04e56c18c0b293d9767c5a6a9
Reviewed-on: https://review.openstack.org/13574
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
James E. Blair