By setting parameters on a pipeline, you can change the default
"Build successful/failed" introductory messages left on Gerrit.
Change-Id: Ie05c8c87a47994faced4ee67fda4ab4e64ea0d20
Reviewed-on: https://review.openstack.org/19677
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Based on voluptuous library.
Basic validation should catch typos, missing or extra attributes.
Can be expanded to do more serious validation (ie, specifying
a comment in a trigger should require the event be comment-added).
Adds a command line option to validate a named layout file and
exit.
(Also add dist/ to .gitignore.)
Change-Id: Ia864ebde1765141d4d1a52bc77033689b6210e81
Reviewed-on: https://review.openstack.org/19443
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Instead, use queue task_done calls to indicate that the scheduler
has finished processing events. This lets the tests know when
the queues are both empty and all requests have been handled.
Add a lock around reporting complete events in fake jenkins jobs
so that waitUntilSettled can be assured that no new events will
arrive.
Directly report LOST builds when a job doesn't exist, rather
than spawning a new thread (which was only done to work around
the lock).
Change-Id: I32ad46648c82d7458fb5be779c62ac5b57857674
Reviewed-on: https://review.openstack.org/19330
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Zuul already treats LOST jobs as failures in much of the code. Do the
same in didAnyJobFail so that changes behind a change with a LOST job
get their jobs cancelled.
Change-Id: I1eb819a6cfb3b3ee0a2c131417b00dab1d83f29d
Reviewed-on: https://review.openstack.org/19270
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
These occur at critical points in the code and should be protected
with exception handlers.
Make sure that a build actually has a start time before calculating
its duration.
Change-Id: I1ff1260da0b7b5095c6b075bd0d78c7a1490686a
Reviewed-on: https://review.openstack.org/19023
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When a build is reported as lost, it has no start_time, so the
code that calculated the run-time for statsd threw an exception.
Correct that by only calculating the run time for jobs that have
really completed.
Add debug lines to onBuild* that would have helped diagnose this.
Keep track of the throwaway threads that are used to report LOST
builds so that the test suite can more correctly know when the
system is settled.
Add a test case for a single non-existent post job.
Add STATSD_* env variables in tox to make sure that the statsd
code paths are executed by tests.
Change-Id: I339443cbad7b0858d70e9180d63dfb701b8eaee9
Reviewed-on: https://review.openstack.org/18792
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
The filtering regexp should not allow greedy matchers (such as .*) but
use non greedy ones (such as .*?). One want to escape dots as well.
Additionally, whenever one want to add several emails, he should use a
YAML list.
Change-Id: I9deea0893151358ddde9b1ec5a40c322c45d7ff2
Reviewed-on: https://review.openstack.org/18350
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Also, minor optimization to email filter matching.
Change-Id: Iedea0fc14b10a8f68ec52e38ceab4b2fb407e364
Reviewed-on: https://review.openstack.org/18515
Reviewed-by: Antoine Musso <hashar@free.fr>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Reports statistics on job runtime, number/rate of jobs run, and
number/rate of individual gerrit events to statsd.
Change-Id: I0db9352610b4c268c95da98fd282d7f463945f90
Reviewed-on: https://review.openstack.org/18628
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
So that status pages can easily link to gitweb urls of ref-update
triggered changes.
Change-Id: Id9b708a473d92c30727f66578a0174f18c657723
Reviewed-on: https://review.openstack.org/18602
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Can be used for nifty ajax-style status pages.
Add optional description field to pipeline.
Change-Id: If5db3f6945f65f038833cbf9c783de5ffef63b49
Reviewed-on: https://review.openstack.org/18579
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add the option to specify the URL pattern left in Gerrit, per-job.
Example: a job that builds documentation and uploads it to a staging
site can leave the URL of the staging site iff the job succeeds.
Change-Id: I34841e61e9116fd8d4de7ac09d8f9cfe36fe78ec
Reviewed-on: https://review.openstack.org/18264
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Some repositories might not have a master branch or could use a
different default HEAD. A git reset was made to remotes/master which
would produce a stacktrace.
Fix https://bugs.launchpad.net/zuul/+bug/1083154
Change-Id: Ie9390c0c5dd707d77fbfa8cff2078a9b31fd31ae
Reviewed-on: https://review.openstack.org/17239
Reviewed-by: Antoine Musso <hashar@free.fr>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
On some setup we might restrict a pipeline to trusted users. The new
email_filter will matches the email coming from the event account
attribute.
In Gerrit, each event hold the account in a different field
name so I have simply added a map to find out the correct field.
email_filter works just like comment_filter, fields are ORed and are
considered to be regex.
Change-Id: I775f67b48d5f162106c024f94fe498a049b3fe94
Reviewed-on: https://review.openstack.org/17609
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
Change-Id: I840e1d1fb66d22d14deb4e3307962c7b7f85c63a
Reviewed-on: https://review.openstack.org/18194
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The gerrit trigger took an sshkey parameter, but the merger,
which uses GitPython, did not honor it. Add a GIT_SSH wrapper
for the merger to use to support arbitrary ssh keys for remote
git operation.
Change-Id: I43f0830905ee7103a2acbfa4b4f90959254f60b2
Reviewed-on: https://review.openstack.org/17269
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
We're starting to use this elsewhere in OpenStack... avoids having to
do pyflakes workaround tricks.
Change-Id: Ic767677e87f30794d49baf100e3ded7555272df6
Reviewed-on: https://review.openstack.org/16851
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
If we put the code for zuul-server into the zuul module, then autodoc
will pick it up for documentation.
Change-Id: Idf2db82f706243c6e69984170b4783f1eece2237
Reviewed-on: https://review.openstack.org/16850
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Our code was expecting the Gerrit web interface to be available at the
root of the Gerrit server which is not always the case since people
could publish the Gerrit web interface under a subdirectory. In such
configuration, the existing assumption prevents Zuul from getting
references information.
This patch adds a new optional configuration variable 'baseurl' to the
'gerrit' section. It lets one override the default:
[gerrit]
server=gerrit.wikimedia.org
baseurl=https://gerrit.wikimedia.org/r
Updated Gerrit._getInfoRefs() to use 'baseurl' instead of 'server' when
creating the url to fetch references from. This is still falling back to
the old default whenever gerrit.baseurl is not set.
Change-Id: Ia9910acf46fe82c425911784412356598b3c7eac
Reviewed-on: https://review.openstack.org/16884
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
This document the required permissions of the Zuul Jenkins user.
Change-Id: Id977e3813f943b032c7628d8cdc663b3bdbb688d
Reviewed-on: https://review.openstack.org/16528
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
After successfully adding openstack versioning to jenkins-job-builder
this add the same support for zuul.
Change-Id: Ia5baab2b0d9392c1b3c70bf890eaf7c6a2ea5c29
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/16219
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
If a large change is fetched, it may arrive in only a pack file,
which tickles the invalid cache bug in git-python.
Fixes bug 1078946.
Change-Id: Ib912967e37c6aed906d86dc35a4b4a65e4f9c3bd
Reviewed-on: https://review.openstack.org/16199
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Change-Id: If1c43a2965c30e56b09a6cf85bcf8959c67d62c6
Reviewed-on: https://review.openstack.org/15927
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
GitPython when fetching refs may improperly read in git progress info
and interpret it as fetch info. When this happens an AssertionError is
thrown. However, since fetching seems to cache the ref locally
subsequent fetches work without running into the progress info problem.
So fetch again if an AssertionError is caught.
Upstream bugfix
https://github.com/gitpython-developers/GitPython/pull/42
Change-Id: I1768b836aade2853d4dd3b2f4cd1fce7ec035524
Reviewed-on: https://review.openstack.org/15219
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Reacting on `change-merged` event definitely requires --force-message to
be passed to `gerrit review` or Gerrit will complain the change is
closed whenever it is also passed --verify X or --code-review Y.
Added a simple example to setup a post-merge independant pipeline.
Left a note about ``change-merged`` lacking the commit sha1. One should
instead use the ``ref-updated`` when the commit sha1 is needed.
Close #1067083
Change-Id: Ic402a0048651d10ac619010fdfd58725374368cd
Reviewed-on: https://review.openstack.org/14476
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: Ieda682233843b4500e8d59d91ac24b36dfd172dd
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/14433
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The addition of ZUUL_COMMIT (commit 81515ad, change I88c38a28)
deprecated the UUID parameter in favor of ZUUL_UUID but did not honor
the new parameter.
This patch add ZUUL_UUID lookup in jenkins_endpoint() and
findBuildInQueue() with a fallback on UUID. Probably only needed on
jenkins_endpoint() when a Jenkins job lack the UUID parameter, but I
think we might as well add it in findBuildInQueue() just to be safe.
Also update the documentation to uses ZUUL_ prefixed variables where
relevant and added a note about GERRIT_PROJECT and UUID being
deprecated.
Change-Id: Ie0426f388cbd3a373af2110b7460aca6ff4f1751
Reviewed-on: https://review.openstack.org/14327
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
We always assume being able to get the references from a specific URL
although that might not be available (for example if Apache is not
properly setup). Logging an error with the URL we attempted to reach and
raising the urllib2 exception will give an obvious hint to the
administrator.
Change-Id: I0465f8ec0749f6f16d64ddb22dd2ad93dbc42c82
Reviewed-on: https://review.openstack.org/13815
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
I hope.
Change-Id: I813fdeb9ecd2e6f64871bc9499b0d0919a9cea83
Reviewed-on: https://review.openstack.org/14171
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
To facilitate using zuul with just the jenkins git plugin, add
the ZUUL_COMMIT parameter. Also, always include ZUUL_REF, whether
the job is pre- or post-commit. If it's pre, it will be a
refs/zuul ref, if it's post, it will be refs/tags or a branch
name.
Change-Id: I88c38a28dcd552b2540095d36caacd10acf167b8
Reviewed-on: https://review.openstack.org/13934
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Waiting until settled was not considering jobs in the jenkins queue.
This corrects that, as well as adding a little more debugging info
for the problematic job that was tickling this.
Change-Id: I1f4c76ff3e4adc830f47156d0740384815d5a51b
Reviewed-on: https://review.openstack.org/14106
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: If46832e6c1d5538fad88c9503c248372cb5b9416
Reviewed-on: https://review.openstack.org/14096
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I6cdd20e6238fafdca626eb2531a3a78c985bb680
Reviewed-on: https://review.openstack.org/13974
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
In the case of ref deletion, the null sha is not very
unique, and could be present in another project. Make
sure to compare the project, ref, and sha to determine
if two ref-updated events are equal.
Change-Id: I6bd4a02f21bd31b91ef369783641ded014f7717c
Reviewed-on: https://review.openstack.org/13839
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When commenting on Gerrit that jobs are beginning include a link to the
zuul status page if one is configured.
Change-Id: Iba5f315f4baa006939dbdcb07b0758e570242ca8
Reviewed-on: https://review.openstack.org/13858
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
I have been playing with zuul on port 8001 using curl:
curl http://127.0.0.1:8001/jenkins_endpoint
Had to `strace -f` the process to find out what was going wrong. Logging
the exception will simply make it easier to track possible issues coming
from the Jenkins Notification plugin..
Change-Id: I461146d0a6e7d63a0d8bedc3d4a6ad19e22548d6
Reviewed-on: https://review.openstack.org/13817
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fixed all pep8 errors.
E125 is ignored because of false alarms.
Change-Id: I4da60409e0095c0896230cd01bda548ed2e3f741
Reviewed-on: https://review.openstack.org/13740
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
To make port setting of gerrit server in zuul.conf really work.
Change-Id: I6f44f21256391fd04e56c18c0b293d9767c5a6a9
Reviewed-on: https://review.openstack.org/13574
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Updated pep8 version requirement to 1.3.3
Fixed E127 errors
All ignores are to be removed in the next sequence of patches
Change-Id: Ia9e922b8873686a0f905f2548cc43d534ee1c912
Reviewed-on: https://review.openstack.org/13642
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Zhongyue Luo <zhongyue.nah@intel.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
This reverts commit afd39bb3cd
This change should be rebased against the dev branch instead and
submitted there. Once the production CI freeze ends immediately
following the OpenStack Folsom release, dev changes will be merged back
into master.
Change-Id: Iabc1a0b287f0a433ca297a0d5cd1e21dd670e1d3
Reviewed-on: https://review.openstack.org/13638
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When running zuul in non-daemon mode (with -d), it is often useful to
abort it using Control + C.
Change-Id: I385a41625633f3120b95c07283bef015824e5853
Reviewed-on: https://review.openstack.org/13571
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I95ef3dde919f4ab5ce471295fb7cc1568e42402a
Reviewed-on: https://review.openstack.org/13526
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When using python-daemon 1.6, the interface has changed in an
uncompatible way. Clark pointed me to Gerritbot which solves that issue
with a simple try / catch block implemented with:
b2be72e69d
So this patch is merely a copy/paste from David "davido" Ostrovsky with
a small workaround for pyflakes issue #13 (we have to prented we are
using the variable holding the module).
Change-Id: Iffdf7fca067734fa9c09b5bddfb13f122e6251a7
Reviewed-on: https://review.openstack.org/13524
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
zuul also need the 'lockfile' and 'python-daemon' python packages.
Change-Id: I16ed4fba9705796f83aa980687de8b9d262eb715
Reviewed-on: https://review.openstack.org/13523
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Updated pep8 version requirement to 1.3.3
Fixed E127 errors
All ignores are to be removed in the next sequence of patches
Change-Id: I266498a2c64a215648a4a598b8cba5669049e37c
Reviewed-on: https://review.openstack.org/13384
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
Some puppet modules have been renamed in openstack-ci-puppet:
* `jenkins_slave` puppet module has been integrated in `jenkins` module
* `openstack-ci-config` has been renamed `openstack_project`
Change-Id: Ie4a35022301c156f74e1fdef859c87319efb60da
Reviewed-on: https://review.openstack.org/13519
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
James E. Blair