Change-Id: I840e1d1fb66d22d14deb4e3307962c7b7f85c63a
Reviewed-on: https://review.openstack.org/18194
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The gerrit trigger took an sshkey parameter, but the merger,
which uses GitPython, did not honor it. Add a GIT_SSH wrapper
for the merger to use to support arbitrary ssh keys for remote
git operation.
Change-Id: I43f0830905ee7103a2acbfa4b4f90959254f60b2
Reviewed-on: https://review.openstack.org/17269
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
We're starting to use this elsewhere in OpenStack... avoids having to
do pyflakes workaround tricks.
Change-Id: Ic767677e87f30794d49baf100e3ded7555272df6
Reviewed-on: https://review.openstack.org/16851
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
If we put the code for zuul-server into the zuul module, then autodoc
will pick it up for documentation.
Change-Id: Idf2db82f706243c6e69984170b4783f1eece2237
Reviewed-on: https://review.openstack.org/16850
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Our code was expecting the Gerrit web interface to be available at the
root of the Gerrit server which is not always the case since people
could publish the Gerrit web interface under a subdirectory. In such
configuration, the existing assumption prevents Zuul from getting
references information.
This patch adds a new optional configuration variable 'baseurl' to the
'gerrit' section. It lets one override the default:
[gerrit]
server=gerrit.wikimedia.org
baseurl=https://gerrit.wikimedia.org/r
Updated Gerrit._getInfoRefs() to use 'baseurl' instead of 'server' when
creating the url to fetch references from. This is still falling back to
the old default whenever gerrit.baseurl is not set.
Change-Id: Ia9910acf46fe82c425911784412356598b3c7eac
Reviewed-on: https://review.openstack.org/16884
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
This document the required permissions of the Zuul Jenkins user.
Change-Id: Id977e3813f943b032c7628d8cdc663b3bdbb688d
Reviewed-on: https://review.openstack.org/16528
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
After successfully adding openstack versioning to jenkins-job-builder
this add the same support for zuul.
Change-Id: Ia5baab2b0d9392c1b3c70bf890eaf7c6a2ea5c29
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/16219
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
If a large change is fetched, it may arrive in only a pack file,
which tickles the invalid cache bug in git-python.
Fixes bug 1078946.
Change-Id: Ib912967e37c6aed906d86dc35a4b4a65e4f9c3bd
Reviewed-on: https://review.openstack.org/16199
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Change-Id: If1c43a2965c30e56b09a6cf85bcf8959c67d62c6
Reviewed-on: https://review.openstack.org/15927
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
GitPython when fetching refs may improperly read in git progress info
and interpret it as fetch info. When this happens an AssertionError is
thrown. However, since fetching seems to cache the ref locally
subsequent fetches work without running into the progress info problem.
So fetch again if an AssertionError is caught.
Upstream bugfix
https://github.com/gitpython-developers/GitPython/pull/42
Change-Id: I1768b836aade2853d4dd3b2f4cd1fce7ec035524
Reviewed-on: https://review.openstack.org/15219
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Reacting on `change-merged` event definitely requires --force-message to
be passed to `gerrit review` or Gerrit will complain the change is
closed whenever it is also passed --verify X or --code-review Y.
Added a simple example to setup a post-merge independant pipeline.
Left a note about ``change-merged`` lacking the commit sha1. One should
instead use the ``ref-updated`` when the commit sha1 is needed.
Close #1067083
Change-Id: Ic402a0048651d10ac619010fdfd58725374368cd
Reviewed-on: https://review.openstack.org/14476
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: Ieda682233843b4500e8d59d91ac24b36dfd172dd
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/14433
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The addition of ZUUL_COMMIT (commit 81515ad, change I88c38a28)
deprecated the UUID parameter in favor of ZUUL_UUID but did not honor
the new parameter.
This patch add ZUUL_UUID lookup in jenkins_endpoint() and
findBuildInQueue() with a fallback on UUID. Probably only needed on
jenkins_endpoint() when a Jenkins job lack the UUID parameter, but I
think we might as well add it in findBuildInQueue() just to be safe.
Also update the documentation to uses ZUUL_ prefixed variables where
relevant and added a note about GERRIT_PROJECT and UUID being
deprecated.
Change-Id: Ie0426f388cbd3a373af2110b7460aca6ff4f1751
Reviewed-on: https://review.openstack.org/14327
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
We always assume being able to get the references from a specific URL
although that might not be available (for example if Apache is not
properly setup). Logging an error with the URL we attempted to reach and
raising the urllib2 exception will give an obvious hint to the
administrator.
Change-Id: I0465f8ec0749f6f16d64ddb22dd2ad93dbc42c82
Reviewed-on: https://review.openstack.org/13815
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
I hope.
Change-Id: I813fdeb9ecd2e6f64871bc9499b0d0919a9cea83
Reviewed-on: https://review.openstack.org/14171
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
To facilitate using zuul with just the jenkins git plugin, add
the ZUUL_COMMIT parameter. Also, always include ZUUL_REF, whether
the job is pre- or post-commit. If it's pre, it will be a
refs/zuul ref, if it's post, it will be refs/tags or a branch
name.
Change-Id: I88c38a28dcd552b2540095d36caacd10acf167b8
Reviewed-on: https://review.openstack.org/13934
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Waiting until settled was not considering jobs in the jenkins queue.
This corrects that, as well as adding a little more debugging info
for the problematic job that was tickling this.
Change-Id: I1f4c76ff3e4adc830f47156d0740384815d5a51b
Reviewed-on: https://review.openstack.org/14106
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: If46832e6c1d5538fad88c9503c248372cb5b9416
Reviewed-on: https://review.openstack.org/14096
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I6cdd20e6238fafdca626eb2531a3a78c985bb680
Reviewed-on: https://review.openstack.org/13974
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
In the case of ref deletion, the null sha is not very
unique, and could be present in another project. Make
sure to compare the project, ref, and sha to determine
if two ref-updated events are equal.
Change-Id: I6bd4a02f21bd31b91ef369783641ded014f7717c
Reviewed-on: https://review.openstack.org/13839
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When commenting on Gerrit that jobs are beginning include a link to the
zuul status page if one is configured.
Change-Id: Iba5f315f4baa006939dbdcb07b0758e570242ca8
Reviewed-on: https://review.openstack.org/13858
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
I have been playing with zuul on port 8001 using curl:
curl http://127.0.0.1:8001/jenkins_endpoint
Had to `strace -f` the process to find out what was going wrong. Logging
the exception will simply make it easier to track possible issues coming
from the Jenkins Notification plugin..
Change-Id: I461146d0a6e7d63a0d8bedc3d4a6ad19e22548d6
Reviewed-on: https://review.openstack.org/13817
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fixed all pep8 errors.
E125 is ignored because of false alarms.
Change-Id: I4da60409e0095c0896230cd01bda548ed2e3f741
Reviewed-on: https://review.openstack.org/13740
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
To make port setting of gerrit server in zuul.conf really work.
Change-Id: I6f44f21256391fd04e56c18c0b293d9767c5a6a9
Reviewed-on: https://review.openstack.org/13574
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Updated pep8 version requirement to 1.3.3
Fixed E127 errors
All ignores are to be removed in the next sequence of patches
Change-Id: Ia9e922b8873686a0f905f2548cc43d534ee1c912
Reviewed-on: https://review.openstack.org/13642
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Zhongyue Luo <zhongyue.nah@intel.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
This reverts commit afd39bb3cd
This change should be rebased against the dev branch instead and
submitted there. Once the production CI freeze ends immediately
following the OpenStack Folsom release, dev changes will be merged back
into master.
Change-Id: Iabc1a0b287f0a433ca297a0d5cd1e21dd670e1d3
Reviewed-on: https://review.openstack.org/13638
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When running zuul in non-daemon mode (with -d), it is often useful to
abort it using Control + C.
Change-Id: I385a41625633f3120b95c07283bef015824e5853
Reviewed-on: https://review.openstack.org/13571
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I95ef3dde919f4ab5ce471295fb7cc1568e42402a
Reviewed-on: https://review.openstack.org/13526
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When using python-daemon 1.6, the interface has changed in an
uncompatible way. Clark pointed me to Gerritbot which solves that issue
with a simple try / catch block implemented with:
b2be72e69d
So this patch is merely a copy/paste from David "davido" Ostrovsky with
a small workaround for pyflakes issue #13 (we have to prented we are
using the variable holding the module).
Change-Id: Iffdf7fca067734fa9c09b5bddfb13f122e6251a7
Reviewed-on: https://review.openstack.org/13524
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
zuul also need the 'lockfile' and 'python-daemon' python packages.
Change-Id: I16ed4fba9705796f83aa980687de8b9d262eb715
Reviewed-on: https://review.openstack.org/13523
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Updated pep8 version requirement to 1.3.3
Fixed E127 errors
All ignores are to be removed in the next sequence of patches
Change-Id: I266498a2c64a215648a4a598b8cba5669049e37c
Reviewed-on: https://review.openstack.org/13384
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
Some puppet modules have been renamed in openstack-ci-puppet:
* `jenkins_slave` puppet module has been integrated in `jenkins` module
* `openstack-ci-config` has been renamed `openstack_project`
Change-Id: Ie4a35022301c156f74e1fdef859c87319efb60da
Reviewed-on: https://review.openstack.org/13519
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
The zuul terminology for queues was changed to pipelines to help clean
up the code and make it easier to read and debug. Update the
documentation to reflect this terminology change.
Change-Id: I8805f7ee201196b9658b8ea3546d4451aed4c700
Reviewed-on: https://review.openstack.org/13312
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Gerrit actually only speaks smart http, but we were assuming
dumb HTTP (which is provided in OpenStack's configuration).
Change-Id: Id299f53a6be6fc1670edf1da4b3353115bf1e31e
Reviewed-on: https://review.openstack.org/13146
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When Zuul tried to launch a non-existent jenkins job, the path
to mark the job as LOST involves calling back into the scheduler
from the launcher while holding the queue lock. That deadlocks,
so instead, spawn a thread to report the job as lost.
Change-Id: I6a5661638e7edba1c56eb0b3d6384283b3ecc4ed
Reviewed-on: https://review.openstack.org/12889
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The GitPython Repo object caches whether a given sha is held in an packed
or loose database. If the remote repository is repacked, that information
could be invalid. Therefore, each time the local repo is updated from the
remote, clear the object database cache.
See this file for the cache in action:
https://github.com/gitpython-developers/gitdb/blob/master/gitdb/db/base.py#L227
Change-Id: I547ba01d8cdc20f431e93e02736b498f744992c0
Reviewed-on: https://review.openstack.org/12597
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
We are not concerned with the code coverage of upstream libraries that
are used in zuul. Restrict code coverage gathering to the zuul package.
Change-Id: I62eb752c5fd0fe0289253c0b5c950382d760ea95
Reviewed-on: https://review.openstack.org/12725
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I6507896e9980eee4ac13ff8bd94f0e32947cd5d8
Reviewed-on: https://review.openstack.org/12279
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Move most of the change queue logic into the model. This should
be a little cleaner, and it should be easier to follow what
happens when a build is complete.
Run launch jobs over the whole queue.
Collapse addChange into the base manager.
Makes reading the logic around adding a change simpler. Several
hooks are provided for the dependent manager subclass to extend
the method around dependent changes.
Move onbuildcompleted and reporting methods to base class.
Simplifies the code path when a build is completed. Move to a
more generalized model where updates are applied to changes as
received, then search for changes ready to be reported, then
search for jobs that need to be launched.
Also,
* test check queue functionality
* assert all queues are empty at ends of tests
* move formatting methods to pipeline/queue classes
* add queue-only mode notification to status screen
Change-Id: I336ac289af6ebd23567ac54b359c9f38af7f2ac7
Reviewed-on: https://review.openstack.org/12277
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fix a bug where change objects were being compared incorrectly
when determining whether or not a change was in the queue
after a failed change had been dequeued. Added a test-case
that simulates the real-world conditions that exposed the bug.
Change-Id: I94a7915353335d80ab42b6c10c19595cb27788ae
Reviewed-on: https://review.openstack.org/12078
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
Change-Id: Iadd7e7f22fa31282d820bac12ba00c8f94cb8369
Reviewed-on: https://review.openstack.org/11941
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
If url_pattern is set in zuul.conf, use it instead of the build
url supplied by jenkins when reporting a change to gerrit.
Change-Id: Ic66b1efb5c7eae0dbbfd1927ee2827d13729b568
Reviewed-on: https://review.openstack.org/11908
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Change-Id: I5a9ccc857c28d458e3a8fbac03a9d29cc49b1da0
Reviewed-on: https://review.openstack.org/11904
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Set ZUUL_TEST_ROOT to put them somewhere other than /tmp.
Change-Id: Ic73ec93f39ffb6693c202bb80bbc0597dd1d7e43
Reviewed-on: https://review.openstack.org/11903
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
When a change at the head of the queue fails, immediately remove
it from the queue and re-launch jobs behind it -- there's no
need to wait for it to finish.
Change-Id: I5d8650c1aa6d56090175a7b90a7baf327f6b8158
Reviewed-on: https://review.openstack.org/11825
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
If a series of dependent changes are in the queue and one of them
fails tests and is dequeued, dequeue any changes behind it that
depend on it since they can't merge.
Change-Id: Iffe7964c2e76bc1649f14b62bf4b3fc391a8c8cb
Reviewed-on: https://review.openstack.org/11820
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Jeremy Stanley
Monty Taylor
James E. Blair
James E. Blair
Antoine Musso
Paul Belanger
Clark Boylan
Zhongyue Luo
unicell