Using the git protocol makes it hard to fetch the repo if internet
access is only possible via a proxy. An easy fix is using the https
protocol. That way the http_proxy vars on the host are automatically
obeyed and the clone via proxy works.
Change-Id: I18d75b5a16c809ac2d7834e91d32617017aea7f8
Github reviews are a new pipeline requirement that is driver specific.
Reviews can be approved, changes_requested, or comment. They can come
from people with read, write, or admin access. Access is hierarchical,
admin level includes write and read, and write access includes read.
Review requirements model loosely the gerrit approvals, allowing
filtering on username, email, newer-than, older-than, type, and
permission.
Brings in an unreleased Github3.py code. Further extends that code to
determine if a user has push rights to a repository.
Documentation is not included with this change, as the docs need
restructuring for driver specific require / reject.
Change-Id: I3ab2139c2b11b7dc8aa896a03047615bcf42adba
Signed-off-by: Jesse Keating <omgjlk@us.ibm.com>
This makes the transition to python3 much smoother.
Change-Id: I9d8638dd98502bdd91cbe6caf3d94ce197f06c6f
Depends-On: If6bfc35d916cfb84d630af59f4fde4ccae5187d4
Depends-On: I93bfe33f898294f30a82c0a24a18a081f9752354
Github reporter can be configured to merge pull reqeusts.
When there are multiple merges called at the same time, it leads to a
situation when github returns 405 MethodNotAllowed error becuase github
is checking the branch mergeability.
When we encounter this situation, we try to wait a bit (2 seconds for
now) and try to merge again.
Pre-release version of Github3.py has to be used, because the latest
released version 9.4 has a bug in merge method. Furthermore the newest
merge method supports specifying exact sha to be merged, which is
desirable to ensure that the exact commit that went through the pipeline
gets merged.
Both are already fixed in the stable branch, but not yet released on
PyPi. See:
90c6b7c2656ef02cb33f
Change-Id: I0c3abbcce476774a5ba8981c171382eaa4fe0abf
Reading historical voluptuous docs it appears that the change to dict
handling occurred in the new 0.10.2 release. There are no earlier 0.10
releases to check against so just require voluptuous>=0.10.2.
Change-Id: I5ade4a8c2d03d5519ae1ed95e133717d5c28d0ad
Every project should have a public and private key to encrypt secrets.
Zuul expects them to already exist under /var/lib/zuul/keys on the
scheduler host. If an operator manages these keys externally, they
should simply be placed there. If they are not found, Zuul will
create them on startup and store them there so they will be found on
the next run.
The test framework uses a pre-generated keypair most of the time to
save time, however, a test is added to ensure that the auto-generate
code path is run.
Co-Authored-By: James E. Blair <jeblair@redhat.com>
Change-Id: Iedf7ce6ca97fab2a8b800158ed1561e45899bc51
It appears newer versions of GitPython have slowed considerably.
Cap GitPython until https://github.com/gitpython-developers/GitPython/issues/605
is resolved.
Change-Id: Ie6c8722e8b607bb50e77fbad59e18363616f7e0d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This will allow us to enter results from all jobs for
use with the openstack-health dashboard.
Depends-On: I08dbbb64b3daba915a94e455f75eef61ab392852
Change-Id: I28056d84a3f6abcd8d9038a91a6c9a3902142f90
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Add a requirement on kazoo and add a Zookeeper chroot to the test
infrastructure.
This is based on similar code in Nodepool.
Change-Id: Ic05386aac284c5542721fa3dcb1cd1c8e52d4a1f
We need to pin parmakio until we fix an expection in
zuul/connection/gerrit.py client.load_system_host_keys()
Traceback (most recent call last):
File "/opt/ansible-role-zuul/git/openstack-infra/zuul/zuul/connection/gerrit.py", line 168, in _run
client.load_system_host_keys()
File "/usr/lib/python2.7/site-packages/paramiko/client.py", line 101, in load_system_host_keys
self._system_host_keys.load(filename)
File "/usr/lib/python2.7/site-packages/paramiko/hostkeys.py", line 101, in load
e = HostKeyEntry.from_line(line, lineno)
File "/usr/lib/python2.7/site-packages/paramiko/hostkeys.py", line 331, in from_line
key = RSAKey(data=decodebytes(key))
File "/usr/lib/python2.7/site-packages/paramiko/rsakey.py", line 58, in __init__
).public_key(default_backend())
File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/__init__.py", line 35, in default_backend
_default_backend = MultiBackend(_available_backends())
File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/__init__.py", line 22, in _available_backends
"cryptography.backends"
AttributeError: 'EntryPoint' object has no attribute 'resolve'
Change-Id: Ifc12a1ec9f26c0b236ed00e128b707c6fba58b1e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
argparse was external in python 2.6 but not anymore, remove it from
requirements.
This should help with pip 8.0 that gets confused in this situation.
Installation of the external argparse is not needed.
Change-Id: Ib7e74912b36c1b5ccb514e31fac35efeff57378d
Note that ansible is licensed under the GPL which may have
implications for the distribution of Zuul as a whole.
Actually use ansible to launch a hello world test.
Also reorganize the launcher directory to be more like
the mergers: a client and server.
Change-Id: I30f7ed2cadbb7d2381ec477e229d19c9c6e41743
Looking at the library, and openstack requirements, there doesn't
appear to be a reason to cap webob. So, remove the cap so we can use
newer versions.
Change-Id: Id19c297b540e9081bcf733dff3a334a4b0f477d8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This patch upgrades zuul to support APScheduler 3.0. For the most
part, 3.0 was a rewrite but our changes seem to be limited.
Change-Id: I0c66b5998122c3f59ed06e3e7b3ab3199f94f478
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
There is no API or behavioral change required, but gear 0.5.7 has
significant performance improvements, enough so that it is required
for OpenStack's installation, so bump the requirement here.
Gear 0.5.6 has a regression that causes Zuul tests (but not actual
operation) to fail.
Also includes an unrelated test race fix: the order in which jobs
completed in the test_crd_check_git_depends test was random. Switch
to releasing them in sequence for easier comparison with the build
history at the end.
Change-Id: I281356a620c017a546ec04a0a767cf34c1fc62bf
Sometimes, even after cloning a new repository, zuul will fail to
merge the change with a stack trace shown below.
I was using GitPython==0.3.2.1. Upgrading it to >= 0.3.3 resolves the issue.
2015-01-22 08:27:48,222 ERROR zuul.Merger: Exception while merging a change:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/zuul/merger/merger.py", line 234, in _mergeChange
commit = repo.merge(item['refspec'], 'resolve')
File "/usr/local/lib/python2.7/dist-packages/zuul/merger/merger.py", line 132, in merge
self.fetch(ref)
File "/usr/local/lib/python2.7/dist-packages/zuul/merger/merger.py", line 145, in fetch
origin.fetch(ref)
File "/usr/local/lib/python2.7/dist-packages/git/remote.py", line 598, in fetch
return self._get_fetch_info_from_stderr(proc, progress or RemoteProgress())
File "/usr/local/lib/python2.7/dist-packages/git/remote.py", line 540, in _get_fetch_info_from_stderr
for err_line, fetch_line in zip(fetch_info_lines, fetch_head_info))
File "/usr/local/lib/python2.7/dist-packages/git/remote.py", line 540, in <genexpr>
for err_line, fetch_line in zip(fetch_info_lines, fetch_head_info))
File "/usr/local/lib/python2.7/dist-packages/git/remote.py", line 252, in _from_line
raise ValueError("Failed to parse line: %r" % line)
ValueError: Failed to parse line: 'Total 7 (delta 0), reused 7 (delta 0)'
This is the more specific GitPython Commit that resolves the issue:
d48ed95cc7
Closes-Bug: #2000130
Change-Id: I1992b492e8c80322cc6ae01f6e2ebc181a9de813
Since keystoneclient pulls in lxml which needs various system
library headers to build its bindings, and swiftclient depends on
keystoneclient, and zuul handles these as optional dependencies,
make them test requirements.
Change-Id: I5c7db10644dcf2810cbfdfe8c11d4790909c04cf
The Zuul client uses babel.dates.format_timedelta which was introduced
with 1.0 http://babel.pocoo.org/docs/changelog/#version-1-0
Ubuntu Precise has 0.9.6 causing the Zuul client to abort because of the
missing method.
Change-Id: I60ed0d61c97091c17a926770ed1a02f1156ec5e0
According to https://wiki.openstack.org/wiki/Python3 dict.iteritems()
should be replaced with six.iteritems(dict).
Added six >= 1.6.0 as new requirement. Removed six from test requirements.
Change-Id: Ifba282b1bde9681268e6e46ffd024c96c6b9be05
Have zuul send signed credentials as part of the job for workers to
consume and upload assets to a defined location.
Zuul currently doesn't care about logs however this change will
suggest a LOG_PATH to builders as a unqiue destination prefix
allowing zuul to know preemptively the destination.
The workers are still required to send a URL of the final location.
Change-Id: I042cdd2dd2407f381cafcabc5c6b83d9b9a9eb00
* Remove mention of Jenkins from the codebase
* Remove unused dependency on python-jenkins
* Update docs to be suggestive of Jenkins but not require
Change-Id: I15f5462466899fb520aa66669f2e0b1d4fa1b98a
* requirements.txt: The latest release of the gear library fixes an
enqueuing/function registration order problem exposed by the recent
zuul merger fan-out implementation. Insist upon it.
Change-Id: Id2275477b12ce62fc485e5769029fe107a82b52d
Voluptuous 0.7.0 introduced a backward API incompatible change in
upstream commit 475adebc:
https://github.com/alecthomas/voluptuous/commit/475adebc
The schema are precompiled and the validate_X have been removed,
voluptuous is now smart enough to detect the type of value being
validated and would call an internal validation method matching the
type.
Commit is contained since 0.7.0:
$ git tag --contains 475adebc
0.7.0
0.7.1
0.7.2
0.8.1
0.8.2
$
I tested it using etc/layout.yaml-sample and added an inexistent
pipeline to the project. The test yield:
voluptuous.MultipleInvalid: extra keys not allowed @
data['projects'][0]['nonpipe']
Debian has recently accepted voluptuous 0.8.2 in testing, so if we want
to package Zuul, we better have to upgrade our voluptuous requirement as
well. Ref: http://packages.qa.debian.org/v/voluptuous.html
Change-Id: I117ea644863b2e4a4dc3429aa81e868573382877
Where we're using the same libraries as OpenStack, sync with the
OpenStack versions. Just to be nice.
Change-Id: I8e90d2a8945d62e962b813c6396f0e7db4e14222
d2to1 pulls in setuptools, which trips the unhappy bugs with setuptools
updating. Move past that and just use new pbr.
Change-Id: I2609eda10ed781116940c3607ff85a14fc4b7f58
* doc/source/zuul.rst: Document SIGUSR2 behavior.
* zuul/cmd/server.py: When SIGUSR2 is received log stack traces for all
active running threads. This is useful for debugging deadlock
situations. Note that this makes use of sys._current_frames which may
not play nice with all implementations of Python.
*tests/test_stack_dump.py: Test the stack dump signal handler with a new
test file, class, and test method.
* requirements.txt: Add argparse to requirements list so that py26 tests
can pass when zuul.cmd.server is imported.
Change-Id: I8ad8155b16f324e832c191f0a619ff89ef804a87
Change-Id: I0385ac9f8336a8c2e45b0fa6ed062b6b23a182aa
Reviewed-on: https://review.openstack.org/33351
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Tobias Henkel