Gear is ready to make a release, but we're about to remove it.
Pin to less than the upcoming release to retain stability while
we finish up removal.
Change-Id: I4822c212f0d1fc018e92d01d162868e274dbd6ba
This change adds a new prometheus_port option to start a metric server
to be scrapped by a prometheus service. By default, the server exposes
process informations.
Change-Id: Ie329df6adc69768dfdb158d00283161f8b70f07a
An SQL database is an additional requirement so we should also include
the client libraries for it.
Change-Id: Id3a8b55928d9ce9e8764f18a64a42bb2f14f5ecb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The release of pyjwt 2.0.0 changed the behavior of some functions, which
caused errors. Fix the errors, use pyjwt 2.0.0's better handling of JWKS,
and pin requirement to 2.X to avoid future potential API breaking changes.
Change-Id: Ibef736e0f635dfaf4477cc2a90a22665da9f1959
It has the capability to index build and buildset results.
With the help of tools like Kibana, advanced analytics
dashboard could be built on top of the Zuul Elasticsearch
index.
Optionally job's variables and zuul_return data can be
exported along with build results under the job_vars and
job_returned_vars fields.
Change-Id: I5315483c55c10de63a3cd995ef681d0b64b98513
Removes outdated workaround on importlib-resources.
This fixes issue with py39+, where stdlib already provides it.
Change-Id: I82264c293ff9072e970e81417658fc934cccea2d
Release 2.0.0 on December 22, 2020 brought breaking changes. Pin to the
last working version until we can fix the breaks properly.
Change-Id: I2e6cc36d373f8c774eda5f5caec9868f74023729
Branch protection rules in github are fn patterns which are currently
matched locally in zuul. This is error prone and can lead in edge
cases to wrong matches resulting in wrong enqueue decisions into gate
pipelines. When requesting branch protection rules in github we also
can request the matching refs along with the rules. This is much safer
since we can plain text match them against the change branch.
Change-Id: Ic995d4b2e16a5d741f0209fa9236959d8f4d10b9
These various exceptions are raised as certain parts of the
configuration are parsed. Although they give you a lot of information
about the issue, they don't give you any context for what part of your
configuration caused the error to be raised.
They are all working on a particular item of the configuration that I
think it would be very useful to show in the error output.
This was inspired by me putting playbooks in the .zuul.d directory,
where they were read as config files. In hindsight it was obvious,
but it took me a while to figure out what was going on. With this it
would have reported something like:
Zuul encountered a syntax error while parsing its configuration in
the repo org/project1 on branch master. The error was:
Configuration item has more than one key. Each zuul.yaml
configuration file must be a list of dictionaries with a
single key, for example:
- job:
name: foo
... blah ...
The incorrect values are:
hosts: all
tasks:
- debug:
msg: blah
name: foo
which would have clued me in immediately.
This truncates the message if it is too long; which might be the case
if dumped a playbook or foreign config file. However I think
practically most errors are typo level on jobs/templates/etc. that
will show nicely.
The YAML version is updated to support non-sorted dumping of the keys.
This is particularly helpful with Python 3.7+ and default ordered
dicts.
Change-Id: I4851a5a796fa452a023e0e6193fc724ae0967a44
Current set of dependencies brings latest cherrypy which is
incompatible with cheroot we need. This pins cherrypy to last
version that is compatible. The current verision of pip will
still install cherrypy, but the upcoming depsolver is unable to
resolve the conflict.
We also add a fail-safe `pip check` that runs the new depsolver
and should prevent a similar regressions from appearing.
Change-Id: I2b0fa43b906d0919854d30da0ee825002444b201
I installed Zuul 18.0 on a CentOS 7 with Python 3.6 from RH SCL repos
which comes with virtualenv==15.1.0. This is effectively too old to
provision the virtual environments for Ansible, likely because that
version of virtualenv hardcodes PIP 9. During the installation,
zuul-executor dies with this error:
Installing collected packages: MarkupSafe, jinja2, PyYAML, pycparser, cffi, six, pynacl, cryptography, bcrypt, paramiko, ansible, XStatic-Bootstrap-SCSS, pyfakefs, XStatic-Patternfly-Bootstrap-Treeview, wcwidth, pyparsing, pyperclip, cmd2, pbr, stevedore, PrettyTable, cliff, XStatic, XStatic-Patternfly, wrapt, debtcollector, msgpack, pytz, netaddr, iso8601, netifaces, Babel, oslo.i18n, oslo.utils, oslo.serialization, itsdangerous, Werkzeug, click, Flask, SQLAlchemy, Flask-SQLAlchemy, XStatic-jQuery, Mako, python-editor, python-dateutil, alembic, Flask-Migrate, pygments, junit-xml, Flask-Script, Frozen-Flask, extras, linecache2, traceback2, argparse, unittest2, fixtures, python-mimeparse, testtools, python-subunit, XStatic-DataTables, ara, os-service-types, urllib3, chardet, idna, certifi, requests, keystoneauth1, jsonpointer, jsonpatch, appdirs, munch, decorator, jmespath, requestsexceptions, dogpile.cache, openstacksdk, dictdiffer, websocket-client, pyasn1, rsa, pyasn1-modules, cachetools, google-auth, oauthlib, requests-oauthlib, kubernetes, python-string-utils, ruamel.yaml.clib, ruamel.yaml, openshift, google-resumable-media, protobuf, googleapis-common-protos, google-api-core, google-cloud-core, google-cloud-storage
Running setup.py install for PyYAML: started
Running setup.py install for PyYAML: finished with status 'done'
Running setup.py install for ansible: started
Running setup.py install for ansible: finished with status 'done'
Running setup.py install for XStatic-Patternfly-Bootstrap-Treeview: started
Running setup.py install for XStatic-Patternfly-Bootstrap-Treeview: finished with status 'error'
Complete output from command /var/lib/zuul/ansible-bin/2.6/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-24z_rrjk/XStatic-Patternfly-Bootstrap-Treeview/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-6ie_p40b-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/lib/zuul/ansible-bin/2.6/include/site/python3.6/XStatic-Patternfly-Bootstrap-Treeview:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/setuptools/__init__.py", line 10, in <module>
from setuptools.extern.six.moves import filter, filterfalse, map
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/setuptools/extern/__init__.py", line 1, in <module>
from pkg_resources.extern import VendorImporter
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3017, in <module>
@_call_aside
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3003, in _call_aside
f(*args, **kwargs)
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3045, in _initialize_master_working_set
dist.activate(replace=False)
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2577, in activate
declare_namespace(pkg)
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2151, in declare_namespace
_handle_ns(packageName, path_item)
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2091, in _handle_ns
_rebuild_mod_path(path, packageName, module)
File "/var/lib/zuul/ansible-bin/2.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2120, in _rebuild_mod_path
orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
This points to https://github.com/pypa/pip/issues/4216, so the fix is
apparently to make sure that there's new enough PIP + setuptools in the
created virtual env, and these versions are determined by host's
virtualenv module (thanks to webknjaz on IRC for explaining this to me).
After a restart zuul-executor runs normally, but as a side effect not
all required Python modules are installed for Ansible's usage. For me,
this meant that (e.g.) `upload-logs-swift` won't work ebcause there's no
"openstack" module.
I don't know how to make zuul-executor error persistent, but let's at
least make the life easier for the next poor sod installing this SW on
a system where "virtualenv" is "too old" -- which is also the latest
available Ubuntu if I'm checking their revision numbers correctly.
Require virtualenv >= 20 while preserving the blacklist from commit
7c6503aba. Given that Zuul's own CI uses PIP to pull the latest
virtualenv for the test suite, make sure that the users get what was
tested.
Change-Id: I2b53e532b63a9ae20e817d9599c46cccf613e425
Replace our tox-py37 job with tox-py38, extend the list of trove
classifiers for Python versions in package metadata, and replace the
"py35" in the tox.ini envlist with just "py3" so that folks running
`tox` unqualified on their systems will use whatever python3
interpreter they have on hand (odds are it's in our supported range
these days). Also uncap python-daemon so we use a version compatible
with Python >=3.8.
Change-Id: I4b382bb89847d3ae2b20292d923c3c2211dfb254
The global tox installation on our test nodes is affected by an
upstream issue[1]. Additionally, the virtualenv created by tox
under those conditions is also affected. To work around this,
pin the version of importlib-resources in the global tox install,
as well as in Zuul's own requirements.
[1] https://gitlab.com/python-devs/importlib_resources/issues/83
Change-Id: I31ed50185a71d867a2ad512ef9b526c5b607ed5c
The canMerge check is executed whenever zuul tests if a change can
enter a gate pipeline. This is part of the critical path in the event
handling of the scheduler and therefore must be as fast as
possible. Currently this takes five requests for doing its work and
also transfers large amounts of data that is unneeded:
* get pull request
* get branch protection settings
* get commits
* get status of latest commit
* get check runs of latest commit
Especially when Github is busy this can slow down zuul's event
processing considerably. This can be optimized using graphql to only
query the data we need with a single request. This reduces requests
and load on Github and speeds up event processing in the scheduler.
Since this is the first usage of graphql this also sets up needed
testing infrastructure using graphene to mock the github api with real
test data.
Change-Id: I77be4f16cf7eb5c8035ce0312f792f4e8d4c3e10
GitPython was capped to <3.0.0 due to increase test times. According
to the upstream issue [1] this should, be fixed upstream. See if we
can uncap it now.
[1] https://github.com/gitpython-developers/GitPython/issues/906
Change-Id: I62a2731c4045f72060a67b8ad3eb89f6b07a0e97
There may have been a behavior change with polling that causes a
surplus of log lines. Avoid that version until we understand more.
Change-Id: I246acda9352267eb6b9be6d1b06fb87b133709ec
virtualenv==20.0.0 introduced new symlinking of "seed" packages that
broke within a bwrap context. We fixed this by forcing those seed
packages to be installed by pip avoiding any symlinking. Since then
upstream virtualenv has realize this was bad default behavior and
virtualenv==20.0.2 does not symlink by default.
Take advantage of this and exclude 20.0.0 and 20.0.1 as they need
special handling. This allows other versions (new and old) to function.
Change-Id: I3b993e72b52e2234e046a657280b008887c535a3
New virtualenv has changed behavior and cli arguments. In particular the
seed packages are by default installed in a common lib location and not
updated. These packages are then symlinked to in the virtualenv. This
causes problems for zuul when we run ansible in bwrap and that common
lib path isn't bind mounted in the container.
We can fix this by telling virtualenv to install the seed packages uses
pip and not its internal bundled app data. This is preferable for Zuul
because then we don't have to manage permissions and locations of a
separate seed package library location.
Note that we force virtualenv to be >=20.0.0 as old virtualenv does not
have this new seeder option.
Change-Id: Ib9b2e0b51e6acd744d1c23b4a27067f0202a679d
CherryPy uses cheroot underneath as http server. The release 8.1.0
introduced a regression which leads to occasional connection resets
while handling concurrent requests [1]. Cap it until there is a
release available that fixes this issue.
[1] https://github.com/cherrypy/cheroot/issues/263
Change-Id: I39716e9c203cc42ab320cb47e090fbc8f67320c9
Instead of preventing newer version which would create conflicts, we
just avoid the known broken versions.
Change-Id: I7e04bd6efddaab9847c01c48a2e6ef4be1a41e4e
This looks like it might be extending our unit test runtime by
50% (from 30 mins to 45, thefore pushing the overall job from
45 to 60+ mins).
Change-Id: I333cc1dbe5b2223e7d6f86ff7cdb6d5f063bda39
Allow an operator to define authorization rules.
Allow an operator to add authorization rules to a tenant.
Add a rule parser and a rule registry.
The authZ engine is not plugged in yet.
Change-Id: I3a86c6c7d62ad2bce68a98dbd2fff18549b94fb9
As a first step towards supporting multiple ansible versions we need
tooling to manage ansible installations. This moves the installation
of ansible from the requirements.txt into zuul. This is called as a
setup hook to install the ansible versions into
<prefix>/lib/zuul/ansible. Further this tooling abstracts knowledge
that the executor must know in order to actually run the correct
version of ansible.
The actual usage of multiple ansible versions will be done in
follow-ups.
For better maintainability the ansible plugins live in
zuul/ansible/base where plugins can be kept in different versions if
necessary. For each supported ansible version there is a specific
folder that symlinks the according plugins.
Change-Id: I5ce1385245c76818777aa34230786a9dbaf723e5
Depends-On: https://review.openstack.org/623927
This updates the github driver to cache PRs by sha using cachetool's
LRUCache.
We make this change because we need to cache closed PRs so can't rely on
the action of closing a PR to remove the PR from the cache. Since we
don't have a good method of evicting entries we fall back to LRU with
a reasonable cache size (2k commits).
Change-Id: I5fb6c8b33f9eed221a8b84e537f02e7dccf2d2df
If the gearman server vanishes (e.g. due to a VM crash) some clients
like the merger may not notice that it is gone. They just wait forever
for data to be received on an inactive connection. In our case the VM
containing the zuul-scheduler crashed and after the restart of the
scheduler all mergers were waiting for data on the stale connection
which blocked a successful scheduler restart. Using tcp keepalive we
can detect that situation and let broken inactive connections be
killed by the kernel.
Depends-On: I8589cd45450245a25539c051355b38d16ee9f4b9
Change-Id: I30049d59d873d64f3b69c5587c775827e3545854
18.0.1 has been released an fixes:
https://github.com/cherrypy/cherrypy/pull/1736
Change-Id: Iffda4376e1393020d54ba6e25d0dda07616d72c6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
With the latest cherrypy version the tests run into the hard test
timeout. This is caused by a header conversion issue introduced in
cherrypy 18.0.0. There is a pull request [1] that fixes this
issue. Once this is merged and released we can uncap it again.
The newest reno release, 2.10.0, requires a feature of newer sphinx without
specifying that the minimum compatibility has changed. Uncap sphinx here
so at least the doc build is not broken, use Sphinx 1.6.1 since that is
the oldest sphinx version supporting what reno needs.
[1] https://github.com/cherrypy/cherrypy/pull/1736
Change-Id: Ic00e664152062d1fc1677e467b638ac2be99aef1
Co-Authored-By: Andreas Jaeger <jaegerandi@gmail.com>
* Aiohttp (and related libraries) have a python support policy
which is causing us problems.
* Cherrypy supports threads which integrates well with the rest
of Zuul.
Change-Id: Ib611df06035890d3e87fc5ad92fdfc7ac441edce
Our custom command.py Ansible module is updated to match the
version from 2.5, plus our additions.
strip_internal_keys() is moved within Ansible yet again.
Change-Id: Iab951c11b23a24757cf5334b36bc8f7d12e19db0
Depends-On: https://review.openstack.org/567007
async-timeout 3.0.0 has released and only supports python 3.5.3 (which
is greater than the Xenial we test on). Pin it for lower pythons
(this is similar to I460652a4468bfa76895a5c563612ff6119c0d483 which
pinned yarl -- in that case it was stated it was 1.1.1 which would be
the final release for python<3.5.3 support; however in this case it's
not clear if maybe there might be more 2.X releases, so hence the
less-than qualifier...)
Change-Id: Ifa2ddedbd4f431d8cd08059e31814b95dc74e368
As described in [1], the latest releases of yarl (brought in via
aiohttp) require python 3.5.3. Pin it for lower versions (which
happens to be 3.5.2 in xenial ... which we test on) to the last
release that works (1.1.1).
[1] https://github.com/aio-libs/yarl/issues/189
Change-Id: I460652a4468bfa76895a5c563612ff6119c0d483
The github status requirements matching and trigger filter are
currently plain text matching based. This currently limits sharing of
pipeline definitions between tenants as zuul reports the status as
'<tenant>/<pipeline>'. This currently makes it necessary to define
trigger filter for each tenant [1] and completely blocks pipeline
requirements.
A solution to this is regex matching which makes it possible to define
the filter once [2].
Further this enables an interesting further use case to trigger on any
successfull status [3]. This makes it easier to cooperate with other
CI systems or github apps which also set a status.
Directly use re2 as this will be used in the future for regex
matching.
[1] Trigger filter snippet
trigger:
github:
- event: pull_request
action: status
status:
- zuul:tenant1/check:success
- zuul:tenant2/check:success
- zuul:tenant3/check:success
- zuul:tenant4/check:success
[2] Regex trigger filter snippet
trigger:
github:
- event: pull_request
action: status
status:
- zuul:.+/check:success
[3] Generic success filter snippet
trigger:
github:
- event: pull_request
action: status
status:
- .*:success
Change-Id: Id1b9d7334db78d0f13db33d47a80ffdb65f921df
This updates both the dependency to ansible 2.4 and also ports in the
needed changes to the command module.
Version 2.4.0 definitely does not work for us because YAML
hosts file parsing is broken, but 2.4.1 and greater should
be fine.
Change-Id: I63f72b45ecb9533eac5ba9eb0eef426beec905e3
The latest change on github3.py broke us [1]. As a temporary quick fix
pin it to a revision prior to this change until we have a proper way
how to deal with this.
[1] Trace:
2018-03-12 12:48:36,299 ERROR zuul.Scheduler: Exception in management event:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/github3/models.py", line 47, in __init__
self._update_attributes(json)
File "/usr/lib/python3.6/site-packages/github3/repos/repo.py", line 2720, in _update_attributes
self.original_license = repo['license']
KeyError: 'license'
Change-Id: I22f62ea6b6d621a8b817542e72fe3a762a79a491
James E. Blair
Tristan Cacqueray
Paul Belanger
Matthieu Huin
Fabien Boucher
Sorin Sbarnea
Ian Wienand
Tobias Henkel
James E. Blair
Jan Kundrát
Jeremy Stanley
Clark Boylan
David Shrewsbury
Jesse Keating
Monty Taylor