The Gatekeeper, or a project gating system
Go to file
Tobias Henkel 1fab39cc4b
Parallelize github event processing
The GitHub driver does an event pre processing before adding a trigger
event to the scheduler. This is currently done single threaded to
ensure that the trigger events are enqueued in order into the
scheduler. A problem is that this pre processing can take a few
seconds which limits the rate of events we can process. In order to
parallelize this while keeping the order of the trigger events we need
to do two things.

We keep consuming the event queue single threaded but instead of
processing and forwarding them directly to the scheduler we process
them in a thread pool and put the futures into a result queue. This
second queue can then again be processed single threaded and maintains
the correct ordering of the events.

Second updating the change cache currently assumes that it runs single
threaded. In order to avoid data races we need to lock this by the
change.

Change-Id: I08f31b99e2a58e51cef0de89edd98c957d7db87f
2019-06-11 10:38:51 +02:00
doc Proposed spec: tenant-scoped admin web API 2019-06-07 09:56:34 -07:00
etc Fix indent error for layout.yaml-sample. 2018-06-20 12:20:23 +08:00
playbooks Install latest git-review from PyPI in quickstart 2019-05-20 17:46:33 +00:00
releasenotes/notes Report tenant and project specific resource usage stats 2019-05-29 04:10:08 +00:00
tests Parallelize github event processing 2019-06-11 10:38:51 +02:00
tools Merge "Use nodejs v10 in testing" 2019-06-10 15:57:12 +00:00
web Cap xterm.js to < 3.14 2019-06-10 19:22:02 +02:00
zuul Parallelize github event processing 2019-06-11 10:38:51 +02:00
.coveragerc Revert "Revert "Switch to stestr"" 2018-05-17 08:33:40 -07:00
.dockerignore Add web/node_modules to dockerignore 2019-01-27 11:23:45 +01:00
.gitignore Fix ignored but tracked .keep file 2018-12-02 09:12:25 +01:00
.gitreview OpenDev Migration Patch 2019-04-19 19:25:28 +00:00
.mailmap Fix pep8 E127 violations 2012-09-26 14:23:10 +00:00
.stestr.conf Revert "Revert "Switch to stestr"" 2018-05-17 08:33:40 -07:00
.zuul.yaml Use nodejs v10 in testing 2019-05-30 14:27:30 -07:00
COPYING Update README and add GPL license 2018-03-19 09:25:52 -07:00
Dockerfile Cleanup executor specific requirements 2019-04-04 08:58:04 +02:00
LICENSE Initial commit. 2012-05-29 14:49:32 -07:00
MANIFEST.in manifest: add zuul/ansible and ansible-config.conf 2019-03-18 08:07:00 +00:00
README.rst Update references for opendev 2019-04-24 12:59:17 +00:00
TESTING.rst tox: Integrate tox-docker 2019-04-03 15:05:42 +01:00
bindep.txt Install virtualenv from source 2019-03-16 11:07:58 +01:00
requirements.txt Install virtualenv from source 2019-03-16 11:07:58 +01:00
setup.cfg Cleanup executor specific requirements 2019-04-04 08:58:04 +02:00
setup.py Partial sync with OpenStack requirements. 2013-09-25 15:30:37 -07:00
test-requirements.txt Cap mypy 2018-12-07 15:00:17 -08:00
tox.ini Merge "tox: Integrate tox-docker" 2019-05-17 05:42:08 +00:00

README.rst

Zuul

Zuul is a project gating system.

The latest documentation for Zuul v3 is published at: https://zuul-ci.org/docs/zuul/

If you are looking for the Edge routing service named Zuul that is related to Netflix, it can be found here: https://github.com/Netflix/zuul

If you are looking for the Javascript testing tool named Zuul, it can be found here: https://github.com/defunctzombie/zuul

Getting Help

There are two Zuul-related mailing lists:

zuul-announce

A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.

zuul-discuss

General discussion about Zuul, including questions about how to use it, and future development.

You will also find Zuul developers in the #zuul channel on Freenode IRC.

Contributing

To browse the latest code, see: https://opendev.org/zuul/zuul To clone the latest code, use git clone https://opendev.org/zuul/zuul

Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/zuul

Suspected security vulnerabilities are most appreciated if first reported privately following any of the supported mechanisms described at https://zuul-ci.org/docs/zuul/user/vulnerabilities.html

Code reviews are handled by gerrit at https://review.openstack.org

After creating a Gerrit account, use git review to submit patches. Example:

# Do your commits
$ git review
# Enter your username if prompted

Join #zuul on Freenode to discuss development or usage.

License

Zuul is free software. Most of Zuul is licensed under the Apache License, version 2.0. Some parts of Zuul are licensed under the General Public License, version 3.0. Please see the license headers at the tops of individual source files.

Python Version Support

Zuul v3 requires Python 3. It does not support Python 2.

As Ansible is used for the execution of jobs, it's important to note that while Ansible does support Python 3, not all of Ansible's modules do. Zuul currently sets ansible_python_interpreter to python2 so that remote content will be executed with Python 2.