61eea2169b
We originally wrote the change list to be a best-effort service for the scheduler check for whether a change is in a pipeline (which must be fast and can't lock each of the pipelines to read in the full state). To make it even simpler, we avoided sharding and instead limited it to only the first 1024 changes. But scope creep happened, and it now also serves to provide the list of relevant changes to the change cache. If we have a pipeline with 1025 changes and delete one of them from the cache, that tenant will break, so this needs to be corrected. This change uses sharding to correct it. Since it's possible to attempt to read a sharded object mid-write, we retry reads in the case of exceptions until they succeed. In most cases this should still only be a single znode, but we do truncate sharded znodes, so there is a chance even in the case of a small number of changes of reading incorrect data. To resolve this for all cases, we retry reading until it succeeds. The scheduler no longer reads the state at the start of pipeline processing (it never needed to anyway), so if the data become corrupt, a scheduler will eventually be able to correct it. In other words, the main pipeline processing path only writes this, and the other paths only read it. (An alternative solution would be to leave this as it was and instead load the full pipeline state for maintaining the change cache; that runs infrequently enough that we can accept the cost. This method is chosen since it also makes other uses of this object more correct.) Change-Id: I132d67149c065df7343cbd3aea69988f547498f4 |
||
---|---|---|
doc | ||
etc | ||
playbooks | ||
releasenotes/notes | ||
tests | ||
tools | ||
web | ||
zuul | ||
.coveragerc | ||
.dockerignore | ||
.gitignore | ||
.gitreview | ||
.mailmap | ||
.stestr.conf | ||
.zuul.yaml | ||
COPYING | ||
Dockerfile | ||
LICENSE | ||
MANIFEST.in | ||
README.rst | ||
TESTING.rst | ||
bindep.txt | ||
reno.yaml | ||
requirements.txt | ||
setup.cfg | ||
setup.py | ||
test-requirements.txt | ||
tox.ini |
README.rst
Zuul
Zuul is a project gating system.
The latest documentation for Zuul v3 is published at: https://zuul-ci.org/docs/zuul/
If you are looking for the Edge routing service named Zuul that is related to Netflix, it can be found here: https://github.com/Netflix/zuul
If you are looking for the Javascript testing tool named Zuul, it can be found here: https://github.com/defunctzombie/zuul
Getting Help
There are two Zuul-related mailing lists:
- zuul-announce
-
A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.
- zuul-discuss
-
General discussion about Zuul, including questions about how to use it, and future development.
You will also find Zuul developers in the #zuul channel on Freenode IRC.
Contributing
To browse the latest code, see: https://opendev.org/zuul/zuul To clone the latest code, use git clone https://opendev.org/zuul/zuul
Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/zuul
Suspected security vulnerabilities are most appreciated if first reported privately following any of the supported mechanisms described at https://zuul-ci.org/docs/zuul/user/vulnerabilities.html
Code reviews are handled by gerrit at https://review.opendev.org
After creating a Gerrit account, use git review to submit patches. Example:
# Do your commits
$ git review
# Enter your username if prompted
Join #zuul on Freenode to discuss development or usage.
License
Zuul is free software. Most of Zuul is licensed under the Apache License, version 2.0. Some parts of Zuul are licensed under the General Public License, version 3.0. Please see the license headers at the tops of individual source files.
Python Version Support
Zuul requires Python 3. It does not support Python 2.
Since Zuul uses Ansible to drive CI jobs, Zuul can run tests anywhere Ansible can, including Python 2 environments.