The Gatekeeper, or a project gating system
Go to file
James E. Blair d4c7d29360 Clarify merger updates and resets
Several changes in an attempt to clarify exactly when updates and
resets should and do happen:

* Remove the repo_state argument from Merger.getRepo()

It was unclear under what circumstances the low-level repo object
honored repo_state (not much).  Remove it entirely and rely on
high-level Merger methods to deal with repo_state.

* Have merger.setRepoState() operate on one project instead of a
  list of items

Part of the reason we were passing repo_state to low-level
methods was to reset the state for required projects in the
executor.  Essentially there were three cases: projects of change
items, projects of non-change items, and projects of neither but
in required-projects.  The low-level repo_state usage only
handled the last, the first is easy, and the second we handled by
creating a list of non-change items and passing it to
setRepoState on the merger.

A simpler method of handling all of that is to reduce it to two
cases: projects of change items (which need to be merged) and the
rest (which need to be restored).  If we do that, we can maintain
a set of projects we've seen while merging in the first case,
then iterate over all the remaining projects and call
setRepoState on each in the second.

* Remove the update call from Repo.reset()

This lets us call Repo.reset() frequently (i.e., at the start of
any operation that writes to the merger's git repo working dir)
without performing a git fetch.  We need to make sure we call
Repo.update() where necessary.

* Remove the reset call from Merger.updateRepo()

This will now only call repo.update(), and even that will only
happen if the repo_state says we should.  So we can safely call
this before any significant operations and know that it will
update the repo if necessary.

* Add an update() call to getRepoState()

Because we removed the update() call from Repo.reset(), we need
to add one here next to the existing call to reset().

* Add a reset call to getFiles()

It relied on the reset in updateRepo.

* Set execution_context to False on the executor's main merger

The execution_context parameter determines whether we manipulate the
origin remotes to point at the previous commit.  This should be set
for mergers that operate on the build work dir, but it should not
be set for the main merger within the executor (so the main merger
behaves just like a standalone merger).  It previous was erroneously
set for the executor's main merger and this change corrects that.

* Add Merger.updateRepo() calls in the merger server merge method

The merger needs to update and reset each repo before merging changes.
Currently _mergeItem resets the repo the first time it encounters it.
But we still need to update the repo.  We don't want to update within
the merger method because the executor performs batch updates in
parallel before starting a merge and we don't want to re-do that work.
So instead we add it to the merger server invocation, so it's only
used in the merger:merge gearman function code path.

Change-Id: I740e958357dc7bf0a6506474c5991da12ab6264e
2021-04-21 14:53:54 -07:00
doc Merge "Move key_store_password to keystore section in zuul.conf" 2021-04-21 19:45:17 +00:00
etc Move key_store_password to keystore section in zuul.conf 2021-04-14 06:42:44 -07:00
playbooks Use ssh-keyscan -4 in quick-start 2021-04-19 16:06:08 -07:00
releasenotes/notes Merge "Move key_store_password to keystore section in zuul.conf" 2021-04-21 19:45:17 +00:00
tests Clarify merger updates and resets 2021-04-21 14:53:54 -07:00
tools Use ZooKeeper TLS in tests 2021-03-08 06:49:57 -08:00
web Merge "web: add benchmark test for build output" 2021-03-13 11:48:23 +00:00
zuul Clarify merger updates and resets 2021-04-21 14:53:54 -07:00
.coveragerc Revert "Revert "Switch to stestr"" 2018-05-17 08:33:40 -07:00
.dockerignore Add web/node_modules to dockerignore 2019-01-27 11:23:45 +01:00
.gitignore Use ZooKeeper TLS in tests 2021-03-08 06:49:57 -08:00
.gitreview OpenDev Migration Patch 2019-04-19 19:25:28 +00:00
.mailmap Fix pep8 E127 violations 2012-09-26 14:23:10 +00:00
.stestr.conf Revert "Revert "Switch to stestr"" 2018-05-17 08:33:40 -07:00
.zuul.yaml Use ZooKeeper TLS in tests 2021-03-08 06:49:57 -08:00
COPYING Update README and add GPL license 2018-03-19 09:25:52 -07:00
Dockerfile Noop change to Dockerfile to trigger image builds 2021-02-19 13:09:18 -08:00
LICENSE Initial commit. 2012-05-29 14:49:32 -07:00
MANIFEST.in Optimize canMerge using graphql 2020-02-28 09:43:56 +01:00
README.rst Support nodes setting 'auto' python-path 2019-09-19 10:28:53 +10:00
TESTING.rst Docs: fix stestr run example 2020-01-21 10:36:07 +01:00
bindep.txt Use ZooKeeper TLS in tests 2021-03-08 06:49:57 -08:00
reno.yaml Add reno configuration settings 2020-07-22 08:45:46 -07:00
requirements.txt Include database requirements by default 2021-02-23 16:38:47 -05:00
setup.cfg Merge "Include database requirements by default" 2021-03-09 23:24:34 +00:00
setup.py Partial sync with OpenStack requirements. 2013-09-25 15:30:37 -07:00
test-requirements.txt Optimize canMerge using graphql 2020-02-28 09:43:56 +01:00
tox.ini Use ZooKeeper TLS in tests 2021-03-08 06:49:57 -08:00

README.rst

Zuul

Zuul is a project gating system.

The latest documentation for Zuul v3 is published at: https://zuul-ci.org/docs/zuul/

If you are looking for the Edge routing service named Zuul that is related to Netflix, it can be found here: https://github.com/Netflix/zuul

If you are looking for the Javascript testing tool named Zuul, it can be found here: https://github.com/defunctzombie/zuul

Getting Help

There are two Zuul-related mailing lists:

zuul-announce

A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.

zuul-discuss

General discussion about Zuul, including questions about how to use it, and future development.

You will also find Zuul developers in the #zuul channel on Freenode IRC.

Contributing

To browse the latest code, see: https://opendev.org/zuul/zuul To clone the latest code, use git clone https://opendev.org/zuul/zuul

Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/zuul

Suspected security vulnerabilities are most appreciated if first reported privately following any of the supported mechanisms described at https://zuul-ci.org/docs/zuul/user/vulnerabilities.html

Code reviews are handled by gerrit at https://review.opendev.org

After creating a Gerrit account, use git review to submit patches. Example:

# Do your commits
$ git review
# Enter your username if prompted

Join #zuul on Freenode to discuss development or usage.

License

Zuul is free software. Most of Zuul is licensed under the Apache License, version 2.0. Some parts of Zuul are licensed under the General Public License, version 3.0. Please see the license headers at the tops of individual source files.

Python Version Support

Zuul requires Python 3. It does not support Python 2.

Since Zuul uses Ansible to drive CI jobs, Zuul can run tests anywhere Ansible can, including Python 2 environments.