The Gatekeeper, or a project gating system
Go to file
James E. Blair 7153505cd5 Fix prune-database command
This command had two problems:

* It would only delete the first 50 buildsets
* Depending on DB configuration, it may not have deleted anything
  or left orphan data.

We did not tell sqlalchemy to cascade delete operations, meaning that
when we deleted the buildset, we didn't delete anything else.

If the database enforces foreign keys (innodb, psql) then the command
would have failed.  If it doesn't (myisam) then it would have deleted
the buildset rows but not anything else.

The tests use myisam, so they ran without error and without deleting
the builds.  They check that the builds are deleted, but only through
the ORM via a joined load with the buildsets, and since the buildsets
are gone, the builds weren't returned.

To address this shortcoming, the tests now use distinct ORM methods
which return objects without any joins.  This would have caught
the error had it been in place before.

Additionally, the delet operation retained the default limit of 50
rows (set in place for the web UI), meaning that when it did run,
it would only delete the most recent 50 matching builds.

We now explicitly set the limit to a user-configurable batch size
(by default, 10,000 builds) so that we keep transaction sizes
manageable and avoid monopolizing database locks.  We continue deleting
buildsets in batches as long as any matching buildsets remain. This
should allow users to remove very large amounts of data without
affecting ongoing operations too much.

Change-Id: I4c678b294eeda25589b75ab1ce7c5d0b93a07df3
2023-03-29 17:12:13 -07:00
doc Merge "Add scheduler run handler metric" 2023-02-15 08:43:19 +00:00
etc Remove "sql connection" backwards compatability for database 2022-01-25 16:07:08 -08:00
playbooks Add noxfile and switch to nox 2022-12-20 08:57:53 -08:00
releasenotes/notes Fix prune-database command 2023-03-29 17:12:13 -07:00
tests Fix prune-database command 2023-03-29 17:12:13 -07:00
tools Fix user creation in Docker test setup script 2023-02-21 13:55:06 +01:00
web Merge "Add project-ssh-key to API docs" 2023-02-15 20:53:54 +00:00
zuul Fix prune-database command 2023-03-29 17:12:13 -07:00
.coveragerc Revert "Revert "Switch to stestr"" 2018-05-17 08:33:40 -07:00
.dockerignore Add web/node_modules to dockerignore 2019-01-27 11:23:45 +01:00
.gitignore Add noxfile and switch to nox 2022-12-20 08:57:53 -08:00
.gitreview OpenDev Migration Patch 2019-04-19 19:25:28 +00:00
.mailmap Fix pep8 E127 violations 2012-09-26 14:23:10 +00:00
.stestr.conf Revert "Revert "Switch to stestr"" 2018-05-17 08:33:40 -07:00
.zuul.yaml Switch to nox-docs 2022-12-20 09:21:39 -08:00
COPYING Update README and add GPL license 2018-03-19 09:25:52 -07:00
Dockerfile Add PBR_VERSION argument to Dockerfile 2023-02-13 15:50:37 -08:00
LICENSE Initial commit. 2012-05-29 14:49:32 -07:00
MANIFEST.in Optimize canMerge using graphql 2020-02-28 09:43:56 +01:00
README.rst Update README to point to Matrix, not IRC 2022-11-08 22:25:10 -08:00
TESTING.rst Update unit test container setup and instructions 2022-08-05 21:00:02 +00:00
bindep.txt Bump up to python3.10 2022-07-01 15:59:48 -07:00
noxfile.py Merge "Switch to sqlalchemy 2.0" 2023-02-14 14:55:16 +00:00
reno.yaml Add reno configuration settings 2020-07-22 08:45:46 -07:00
requirements.txt Switch to sqlalchemy 2.0 2023-02-01 09:18:59 -08:00
setup.cfg Add noxfile and switch to nox 2022-12-20 08:57:53 -08:00
setup.py Partial sync with OpenStack requirements. 2013-09-25 15:30:37 -07:00
test-requirements.txt github: more complete mocking for app setup 2021-09-23 19:53:48 +10:00
tox.ini Add noxfile and switch to nox 2022-12-20 08:57:53 -08:00

README.rst

Zuul

Zuul is a project gating system.

The latest documentation for Zuul v3 is published at: https://zuul-ci.org/docs/zuul/

If you are looking for the Edge routing service named Zuul that is related to Netflix, it can be found here: https://github.com/Netflix/zuul

If you are looking for the Javascript testing tool named Zuul, it can be found here: https://github.com/defunctzombie/zuul

Getting Help

There are two Zuul-related mailing lists:

zuul-announce

A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.

zuul-discuss

General discussion about Zuul, including questions about how to use it, and future development.

You will also find Zuul developers on Matrix <https://matrix.to/#/#zuul:opendev.org>.

Contributing

To browse the latest code, see: https://opendev.org/zuul/zuul To clone the latest code, use git clone https://opendev.org/zuul/zuul

Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/zuul

Suspected security vulnerabilities are most appreciated if first reported privately following any of the supported mechanisms described at https://zuul-ci.org/docs/zuul/user/vulnerabilities.html

Code reviews are handled by gerrit at https://review.opendev.org

After creating a Gerrit account, use git review to submit patches. Example:

# Do your commits
$ git review
# Enter your username if prompted

Join us on Matrix to discuss development or usage.

License

Zuul is free software. Most of Zuul is licensed under the Apache License, version 2.0. Some parts of Zuul are licensed under the General Public License, version 3.0. Please see the license headers at the tops of individual source files.

Python Version Support

Zuul requires Python 3. It does not support Python 2.

Since Zuul uses Ansible to drive CI jobs, Zuul can run tests anywhere Ansible can, including Python 2 environments.