zuul
/
zuul
Code Issues Proposed changes
The Gatekeeper, or a project gating system
764 Commits 1 Branch 48 Tags 139 MiB
Python 89.7%
JavaScript 9.4%
Shell 0.5%
CSS 0.3%
Go to file
Antoine Musso d9ac4c18b1 Zuul references cleaner 
Zuul mergers create a vast number of git references under /refs/zuul
which are never garbage collected.

With hundred of thousands of references, that makes git fetch operations
very slow since git uploads all references to Gerrit to synchronize the
Zuul maintained repository.  On one of Wikimedia busy repository
(mediawiki/core) we had 55000 such references and it can take up to 18
seconds for a fetch to complete.  I have seen occurences of a merge
taking 2 minutes to complete.

As such, this tiny script clears out references for which the commit date
of the pointed commit object is older than 360 days (the default).

It is not perfect since a recent reference can well point to an old
object.  That would be the case on repositories that are barely active.
In such case the ref will be gone despite it being recently created.

A better way would be to vary Zuul references by using month/day which
will let one easily garbage collect them.  But I am being lazy and that
would not let us clear out references using the current scheme.

Example usage:

 zuul-clear-refs.py --verbose --dry-run --until 90 /srv/zuul/git/project

Would show a list of references pointing to commit dates older than 90
days and output a message whenever the script would delete them.

Hint about the utility in our merger documentation.

Reference:
 https://phabricator.wikimedia.org/T70481

Change-Id: Id4e55f5d571ebd5e8271e516f53f8e05c1f78c1a
 		2015-07-20 18:57:04 +02:00
doc Zuul references cleaner 2015-07-20 18:57:04 +02:00
etc Merge "Status: Fix JSHint violations and commit settings" 2015-07-01 09:00:04 +00:00
tests Check that empty reports aren't sent 2015-06-15 08:07:19 +00:00
tools Zuul references cleaner 2015-07-20 18:57:04 +02:00
zuul Merge "Add uuid to the launch log record" 2015-07-02 21:24:41 +00:00
.gitignore Ignore .venv directory 2013-12-03 17:23:54 -08:00
.gitreview Updated .gitreview location 2012-12-16 20:34:13 +00:00
.mailmap Fix pep8 E127 violations 2012-09-26 14:23:10 +00:00
.testr.conf Add ability to skip log capture. 2013-07-02 17:47:23 +00:00
LICENSE Initial commit. 2012-05-29 14:49:32 -07:00
MANIFEST.in Migrate to pbr. 2013-06-25 19:04:30 +00:00
NEWS.rst Split the merger into a separate process 2014-02-17 11:47:15 -08:00
README.rst Switched documentation to point to storyboard. 2014-11-11 15:59:24 -08:00
TESTING.rst Support multiple triggers 2013-08-01 11:56:52 -07:00
requirements.txt Update to gear 0.5.7 2015-05-07 12:00:40 -07:00
setup.cfg Replace ci.o.o links with docs.o.o/infra 2015-05-14 21:38:19 +00:00
setup.py Partial sync with OpenStack requirements. 2013-09-25 15:30:37 -07:00
test-requirements.txt Remove docutils pin 2014-10-14 20:32:02 +02:00
tox.ini Fix pep8 issues 2015-01-21 19:38:33 +11:00

README.rst

Zuul

Zuul is a trunk gating system developed for the OpenStack Project.

Contributing

To browse the latest code, see: https://git.openstack.org/cgit/openstack-infra/zuul/tree/ To clone the latest code, use git clone git://git.openstack.org/openstack-infra/zuul

Bugs are handled at: https://storyboard.openstack.org/#!/project/679

Code reviews are, as you might expect, handled by gerrit. The gerrit they use is http://review.openstack.org

Use git review to submit patches (after creating a gerrit account that links to your launchpad account). Example:

# Do your commits
$ git review
# Enter your username if prompted