zuul/zuul
Simon Westphahl 8bcfb8bc4a Correctly limit buildsets with multiple refs
The current SQL query will not correctly limit the number of buildsets
when some of the buildsets are related to multiple refs (circular
dependencies). The problem is that LIMTI works on number of rows, but we
want to limit only on the number of buildsets.

This corrects the problem by using a subquery to identify the distinct
buildsets, limiting that, and then querying for all of the information
about those buildsets.

This also updates the methods which perform the same function for builds,
even though we are not yet seeing an issue in practice.  It is
theoretically possible to call the getBuilds method with 'provides' and
'limit' arguments, which would produce the same problem as the buildsets
query.  That is not possible in practice, as the REST API doesn't support
provides, and the scheduler which does pass the provides argument doesn't
pass limit.  However, that could easily change in the future.  Additionally,
this future-proofs us if we add more queryable one-to-many relationships
to builds in the future (such as if we linked builds to multiple refs).
Also, it's easier to maintain these methods if they follow the same pattern.

There does not appear to be a performance loss in either mysql or postgres
with local testing on large data sets.  There may actually be an improvement
(but it's within the margin of error, so hard to say).

The index hints previously needed for mysql appear to no longer be
necessary and are removed.

Change-Id: Ib19e4cb8171f5d4d2873fb6b9c0301eb5d4ee43d
Co-Authored-By: James E. Blair <jim@acmegating.com>
2024-03-25 13:31:19 -07:00
..
ansible Merge "Enable TCP keepalives in zuul_stream" 2024-02-27 18:14:12 +00:00
cmd Merge "Fix validate-tenants isolation" 2024-02-28 02:46:55 +00:00
connection Use the GitHub default branch as the default branch 2023-08-23 11:07:08 -07:00
driver Correctly limit buildsets with multiple refs 2024-03-25 13:31:19 -07:00
execution_context List process ids in bwrap namespace 2023-06-28 13:31:06 -07:00
executor Merge "Add zuul_unreachable ansible host group" 2024-03-25 18:26:14 +00:00
lib Make ansible package check more robust 2024-03-25 11:25:35 -07:00
manager Merge "Emit per-branch queue stats separately" 2024-03-25 19:22:37 +00:00
merger Merge "Only return the latest config for project-branch" 2024-03-11 10:57:17 +00:00
reporter Finish circular dependency refactor 2024-02-09 07:39:40 -08:00
source Fix bug with cached merge modes in TPC 2023-11-16 09:37:08 +01:00
sphinx docs: remove zuuldocs domain 2020-08-04 16:10:09 +10:00
trigger Refactor configuration error handling 2023-10-30 16:19:45 -07:00
vendor Add some pipeline processing stats 2022-01-25 08:16:52 -08:00
web Add script to generate openapi spec 2024-03-09 11:25:40 -08:00
zk Remove most model_api backwards-compat handling 2024-02-09 07:39:55 -08:00
__init__.py Initial commit. 2012-05-29 14:49:32 -07:00
_setup_hook.py Revert "Revert "Create zuul/web/static on demand"" 2020-07-07 19:53:33 +02:00
change_matcher.py Remove most model_api backwards-compat handling 2024-02-09 07:39:55 -08:00
configloader.py Use ProjectNotFoundError 2024-03-18 15:09:47 -07:00
exceptions.py Handle dependency limit errors more gracefully 2024-03-19 14:37:26 +01:00
model_api.py Finish circular dependency refactor 2024-02-09 07:39:40 -08:00
model.py Merge "Add a zuul.buildset_refs variable" 2024-03-25 10:09:16 +00:00
nodepool.py Add job name back to node request data 2024-03-07 08:02:30 +01:00
scheduler.py Use NodesetNotFoundError class 2024-03-18 15:03:58 -07:00
version.py Convert pkg_resources usage to importlib 2024-03-20 09:17:19 -07:00