1f026bd49c
This change completes the circular dependency refactor. The principal change is that queue items may now include more than one change simultaneously in the case of circular dependencies. In dependent pipelines, the two-phase reporting process is simplified because it happens during processing of a single item. In independent pipelines, non-live items are still used for linear depnedencies, but multi-change items are used for circular dependencies. Previously changes were enqueued recursively and then bundles were made out of the resulting items. Since we now need to enqueue entire cycles in one queue item, the dependency graph generation is performed at the start of enqueing the first change in a cycle. Some tests exercise situations where Zuul is processing events for old patchsets of changes. The new change query sequence mentioned in the previous paragraph necessitates more accurate information about out-of-date patchsets than the previous sequence, therefore the Gerrit driver has been updated to query and return more data about non-current patchsets. This change is not backwards compatible with the existing ZK schema, and will require Zuul systems delete all pipeline states during the upgrade. A later change will implement a helper command for this. All backwards compatability handling for the last several model_api versions which were added to prepare for this upgrade have been removed. In general, all model data structures involving frozen jobs are now indexed by the frozen job's uuid and no longer include the job name since a job name no longer uniquely identifies a job in a buildset (either the uuid or the (job name, change) tuple must be used to identify it). Job deduplication is simplified and now only needs to consider jobs within the same buildset. The fake github driver had a bug (fakegithub.py line 694) where it did not correctly increment the check run counter, so our tests that verified that we closed out obsolete check runs when re-enqueing were not valid. This has been corrected, and in doing so, has necessitated some changes around quiet dequeing when we re-enqueue a change. The reporting in several drivers has been updated to support reporting information about multiple changes in a queue item. Change-Id: I0b9e4d3f9936b1e66a08142fc36866269dc287f1 Depends-On: https://review.opendev.org/907627
61 lines
2.9 KiB
YAML
61 lines
2.9 KiB
YAML
---
|
|
prelude: >
|
|
This release includes a significant refactoring of the internal
|
|
handling of circular dependencies. This requires some changes for
|
|
consumers of Zuul output (via some reporters or the REST API) and
|
|
requires special care during upgrades. In the case of a
|
|
dependency cycle between changes, Zuul pipeline queue items will
|
|
now represent multiple changes rather than a single change. This
|
|
allows for more intuitive behavior and information display as well
|
|
as better handling of job deduplication.
|
|
upgrade:
|
|
- |
|
|
Zuul can not be upgraded to this version while running. To upgrade:
|
|
|
|
* Stop all Zuul components running the previous version
|
|
(stopping Nodepool is optional).
|
|
|
|
* On a scheduler machine or image (with the scheduler stopped)
|
|
and the new version of Zuul, run the command:
|
|
|
|
zuul-admin delete-state --keep-config-cache
|
|
|
|
This will delete all of the pipeline state from ZooKeeper, but
|
|
it will retain the configuration cache (which contains all of
|
|
the project configuration from zuul.yaml files). This will
|
|
speed up the startup process.
|
|
|
|
* Start all Zuul components on the new version.
|
|
- The MQTT reporter now includes a job_uuid field to correlate retry
|
|
builds with final builds.
|
|
deprecations:
|
|
- |
|
|
The syntax of string substitution in pipeline reporter messages
|
|
has changed. Since queue items may now represent more than one
|
|
change, the `{change}` substitution in messages is deprecated and
|
|
will be removed in a future version. To maintain backwards
|
|
compatability, it currently refers to the arbitrary first change
|
|
in the list of changes for a queue item. Please upgrade your
|
|
usage to use the new `{changes}` substitution which is a list.
|
|
- |
|
|
The syntax of string substitution in SMTP reporter messages
|
|
has changed. Since queue items may now represent more than one
|
|
change, the `{change}` substitution in messages is deprecated and
|
|
will be removed in a future version. To maintain backwards
|
|
compatability, it currently refers to the arbitrary first change
|
|
in the list of changes for a queue item. Please upgrade your
|
|
usage to use the new `{changes}` substitution which is a list.
|
|
- |
|
|
The MQTT and Elasticsearch reporters now include a `changes` field
|
|
which is a list of dictionaries representing the changes included
|
|
in an item. The correspending scalar fields describing what was
|
|
previously the only change associated with an item remain for
|
|
backwards compatability and refer to the arbitrary first change is
|
|
the list of changes for a queue item. These scalar values will be
|
|
removed in a future version of Zuul. Please upgrade yur usage to
|
|
use the new `changes` entries.
|
|
- |
|
|
The `zuul.bundle_id` variable is deprecated and will be removed in
|
|
a future version. For backwards compatability, it currently
|
|
duplicates the item uuid.
|