Commit Graph

9673 Commits (master)
 

Author SHA1 Message Date
Zuul 2bdc98b6d3 Merge "Fix setting autoholds through API with change supplied" 15 hours ago
Zuul cc2db2aeb5 Merge "Document the source of the afs 0x40084301 ioctl magic number" 1 day ago
Radosław Piliszek a43d65d87a [docs] Mention that paused dependencies are OK
Change-Id: I435a7711b56dfd3f2863a8c699eb949a613a2de2
2 days ago
Zuul c15ca00bab Merge "Use buildx to build images" 2 days ago
Zuul b78aa9b6d4 Merge "Web: remove config error drawer" 3 days ago
Zuul 0b5795b808 Merge "Web: convert config errors to table" 3 days ago
Clark Boylan ff166e3ea9 Document the source of the afs 0x40084301 ioctl magic number
During debugging of ioctl failures one of the things we explored was
that this magic number may no longer be correct. Turns out it is
correct, but documenting the source of this value may aid future
debugging.

Change-Id: I87504ee5763bbdc819e68f9defee3df5277eec51
4 days ago
James E. Blair eb550597b0 Use os.open with setpag
When we open the ioctl file to run the openafs setpag syscall,
we previously used the high-level open method, which apparently
issues an unwanted TCGETS ioctl which crashes the program with
a kernel error under certain versions of python+openafs+linux
(3.10.6, 1.8.9, 5.15.0, respectively).

Switch to a low-level open to avoid this call.

Change-Id: I5e08a6020cf6cd4ad2a0084effb697aa39dae9c6
4 days ago
James E. Blair 9e0f2b5694 Fix setting autoholds through API with change supplied
The set autohold api endpoint incorrectly handled supplied values
such that if the user supplied a change without a ref it would always
use the default ref (.*).  This corrects the case handling and
adds tests.

Change-Id: I1ae14c327fd8fd2b866013d4d5078a9fbd85f843
1 week ago
Zuul 318df3a9ba Merge "Document backing up and restoring project keys" 1 week ago
Zuul 0a01f11786 Merge "Add more info to merge mode selection errors" 1 week ago
Zuul d9a506e384 Merge "Replace docker by podman for quick-start" 1 week ago
James E. Blair b63fc7bed6 Web: remove config error drawer
This changes the bell icon at the top to link directly to the
config-errors page.

Currently, the bell icon opens a notification drawer which shows
one rendition of a list of config errors, and if the user clicks
on any error, it navigates to the config-errors page which shows
another rendition.

Rather than having two ways of viewing config errors, let's just
have one.

Change-Id: I630fb2311ec5fa47c8a0dbf8a838c4e0453f32e1
1 week ago
James E. Blair bcae20f897 Web: convert config errors to table
This updates the config-errors page to show errors in table format
along with a filter bar.

This allows users to see all of the errors at a glance, or filter
by project or error type.

The table rows show a summary of error information, and each row
can be expanded to show the full error message.

Change-Id: Ie019d4193c730fe84f4bc2827106825093fa68a4
1 week ago
Jeremy Stanley 1fe8c37bf9 Document backing up and restoring project keys
Administrators may not realize until it's too late that it would
have been better for them to save copies of project keys in case of
a disaster impacting the keystore in ZooKeeper. Zuul provides the
tools necessary to facilitate this, so document them prominently in
the places newcomers are likely to look when first installing or
learning to operate the software.

Change-Id: Iaf8bc9cbcdb885cd6b9648ec4080c144d1fb4ccf
1 week ago
Clark Boylan 1d8ce7737e Add more info to merge mode selection errors
It can be confusing to know what merge modes are considered valid for a
project when Zuul tells you: "Merge mode foo is not supported by project
bar." Update this error message to include the list of merge modes that
Zuul considers to be valid for the project to reduce confusion.

Change-Id: If0d5cf1b06a93f3da507fba43077a5a9e327ca5e
1 week ago
Zuul b01446e8a8 Merge "[doc] Fix the description of the "shadow" attr" 2 weeks ago
James E. Blair 84e0e76e2f Add error information to config-errors API endpoint
This is the first in a series of changes to improve the usability
of the web view of config errors.  The end goal is to be able to
display them in a more structured manner.  A secondary goal is to
eventually add warnings (eg, deprecation warnings) which is
really only feasible if we have structured presentation of
errors.

This change does the following:

* Adds severity and error names to existing configuration errors
* And makes them available via the config-errors API endpoint
* Reduces the call sites for the error accumulator
  (LoadingErrors.addError)
* Unifies the calling convention for the accumulator
  (we stop passing in Exception objects)

Change-Id: Ia17dd3e7ad8cdfa8a07bb03b871078415d0c145e
2 weeks ago
Zuul 3489d22290 Merge "Conditionally render ANSI console text with a black background" 2 weeks ago
Zuul c8f88a8154 Merge "Ensure cycle dependencies are enqueued ahead" 2 weeks ago
Simon Westphahl 381ba7c24f
Ensure cycle dependencies are enqueued ahead
This change fixes a bug related to circular dependency resolution where
non-cycle changes could be enqueued between changes of the same cycle.

This violated the invariant assumption that changes of the same
dependency cycle are enqueued in sequence. This could cause the pipeline
processor to loop indefinitely under certain conditions.

The idea behind this fix is to treat all unprocessed dependencies of
other changes in the same cycle as if they were direct dependencies of
the current change. By that we will try to enqueue dependencies of any
change in the cycle ahead of the whole cycle.

Change-Id: I3eeb9fc9f6fca73982ce01d180dca9f58868bff3
2 weeks ago
Zuul 4de0cdf033 Merge "Only use "master" as default branch if not set" 3 weeks ago
James E. Blair 68d705bad8 Conditionally render ANSI console text with a black background
We render ANSI colors in 3 places:
* Streaming console
* Build task summary tab
* Build console tab

(We do not render ANSI colors in the Build log tab.)

In the streaming console, the default color scheme is white-on-black
which is probably what most devs expect when they render color output.
The other two locations maintain the black-on-white Patternfly default
scheme.  This means that some colors (eg yellow) are difficult to read.

In order to accommodate cases where ANSI colors are used, the
color scheme of the preformatted sections in the Build task summary or
Build console tabs is changed to white on black iff an ANSI color
sequence is detected.  Otherwise, it remains the Patternfly default
of black on white.

Change-Id: If99d09f614c51495af4ee400891b9a8f2e217db8
3 weeks ago
James E. Blair e342d25cb9 Use buildx to build images
This is required in order to have speculative image builds with
images on quay.io.

Change-Id: I993ee98e160c27f98a363779ccd0630895935712
3 weeks ago
Tristan Cacqueray 0e3756bd5c Replace docker by podman for quick-start
Change-Id: Ic6ea0e350e7a6aae656823b4294e49ee871b3329
Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/883593
3 weeks ago
Zuul bb01c1613f Merge "Handle zuul.yaml files with only comments" 3 weeks ago
Zuul e812ce6a3d Merge "Add missing event id to management events" 3 weeks ago
Zuul 1edf5b6760 Merge "Fix delete-pipeline-state command" 3 weeks ago
Zuul 0bd76048d1 Merge "Correctly complete build on transient errors" 3 weeks ago
Zuul 1e41732aa3 Merge "Improve FrozenJob.isEqual" 3 weeks ago
Zuul ad532eb3d0 Merge "Allow duplicate queue definitions on same project branches" 3 weeks ago
Zuul aa74fb9ec3 Merge "Expose max. attempts to the job as a Zuul variable" 3 weeks ago
Zuul 37959752b4 Merge "Fix bug around Github token expiration" 3 weeks ago
James E. Blair d0776916de Refresh builds even less
Change I3824af6149bf27c41a8d895fc682236bd0d91f6b intended to refresh
builds from ZK only when necessary.  Before that change we would
refresh them only if they did not have a result (because once they
have a result they won't change).  That change should have reduced
that so that in the cases where we don't have a result yet, we
still only refresh if the build has changed.

In other words, that change should have used an "and" instead of
an "or".

Logically, if builds can't change after they have a result, then
the check of whether we have a result is not necessary.  So rather
that change the operator, we can just drop the build.result check
altogether and rely on the object update check.  This has the effect
of making the code more future proof as well, in that we remove
the assumption that the build will never change after receiving a
result.

This change also surfaced a bug in the original implementation:
because refreshing the Build objects happens inside the deserialize
method of the BuildSet object, the BuildSet has not actually updated
its build_versions variable from ZK yet, which means our comparisons
in shouldRefreshBuild were using outdated data.  To correct, we now
pass in the newly deserialized value.  And the same for Jobs.

Change-Id: Ie688f2ee0343cab5d82776ccfc7b0f2edc5f91e5
3 weeks ago
Clark Boylan c1b0a00c60 Only check bwrap execution under the executor
The reason for this is that containers for zuul services need to run
privileged in order to successfully run bwrap. We currently only expect
users to run the executor as privilged and the new bwrap execution
checks have broken other services as a result. (Other services load the
bwrap system bceause it is a normal zuul driver and all drivers are
loaded by all services).

This works around this by add a check_bwrap flag to connection setup and
only setting it to true on the executor. A better longer term followup
fixup would be to only instantiate the bwrap driver on the executor in
the first place. This can probably be accomplished by overriding the
ZuulApp configure_connections method in the executor and dropping bwrap
creation in ZuulApp.

Temporarily stop running the quick-start job since it's apparently not
using speculative images.

Change-Id: Ibadac0450e2879ef1ccc4b308ebd65de6e5a75ab
3 weeks ago
Zuul deab00c8bd Merge "Use bwrap --disable-userns if possible" 3 weeks ago
James E. Blair 38423ed883 Fix JWT auth test and alembic migration
A new version of pyjwt was released which alters an internal call
it makes to urllib which we have mocked in the unit tests.  Our
mock must be updated to match.

The previous version would call urlopen with a string url argument,
newer versions call it with a Request object (urllib accepts both).

This change updates the mock to accept both.

There has also been a recent alembic release which alters the function
signature of alter_column so that most arguments are now required to
be keyword arguments.  This causes one of our migrations to fail
since it was not passing the new type in as a positional argument.

Notably, the type was not the next positional argument in the
signature ("nullable" is), so this explains why we had two
"change patchset to string" migrations: the first one did not actually
work.

This change alters the migration to do what it effectively did rather
that what it was intended to do.  The later migration continues to
correct the error.

Change-Id: I10cdffa43147f7f72e431e7c64e10e9416dfb295
3 weeks ago
Clark Boylan 0937872119 Use bwrap --disable-userns if possible
Newer bwrap has added the ability to disable additional nested user
namespace creation from with the bwrap execution context. Take advantage
of this feature in Zuul if we are able to in order to fortify Zuul's
security position.

In particular we need two conditions to take advantage of this. 1) bwrap
must be new enough to support the feature (>=0.8.0) and 2) we must be
running with user namespaces enabled. We explicitly check for both
conditions and add the appropriate invocation flags to bwrap when the
conditions are met.

Change-Id: Idf933a0847cb8570b551892186ca9c0057be127f
3 weeks ago
Zuul a914670514 Merge "Add statement timeouts to some web sql queries" 1 month ago
Simon Westphahl 9c9d69e058
Fix bug around Github token expiration
Even after increasing the grace time for Github app installation tokens
to 5min we were still seeing exceptions related to expired app tokens.

Upon furhter investigation it turned out that the current grace time had
no effect at all since we passed the *adjusted* expiry time to the
Github client, which takes it at face value and raises an exception if
the expiry time is exceeded.

To fix this we'll store the original expiry time in the token cache and
pass that directly to the Github cliendt. We then adjust the cutoff time
by the 5min grace time when checking if a token should still be
considered valid.

Change-Id: I56f51df1d57e4dd7f1f85eba4af28c2a7318ddd1
1 month ago
Zuul 51194abf56 Merge "web: add dark mode and theme selection" 1 month ago
Simon Westphahl 515e6ab8f7
Correctly complete build on transient errors
Use the `completeBuild()` method for reporting the result on transient
errors during repo update instead of the old `sendWorkComplete()` API.

2023-04-28 12:29:49,173 ERROR zuul.AnsibleJob: [e: ...] [build: ...] Exception while executing job
Traceback (most recent call last):
  File "/opt/zuul/lib/python3.10/site-packages/zuul/executor/server.py", line 1144, in do_execute
    self._execute()
  File "/opt/zuul/lib/python3.10/site-packages/zuul/executor/server.py", line 1345, in _execute
    self.job.sendWorkComplete(
AttributeError: 'FrozenJob' object has no attribute 'sendWorkComplete'

Change-Id: I05be093f80f015463f727f55154e16202821e961
1 month ago
Zuul bbdbe81790 Merge "Add Gerrit pipeline trigger requirements" 1 month ago
Zuul 8dcc69fbf0 Merge "Add GitHub pipeline trigger requirements" 1 month ago
Clark Boylan 57cdfc978f Add container creation details
This adds extra bits of info that the ensure-quay-repo role expects to
be present.

Change-Id: I581c667515b5ee2eec4ee79e9f2b485247711195
1 month ago
James E. Blair 546ad5353a Add Gerrit pipeline trigger requirements
This updates the Gerrit driver to match the pattern in the GitHub
driver where instead of specifying individual trigger
requirements such as "require-approvals", instead a complete ref
filter (a la "requirements") can be embedded in the trigger
filter.

The "require-approvals" and "reject-approvals" attributes are
deprecated in favor of the new approach.

Additionally, all require filters in Gerrit are now available as
reject filters.

And finally, the Gerrit filters are updated to return
FalseWithReason so that log messages are more useful, and the
Github filters are updated to improve the language, avoid
apostraphes for ease of grepping, and match the new Gerrit
filters.

Change-Id: Ia9c749f1c8e318fe01e84e52831a9d0d2c10b203
1 month ago
James E. Blair 1a4ec7e926 Add GitHub pipeline trigger requirements
This mimics a useful feature of the Gerrit driver and allows users
to configure pipelines that trigger on events but only if certain
conditions of the PR are met.

Unlike the Gerrit driver, this embeds the entire require/reject
filter within the trigger filter (the trigger filter has-a require
or reject filter).  This makes the code simpler and is easier for
users to configure.  If we like this approach, we should migrate the
gerrit driver as well, and perhaps the other drivers.

The "require-status" attribute already existed, but was undocumented.
This documents it, adds backwards-compat handling for it, and
deprecates it.

Some documentation typos are also corrected.

Change-Id: I4b6dd8c970691b1e74ffd5a96c2be4b8075f1a87
1 month ago
Zuul 0897279cb9 Merge "Optimize layout generation with many job variants" 1 month ago
Zuul c4a6f6fcd6 Merge "Further fix getting topic changes by git needs" 1 month ago
Zuul aad1fe11a6 Merge "Fix getting Gerrit topic changes with git needs" 1 month ago