Zuul now requires at least python 3.6 so use the bionic image for the
zuul-stream-functional tests as well.

Change-Id: Iba47cd9733e97fba5b0471fdcfc69588f03f85d9

We have removed support for Ansible 2.7 so we should remove the
zuul-stream-functional-2.7 job as well.

Change-Id: If6657c8392ab55052a2fd9981e5d333c254dfbae

I recently had an issue using our nodepool container jobs from the dib
repository.  Nodepool jobs define a base container job and then two
variants; one for installing with released tools and one for
installing with siblings from git.  In this case, you never really
want to inherit from the base job; you want to choose one of the two
variants.  Although the base job was marked abstract, it didn't really
stop me accidentally inheriting from it directly.

This proposes an "intermediate" flag that would be applied to such a
base job.  In a case such as above, it would have raised an error and
alerted me to use one of the two other jobs.

Change-Id: Ifbb9ffa65f86a6b86b63a38e3234a12b564ba3c1

Change-Id: I3d197996597c70165edbf2e26f621391c87873e6

When required status checks are missing this is not mentioned in the
logs which can make it difficult to find out from the logs why a
change didn't enter the gate.

Change-Id: Ifc8a5d5406acbada1ec8d9d07ce03887c4e7b261

We've noticed that if zuul executors (and presumably mergers) don't shut
down gracefully that they may leak git index.lock files in the .git dirs
of the merger repos. Since these repos should be dedicated to zuul's use
without outside interference we can reasonably safely remove any present
index.lock files when starting zuul mergers (and executors).

This implementation does an os.walk under the merger repos root looking
for .git dirs and once it has found them checks for any index.lock
files. This happens before starting the gearman worker which should
avoid any races with these resources.

Change-Id: Ie043453bcdf4500a3718da6f705c882431acafdf

'approvals_left' key is not present when 'Required Merge Request
Approvals' feature isn't available. With enterprise edition,
'approved' key can not be used since the related value is always true
when 'approvals_required' is equal to zero.

The exception was:

    Traceback (most recent call last):
      File "zuul/driver/gitlab/gitlabconnection.py", line 240, in run
	self._handleEvent()
      File "zuul/driver/gitlab/gitlabconnection.py", line 230, in _handleEvent
	event=event)
      File "driver/gitlab/gitlabconnection.py", line 510, in _getChange
	self._updateChange(change, event)
      File "driver/gitlab/gitlabconnection.py", line 521, in _updateChange
	change.project.name, change.number, event=event)
      File "zuul/driver/gitlab/gitlabconnection.py", line 560, in getMR
	mr['approved'] = mr_approval_status['approvals_left'] == 0
    KeyError: 'approvals_left'

Change-Id: I07303ae3309937046b69afe18e9e183853fd448f

1.
gerritconnection: changes are using "data.change.branch" to fill
event.branch which doesn't have 'refs/heads/' prefix

branch creation and deletion were using "data.refUpdate.refName"
to fill event.branch which has the complete refname: it starts
with 'refs/heads/'

As a result, in the Scheduler event process queue, the cache was not
properly cleared when calling Abide's clearUnparsedBranchCache,
called using event.branch and not change.branch.

change.branch is already correctly set to the relative refname

When reconfiguring a tenant, it could hold data which were removed
and reconfigured tenant could still use data that had disappeared

This patch removes the prefix "refs/heads/" when setting event branch
name in case of deletion or creation

note: it's not possible to use event.branch to set change.branch in
gerritconnection.getChange as event objects doesn't have a constant base
type.
it can be either:
- GerritTriggerEvent -> TriggerEvent -> object
- DequeueEvent -> ManagementEvent -> object
the latter doesn't have a branch attribute

2.
Setting event.branch is moved to any 'ref-update' event, so it will now
be set when newrev and oldrev are both not null. This doesn't have and
should not have any impacts

Change-Id: Ie60b382b23074cc9feff0648e786ddaf0d3454aa

PF4 broke the horizontal scrolling behaviour of the pre/table
combination used on this page. This restores it by setting the overflow
attribute on the pre element.

Change-Id: I038942d235fd13cae90ed82429558f441f1dcbb4

We seem to be putting the actual error message as the header and the
title (the project + error file) as the body.  Swap them around.

The error is preformatted text with embedded newlines.  Wrap it in a
<pre> with pre-wrap so it displays correctly.

Change-Id: I9fafafe38a2b24ac8558f97000225f7b53fab9bf

Since we introduced PF4 together with the Drawer component to show the
config errors, we are not able to scroll the page by using the keyboard.
So far, no workaround really did the trick and every one came with
another drawback in different areas.

So we decided to ditch the Drawer component and use a Modal as
replacement. This should still serve the use case as we have "something
popping up when clicking on the bell icon", but this something doesn't
break the layout and/or the scrolling behaviour.

Additionally this change removes the "height: 100%" CSS rule on the root
element. In combination with the removed drawer this should restore the
scrolling behaviour via keyboard without introducing any further
drawbacks like wrong tabbing or focus behaviour.

For additional background information on this topic, please refer to the
changes [1], [2] and [3]. There is also a PF4 issue on Github available
[4].

[1] https://review.opendev.org/#/c/743239/
[2] https://review.opendev.org/#/c/743917/
[3] https://review.opendev.org/#/c/742759/
[4] https://github.com/patternfly/patternfly-react/issues/4624

Change-Id: I45df9cabc96d2d7fb4ddebdf1c72d0dd8a3adfd2

The original change was reverted to fix a scrolling bug that was introduced by a different change. Using this commit in combination with [1] should restore the old behaviour.

Applying [2] on top of them should finally get rid of the scrolling issues.

[1]: https://review.opendev.org/#/c/750361/
[2]: https://review.opendev.org/#/c/750322/

This reverts commit b4b5b9fd58.

Change-Id: Icd498314762a8edca751413b7ee07b9b72317c5b

This accidentally broke the layout of the build result page and the tabs only reachable by scrolling horizontally in case a build log contained very long log lines.

This reverts commit 8d0f440e1b.

Change-Id: Ifcdf5c737ef6f2e178c9a4531ead81e8c0fe6505

Branch protection rules in github are fn patterns which are currently
matched locally in zuul. This is error prone and can lead in edge
cases to wrong matches resulting in wrong enqueue decisions into gate
pipelines. When requesting branch protection rules in github we also
can request the matching refs along with the rules. This is much safer
since we can plain text match them against the change branch.

Change-Id: Ic995d4b2e16a5d741f0209fa9236959d8f4d10b9

Change I99f0c8edaae2185c5dbf855398ace2522237226d allowed offloading the
repo reset to a process pool. We can take advandage of this also for the
the repo reset during merge as this is also taking longer under high
load.

Change-Id: I14e704ab818c8c2e0405f0adfc55cee564ed4d1e

A recent change to introduce user preferences in the client-side
web app had an error when detecting the case where a user switched
from having auto-reload of the status page disabled to enabled.
It only checked that the autoreload setting was enabled and there
was no timer currently set; if that was true, then it would force
a refresh.  This check was performed every time the component was
updated, so each time a status update happened, the component would
naturally be updated as part of the process, run this check, see
that autoreload was enabled, and then force an update regardless
of the visibility setting.

This change ensures the forced update only happens in the specific
case that the user has changed the autoreload setting, thereby
breaking the loop described above.

Change-Id: Ie7ef175536bb20033cede5e93748cda08c3d918d

Github responds with a 500 error if the diff is too large to make
sense. This is currently handled by the retry handler with backoff and
can significantly delay event processing. However Github will never
respond with something different so this needs to be excluded from the
retry handler.

Further it has been revealed that our error handling code incorrectly
results in an empty changed files list. However in this case we must
set the changed files to None so zuul takes care of this itself.

Change-Id: Ie825a8801032d5a1ec66afb1244aa3b571fd1d39

In the past source_event has been used to improve merge messages. This
is being attached to changes in getChange. Since there can be any
event on a change this has been wrong and is now unused. Thus remove
this variable. Further it currently even creates a problematic memleak
when doing dequeue commands without the change being in any queue. In
this case the exception trace including all local variables of the
frames (e.g. Tenant) are attached to the event and thus to the
cached change which will never be cleared unless there is another
event incoming for this change).

Change-Id: I23e59e66f485bf46493d7a5bdf7bc29966e56993

When we hit an exception in a management event we attach it to the
event so the sender of the event can get the stack trace back. However
the traceback also contains the local variables of all frames part of
the traceback. Those also can contain references to Tenant objects
which will be blocked for the gc until the event is (hopefully)
collected as well. Since we're only interested in the stack traces and
don't need the local variables clear the frames before attaching the
traceback to the event.

Change-Id: I57854ac4b9d1405d2b7b86f27149f87af968e77a

When the scheduler looses its zk session it resubmits all lost node
requests as new ones. However it didn't stop the watch for the old one
which keeps being registered in Kazoo. The watch contains the
NodeRequest object since it's bound to the callback. Thus by leaking
the Watch we also leak the NodeRequest, the attached BuildSet,
QueueItem and finally the Tenant and Layout as well.

This can be fixed by stopping the watch in this case.

Change-Id: I3b05ec92816ab5eb06ad40dfad85ddfebfbf2cc4

The boolean "held" attribute is set to True if a build triggered
a autohold request, and its nodeset was held.
Allow filtering builds by "held" status.

Change-Id: I6517764c534f3ba8112094177aefbaa144abafae

There are now increasing numbers of dependencies like cryptocraphy
that will drop support for Python 3.5 soon. Thus we should consider
dropping the support for Python 3.5 as well.

Change-Id: I830ec3e97cfaac5d336d02755fe788572f36fb07

Ansible 2.7 is in security fix only maintenance mode since quite some
time and will be end of life soon. It further blocks upgrade of zuul
to Python 2.8 due to incompatibilities. Thus drop support.

Change-Id: I13802db3314450ad149fdadacd1e2e70dd8468ef
Depends-On: https://review.opendev.org/727345