This lets us use the build UUID as the job temp dir name
so that the finger log streamer can search the job root dir
to find the requested log by build ID.
Also, since the JobDir parameters aren't really optional, just
make that explicit.
Change-Id: Ifeb9f37c6c9c1ce792079e63a6c507461081f03c
It's ugly when we print:
ok: [ubuntu-xenial]Ran in 0:00:05.475300 seconds. Start: 2017-05-26
20:21:40.805825 end: 2017-05-26 20:21:46.281125
It would be better if we print:
ok: [ubuntu-xenial] Runtime: 0:00:05.475300 Start: 2017-05-26
20:21:40.805825 End: 2017-05-26 20:21:46.281125
Note the space before Runtime and the capital E on end to match the capital
S on Start.
Change "Ran in {delta} seconds." to "Runtime: {delta}" - because
0:00:05.475300 isn't in units of seconds - it's a time with hours,
minutes, seconds and microseconds.
Change-Id: I9c61a1d81d8f9f98bb1646674c7e6830b3e85619
Change I453754052b75de3d4fac266dc73921c7ce9c075f had the effect that
no source connection was configured in the merger. This can be fixed
by changing the type check from issubclass to isinstance. Further also
the zuul and timer drivers don't need to be configured by the merger.
This also adds a test case for that.
Change-Id: If437123c3e8a0d63c5f852619e214cefa0892ce3
Using the git protocol makes it hard to fetch the repo if internet
access is only possible via a proxy. An easy fix is using the https
protocol. That way the http_proxy vars on the host are automatically
obeyed and the clone via proxy works.
Change-Id: I18d75b5a16c809ac2d7834e91d32617017aea7f8
Instead of returning delta, ret, start and stop in a dictionary, make a
special purpose banner for it. Also, don't put ret in the banner - these
are shell commands, and this is the OK handler ... ret == 0 by
definition.
Change-Id: I99326146ed6f3651c38c3aa88eb52df912e71d06
Logging gets unreadable because each command run winds up reporting back
a poorly formatted version of the entire script, which is often -x'd
anyway, as well as the full stdout, which we just streamed into the log.
Strip these so that the result dict is clear and contains useful info.
A followup should likely take start, end, delta and rc and make a nicely
formatted single message.
Change-Id: I57b6f5a12474e75a7f30a4e95e93a007965a2f13
This is behaving differently in python3 in a way that is not
completely understood yet, but does not appear to reflect an
actual leak.
Disable enforcement of the check, but leave the check along with
some extra debugging in place so we can observe it.
Change-Id: Ib19a0fec72a7eaa7122ddd94a95ad2058b164048
In zuul v2, prepareRef waited until the merger had completed its
work before telling the pipeline manager that the item was ready.
This was important as it ensured there was a zuul ref on a merger
for jobs to be able to fetch from. We dropped that in v3 because
the jobs would build their own repo states on the executor, however,
we're about to put something similar back in that we're going have
the merger return the complete repo state so that the executors can
later use that to reconstruct the work the merger performed.
Because prepareItem (essentially the v3 version of prepareRef) did
not wait for the merger result before declaring the item ready, we
can end up launching jobs before the merger completes. This isn't
a big deal, yet, but it could still produce surprising results and
make for extra work. This change restores the old behavior, as well
as makes a test less suceptible to race conditions (again, which
are more likely to be seen in future changes).
Similarly, we also lost setting ready to false in the case of a merge
conflict (a behavior which was present in v2). This corrects that
as well to ensure that we don't execute jobs if there is a merge
conflict.
Because some tests were written with the assumption that only one
merger is running, and we are now running two (a dedicated merger and
one in the executor), they are more likely to fail due to race
conditions, especially now that we are waiting for mergers to complete
before executing jobs. This change also removes the dedicated merger
from the test infrastructure so we rely only on the merger in the
executor.
Change-Id: I6d4da3d901d99290ef99b478ea9a4659058fe839
This adds a comment to PRs when a successfully tested change fails
to merge into an upstream repository due to a Github API error. It
uses the merge_failure_message to format said message.
Change-Id: Ieed299a71133ebd88e2c31d6bd0c54390c0901ba
Signed-off-by: Adam Gandelman <adamg@ubuntu.com>
The change.id given to the status.json output for github PRs is a full
"PR#,SHA" combo. This gives it uniqueness when we can have multiple
commits and versions of commits per PR.
When we display this in the status page though i think all we care about
is the PR number, so trim it out and just display that.
Change-Id: I73385a6b0743a6764fdfdd51cd1d8ee02b71c53c
Signed-off-by: Jamie Lennox <jamielennox@gmail.com>
This updates the github source to only use the most recent review
from each user reviewing a PR. This avoids having obsolete negative
reviews trump a newer positive review.
I've added a new test for good measure to ensure it works for the
github case, where we are doing some conversion of github formatted
timestamps to internal timestamps.
Change-Id: I5607901def856c9363ec786a4116bfec19c9c97c
Co-Authored-By: Jesse Keating <omgjlk@us.ibm.com>
Signed-off-by: Adam Gandelman <adamg@ubuntu.com>
Github reviews are a new pipeline requirement that is driver specific.
Reviews can be approved, changes_requested, or comment. They can come
from people with read, write, or admin access. Access is hierarchical,
admin level includes write and read, and write access includes read.
Review requirements model loosely the gerrit approvals, allowing
filtering on username, email, newer-than, older-than, type, and
permission.
Brings in an unreleased Github3.py code. Further extends that code to
determine if a user has push rights to a repository.
Documentation is not included with this change, as the docs need
restructuring for driver specific require / reject.
Change-Id: I3ab2139c2b11b7dc8aa896a03047615bcf42adba
Signed-off-by: Jesse Keating <omgjlk@us.ibm.com>
This adds support for triggering on github status updates.
Config schema for the github trigger has been updated to accept a list
of statuses, in the "github_user:context:status" format.
Change-Id: I15aef35716ddbcd1e66f84a73d27ca2689c936e4
Co-Authored-By: Jesse Keating <omgjlk@us.ibm.com>
Signed-off-by: Adam Gandelman <adamg@ubuntu.com>
When creating a github pull request change object, the statuses of the
head commit are fetched, and the latest status of each user and context
set is appended into a list of statuses. The string is
'user:context:state', where state can be success, error, or failure.
Context is freeform, and the user is the login of the entity that set
the status.
Tests have been added to validate that github based requirements on
pipelines are honored.
Change-Id: I45abbd6cbddd36b8491bdf9bb8d545216537ad2f
Signed-off-by: Jesse Keating <omgjlk@us.ibm.com>
The sql reporter doesn't not need to be enabled on the merger
or the executor since it will fail if only the zuul scheduler host
is authorized to connect to the sql database.
Change-Id: I453754052b75de3d4fac266dc73921c7ce9c075f
Gerrit may emits multiple events in a single packet, in particular the
change-created and ref-updated events. This results in a single poll
interruption and the GerritWatcher will only process the first event
until there is more activity on the event streams.
To correct this, the _read method needs to read all the lines already
available in the paramiko channel file before polling the file.
Story: 2000816
Task: 3401
Change-Id: I93b55a6491353aee8c954d349cef658c09d1ff2d
If the message has double quotation marks, Gerrit will not be able
to parse it. This applies shell quotes to the message before
sending it.
Change-Id: Ieda76b0f64346ab1388affead1a17630af61a5e6
As we expand the Github driver, we're seeing a need to specify driver-specific
pipeline requirements. To accomplish this, bump the require/reject pipeline
keywords down a level underneath connection names. This lets users specify
per-source pipeline requirements.
This adds new API methods for sources to create the new pipeline filters
(by returning instances or subclasses of RefFilter, which used to be called
ChangeishFilter).
This change also creates and/or moves driver-specific subclasses of EventFilter
and TriggerEvent in(to) their respective drivers.
Change-Id: Ia56c254e3aa591a688103db5b04b3dddae7b2da4
David Shrewsbury