This internal-only attribute is basically the same as "ref" but
spelled differently only in the case of a change. Just use
the "ref" name in all cases for improved developer sanity.
Change-Id: I476f8d32dae37309ab0c9e11c8a5337b213f985e
When variables such as branch, tag, oldrev, and newrev are not
applicable, ensure that they are not defined at all as Ansible
variables.
Update the documentation to be more clear on oldrev and newrev.
Change-Id: I56d9a6496467de2f328eb9a43cc52c5f7ef73310
If the loop item has a result msg - we should display it.
If the loop item doens't have a result msg - there is no other
identifying feature than the loop variable - which may be a full dict or
other json-object. In that case, pretty print it rather than one-lining
it. This may not be perfect - but let's give it a try.
Change-Id: Ib9a4dc0ac7783559151f12996144941e534633ee
In v2 there was a way for a job to see all the changes being tested
with it. This is more complicated in v3 with the addition of
multiple sources. Fortunately, we can pass all the information we
need to accurately represent the situation to the job via an
data structure in ansible variables.
Change-Id: Ibca2065e9014ec67167057f89ef07d36103f89c7
We're providing more than one uuid to the job, so to make it clear
that one is for the build, and one is for the buildset, use those
words.
Change-Id: If9ebdc783dcef4f8dea9fa491b40ae49416e5cf1
So that we may re-use the same jobs for pre and post merge tests,
enqueue an item for every branch of every timer-triggered project
and checkout that branch before running the job. This means that
rather than having a job for gate plus a job for each stable branch,
we hav just have a single job which runs with different content.
The old method is still supported using override branches.
This updates the model to include Change, Branch, Tag, and Ref
objects which can be used as the value of Item.change. Branch,
Tag, and Ref are all very similar, but the distinction may help
us ensure that we're encoding the right information about the items
we are enqueing. This is important for branch matching in pipelines
and is also used to provide job variables.
Change-Id: I5c41d2dcbbbd1c17d68074cd7480e6ab83f884ea
Fact gathering does not have a variable_manager yet, so we need to put
in an if condition for localhost fact gathering.
Additionally, simple errors, such as our local code is prohibited
message, wind up just being {'msg': 'Error Message', 'failed': True}. We
don't need to emit that as a json dict- we can TOTALLY emit that as, you
know, a string after ERROR. So it'll look like:
2017-07-21 06:38:28.982788 | localhost | ERROR: Executing local code is prohibited
Story: 2001129
Task: 4838
Change-Id: Ie9f3bd917bf75f5a566a8a1a09ccf9f29786d82f
Turn on fact caching and smart gathering but not for localhost! We do
not want to leak information about zuul-executors to untrusted
playbooks.
Change-Id: I40941c0f15d801d91c60ff5af33d047044052154
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This meant bubblewrap needed to have a zuul user on the host system,
which when gathering facts on localhost would cause an ansible
failure. Additionally, this no longer is needed because our logging
is now handled by zuul_stream.
Change-Id: I75cd050e26438dbe6d47ff216ffe6e08b9687aca
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We were short-circuiting loading some changes which appeared in
our dependent change query history because we were only looking
at change numbers, not change+patchset. Record both in the history
to avoid this problem.
Note, this could likely be simplified further, but rather than doing
so now, I'll leave that for when we implement cross-source dependencies,
which is very likely to change this further.
This also corrects a defect in the fake gerrit change test fixture.
Change-Id: Ie1d9fe0dfd32032caf4ff8c90bded30d5e7bf886
We were trying to be clever and only emit a space before playbook start
if it was not the first. WHOOPS. Instance variables to not persist
across process boundaries - so the stats call cannot set a variable for
the playbook_on_start to read.
Change-Id: I5fb19102d4e5903f3a49c16df5701a70b93b0b1f
Pass in and use info from the JobDirPlaybook rather than trying to strip
path elements from the playbook name.
Change-Id: Ifcd6f05e27c987d40db23b3dcec344c2eb786d7c
This adds a static html page to zuul-web that a browser can be pointed
to for streaming log files. To leverage this in the status UI the
scheduler sets the build url to this html page adding build uuid,
logfile and optionally a different url for accessing the websocket.
Tobias has to run his websocket streamer on a different domain than
the other things, via proxy things - so the url zuul-web is serving
for the static file isn't the same as what it is for the websocket
from a consumer perspective. So introduce a config variable for
zuul-web that allows setting an explicit url for that. If it's not
set, use the relative path from static/stream.html to console-stream.
Further to not throwing away the finger url retail this as additional
field in status.json. With this a later change to the status ui could
let the user choose between html and finger log streaming.
Co-Authored-By: David Shrewsbury <shrewsbury.dave@gmail.com>
Co-Authored-By: Monty Taylor <mordred@inaugust.com>
Co-Authored-By: Tobias Henkel <tobias.henkel@bmw.de>
Change-Id: I2da7979f448934abe3d41f3a5e50d09004fcccc2
We have spacer lines after task banners that we don't need.
We don't have spacer lines between a PLAY RECAP and the next PLAY which
we do need.
We have a spacer line before a PLAY RECAP which doubles with the
post-task spacer.
We don't need args printed in task banner lines.
Change-Id: I678690bbb4fc41d6028b5e86f43574989c24f8b6
The play banner is very busy. Let's just emit the information about the
execution phase on it's own line. ALSO - let's use PRE-RUN, RUN and
POST-RUN, since those are actually the terms from zuul.yaml. Ignore the
index count in the text log - it's not super interesting. It IS
interesting in the json - but we'll deal with that later.
There is now an annoying pile of copy-pasta between zuul_stream and
zuul_json that we should put into a shared library location. But let's
do that as a follow up.
Change-Id: I6e3abd4c2c7c1967435815fab4f9aaa333103397
The json output wasn't doing its read/append/write cycle properly,
leading to:
{
"plays": {
"plays": {
"plays": {
"plays": {
"plays": {
"plays":
Which, while amusing, isn't really what we wanted.
Add a wrapping layer that contains playbook info. The original format
has a list of play results but only one stats section, so it's not
actually suitable for appending a list of plays anyway.
Change-Id: I49394e9faa8027a21e5ef6919c0f75a4473f51a9
In the case that we're using an implied role, strip 'ansible-' and
'role-' from the repo name when using it as the implied role name.
If this doesn't work for any repos, they can always specify the
role name explicitly.
Change-Id: Ibf4b1852a811542de01f618fa3c5dcde3f57e27c
Add the project in which a job is defined as an implicit role project.
This is a convenience for job authors who may want to put roles in
the root of the project (ie, not adjacent to job playbooks, since,
after all, the roles may be useful outside of the job playbooks). In
that case, they will not need to specify the job's own project in the
roles: section of the job.
Change-Id: Ia382c2da9f7eb7139ceb0b61cb986aace8dc8d8f
There are some errors that the executor may encounter where it will
be unable to, or refuse to, run a job. We know that these errors
will not be corrected by retrying the build, so return them as
errors to the user. The build result will be "ERROR" and the message
which is brief, but hopefully sufficient to illuminate the problem,
will be added to the job report.
Change-Id: Iad486199de19583eb1e9f67c89a8ed8dac75dea1
Story: 2001105
Story: 2001106
We have the full json output if we need it which contains all of the
results - but in most cases we do not need the full result dict on
success and it makes reading logs quite noisy. (hi stat)
Change-Id: I153bcdf0ff4826d34bc83c68ac1cefc4d264da01
Tried first with the upstream callback plugin, but it is a stdout
plugin, so needs to take over stdout to work. We need stdout for
executor communication. Then tried subclassing- but the magical ansible
module plugin loading fun happened again. Just copy it in and modify it
slightly for now.
We add playbook, phase and index information. We also read the previous
file back in and append to it on subsequent runs. This may be a memory
issue. HOWEVER - the current construction will hold all of an individual
play in memory anyway. Most of our content size concerns are around
devstack jobs where the bulk of the content will be in a single playbook
anyway - so although ram pressure may be a real thing - we may need to
solve it on the single playbook level anyway. But for now, this should
get us the data.
Change-Id: Ic1becaf2f3ab345da22fa62314f1296d76777fec
Publishing the inventory as part of publishing logs would be handy
in debugging that everything is as one expects. However, secrets
currently go into the inventory, which means publishing it is not safe.
Write the secrets to their own file.
Also, Return errors if users try to define zuul vars or secrets. To
support that, we need to pass zuul vars as a top-level param, not inside
of vars already.
Change-Id: If58b89882a817ff219ed5f8faf2bde31cc8e1a6a
When running the zuul command without arguments it fails with AttributeError
'Namespace' object has no attribute 'func'. This change prints the usage() when
no argument are used and also sets the default show sub command to running_jobs.
Change-Id: I47c8b4c32a8ce9bdcfeca6719a03794feab477f0
This change adds a tenant column to the zuul_buildset sql table to be able
to filter results based on tenant.
Change-Id: I4eec8e9a32883c54da953b7279e0f8a8c86b943b
James E. Blair