If the loop item has a result msg - we should display it.
If the loop item doens't have a result msg - there is no other
identifying feature than the loop variable - which may be a full dict or
other json-object. In that case, pretty print it rather than one-lining
it. This may not be perfect - but let's give it a try.
Change-Id: Ib9a4dc0ac7783559151f12996144941e534633ee
Fact gathering does not have a variable_manager yet, so we need to put
in an if condition for localhost fact gathering.
Additionally, simple errors, such as our local code is prohibited
message, wind up just being {'msg': 'Error Message', 'failed': True}. We
don't need to emit that as a json dict- we can TOTALLY emit that as, you
know, a string after ERROR. So it'll look like:
2017-07-21 06:38:28.982788 | localhost | ERROR: Executing local code is prohibited
Story: 2001129
Task: 4838
Change-Id: Ie9f3bd917bf75f5a566a8a1a09ccf9f29786d82f
We were trying to be clever and only emit a space before playbook start
if it was not the first. WHOOPS. Instance variables to not persist
across process boundaries - so the stats call cannot set a variable for
the playbook_on_start to read.
Change-Id: I5fb19102d4e5903f3a49c16df5701a70b93b0b1f
Pass in and use info from the JobDirPlaybook rather than trying to strip
path elements from the playbook name.
Change-Id: Ifcd6f05e27c987d40db23b3dcec344c2eb786d7c
We have spacer lines after task banners that we don't need.
We don't have spacer lines between a PLAY RECAP and the next PLAY which
we do need.
We have a spacer line before a PLAY RECAP which doubles with the
post-task spacer.
We don't need args printed in task banner lines.
Change-Id: I678690bbb4fc41d6028b5e86f43574989c24f8b6
The play banner is very busy. Let's just emit the information about the
execution phase on it's own line. ALSO - let's use PRE-RUN, RUN and
POST-RUN, since those are actually the terms from zuul.yaml. Ignore the
index count in the text log - it's not super interesting. It IS
interesting in the json - but we'll deal with that later.
There is now an annoying pile of copy-pasta between zuul_stream and
zuul_json that we should put into a shared library location. But let's
do that as a follow up.
Change-Id: I6e3abd4c2c7c1967435815fab4f9aaa333103397
The json output wasn't doing its read/append/write cycle properly,
leading to:
{
"plays": {
"plays": {
"plays": {
"plays": {
"plays": {
"plays":
Which, while amusing, isn't really what we wanted.
Add a wrapping layer that contains playbook info. The original format
has a list of play results but only one stats section, so it's not
actually suitable for appending a list of plays anyway.
Change-Id: I49394e9faa8027a21e5ef6919c0f75a4473f51a9
We have the full json output if we need it which contains all of the
results - but in most cases we do not need the full result dict on
success and it makes reading logs quite noisy. (hi stat)
Change-Id: I153bcdf0ff4826d34bc83c68ac1cefc4d264da01
Tried first with the upstream callback plugin, but it is a stdout
plugin, so needs to take over stdout to work. We need stdout for
executor communication. Then tried subclassing- but the magical ansible
module plugin loading fun happened again. Just copy it in and modify it
slightly for now.
We add playbook, phase and index information. We also read the previous
file back in and append to it on subsequent runs. This may be a memory
issue. HOWEVER - the current construction will hold all of an individual
play in memory anyway. Most of our content size concerns are around
devstack jobs where the bulk of the content will be in a single playbook
anyway - so although ram pressure may be a real thing - we may need to
solve it on the single playbook level anyway. But for now, this should
get us the data.
Change-Id: Ic1becaf2f3ab345da22fa62314f1296d76777fec
Don't emit results when no_log is set. That's impotant.
If the result is a message and a rc, emit the message - since
that's the important message for the user.
If messages are multi-line, split them and show them multi-line.
If the results have an "invocation" in them, suppress it.
This will still result in synchronize calls potentially showing a lot of
output - but it should be in a readable format. If output from a
synchronize call is not desired or two large, it can be marked no_log.
Change-Id: I90cf5de8eb4479773aee97b5590ac125c06dc00e
During dumping the result_dict it we iterate over a copy of it and
filter out some variables. This results in the error below [1]. The
reason is the result_dict.keys() doesn't return a new set of the
existing keys but a view of them which is changed then during
iteration. This can be fixed by creating a list of the keys and
iterating over that.
[1] Log:
[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin
(<ansible.plugins.callback.zuul_stream.CallbackModule object at
0x7f5d41b73b70>): dictionary changed size during iteration
Change-Id: I2ad148002dbe471296b4c1b0e448f2cc03ce2913
The KeysView we get from the hostvars is not indexable and throws an
error [1] when trying to access via an index. This can be fixed by
using an iterator and getting one item.
[1] Log:
[WARNING]: Failure using method (v2_playbook_on_play_start) in callback plugin
(<ansible.plugins.callback.zuul_stream.CallbackModule object at
0x7f5d41b73b70>): 'KeysView' object does not support indexing
Change-Id: I0c95b55a3b26856e9d101db95682efa0e0694523
We pass the execution phase and the index of the phase now, so emit it
into the play banners in the log. This will allow us to post-process the
logs and put in smart things like "collapse pre tasks".
Also include information about include statements.
Also rename zuul_execution_phase_count to zuul_execution_phase_index -
mainly just because it's an index not a count. My shed is red.
Change-Id: I975ed9547bbcdbb70d5a25c9be398888bdcdb07a
This loads a json file (work/results.json) that the job can write
to. It will be loaded by the executor after the job completes and
returned to the scheduler.
We can use the data in this file as the reported log URL for the
build. Later we can use it to supply file/line comments in
reviews.
Change-Id: Ib4eb743405f337c5bd541dd147e687fd44699713
The check 'if not ret' not only matches a missing return code but also
the value 0 which is actually a successful run. Thus successful
commands end with an error 'Something went horribly wrong during task
execution'. This can be fixed by explicitly checking for None.
Also adds a successful shell task to the test_playbook test case which
fails without this patch.
Change-Id: If1d0721574a82e247659ab0f865ae6acfe12a6be
If there's an exception, we might not write the final lines to the log.
Put those in a finally block.
Change-Id: I0a294af3874c53543176a7bd9c4b89253717b83c
Using python to collect data for later display is useful, as it allows
us to control what's output and what isn't. That's defeated by the
default behavior of logging all of the result parameters. Add a flag we
can set in result dicts that will cause us to discard the parameters
from the log on success. On failure we don't want to do this because on
failure all the debugging we can get is great.
Change-Id: Ic2295c7a8be30144fe47c5bd385e827bfd8095f2
We do this same fallback in other places, and see warnings in the
executor log about hitting this.
Change-Id: Ic9d27a7d3299c8f32fb6d1104dbda4bf5b76c54a
This shouldn't really ever happen for real in zuul, but just in case we
do something in a trusted playbook somewhere like setting ansible_host
to localhost, the code should at least do the right thing. (This comes
up more in doing local testing of plugins)
Change-Id: I638fa5f8cd69cd46823ea42bc4e257972d125945
Items in loops get their own callback events, so we need to handle them.
Additionally, the final item and the event finished event come out of
sequence - so we need to save a note in the event finished event and
then emit it after the following item.
Finally, we'd have a todo list item from ansiblefest to copy not
directly modify the result objects we get because they're shared. Turns
out this is immediately necessary with items, because popping
stdout_lines in one item totally borks the further items (whoops)
Change-Id: Ia70239f240276b10ad09d4bda57310ebcbdb1f4d
When gather_facts is explicit and then is run, it shows up as a 'setup'
task in the execution. It's noise in the log.
Change-Id: I1a9776dcb4c832827d394f5e8f425bd21efdd089
This caused us to fail to log lines when running commands on
localhost (because in that situation, we use python3).
Change-Id: If9f5cf6ed62cb24a6fc16ebf189d36a752fa66d5
It gets decoded as utf-8 on the other side of the socket, and it's being
written to the file as utf-8 - so there's not reason to read it in to a
string before sending it over the wire.
Change-Id: Iad3d33108835fad3ceae0eec38985659449a6452
For local non-command and non-shell tasks there is an access to
stdout_lines which is not assigned before.
Change-Id: Ifaa88a5b7399514afd1a9babe195a70a4a0e9fae
These are causing WILDLY strange issues on ze01. Revert them until we
understand.
Revert "Update run_command to latest ansible"
This reverts commit 1dbf5f96b5.
Revert "Sync command from ansible"
This reverts commit 429428c020.
Change-Id: Ib481160312216a4dbc9c3184d31cbc35b5b36371
Monty Taylor