096e47dc43
If a role is applied to a host more than once (via either play roles or include_roles, but not via an include_role loop), it will have the same task UUID from ansible which means Zuul's command plugin will write the streaming output to the same filename, and the log streaming will request the same file. That means the file might look this after the second invocation: 2022-05-19 17:06:23.673625 | one 2022-05-19 17:06:23.673781 | [Zuul] Task exit code: 0 2022-05-19 17:06:29.226463 | two 2022-05-19 17:06:29.226605 | [Zuul] Task exit code: 0 But since we stop reading the log after "Task exit code", the user would see "one" twice, and never see "two". Here are some potential fixes for this that don't work: * Accessing the task vars from zuul_stream to store any additional information: the callback plugins are not given the task vars. * Setting the log id on the task args in zuul_stream instead of command: the same Task object is used for each host and therefore the command module might see the task object after it has been further modified (in other words, nothing host-specific can be set on the task object). * Setting an even more unique uuid than Task._uuid on the Task object in zuul_stream and using that in the command module instead of Task._uuid: in some rare cases, the actual task Python object may be different between the callback and command plugin, yet still have the same _uuid; therefore the new attribute would be missing. Instead, a global variable is used in order to transfer data between zuul_stream and command. This variable holds a counter for each task+host combination. Most of the time it will be 1, but if we run the same task on the same host again, it will increment. Since Ansible will not run more than one task on a host simultaneously, so there is no race between the counter being incremented in zuul_stream and used in command. Because Ansible is re-invoked for each playbook, the memory usage is not a concern. There may be a fork between zuul_stream and command, but that's fine as long as we treat it as read-only in the command plugin. It will have the data for our current task+host from the most recent zuul_stream callback invocation. This change also includes a somewhat unrelated change to the test infrastructure. Because we were not setting the log stream port on the executor in tests, we were actually relying on the "real" OpenDev Zuul starting zuul_console on the test nodes rather than the zuul_console we set up for each specific Ansible version from the tests. This corrects that and uses the correct zuul_console port, so that if we make any changes to zuul_console in the future, we will test the changed version, not the one from the Zuul which actually runs the tox-remote job. Change-Id: Ia656db5f3dade52c8dbd0505b24049fe0fff67a5
72 lines
2.8 KiB
Python
72 lines
2.8 KiB
Python
# Copyright 2016 Red Hat, Inc.
|
|
#
|
|
# This module is free software: you can redistribute it and/or modify
|
|
# it under the terms of the GNU General Public License as published by
|
|
# the Free Software Foundation, either version 3 of the License, or
|
|
# (at your option) any later version.
|
|
#
|
|
# This software is distributed in the hope that it will be useful,
|
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
# GNU General Public License for more details.
|
|
#
|
|
# You should have received a copy of the GNU General Public License
|
|
# along with this software. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
import imp
|
|
import os
|
|
|
|
import ansible.plugins.action
|
|
|
|
|
|
def _full_path(path):
|
|
return os.path.realpath(os.path.abspath(os.path.expanduser(path)))
|
|
|
|
|
|
def _fail_dict(path, prefix='Accessing files from'):
|
|
return dict(
|
|
failed=True,
|
|
path=path,
|
|
msg="{prefix} outside the working dir {curdir} is prohibited".format(
|
|
prefix=prefix,
|
|
curdir=os.path.abspath(os.path.curdir)))
|
|
|
|
|
|
def _import_ansible_action_plugin(name):
|
|
# Ansible forces the import of our action plugins
|
|
# (zuul.ansible.action.foo) as ansible.plugins.action.foo, which
|
|
# is the import path of the ansible implementation. Our
|
|
# implementations need to subclass that, but if we try to import
|
|
# it with that name, we will get our own module. This bypasses
|
|
# Python's module namespace to load the actual ansible modules.
|
|
# We need to give it a name, however. If we load it with its
|
|
# actual name, we will end up overwriting our module in Python's
|
|
# namespace, causing infinite recursion. So we supply an
|
|
# otherwise unused name for the module:
|
|
# zuul.ansible.protected.action.foo.
|
|
|
|
return imp.load_module(
|
|
'zuul.ansible.protected.action.' + name,
|
|
*imp.find_module(name, ansible.plugins.action.__path__))
|
|
|
|
|
|
def _sanitize_filename(name):
|
|
return ''.join(c for c in name if c.isalnum())
|
|
|
|
|
|
# Ansible assigns a unique id to every task (Task._uuid). However, if
|
|
# a role is included more than once, the task object is re-used. In
|
|
# order to provide unique log ids for the Zuul command log streaming
|
|
# system, this global dictionary is used to map keys that are derived
|
|
# from tasks (task._uuid concatenated with the host name) to a counter
|
|
# which is incremented each time the task+host combination is
|
|
# encountered. Ansible will not run more than one task on a host
|
|
# simultaneously, so this should be sufficiently unique to avoid
|
|
# collisions.
|
|
#
|
|
# We use a global dictionary defined here so that zuul_stream can
|
|
# write to it and zuul.ansible.command modules can read it. Note that
|
|
# the command module operates after a fork and therefore it should be
|
|
# treated as read-only there.
|
|
ZUUL_LOG_ID_MAP = {}
|