The stage-output role had two assumptions:
1) The zuul_copy_output variable would always be defined
2) The role would be able to sudo on the remote node
To make it easier for users who want to use this in a base job,
remove both of those assumptions.
1) We now supply a default empty dict if the variable is not defined
so that the role does not fail with an error.
2) We check to see if we can sudo on the remote node, and if we can,
we do when copying files; otherwise we don't, and assume that the
user will only specify files they have access to (if they don't then
the copy will fail).
Change-Id: I5428c44adfafac4872342b59a92de311f41687b6
With the arrival of ansible-lint 4, Jinja2 variable expansions must
include spaces before and after the variable name inside the
brackets.
Adjust the new violations accordingly and remove the rule
206 exclusion.
Change-Id: Ib3ff7b0233a5d5cf99772f9c2adc81861cf34ffa
The archival task currently relies on the content of all_sources,
but some files are renamed afterwards if their extensions match
one of the special extension of the list.
This means that few files from all_sources are not found and then
not compressed.
Change then the logic: simply discover all files inside logs
and compress all of them.
Change-Id: I7d34d7d90849736b7b842c0bdd67492816f98ebc
This should fix a syntax issue on ansible 2.7.
Change-Id: I006ff64c40875b8f1edd1e9f9bdfe509e2bdb295
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
If the stage_dir/logs directory was a symlink, Ansible would refuse
to perform the chmod. However, if we tell Ansible that it is a
directory, it's fine. This (perhaps curiously) does not tell
Ansible to turn the symlink into a directory. It just tells it
that everything is okay.
Change-Id: Idbd78b97b95ce50b23dda8f384e32359d31013e3
So that this can more easily work with files like '/var/log/syslog',
make it so that if the user specifies "logs_txt" then the file gets
a .txt extension before being placed in the logs directory.
The existing extensions_to_txt variable doesn't work for this because
it requires an existing extension.
Change-Id: I34fea0c44030c04a5540d6b62976557143289196
The 'linters' tox environment was not running on roles, it was only
running on playbooks. This change adds a command to the linters
environment to ensure all roles are linted.
Since these weren't being linted, there were some problems with them.
The first was a warning about usage of the shell module. Both of these
usages seemed appropriate, so this patch adds a skip_ansible_lint tag to
each task that was failing. The second is a warning "no action detected
in task" for zuul modules. This is due to the fact that the linter
cannot find the custom module. One option is to set the ANSIBLE_LIBRARY
path to point to the zuul ansible library directory[1], but the linter
virtualenv does not actually have zuul installed. Instead, we just
disable the linter for the failing tasks.
This also cleans up a comment in the tox file that was referring to a
nonexistent zuul job.
[1] https://github.com/willthames/ansible-review/issues/16
Change-Id: Ie49da9a09733b623bb25c5a4c8aa07eacacf4b33
We only rename the extensions for files specified in zuul_copy_output
but we should rename files in folders specified in zuul_copy_output as
well. In addition we need to rename files or folders that start with a
'.' which would otherwise not be visible once uploaded to logs.
Change-Id: Id55ab51019d10d5ccbc6531549758ac71ad276be
The stage-output role accepts a list of extensions to be replaced as txt
so that they can be seen directly from within a web browser.
Using a list means that children jobs cannot add to the parent setting,
only overwrite. Adding support for dicts while preserving, at least for
now, support for lists - to avoid breaking existing implementations.
This reverts change 844b3333aae02d596061386b5a1a2fbc821950be.
Change-Id: I5052c873f0a5da7ecaa38627e8900af86bbec2f9
Needed-By: https://review.openstack.org/539686
Needed-By: https://review.openstack.org/539708
Needed-By: https://review.openstack.org/539854
This reverts commit 146fa7b2e616ec4f2ba15e8445a666a086549cc3.
This does not work yet as planned. Let's revert for now and test properly again.
Change-Id: Idac7e398fe4854769589411f22f7c4ce7cb363f6
Two tasks in stage-output "Register sources" and "Set source and
destination for files and folders" spam the whole output of "stat"
to be printed to console for each and every file to be staged.
The info is printed twice and it's hardly valuable for debugging purposes.
The following task, "Build a list of source, dest dictionaries for text
file", prints a more compact list of all files and folders that are going
to be staged, which I believe is enough for debugging.
To solve this add no_log to the two tasks mentioned above.
Change-Id: Id52047c429233c6684f2fca5537674f91fb2603d
The stage-output role accepts a list of extensions to be replaced as txt
so that they can be seen directly from within a web browser.
Using a list means that children jobs cannot add to the parent setting,
only overwrite. Adding support for dicts while preserving, at least for
now, support for lists - to avoid breaking existing implementations.
Change-Id: Ifc4b2070c5844e51acb9ad5d512e9f25f5bdd7ef
If an artefact is a symlink, dereference it before coying it as
just copying the link may render it broken one the staged folder
is synced to the executor.
Change-Id: I5ad4dfccbc1389da3afc53f3c866d3475e006db6
This role does roughly what save_dir and save_file do in d-g
functions, only it can be used for a list of files and/or
folders. Jobs can use this role to rename artifacts and place
them in a staging folder which is then used for mass sync to
the executor.
This is typically used in post - the zuul user won't necessarily
have write access to the folders with the artifacts - so it's
easier to stage everything to a folder owned by the zuul user.
Change-Id: Id45a50821539985710da7172a9185fd91e803603