zuul/zuul-jobs
Code Issues Proposed changes

Stop compressing files during intermediate steps part 2

Browse Source 
Stop compressing files that would otherwise be handled uncompressed in
intermediate log handling. The reason for this is final log/artifact
upload should decide if a file need to be compressed or not as some file
hosting services (like swift) may not return the expected data if we get
this wrong.

This was noticed when we used swift to store intermediate tar.gz
tarballs before final upload to permanent storage. When retrieving the
tar.gz files from swift we got them back as uncompressed .tar files
(because swift was being helpful) and then later when we try to
uncompress these files we break.

Instead we'll let the upload to swift (or other storage) decide if files
should be compressed further than their existing state.

This is the second change in the stack and update the stage-output role.

Change-Id: I2fa6f123d23568ba1537a7c26d2672f04110c35b
This commit is contained in:
Clark Boylan 2020-01-06 13:19:23 -08:00
parent bbca430417
commit a6f2ca827e
2 changed files with 7 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
roles/stage-output/README.rst
View File

@ -60,6 +60,11 @@ intended to be used before output fetching in a base job's post-playbook.
zuul.conf --(staged as)--> zuul_conf.txt zuul.conf --(staged as)--> zuul_conf.txt
.. zuul:rolevar:: stage_compress_logs .. zuul:rolevar:: stage_compress_logs
:default: True :default: False
When True, files staged as logs will be compressed individually. When True, files staged as logs will be compressed individually.
Note this option is deprecated as final log storage should control
whether or not contents are compressed. The reason for this is certain
services like swift may serve compressed files like .tar.gz tarballs
uncompressed when you want them to be compressed when served in this
way.

2
roles/stage-output/defaults/main.yaml
View File

@ -1,3 +1,3 @@
stage_dir: "{{ ansible_user_dir }}" stage_dir: "{{ ansible_user_dir }}"
extensions_to_txt: extensions_to_txt:
stage_compress_logs: true stage_compress_logs: false