11 Commits

Author SHA1 Message Date
Clark Boylan
a9a2f5ab50 Be more cautious initing mimetypes
We use mimetypes to set file mime types for upload to log server
locations. We also override yaml files mime type to text/plain when
doing so. If we then call mimetypes.init() again this overrides the
previous yaml update to text/plain. This wasn't a problem until
python3.12 but python3.12 (on Ubuntu Noble specifically) seems to import
test cases in a different order which results in the mimetypes.init()
call in generate_manifest.py overriding the yaml mimetype set by the log
upload test cases.

Simply check if mimetypes is already inited before we init it again
which should avoid the problem entirely.

Note that this is likely to only ever by a testing issues as typically
ansible wouldn't import all of this code together but the unittest
runner does.

Change-Id: Ifb9137ddd89713cad546129c462ad94315100940
2024-06-26 15:07:32 -07:00
James E. Blair
9e22cfdb0f Remove shebang from all python ansible modules
This commit in Ansible:
9142be2f6c

now allows Python modules to specify their interpreter with the shebang.
We expect our roles to use the discovered python interpreter on remote
nodes, and on the executor, we need them to use the virtualenv.  Removing
the specific shebang accomplishes this under Ansible 6, and has no effect
under older versions of Ansible.

Without this, for example, the log upload roles would not have access to
their cloud libraries.

Also update our ansible/cli check in our module files.  Many of our modules
can be run from the command line for ease of testing, but the check that we
perform to determine if the module is being invoked from the command line
or Ansible fails on Ansible 5.  Update it to a check that should work in
all 4 versions of Ansible that Zuul uses.

Change-Id: I4e6e85156459cca032e6c3e1d8a9284be919ccca
2022-09-15 13:48:10 -07:00
Daniel Blixt
dfbc8e2ec8 Use urlencoded filenames in test fixtures
When checking out repo to Windows environment, some files in test
fixture file trees do not get checked out. By setting renaming those
files with an urlencoded name, and populating a temporary directory
with corresponding file structure and decoded filenames during test by
a fixtures script, this will not cause checkout problems for those who
 want to read the repo in Win environment

Change-Id: If354eaf3f0bf2e86ddc05e3750aa5f0067dd9e21
2021-01-20 08:37:26 +01:00
Monty Taylor
635be2a3ae Fix cleanup of symlink fixtures
These aren't all getting cleaned up, which winds up breaking
the second runs. Instead of doing addCleanup with a method that
does the loop again, which can fail in the middle and not
clean up subsequent files, add an individual cleanup when we
add the symlink. This results in all of the symlinks consistently
being cleaned.

Change-Id: Id5a5b09c830ad2ad3bb0b77fb9dbdc494c629824
2020-02-24 17:17:17 +00:00
James E. Blair
04daed7bc4 Move index_links to correct task in generate_zuul_manifest
This argument is intended for the generate role rather than the
artifact return.

Change-Id: Ie5af37e2f55faa758371327d508f0e255694edd7
2020-02-05 13:06:09 -08:00
James E. Blair
2aa0dbcf29 Add index_links option to zuul manifest
This allows a deployer to indicate that they prefer the Zuul
dashboard to link to "dir/index.html" rather than just "/dir" when
displaying directory entries in the log browser.  This helps with
log storage systems which do not automatically serve index.html
files at directory urls.

Change-Id: I0444798c689667e51d8761a20b661f8a9b90323d
2020-02-05 07:19:20 -08:00
Mohammed Naser
f3615abc01 Add tests for manifest generation for missing files
This is a patch that adds test to make sure that we handle FileNotFound
exceptions during walks.

Change-Id: I6d3513ddb4a705a28043065b1398b75018ee3115
2019-09-12 19:39:09 +02:00
Jens Harbott
dd40f5ed9f Fix handling of dangling symlink on manifest generation
There may be broken symlinks within the log directories, those fail with
an error when os.stat() is executed on them. Let's just skip those.

Change-Id: I3e6982c53a08f57ac0e592b8a0041bbb39812d1f
2019-08-26 13:57:03 +00:00
James E. Blair
ea00b6aa5a Use human-readable names for artifact returns
This will cause links to appear on the build page with text like
"Docs preview site".

Change-Id: I65f8990b65607e72190a871409c3e31014ce86df
2019-08-12 07:13:27 -07:00
James E. Blair
02d9a29562 Fix typo in generate-zuul-manifest role
Change-Id: I763a7a6e95f08ba07b38ed6b1701fe16e2548471
2019-07-23 11:58:01 -07:00
James E. Blair
a74ff55816 Add generate-zuul-manifest role
Also, pin gitpython since it's gone py3-only.

Change-Id: I0d626945908ec4df785aea793f6243c6fdfbdb14
2019-07-23 09:43:55 -07:00