This removes redundant ssh known hosts prep in the
multi-node-known-hosts role as we've seen these tasks take significant
amounts of time due to the high cost of ansible looping. According to
the change [0] that added the system wide known_hosts setting this was
not originally done to avoid breaking testing in openstack-zuul-jobs.
Since then those tests have been moved to zuul-jobs [1] and I've updated
them with this change.
This should cut the time to run this role by about half.
[0] https://review.opendev.org/c/zuul/zuul-jobs/+/548642
[1] https://review.opendev.org/c/zuul/zuul-jobs/+/668767
Change-Id: I90492bbb5ae15fd3b36a54071c4eef891f53b405
Support setting authToken to use a token for upload rather than
username/password.
This is "based" on Id0c47d088d6e8febbae3c96caecc81ebe247754e which
does the same for pypi. There we run a test by uploading to the test
pypi server, I'm not aware of a similar thing for npm.
Change-Id: Ie8610bdfaa33b2120a61802c12e6ba534fe55464
This replaces ansible loop tasks which rename log files to have a .txt
suffix with an ansible module task. The reason for this is each ansible
loop iteration can be very slow; up to 3 seconds per task. When
many log files need to be renamed this adds up quickly. The module can
do all of the renames in a single python process that should be much
quicker.
Note the old interface expected a list of extensions to rename. It then
converted that list of extensions into a regex used to find the files to
rename. The new code does not use regexes. It is possible someone abused
the old interface to hijack the regex and do something extra fun, but
that was not documented nor likely to be safe.
Change-Id: I7033a862d7e42b3acd8ad264aefca50685b317ff
This adds extra . separators to the file we check is renamed. This came
up as a real world issue with devstack when testing the follow on update
to stage-output. We put this in a separate change to ensure the old and
new behavior is consistent.
Change-Id: I3b7504cfd90a4e7523ce88c50e90b2e9004e05ee
This updates our stage-output test to exercise the extensions_to_txt
behavior. Do this as we've seen the performance of these tasks is quite
poor. Testing will help us improve this without breaking things.
Change-Id: I35844fa438773b4789e7f1e4d223bd59d5df7ba7
The new 6.5 release seems to have some issues with our code base (see
linked issue). Pin to 6.4 for now.
Change-Id: I9516314f7bd2924bd50456a3f850f6151678e95f
This was added with I2ab11bb45b6b2a49d54db39195228ab40141185c. I
don't think we need to match on the
zuul-tests.d/python-roles-jobs.yaml file; it means that if you update
one job (e.g. say for a new platform like
Id107ff96c75a11d03a0bde30f6011f2b44359f23) it unnecessarily runs the
pyenv test for every platform. Remove the matcher.
Change-Id: I8d24ed8b172304b829668cf768e66e7810c58ef2
The `fail_json()` method of the Ansible module expects a msg argument.
https://docs.ansible.com/ansible/latest/reference_appendices/module_utils.html
Traceback (most recent call last):
File \\"<stdin>\\", line 102, in <module>
File \\"<stdin>\\", line 94, in _ansiballz_main
File \\"<stdin>\\", line 40, in invoke_module
File \\"/usr/local/lib/python3.10/runpy.py\\", line 224, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File \\"/usr/local/lib/python3.10/runpy.py\\", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File \\"/usr/local/lib/python3.10/runpy.py\\", line 86, in _run_code
exec(code, run_globals)
File \\"/var/cache/zuul-executor/jobs/2f077ccd11b844e587a4d220ed22f36c/work/tmp/ansible_zuul_s3_upload_payload_l3h9evxc/ansible_zuul_s3_upload_payload.zip/ansible/modules/zuul_s3_upload.py\\", line 366, in <module>
File \\"/var/cache/zuul-executor/jobs/2f077ccd11b844e587a4d220ed22f36c/work/tmp/ansible_zuul_s3_upload_payload_l3h9evxc/ansible_zuul_s3_upload_payload.zip/ansible/modules/zuul_s3_upload.py\\", line 313, in ansible_main
File \\"/var/cache/zuul-executor/jobs/2f077ccd11b844e587a4d220ed22f36c/work/tmp/ansible_zuul_s3_upload_payload_l3h9evxc/ansible_zuul_s3_upload_payload.zip/ansible/module_utils/basic.py\\", line 2183, in fail_json
AssertionError: implementation error -- msg to explain the error is required
Change-Id: I30a50ebd3c4a2553142a67ecb1bbb1faa2b3c527
We should use the oldest release where we run centos stream 9 jobs which
is wallaby. We set up xena in the past as we backported cs9 jobs to xena
first but now we should switch to wallaby or we may hit issues in
wallaby jobs when the ovs version in xena is higher that in wallaby (as
now).
Change-Id: I18dab9701e96b25396b45df221f54fe94c1ab36b
This reverts commit c4a666991207f9434a819da9bac8b3603ad3d66e.
The change was originally reverted because it broke testing on Jammy.
Jammy broke because it was attempting to install cri-o using the Bionic
method via a PPA. This is no longer valid for newer Ubuntu, and Focal
uses a different process too. We update the cri-o installation for Jammy
to match Focal and make this method the default so that new Ubuntu
releases default to modern installation methods. Bionic continues to
install from a PPA.
Additionally we bump the cri-o version from 1.15 to 1.24 to get a
version that has packages for both Focal and Jammy. This new version
requires we also install cri-o-runc separately as they don't seem to
have proper package dependencies in place between these two packages.
The crio systemd service fails to start without cri-o-runc installed.
Change-Id: Ic29576e26be094744cc1b169a3c8f0bca814f089
This sorts the supported test platforms so that we can more easily
modify them in a future change.
Change-Id: I3a3a0347b89ab90d6ee74bc3070f7ad3604d59df
This was apparently not tested on Ubuntu jammy and has broken
Nodepool testing.
https://zuul.opendev.org/t/zuul/build/9ae631c37a384b4bb63515c6c5f04a00
Revert "ensure-kubernetes: pull cri-dockerd systemd from tag"
This reverts commit ad0ea28b6a2cbc8d68ee1a05e6a1d2712102dca4.
Revert "ensure-kubernetes: install cri-dockerd; fix networking"
This reverts commit 08c922fd988e98bd93d5a6d488a98e5b76797e18.
Change-Id: Ic20b8840c478b6d81626f728b8661ef5946e12d4
I noticed this by accident when I ran ansible-lint over this repo from
an outside context; it didn't use the .yamllint in here and started
compalining about eof whitespace.
After scratching my head for a bit as to why this didn't fail here, I
realised we've allowed various newlines since the initial commit
I936fe2c997597972d884c5fc62655d28e8aaf8c5.
Remove this and just use the default eof rules, and fixup the
whitespace as required. This is fairly unimportant, but is nice for
consistency.
Change-Id: Idb46a1f39ba798b0bf70eaa27b4c6b4758ce3d26
Progressive mode
... makes the linter return a success even if some failures are
found, as long the total number of violations did not increase since
the previous commit.
I have found what I think is inconsistent matching of the errors
between runs and I'm not sure it isn't hiding problems. We are
linter-clean and gate on the linter passing, so we don't need to do
this two-pass system.
Change-Id: I47be01a095d80dfb4d15f90da7bce49c3d42a2dd
This updates to ansible-lint 6. Some prior changes have updated for
the bulk of the rules, a couple of noqa points are added here. Some
updates to the skipped rules are added. This should have no
operational change, the only updates are cosmetic.
Change-Id: I165677bbb904f92292df00f7b9b27f8f3573aeb0
This is preparation for a later version of ansbile-lint, which finds
missing names on blocks. This seems a reasonable rule, and the
Ansible manual says [1]
Names for blocks have been available since Ansible 2.3. We recommend
using names in all tasks, within blocks or elsewhere, for better
visibility into the tasks being executed when you run the playbook.
This simply adds a name tag for blocks that are missing it. This
should have no operational change, but allows us to update the linter
in a follow-on change.
[1] https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html
Change-Id: I92ed4616775650aced352bc9088a07e919f1a25f
This fixes a number of places where we do not have spaces between
filters. I think that this is a reasonable rule for readability (I
also think it probably was enforced, but maybe later versions got
better at detecting it?).
These are detected by a later version of Ansible lint; this change
should have no operational change to any roles but prepares us to
update in a follow-on change.
Change-Id: I07e1a109b87adce86f483d14d7e02fcecb8313d5
This marks this particular ignore_errors as safe. Generally I think
the rule is good, in that tasks get marked as failed when using
ignore_errors which is confusing and hard to debug. But in this case
it's because of missing hostnames
(I8d67d8aa284b6ce9ae012608e8f1b12784ce836b) and not something we can
catch easily.
Change-Id: I9b5d6a85254f689e97bca91951632b4dea60cfde
In this repo we name the loop variables. Although this is a test
playbook, it's good for consistency. This is picked up by a later
version of ansible-lint. This should have no operational change.
Change-Id: I084a1e8515fe1fda039190fe6518512ebf03217e
I don't think this testing has been run of Focal before, and the ssh
there is more picky about trailing newlines in the ssh private key.
Ensure it has a newline to avoid problems.
While we're here, clean up an old unused argument
Change-Id: If300083b9bc8e5538dcb1eeeaa896ba22c21232e
Some of our jobs have started failing with
error: Multiple top-level packages discovered in a flat-layout:
['roles', 'playbooks'].
... <and so on> ...
which seems to be a combination of jobs that don't run often and some
recent-ish version of setuptools/pip. From what I can tell, what we
want to do here is disable autodetection as what we have isn't really
a python distributable package.
Also fix the two places this is monkey-patched in over the existing
file.
Change-Id: I2a0dfbbedbb9bddd34b6af691118cf7c422a82b0
When we added this test I used ensure-pip in the ensure-twine role to
pull in the pip dependency. But we soon realised that ensure-twine
ran in a rootless context so couldn't install packages, and reverted
it with I2cf4224228860b8a2cddd360636bb8633967b07e.
However, this testing wasn't fixed up. Use ensure-pip in the test
playbook before we run the test, and add a note to ensure-twine that
it runs without privileges.
Change-Id: I6aef14b188dce8c4de774b09ddad718fec8fbf6c
For ... reasons ... kubernetes deprecated docker as a container
runtime, and requires this cri-dockerd daemon to make docker cri-ish
enough to work. Install and start it so the docker path keeps
working, although long-term I guess they're saying to move away from
this (from what I read it the "none" driver will also have problems
with cgroupsv2, which makes it's future on Jaunty look interesting).
Honestly I don't really know why the cri-o now needs the
--network-plugin flag. Without it I get
X Exiting due to RUNTIME_ENABLE: unknown network plugin:
which isn't described anywhere I can see. Improvements welcome :)
Change-Id: I8ff34fa116aca14abee7e71f510bc49ffc547524