I noticed this by accident when I ran ansible-lint over this repo from
an outside context; it didn't use the .yamllint in here and started
compalining about eof whitespace.
After scratching my head for a bit as to why this didn't fail here, I
realised we've allowed various newlines since the initial commit
I936fe2c997597972d884c5fc62655d28e8aaf8c5.
Remove this and just use the default eof rules, and fixup the
whitespace as required. This is fairly unimportant, but is nice for
consistency.
Change-Id: Idb46a1f39ba798b0bf70eaa27b4c6b4758ce3d26
In this repo we name the loop variables. Although this is a test
playbook, it's good for consistency. This is picked up by a later
version of ansible-lint. This should have no operational change.
Change-Id: I084a1e8515fe1fda039190fe6518512ebf03217e
I don't think this testing has been run of Focal before, and the ssh
there is more picky about trailing newlines in the ssh private key.
Ensure it has a newline to avoid problems.
While we're here, clean up an old unused argument
Change-Id: If300083b9bc8e5538dcb1eeeaa896ba22c21232e
Some of our jobs have started failing with
error: Multiple top-level packages discovered in a flat-layout:
['roles', 'playbooks'].
... <and so on> ...
which seems to be a combination of jobs that don't run often and some
recent-ish version of setuptools/pip. From what I can tell, what we
want to do here is disable autodetection as what we have isn't really
a python distributable package.
Also fix the two places this is monkey-patched in over the existing
file.
Change-Id: I2a0dfbbedbb9bddd34b6af691118cf7c422a82b0
When we added this test I used ensure-pip in the ensure-twine role to
pull in the pip dependency. But we soon realised that ensure-twine
ran in a rootless context so couldn't install packages, and reverted
it with I2cf4224228860b8a2cddd360636bb8633967b07e.
However, this testing wasn't fixed up. Use ensure-pip in the test
playbook before we run the test, and add a note to ensure-twine that
it runs without privileges.
Change-Id: I6aef14b188dce8c4de774b09ddad718fec8fbf6c
Simplify the update procedure by tagging the checkout with the current
unix timestamp, so we should always having an increasing version
number. This means we can do the test upload unconditionally in the
check gate. Given this runs very infrequently, this shouldn't be a
burden.
Change-Id: I12c222802297d760296d35651db8d22fedb6dab4
This does a full upload of the sandbox project to the test.pypi.org
service.
It uses the inline token with the caveats noted in the comment. Since
you can't upload the same version twice, always running this in the
check job will mean it will work the first time, but then fail (unless
something has been committed to the sandbox to update the version
number). As noted, the idea is that we develop the change and then do
one run that does a full test, and reviewers can check that before
committing. There's no real way to use a secret here because we
actually want this to run at check time, not in any post pipeline
where it would already be in production. We don't want to do
something like require committing something to sandbox every time you
run this in the check queue, etc. Given how much this is actually
updated (most of it hasn't been touched since 2019) I think this is a
reasonable compromise.
Change-Id: Ida4ad07c82a6301107c938565656988aba3bf250
Very basic testing to include the ensure-twine and upload-pypi role.
This really just does a syntax and sanity check.
Change-Id: Ie40ec32b3b2ffc4646301c6e145c527fe6a3bae2
First up we need python2.7 specific msrest version as azure-storage-blob
depends on it and it doesn't properly specify python versions for its
releases.
Next we cannot use zuul/zuul to test ensure-pip can install packages
because zuul/zuul requires python3.8 or newer now. Some platforms we run
ensure-pip on still only have python3.6. We address this by using
zuul-client instead.
Change-Id: I3d91fd3f85e3d7845ab2d7615a1ebfb614064573
The input to encrypt files may be a list of paths so our validation has
to evaluate and state each list entry separately. Without this we fail
beacuse the list of paths is treated like a single path and that does
not stat resulting in early failure.
Change-Id: Ibe3f6b162c3adad928708464ea03ddded2f4c683
Our wheel mirrors contain pluggy-1.0.0-py2.py3-none-any.whl which pip
sees as a sign to install it under py2.7 although the wheel is only
meant for >=py3.6. Add a cap to the test command to install some older
version instead.
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I32d2c29988f92124e7dcdf462b2dd07cfd68b4d2
I74b9de7092f182c942a58ac7a46b9fbd791889de hit a common ansible gotcha
where it likes to strip the trailing newline after a {% endif %}.
This has resulted in invalid lines in our sources.list.
Unfortunately we miss this because it still exits with 0. Add a
simple test looking for warning output.
Change-Id: I46d393a5e67d10a52c4dcca803176ff368a4b5bd
If a key is in our existing keyring has an expiry date (or, has
expired), always import the provided value again as it may be
refreshing the value.
Add an expiring key to test the matching; although on an ephemeral
node we're importing always anyway.
Also update the file test to a stat -- this is better than a weird
error from gpg later.
Change-Id: I8e7bc38c68c224795630b90a1b989098a7661491
It looks like 9-stream don't want to support virtualenv. Fail the
ensure-virtualenv role on this platform, and account for this in
testing.
People should use ensure-pip and venv which is portable everywhere.
Change-Id: Ifae93c1eeb96792aa26a624574d595d77cb58c4b
To match change I2870450ffd02f55509fcc1297d050b09deafbfb9 in Zuul.
This does not use the versioning feature due to the nature of this repo.
This also corrects a reference which is now an error.
Change-Id: Ia1d31df932b447f11bc588925de9974d4f6dfc7d
Since the security update we are not able to modify
zuul.executor.log_root variable in the role testing.
Replace this with explicit log folder creation on the test node.
Change-Id: Ie6cff6f9e9c5594167ddda6cd345f9c3d9f2f470
This updated branch fixes a few issues and should ensure display with
pcp-dstat (I9234b226242f145db9c235797649202aa530ec74)
Add an override so we pull it
Change-Id: I0d5e1b567c364a9e6c7aa0b95de17abffaef0434
Update the alternate tox.ini files in test-playbooks/python to
trigger configuration we saw expose a bug in the siblings lib's
showconfig parser when a tox.requires value resulted in pip
activity. Update its minversion to the earliest tox which provides
the tox.requires configuration option, for clarity.
While we're there, switch to using tox_config_file instead of
overriding zuul_work_dir since that's now an option for this role
and was previously not exercised in the test job.
Change-Id: I521580da978903cd116c872447b1c028c470e63d
In change Iafeb88eaf9a596603ad4d2134a4574345d5189ab we fixed a bug
with handling of `tox --showconfig -vv` in the siblings tasks for
the tox role. This arose because of the desire to add tox_extra_args
to all tox invocations in Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
because we currently set a default of '-vv' for the tox_extra_args
rolevar. This fix could easily regress in the future if we were to
ever change the default value, so explicitly test with it in the
zuul-jobs-test-tox job by adding it to the python/tox test playbook.
Change-Id: Ib6ddf7d188904193a5ecd520acac1d676e5d78ae
Work around lack of SNI support in old distutils versions shipped
with Python on platforms like CentOS 7 and Ubuntu 16.04 LTS by
installing PBR first so that distutils won't be compelled to do so.
Warehouse (PyPI) ceased supporting clients without SNI support in
March of this year.
Change-Id: Ic741d9a87c2ca3a5249cd03c3cbd38e2ad1f46a1
Because buildset registries may be used by jobs that finish before other
jobs are finished using the buildset registry we must be careful not to
expose the registry credentials in the jobs that finish sooner.
Otherwise logs for the earlier job runs could potentially be used to
poison the registry for later jobs.
This is likely currently incomplete. Other Zuulians should look over it
carefully to ensure we're covering all the bases here.
The cases I've identified so far are:
* Setting facts that include passwords
* Reading and writing to files that include passwords (as content may be
logged)
* Calling modules with passwords passed as arguments (the module
invocation is logged)
I've also set no_log on zuul_return that passes up credentials because
while the logging for zuul_return is minimal today, I don't want to
count on it remaining that way.
We also use the yet to be merged secret_data attribute on zuul_return to
ensure that zuul_return itself does not expose anything unwanted.
Finally it would be great if others could check over the use of
buildset_registry variables to make sure there aren't any that got
missed. One thing I'm not sure of is whether or not when conditionals
get logged and if we need to be careful about their use too.
Temporarily remove some buildset-regitry jobs which are in a catch-22.
Change-Id: I2dea683e27f00b99a7766bf830981bf91b925265
This change enables using fetch-translation role along with
the fetch-output role. By default the role still synchronizes
artifacts back to the executor.
Change-Id: I85c021706c1fa20f8d28b3a1f56c9435ac3836d5
Properties can be used to tag files.
When another system is using a property filter, being able to set
the properties for zuul artifacts is very convenient.
Change-Id: Ib16ca0f6b532649daa77aa26a8ffa29b78429b71
The stage-output role had two assumptions:
1) The zuul_copy_output variable would always be defined
2) The role would be able to sudo on the remote node
To make it easier for users who want to use this in a base job,
remove both of those assumptions.
1) We now supply a default empty dict if the variable is not defined
so that the role does not fail with an error.
2) We check to see if we can sudo on the remote node, and if we can,
we do when copying files; otherwise we don't, and assume that the
user will only specify files they have access to (if they don't then
the copy will fail).
Change-Id: I5428c44adfafac4872342b59a92de311f41687b6
Role copied and modified from ensure-podman
As focal doesn't exist for project atomic ppa [1]
Install is performed from opensuse repository only
[1] http://ppa.launchpad.net/projectatomic/ppa/ubuntu/dists/
Change-Id: I72fc2e68768664b80c39bd47295330131337d8b5
Fixes several rules with test-playbooks and remove path exclusions.
Since we introduced the progressive mode, exclusions are no longer
needed and not haivng them will assure newly added code follows
the same guidelines.
Change-Id: I0a72d34aff8cf23172e26c7f44e0f61571ec74b6
On Centos8, during the docker-ce installation, the docker.socket service
is start with a bogus state:
docker.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is not functional until restarted.
Later, when the `Assure docker service is running` task tries to start
the service, it fails with the following error:
dockerd[29743]: failed to load listeners: no sockets found via socket activation: make sure the service was started by systemd
Example:
https://0c7366f2ce9149f2de0c-399b55a396b5093070500a70ecbf09b9.ssl.cf1.rackcdn.com/410/c233496b96c70cfc6204e75d10116a96b08d4663/check/ansible-test-sanity-docker/787388f/ara-report/index.html
Another example: https://github.com/kata-containers/tests/issues/3103
Also: Remove use of kubectl --generator=run-pod/v1
This has been deprecated since 1.17 and removed since 1.20. run-pod wound
up being the only generator that did anything, so this parameter became a
no-op. This has to be squashed into this commit to unbreak the gate.
Change-Id: I666046fe2a3aa079643092c71573803851a67be2
Reading the installation guide for podman, they reference opensuse.org
as the official package repos for ubuntu:
https://podman.io/getting-started/installation
Using this repo allows us to pull in much newer version of podman on
ubuntu. The current PPA package repo hasn't been updated since late
2019.
Change-Id: Ie34419184925a4bcf30422a782e6a238c11f2319
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
- bumps ansible-lint to 5.0
- updates our custom rules to make them compatible with 5.0
- replace custom module mocking with native ansible-lint ones
- remove custom call of ansible-playbook --syntax-check as now this
is done by ansible-lint
- assured molecule vars are hosted under a vars/ folder in order to
avoid confusing linter detection.
- replaced custom rule for loop var names in role as now this this an
optional core feature of the linter (see config)
- replaced custom rule no-same-owner with opt-in one (see config)
Change-Id: I233fae8c9036d295968a97ee80e07fde8846c633
This change adds a new attribute to setup zookeeper TLS.
It also adds support for Debian-derived distros.
Change-Id: Ifb5fc51f3b66be0b2dd1b8003507e21d8afe16fc
It is perfectly valid to want to add a value to a file that
does not exist yet, even the path may be missing.
This fixes last night regression when installing docker no longer
creates the /etc/docker folder, causing our MTU update to fail.
Change-Id: I0f037d1d6664de3c3b777aaf6da9cd7c3e8bb15f
Reference: https://review.rdoproject.org/zuul/builds?job_name=tox-py36-ci-config&project=rdo-infra/ci-config
This new role will be used to replace our upload-docker-image role in
the future.
Change-Id: I0e2b0cca6575255520aa6d4d48a12128ab5f46cc
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The merge-output-to-logs role is not doing anything in this test. As
described inline, this merges files in
zuul.executor.work_root/<docs,artifacts> into the
zuul.executor.log_root directory so they are available in change
results.
Since this job doesn't publish anything there, this role is unused.
merge-output-to-logs currently can't run because it tries to run shell
scripts on the executor. Thus we can remove this unused role and
restore the job.
Change-Id: I1afc905aa8d9c420bed316e99760ad7ad1d838ce