1674 Commits

Author SHA1 Message Date
Sorin Sbarnea
9baebe3684 yamlint: EOF newlines and comments indent
Fixed two rules which where temporary disabled during introduction
of the linter.

Change-Id: Icd1e1b40b1e8207ab5ff7088a48e8f0a800e3aa8
2020-05-14 08:42:17 +00:00
Zuul
6c978d0746 Merge "Add zuul_work_dir to run-test-command" 2020-05-13 19:04:20 +00:00
Zuul
a365cf53a8 Merge "fetch-sphinx-tarball: add missing zuul_success default" 2020-05-13 19:04:18 +00:00
Zuul
d17cc7b522 Merge "fetch-sphinx-output: introduce zuul_use_fetch_output" 2020-05-13 18:59:10 +00:00
Zuul
92dbfb6211 Merge "ansible-lint-rules: Fix bad path and filename" 2020-05-13 18:33:00 +00:00
James E. Blair
a3ba8e309a Add zuul_work_dir to run-test-command
To make it so that a repo can host a simple command which is an
integration test of multiple projects, add the semi-standard
zuul_work_dir variable to the job.

Change-Id: I265469ac9c12c3a38a11b243f3ff747d27141f21
2020-05-13 11:31:01 -07:00
Zuul
adf84f02db Merge "Set node_version in js-build base job" 2020-05-13 16:40:13 +00:00
Monty Taylor
a8722a38ed Set node_version in js-build base job
Defaults in one role do not automatically transfer to
roles via include_role, and applying things correctly
can lead to recursion issues:

  https://github.com/ansible/ansible/issues/36274

Just set the default in the job.

Change-Id: Iabefd80c8fc40a61a601b082f265c93af643cd83
2020-05-13 11:10:54 -05:00
Andreas Jaeger
d2bcdf157e Fix nodejs-npm-run-test
pre-test.yaml was renamed in I33c12cb0d9516bdffef7d8d04af4dbcb03ed8355
to browser.yaml and we missed one place. Follow rename.

Change-Id: I628dbe7481ea0277e564af057ae57fdf2ce51820
2020-05-13 11:02:48 +02:00
Zuul
b8a24d635f Merge "Split browser and non-browser targetting js test jobs" 2020-05-12 17:10:19 +00:00
Tristan Cacqueray
6ac719ba2f fetch-sphinx-tarball: add missing zuul_success default
This change prevents an issue in test-jobs where zuul_success is undefined.

Change-Id: I408e9f957ff3151d39f7b5b2ea038255570baf4d
2020-05-12 16:54:32 +00:00
Tristan Cacqueray
59bacd1518 fetch-sphinx-output: introduce zuul_use_fetch_output
This change enables using fetch-sphinx-output role along with
the fetch-output role. By default the role still synchronizes
artifacts back to the executor.

Change-Id: I81c8a2faf566be694dbf1c59cbbd157dc4df5146
2020-05-12 16:38:26 +00:00
Monty Taylor
0caa85cc48 Split browser and non-browser targetting js test jobs
Some javascript operations, like running tests for web-apps, need
browsers. Javascript packages that are not targetting browsers
do not.

Make a -browser version that installs the browsers and xvfb, and
a non-browser version that does not.

Change-Id: I33c12cb0d9516bdffef7d8d04af4dbcb03ed8355
2020-05-12 11:31:42 -05:00
Zuul
9d18ae4326 Merge "Extract ensure-javascript-build-tool role" 2020-05-12 16:28:17 +00:00
Zuul
bc7002ddc6 Merge "Fail and direct user to use ensure-* version of roles" 2020-05-12 13:42:47 +00:00
Zuul
7730bf0718 Merge "Add new non-npm specific javascript jobs" 2020-05-12 13:37:06 +00:00
Monty Taylor
5973c9b7b1 Extract ensure-javascript-build-tool role
The pre-playbook for the javascript jobs got complex. Extract it
to a role so we can better document it.

Also - stop installing javascript depends in pre - the depends
declared in a patch should be tested as part of the patch.

Change-Id: I50a483f223620cd3f9ecd82887062cfc9ac64b7f
2020-05-12 07:53:14 +02:00
Monty Taylor
202cce830e Add new non-npm specific javascript jobs
We have a bunch of jobs that are built around the npm role, but
for projects using yarn, that can lead to ignoring yarn.lock.

For projects with a yarn.lock, we can assume the user wants to
use yarn. Make a new js-package-manager role that can detect
if that's the case and otherwise use npm. Make an js_build_tool
parameter that allows the user to override that auto-detection.

Make a whole new suite of jobs that do this behavior, do not have
npm in their name, and default to the latest node LTS, version 14.

Don't install yarn if we're not going to use yarn. Also allow people
who want to use yarn but don't have a yarn.lock to override
js_build_tool everywhere we do that logic.

Mark the old jobs deprecated.

Shift the npm and yarn roles to use the new js-package-manager role
with defaults set.

Change-Id: I8013228ca05607a69f390a9bb75991fc6543f865
2020-05-12 07:50:19 +02:00
Zuul
1ab22a3b44 Merge "haskell-stack-test: add build target job variable" 2020-05-11 16:20:03 +00:00
Zuul
8c49060efe Merge "cabal-test: add build target job variable" 2020-05-11 16:20:02 +00:00
Zuul
d92b1952b5 Merge "ensure-pip: use full python3 path" 2020-05-11 15:29:50 +00:00
Zuul
2d28b87a28 Merge "Revert "Revert "ensure-tox: use venv to install""" 2020-05-11 15:29:49 +00:00
Zuul
d6eba83f36 Merge "Add explanatory comment to tox siblings" 2020-05-11 14:46:39 +00:00
Albin Vass
d73f16690e Add explanatory comment to tox siblings
Change-Id: I12562f6af25bf56c5489bc436f91017fa25bb474
2020-05-11 15:32:09 +02:00
Ian Wienand
b6b6e36368 ensure-pip: use full python3 path
As seen by Iaa3ecd05b64af6dd9b2ee17a39bcbe6cde8686ba "python3 -m venv
--help" doesn't actually tell you if the command will work, so it's
really not helpful.  Modify this to run the venv from the python in
the path.  This also expands the comments around why a full path is
necessary.

Change-Id: I03bded0345ff734fd6c5d02fdd44ed1cc14360d2
2020-05-11 13:40:18 +10:00
Ian Wienand
9c04709719 Revert "Revert "ensure-tox: use venv to install""
This reverts commit bac2bf6c45d6dc17590aa9508be998daa940f0b1.

The problem was really that ensure-pip was giving us a
"ensure_pip_virtualenv_command" that didn't work on some systems.
Iaa3ecd05b64af6dd9b2ee17a39bcbe6cde8686ba fixes this underlying issue,
so we can revert to the original change that uses that to install the
tox environment.

Change-Id: I8ce9dceb721474d3220f6e72409481dc89875ee0
2020-05-11 11:10:09 +10:00
Ian Wienand
bbb149ebf9 ensure-pip: always check for python3-venv on Debuntu
As explained inline, we can have systems where pip is installed, but
python3-venv isn't.

This role is assuming that "python3 -m venv --help" indicates that you
will be able to create a working venv with this command, but this is
unfortunately incorrect.  On Debuntu this requires the python3-venv
package as well.  Put in an unconditional probe for this in the
workarounds, and install it if required.

Change-Id: Iaa3ecd05b64af6dd9b2ee17a39bcbe6cde8686ba
2020-05-11 11:10:04 +10:00
Monty Taylor
6ab78932a4 Set ignore_basepython_conflict to true
Otherwise we miss that we're not using 3.8 in the 38 env.

Fixes the cgi.escape issue that we didn't catch earlier.

Change-Id: I604bd18b70950580f60714598624d493f4a5060f
2020-05-09 20:43:52 +00:00
Mohammed Naser
3fc4f44958 pyenv: install libffi-dev
Without this package, you'll get an error similar to this:

  ModuleNotFoundError: No module named '_ctypes'

Change-Id: If1b7b66217a33c9001a20449d695cd91dfa2e3c6
2020-05-09 14:03:51 -05:00
Mohammed Naser
1e72ab0a13 tox: run ensure-python first
We currently run ensure-python after running ensure-tox which means that
both ensure-pip and ensure-tox ran against the system Python instead of
the version that's created by ensure-python.

Depends-On: https://review.opendev.org/726549
Change-Id: Id2100bb5f981944223063cf4b2a5a568cc1dc366
2020-05-09 17:56:37 +00:00
Mohammed Naser
b7554e7104 bindep: add gcc
In order to build subprocess32 which comes in as a dependency, you need
gcc or otherwise you'll end up with:

  configure: error: in `/tmp/pip-install-bgksUt/subprocess32':
  configure: error: no acceptable C compiler found in $PATH

Change-Id: I9b213899f9636fdbc7c054c15ccae4d88b150d40
2020-05-09 13:49:03 -04:00
Monty Taylor
5f31625a94 Add --all to skopeo copy from insecure registry
skopeo needs to be told to copy all instances of a given image,
otherwise it just grabs one of them.

https://github.com/containers/skopeo/pull/741

Change-Id: If78ad50602e745ef7747d983b987cf964ff6e67f
2020-05-08 15:25:12 -05:00
Albin Vass
c96f4f5a16 Fail and direct user to use ensure-* version of roles
Tuesday the 12th May is two weeks after announcing deprecation of the
install-* roles [1] and the date we should starting throwing an error
message to direct the user to use the renamed ensure-* role instead.

[1]
http://lists.zuul-ci.org/pipermail/zuul-announce/2020-April/000071.html

Change-Id: Ie2474c5c0704aa44f51bdd5d7afddb8c7e4e578b
2020-05-08 19:03:06 +00:00
Albin Vass
c9db6cca96 ansible-lint-rules: Fix bad path and filename
Change-Id: I02b937b8d68600062a25731ae5d4b43f33c8c4a9
2020-05-08 19:38:58 +02:00
James E. Blair
bac2bf6c45 Revert "ensure-tox: use venv to install"
This reverts commit 6d78fc4f900415b4e33db1653a8be64ee62a6ee6.

In some environments, this error is being seen:

2020-05-07 18:36:02.139422 | debian-buster |   "msg": "stdout: The virtual environment was not created successfully because ensurepip is not\navailable.  On Debian/Ubuntu systems, you need to install the python3-venv\npackage using the following command.\n\n    apt-get install python3-venv\n\nYou may need to use sudo with that command.  After installing the python3-venv\npackage, recreate your virtual environment.\n\nFailing command: ['/home/zuul/.local/tox/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']\n\n"

Change-Id: I6d38bf16cc38020c53815dfa6ab94f8cab4de0a2
2020-05-08 07:36:40 -07:00
Tristan Cacqueray
16673b8dc1 haskell-stack-test: add build target job variable
This change adds support for custom target and adds a
separate build step to better report build failure, as
opposed to a single test failure.

Change-Id: I9e7a5e14b669dcbacd26861595358f6a4c04b585
2020-05-08 14:11:11 +00:00
Tristan Cacqueray
7fdf0cc3ab cabal-test: add build target job variable
This change adds support for custom target and removes the
configure step which is not necessary.
This change also uses the copr of cabal-install to get
a more up-to-date version.

Change-Id: I538e2b24abb66181d1ead2779bf6aa3440ee58ff
2020-05-08 13:38:17 +00:00
Monty Taylor
64741aa73d Don't upload to the tag with buildx - only to the change tag
This is a copy/pasta error from pulling this logic over from build.
We need to upload to dockerhub but only to the change_ version
of the tag.

Change-Id: I19d1b326ad2c5b1cef846f47e667f3c3480582e7
2020-05-07 18:17:59 -05:00
Zuul
fa22b3d9a0 Merge "Process siblings in upload-image push" 2020-05-07 22:21:19 +00:00
Zuul
7c0b94bcf7 Merge "Actually include platform in the upload build" 2020-05-07 22:21:17 +00:00
Monty Taylor
bd1ee92542 Process siblings in upload-image push
Otherwise the context might be different and we might have to
rebuild, which would be silly. I mean, not like this isn't
silly already.

Change-Id: I3651390581ad1b4ae66ed336825f60284bd7c4ad
2020-05-07 17:01:35 -05:00
Monty Taylor
53118e79ea Actually include platform in the upload build
It's awesome, we used buildx to push to docker hub. What we forgot
to do is pass the arch argument.

Change-Id: I99025544f7a8aea651e9562041af859a38cddbfc
2020-05-07 16:57:50 -05:00
Zuul
289faf8612 Merge "Add docker_dockerfile to upload-docker-image defaults" 2020-05-07 17:25:35 +00:00
Monty Taylor
48afa2248f Add docker_dockerfile to upload-docker-image defaults
We document it in the docs as a common thing, but we never needed
it. After the previous patch to use buildx routines for upload,
we need to define the default.

Change-Id: Idf0ed40d125c8cfc2eb4a381035b154c9bc2c1c5
2020-05-07 12:07:36 -05:00
Zuul
7e94acf697 Merge "Remove opensuse-15-plain testing" 2020-05-07 16:06:23 +00:00
Zuul
8478544a62 Merge "Check blocks recursively for loops" 2020-05-07 16:02:14 +00:00
Zuul
98052c45ee Merge "Upload images to dockerhub with buildx when using buildx" 2020-05-07 15:01:03 +00:00
Zuul
480ec972a6 Merge "Made sequence indent consistent" 2020-05-07 14:42:09 +00:00
Monty Taylor
dfb3884440 Upload images to dockerhub with buildx when using buildx
If we build multi-arch images with buildx we also need to push to
dockerhub using buildx, because otherwise we're just pushing
the single-arch image we fetched back from the buildset registry
for the local cache.

Change-Id: If8b95a708e4f0d24e959317b803f5c9379a8b62b
2020-05-07 09:23:56 -05:00
Sorin Sbarnea
4af438d136 Made sequence indent consistent
Change-Id: I5ea4232ca4fd6e03d5b5f72eb6704bee84d04ea5
2020-05-07 14:13:55 +01:00