1017 Commits

Author SHA1 Message Date
Zuul
a7c7d4fa16 Merge "Add a markdownlint job and role" 2020-01-31 17:05:16 +00:00
Andrii Ostapenko
fc27907b98 Fix periodic image build jobs
Don't try to create change tag if zuul.change is not defined that is the
case for periodic builds.

Change-Id: I5a7e02caf0f502e90e0477daeee868bf06dfbb5c
2020-01-28 14:19:30 -06:00
Clint Byrum
33580f22b0 Add a markdownlint job and role
This tool will ensure that your markdown files follow some best
practices.

Change-Id: I7bcc70443dbe5fa31e3cc1139d608834c00851b9
2020-01-27 12:23:24 +00:00
Zuul
be8308c7ac Merge "Update roles/prepare-workspace-openshift/README.rst" 2020-01-24 17:08:21 +00:00
Zuul
37b53a4d70 Merge "Return subunit test report as artifact" 2020-01-24 10:04:02 +00:00
Zuul
4facd5ddc7 Merge "fetch-tox-output: introduce zuul_use_fetch_output" 2020-01-24 08:36:57 +00:00
Zuul
5ea9bac2df Merge "Add prepare-workspace-openshift role" 2020-01-23 16:19:37 +00:00
Andreas Jaeger
9b655c8d5d Update roles/prepare-workspace-openshift/README.rst
Use a proper link to the referenced role and mention in the short
description that this is related to OpenShift.

While it might work with any Kubernetes, it uses an
openshift tool, oc.

Change-Id: Ia4fba00c8a3adec4dc26a2d32a21df8ff548a120
2020-01-23 17:09:03 +01:00
Andreas Jaeger
ba12fb98a0 Fix ansible-2.9 skipped problem
ansible-2.9 does not allow "|skipped" anymore, it needs "is skipped"
instead, fix the only usage.

See also Ic0c93f2ea896ead76922694e6150882d8a280daf and
Id6c62da8f8a94c270d8a5d5981683a9d8c4e7ad6.

Change-Id: I597c92d2f6dce31dab1ac4bf52bb534c8146b33b
2020-01-23 07:47:49 +01:00
Zuul
bf083b71fd Merge "Revert "Make ara-report role to zuul_return an artifact"" 2020-01-23 06:44:59 +00:00
Paul Belanger
0f0d9bd2e2 Revert "Make ara-report role to zuul_return an artifact"
This seems to introduce the following error with Ansible 2.9

 "The conditional check 'not ara_generated | skipped' failed. 
 The error was: template error while templating string: no filter named 'skipped'. 

This reverts commit 262c16b30583f634588f5a52bd7aab944687a8a4.

Change-Id: Id6c62da8f8a94c270d8a5d5981683a9d8c4e7ad6
2020-01-23 00:39:47 +00:00
Zuul
0ed3f04dd1 Merge "Make ara-report role to zuul_return an artifact" 2020-01-22 10:33:15 +00:00
Zuul
b37367637e Merge "ensure-tox: Output tox version" 2020-01-21 21:32:25 +00:00
Ian Wienand
9e5907f4cc ensure-tox: fix pipe race
The

 command -v pip pip3 | head -n1

introduced with Ie50928c9b782ea84db916bb1441567e1206ff466 has a very
subtle race; if "pip" and "pip3" exists and there are two lines
output, the "head -n1" will exit and depending on scheduling the
"command" might write to a broken pipe (this manifests as exit code
141).

Move this to a more explicit if statement.

Co-Authored-By: Jens Harbott <j.harbott@x-ion.de>
Change-Id: I80823a7bc6351925d6f0b20bdebca3eafef0b27d
2020-01-22 07:49:35 +11:00
David Shrewsbury
4538583b1a ensure-tox: Output tox version
This can be useful for debugging.

Change-Id: Ic5ffb46cf2f583cd6dd04289e55f40f8acd3d9bf
2020-01-21 15:00:05 -05:00
Andreas Jaeger
5eb4c257f5 fetch-sphinx: Exclude doctrees directory
The sphinx .doctrees directory is not needed for publishing. By default
sphinx puts it below html. Some projects add to sphinx-build a "-d"
option to put the content elsewhere and not publish it.

This change assures that a .doctrees directory below "html" will not put
stored as artifact and published later.

Statistics: In all of OpenDev, we have:
* 511 repos putting .doctrees elsewhere and thus not publishing it
* 631 repos  no caring and thus publishing .doctrees

The size of .doctrees directory is not neglectable: 2MB for
system-config, 14 MB for nova.

Change-Id: I63b581164aab84352d9e80278cf4a8d634ff28af
2020-01-21 08:32:26 +01:00
Tobias Henkel
3b0fa71dd2
Return subunit test report as artifact
When returning the subunit test result as an artifact it will be
directly linked on the build result page.

Change-Id: I1a83288ac0857333f87cd085b3e0243bd3b92fc2
2020-01-18 13:31:51 +01:00
Sorin Sbarnea
2410119ac9 ensure-tox: improve pip detection
Fixed issues failing to install tox on python3 only systems which have
only pip3 executable and not the pip one.

Needed-By: https://review.rdoproject.org/r/#/c/24584/
Change-Id: Ie50928c9b782ea84db916bb1441567e1206ff466
2020-01-16 21:06:07 +00:00
Zuul
fc9d36ffe4 Merge "Stop compressing files during intermediate steps part 2" 2020-01-15 16:48:02 +00:00
Tristan Cacqueray
56ee80234c fetch-tox-output: introduce zuul_use_fetch_output
This change enables using fetch-tox role along with
the fetch-output role. By default the role still synchronizes
artifacts back to the executor.

Change-Id: Iffe5ffc52ee4c765b2e23ab8b3235b6bc7c510d8
2020-01-13 21:13:55 +00:00
Zuul
33fc86060b Merge "Fix typo in helm role" 2020-01-11 08:45:30 +00:00
Zuul
5fb6ba9dfe Merge "helm-template: Add role to run 'helm template'" 2020-01-11 08:45:29 +00:00
Zuul
b864698116 Merge "collect-container-logs: add role" 2020-01-10 21:56:26 +00:00
James E. Blair
3748623564 Fix typo in helm role
s/go/helm.

Change-Id: I840fff4e556df4b99994830755daddbe54222bb2
2020-01-10 13:26:51 -08:00
Zuul
e5f39018e4 Merge "install-go: bump version to 1.13.5" 2020-01-10 17:42:50 +00:00
Mohammed Naser
8ae49163f3 helm-template: Add role to run 'helm template'
This role runs Helm template of a specific chart and then makes
sure that all resources become ready afterwards.

Change-Id: I62e160eac673d13bd1018b58fd36cae54e1482b4
2020-01-10 17:31:05 +00:00
Mohammed Naser
228033e78a collect-container-logs: add role
We use this in a few different places and it's really useful
to collect all the logs of all containers.

Change-Id: Idc46a47f444bf48cd040f4f9724f3a6ee8bc8f8e
2020-01-09 19:52:01 -05:00
Zuul
96300f3543 Merge "Move chart-testing to a role and disable version check" 2020-01-09 22:31:14 +00:00
Zuul
14478de03f Merge "Add basic Helm jobs" 2020-01-09 21:19:44 +00:00
James E. Blair
ffb6406a40 Move chart-testing to a role and disable version check
Move the chart-testing task to a role (so that it is easily
reconsumable) and update the default arguments to disable checking
for version bumps (instead, we expect to rely on setting the version
when building a release).

Change-Id: I85482dc704c189222c92fd4174f733382ebf69d1
2020-01-09 11:46:41 -08:00
Zuul
1e378f4071 Merge "Add cri-o support to use-buildset registry" 2020-01-09 16:49:54 +00:00
Clark Boylan
a6f2ca827e Stop compressing files during intermediate steps part 2
Stop compressing files that would otherwise be handled uncompressed in
intermediate log handling. The reason for this is final log/artifact
upload should decide if a file need to be compressed or not as some file
hosting services (like swift) may not return the expected data if we get
this wrong.

This was noticed when we used swift to store intermediate tar.gz
tarballs before final upload to permanent storage. When retrieving the
tar.gz files from swift we got them back as uncompressed .tar files
(because swift was being helpful) and then later when we try to
uncompress these files we break.

Instead we'll let the upload to swift (or other storage) decide if files
should be compressed further than their existing state.

This is the second change in the stack and update the stage-output role.

Change-Id: I2fa6f123d23568ba1537a7c26d2672f04110c35b
2020-01-08 09:24:45 -08:00
Clark Boylan
bbca430417 Stop compressing files during intermediate steps part 1
Stop compressing files that would otherwise be handled uncompressed in
intermediate log handling. The reason for this is final log/artifact
upload should decide if a file need to be compressed or not as some file
hosting services (like swift) may not return the expected data if we get
this wrong.

This was noticed when we used swift to store intermediate tar.gz
tarballs before final upload to permanent storage. When retrieving the
tar.gz files from swift we got them back as uncompressed .tar files
(because swift was being helpful) and then later when we try to
uncompress these files we break.

Instead we'll let the upload to swift (or other storage) decide if files
should be compressed further than their existing state.

This is the first change in the stack that updates this behavior for
fetch-subunit-output.

Change-Id: I5727b270d6d34c256fe78a443baa1ad6c3474108
2020-01-08 09:24:43 -08:00
Mohammed Naser
b476e34ab0 Add basic Helm jobs
This adds a simple job which runs the `chart-testing` linter against
a repository.

Change-Id: I974c64343e88c235a6dc887d5378f7db2ee3b1fc
2020-01-07 17:12:15 +00:00
James E. Blair
b6cb191295 Add cri-o support to use-buildset registry
This lets use-buildset-registry notify cri-o about the new
registries.conf file if it is being used as the container backend
for k8s.

Change-Id: Ia1805519ab4b6bb5f79df0492f702effc6a3e024
2020-01-07 07:35:28 -08:00
Clark Boylan
0e23325a12 Swift upload logs without encoding type for gzip files
When retrieving gzipped files out of swift with gzip encoding set some
swift implementations return a decompressed version of the file if your
client cannot accept gzip encoding. This causes problems when the file
you want is actually compressed like a .tar.gz. Instead we avoid setting
the encoding type of these files forcing swift to give them back as is.

This change should only be approved after confirming its parent is
tested and working.

Change-Id: Ibcf478b572ba84273732e0ede17bf92bddd8c36f
2020-01-06 13:50:01 -08:00
Clark Boylan
9352276cc2 Test swift upload logs without encoding type for gzip files
When retrieving gzipped files out of swift with gzip encoding set some
swift implementations return a decompressed version of the file if your
client cannot accept gzip encoding. This causes problems when the file
you want is actually compressed like a .tar.gz. Instead we avoid setting
the encoding type of these files forcing swift to give them back as is.

This change will be followed by a change that updates the production
role.

Change-Id: Ica27cf070d50fcafd879b6f8ccc3964afa9bf5e6
2020-01-06 13:50:01 -08:00
Zuul
e21bed0c48 Merge "Add pypi_fqdn to differentiate it package mirrors" 2020-01-06 16:36:58 +00:00
Mohammed Naser
e38f43abd9 install-go: bump version to 1.13.5
1.31.1 contained some critical fixes regarding go module
handling, let's just jump to the latest minor release.

Change-Id: I829d3a1550d343e778e79b519925099651392eb1
2019-12-23 23:20:12 -05:00
Clark Boylan
b62c488eab Fix ansible use of filters and tests
Ansible 2.9 requires the use of "version" as a filter and
"version_compare" has been removed. Also tests cannot be used as
filters which means we cannot do when: foo | test and must do
when: foo is test instead. Make these fixes.

Both changes should be backward compatible to ansible 2.5:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#version-comparison
https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.5.html#jinja-tests-used-as-filters

Change-Id: Id95f674a485877db2a7924994366d1c6c591a684
2019-12-17 10:49:57 -08:00
Clark Boylan
a554a401ea Use present for package state instead of installed
Ansible 2.9 does not accept the "installed" state so we need to update
occurences of that to "present".

Change-Id: I13c95794efc63697c2418334ded7c6912bd7b59f
2019-12-17 08:24:05 -08:00
Zuul
5b0644185f Merge "Adds go jobs and roles." 2019-12-13 17:50:51 +00:00
Zuul
8a8adfb58f Merge "openshift speculative containers" 2019-12-11 19:17:27 +00:00
Clark Boylan
f418a8d533 Fix python3 compat in tox siblings handling
This fixes a byte vs str problem on python3 in the tox role's sibling
library.

As part of this fix add unit testing of the function that was failing
under python3 and functional testing of the role that covers siblings.

Change-Id: I2df4ddce91dfd53f5c4177a19288e97120bc3e3b
2019-12-10 15:27:41 -08:00
Zuul
641baedc4c Merge "build-docker-image: fix up siblings copy" 2019-12-09 23:49:16 +00:00
Zuul
939a09387d Merge "build-container-image: support sibling copy" 2019-12-09 23:42:30 +00:00
Zuul
7f6710385f Merge "Fixes tox log fetching when envlist is set to 'ALL'" 2019-12-09 00:18:55 +00:00
Ian Wienand
599954715e build-docker-image: fix up siblings copy
There are a number of issues with this.  Firstly, it needs to copy the
parent directories to make a heirarchy in the .zuul-siblings
directory.  The current "cp -r" was only copying the final directory.
Switch into the source directory and use "--parent" to do this.

Also, it should be copying into the context dir.  Add the
{{ item.context }} to the path where appropriate.

Make new testing image that copies in files from the siblings.
Because COPY will fail if the sources aren't there, this is like an
assert that we copied it correctly.

Change-Id: I9f3b0a1f71d20cf7511f224648dd2fa51a039015
2019-12-09 11:14:21 +11:00
Ian Wienand
29d55cbf17 build-container-image: support sibling copy
Add support for "siblings:" to the build-container-image role.  This
is a port of change I079d823e7194e15b1b496aea0f53f70f6b563f02 from the
build-docker-image role.

Note that we found a couple of issues with this role, fixed with
I9f3b0a1f71d20cf7511f224648dd2fa51a039015.  However,
build-container-image has merged before this fix, so thess fixes now
follow this change.  To avoid squashing this into something confusing
this update incorporates those fixes.  However testing is left to the
follow-on change because the extant build-docker-image role would fail
the corrected testing.  They should be merged roughly together.

One minor thing is the removal of an additional duplicated "tags:"
entry from the variable documentation.

Change-Id: I71e2f2ab29446bd113644ce8c75378b8d3ef8ba6
2019-12-09 11:13:42 +11:00
James E. Blair
6a9aa82fcc openshift speculative containers
Change-Id: I81f7be6065a74f45386ea735a5291d48b7e042ce
2019-12-06 13:50:25 -08:00