412 Commits

Author SHA1 Message Date
Monty Taylor
b034c85ab1
Update fetch sphinx output to use sphinx vars
The sphinx roles takes source and build parameters. Update
fetch-sphinx-output to also honor them so that a job can set them and
have both build and fetch honor them.

Change-Id: Ic9fc6648a25336b968e8727bb8f4fd4cfd1548bb
2017-11-20 18:00:31 -06:00
Monty Taylor
fc4bbd8f07
Add support for warning-is-error to sphinx role
Sphinx currently does not have a way to set warning-is-error in a config
file for builds that do not use python setup.py build_sphinx. It's
perfectly reasonable to use Sphinx for non-python languages, but putting
a setup.cfg into those projects is a bit weird. While we wait on getting
a config setting upstream, read the value out of setup.cfg ourselves if
it exists. This will let existing python projects with the setting work
as we expect, but will also let us just set a zuul variable for
non-python projects if we decide to not just put a setup.cfg in them.

Change-Id: Ie65dcb42c48e6a962f6715f7483ef3758caf2965
2017-11-20 18:00:26 -06:00
Monty Taylor
b493ca62dd
Add general sphinx and reno jobs and role
Sphinx jobs and reno jobs need basically the same thing for
dependencies. So make a new role, ensure-sphinx, which is
parameterizable enough that it can be used by both reno and sphinx jobs.

Make build jobs for both releasenotes and sphinx docs, as both of these
things are perfectly valid things to do in both OpenStack and
non-OpenStack contexts. We'll add an openstack specific job in
openstack-zuul-jobs that uses these as parents but adds the requirements
repo and constraints file settings.

Some of the pip commands here can be improved once
https://github.com/ansible/ansible/pull/33098 lands and is released,
which would allow specifying --user and -c as parameters to the pip
module.

Change-Id: Idd7caf7d88b56d61872906027b4ce7d743572ded
Needed-By: I57de14580f39b9e1c11a587b51b44b61b02c84da
2017-11-20 17:59:28 -06:00
Mohammed Naser
d0851b0b4a Add role to build Puppet module
This patch adds an Ansible role which can be used to build Puppet
modules.

Change-Id: I45761c2d111640733ddec8b4c08066655b6944e2
2017-11-13 18:33:50 -05:00
Andreas Jaeger
560a907b06 Add npm-docs job
Add a job for building javascript documentation using Node.

Change-Id: I4517b9fb5d9b17b099e2a5ab138688caaa976be9
2017-11-09 21:31:50 +01:00
Zuul
b4bd1f0a74 Merge "Collect npm-shrinkwrap for javascript" 2017-11-08 18:02:20 +00:00
Andreas Jaeger
5e4797bda7 Add nodejs-npm jobs
Add native nodejs jobs for lint and test.

Needed-By: Ib38294ab48ef158a248637de79970ef60bc44f7f
Change-Id: I45294049d9228020b70f6b18265e21fa9385f782
2017-11-05 07:41:32 +01:00
Andreas Jaeger
238b2aaf6d Collect npm-shrinkwrap for javascript
Collect npm-shrinkwrap.json if it exists.

Change-Id: I149dc99b1e9f08403a7f5c16ebf38dd7e734d975
2017-11-04 20:25:51 +01:00
Emilien Macchi
d6d835e414 version-from-git: fix logic with tags
When there is no commit since the tag, we don't want the scm_sha part of
project_ver, but just the scm_tag. This patch fix the logic.

Change-Id: I89e59049fe65ebc49fc03f205affa68b8fa5beb9
2017-11-03 16:05:53 -07:00
Zuul
00c8d67f69 Merge "Add a generic stage-output role" 2017-11-02 21:49:28 +00:00
Andrea Frittoli (andreaf)
786c48f051 Add a generic stage-output role
This role does roughly what save_dir and save_file do in d-g
functions, only it can be used for a list of files and/or
folders. Jobs can use this role to rename artifacts and place
them in a staging folder which is then used for mass sync to
the executor.

This is typically used in post - the zuul user won't necessarily
have write access to the folders with the artifacts - so it's
easier to stage everything to a folder owned by the zuul user.

Change-Id: Id45a50821539985710da7172a9185fd91e803603
2017-11-02 07:22:26 +00:00
David Moreau-Simard
0bb84bc58e
Persist iptables rules
We configured iptables rules but did not persist them.
This meant that rules would be flushed when restarting iptables or
the instance.

Change-Id: I9d90f55323a33d6a0f0dda1f7ab25d10984fa6cb
2017-10-31 17:39:32 -04:00
David Moreau-Simard
efd90dd2f9
Authorize the multi-node-bridge network in iptables if there's one
This allows packets to flow freely between things that bind IPs on the
network bridge.

Change-Id: Iaacdca31b27b7033ec08499bcea7061b9d3ec279
2017-10-31 14:25:56 -04:00
Zuul
180fa799be Merge "add more debugging to the upload-pypi role" 2017-10-30 14:48:32 +00:00
Zuul
c216b86625 Merge "Simplify ansible invocations in tox.ini" 2017-10-29 21:15:20 +00:00
James E. Blair
f28cc33bc3 Revert "Move to dictionary list of projects zuul._projects"
This reverts commit df987735730e712ea3bc57f99d3c96cb561c43a9.

The order in which we iterate over items in the "determine local HEAD"
task is not always the order in which we iterate over mirror-workspace-
git-repos.

Change-Id: I181d6a2cfa2c3104730f20362bbdc10fa582339f
2017-10-29 10:37:31 -07:00
James E. Blair
fa896e2125 Zuul: add file extension to playbook path
Zuul now supports including the file extension on the playbook path
and omitting the extension is now deprecrated.  Update references
to include the extension.

Change-Id: I01feaef57f6b376844dabc2eed080010632a776c
2017-10-28 18:18:27 -07:00
Ian Wienand
df98773573 Move to dictionary list of projects zuul._projects
Zuul exports the projects list as a dictionary in _projects in the
depends-on.  The idea is that we switch these jobs to that, and do
some testing.  When ready, we switch "projects" to be the dictionary
view, and revert "_project" calls here to "project".

Change-Id: Id9a7c137ca5bed25d81087201091157c8401576a
Depends-On: I4476b9d4915d107e29b91229287865bff0ada305
2017-10-28 17:46:41 +02:00
Andreas Jaeger
66ec61d67c Simplify ansible invocations in tox.ini
We have no legacy files in here, so no need to skip them.
Also, sync invocations of ansible lint and ansible syntax checks.

Change-Id: I003cf6a46d1dc376d83118e5e5c405aa54d33f22
2017-10-28 16:56:30 +02:00
Zuul
c7c56a580e Merge "Ansible linting fixes" 2017-10-28 14:40:37 +00:00
Paul Belanger
735a8f7c9a
Create fetch-puppet-module-output role
We'll be using this for puppet(-branch)-tarball jobs to publish to
artifacts site.

Change-Id: I4da68c8b1f89e2e04f4b94eebf29905c24aea474
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-10-27 14:15:36 -04:00
Andreas Jaeger
0506323c2b Ansible linting fixes
Fix a few problems noticed when using this in project-config:
* Ignore apt-get warning in configure-mirrors
* All tasks should be named in upload-logs, emit-job-header
* use git_config in mirror-workspace-git-repos
* Ignore git warnings in mirror-workspace-git-repos

Change-Id: Ibcdc8d5a42fb081b5faa0cc89460e4cf88d369ba
2017-10-27 19:18:15 +02:00
Andreas Jaeger
7c4734f398 Run ansible-lint on repo
This change follows  I16186c929e7d0e6e34b35271559e555255a52b00 to run
openstack-zuul-jobs-linters on the repository.

Also, silence remaining lint role.

Depends-On: I46c53229f878fc707bab627654d820b891d9f625
Change-Id: I76abf21ea51c5f7553639475e408e06768fb1016
2017-10-27 07:49:02 +02:00
Andreas Jaeger
6a278c011c Silence ansible-lint
These warnings are ok, disable them.

Change-Id: Ida5ecc81fdaed137ac10baa51643962a5e97b89d
2017-10-27 06:10:31 +02:00
Zuul
1d2464f826 Merge "Remove pep8/pyflakes" 2017-10-26 05:55:29 +00:00
Zuul
d7e5a849a7 Merge "Silence ansible-lint" 2017-10-25 16:39:10 +00:00
Andreas Jaeger
d8b9239324 Silence ansible-lint
Add tags to tell ansible-lint to ignore the problems found since
those are false positives.

Needed-By: I16186c929e7d0e6e34b35271559e555255a52b00
Change-Id: I59e5d41a9539671ee5aab9b9036379eb9101d021
2017-10-25 13:17:16 +00:00
Dirk Mueller
0dd2f84d0b Remove pep8/pyflakes
flake8 pulls the right versions that it needs in already,
so listing them explicitly is redundant.

Change-Id: I3441696f055a51c18828d2b345c269c69c1b0e8c
2017-10-25 14:26:34 +02:00
Zuul
e617405939 Merge "Follow redirects when triggering readthedocs build" 2017-10-24 18:02:28 +00:00
Jens Harbott
cc402a0b26 Follow redirects when triggering readthedocs build
The URL we post to returns a redirect to
https://readthedocs.org/projects/$short_name/builds/ so we have to
follow that.

Change-Id: I26c907df0ab4ec6eb40bce74f590cfab6eec38c7
2017-10-24 08:58:30 +00:00
James E. Blair
cb7efe3198 Merge "Revert "Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header"" 2017-10-23 23:21:02 +00:00
Mohammed Naser
27755ba83f Revert "Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header"
This reverts commit cb8c554f6e0a74eea331e1b3bb1847dbf4ae9233.

Change-Id: I8ef6c6778201b75e198c9a120242690573c320c6
2017-10-23 23:18:32 +00:00
Zuul
96bd241343 Merge "Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header" 2017-10-23 22:55:44 +00:00
Doug Hellmann
b5e6652c72 add more debugging to the upload-pypi role
Show the files discovered for upload.

Change-Id: I74279e8901f7790d935ce664b8026a4acf8b7ba4
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-10-23 13:25:26 -04:00
James E. Blair
9ab259a24d
Support upper-constraints in tox-siblings
Change-Id: I7c8b5ff82603c5ba4aba3db64f16ac9feada089e
2017-10-21 13:21:19 +02:00
Monty Taylor
98c9285361
Change tox-siblings disablement to allow opt-in
The relevant section is already protected by a variable. Instead of
commenting out the call, set the default value of the flag to false.
This allows people who need the functionality and are not broken by the
existing issue to use it.

Change-Id: Ifa5cc3e335b9e9ad52c4c5c9ab71aaea70d4c9aa
2017-10-21 13:18:49 +02:00
David Moreau-Simard
cb8c554f6e Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header
These are helpful to understand the context of a running job before
logs are collected and made available.

Change-Id: I33f3f425a8008534c617422a7f77255a34f86198
Depends-On: I8d014464b3ed580d4a0fc6dbdc9a42601ff107e5
2017-10-20 20:47:59 +00:00
Andreas Jaeger
f0098374b4 Fix publish location for translations
We publish the complete content of translation-source directory to
http://tarballs.openstack.org/translation-source . Fix
the location from where to get the files.

Change-Id: Icb17dbb3269c569ad8bc5343ec1eed10c438d91d
2017-10-20 18:42:06 +02:00
James E. Blair
b8b54ab701 add-sshkey: make the private key owner-writable
Otherwise, it is necessary to change perms to remove it later.

Change-Id: I405cec0a1434a95a0a0e3b60b660cdcadc9e9b24
2017-10-19 14:23:28 -07:00
James E. Blair
5f75a2d004 Add roles to add/remove a GPG key
Current usage of gpg keys involves a single role that adds, signs,
and removes the key all in one.  Some jobs may need the GPG key
installed normally onto the remote host, then left in place, and
later removed.  This change facilitates that.

Change-Id: I2f13f0c4de91808ba1bbdcc0fd20a547e43d602b
2017-10-19 13:26:16 -07:00
James E. Blair
361b11f3e3 Add-sshkey: remove pubkey if present
A mismatched private/public pair causes openssh to fail.  Go figure.

Change-Id: Ic5a51cc5a0b0d04bb2bb617c08f7d5425c9cf36f
2017-10-19 11:52:15 -07:00
Jens Harbott
66940dc5f5 The copy module needs dest instead of path
Change-Id: I797766dc8406039732d5343fa63c6bb9e4221af6
2017-10-19 18:15:13 +00:00
Zuul
7128e30c65 Merge "Use copy instead of file for add-sshkey task" 2017-10-19 17:58:05 +00:00
Zuul
f3f2d822e7 Merge "Disable tox-siblings" 2017-10-19 17:48:40 +00:00
Jens Harbott
a1fda7fb7a Use copy instead of file for add-sshkey task
Also set a safe mode just in case.

Change-Id: Ica6ca8f5e39f64f8d944d1b7b9fd9a7229e8b988
2017-10-19 17:22:27 +00:00
Ian Wienand
75d7e47287 Correct path to coverage job
Seems we added an extra "cover" in
Ibad091f7630832eb993133b0f43a60931b3dfd75

Also, correct the copy path so we copy the directory "cover"
separately

Change-Id: I4966342efd1a3d00c47394de054f81a1d89183c7
2017-10-19 14:49:53 +11:00
James E. Blair
57de2abeca Disable tox-siblings
The lack of support for constraints makes this problematic in
nearly all of the openstack jobs with project dependencies (eg,
neutron, horizon).

The role ends up installing dependencies past what's listed in
constraints.

Disable it for now (the jobs in question still have their v2-era
custom install commands) until constraints support can be added.

Change-Id: I9777bb0ba27c8d181b510e3c5febc83f5ba7142c
2017-10-18 16:08:50 -07:00
Tom Barron
74a40ec89e Collect output from coverage job
Add a post-run playbook to tox-cover job to collect the coverage
report detail and copy it to the log root of the executor.

Change-Id: Ibad091f7630832eb993133b0f43a60931b3dfd75
2017-10-17 21:58:36 -04:00
Zuul
ecd0d55ce1 Merge "Update bindep role defaults to be more sane" 2017-10-17 23:15:28 +00:00
James E. Blair
a1c25d6fc8 upload-pypi: make repository url optional
The current default URL returns an error.  The error message suggests
that we just omit the URL entirely from the config and let twine
handle it.  That's what this patch does.

Change-Id: I83b2eb30e3c93690dbe03052cd3f20a73c110350
2017-10-17 14:13:24 -07:00