Commit Graph

120 Commits

Author SHA1 Message Date
Ian Wienand
0913b3ed7e build-wheel-cache: update README to be more accurate
Change-Id: I724b36f3d4696e8f5ce2f6cc396f6bd09a30a841
2020-05-28 15:59:41 +00:00
Ian Wienand
033570b572 build-wheel-cache: Add virtualenv
Ensure we have the virtualenv package as it is used by the wheel build
script.

Change-Id: I05b8a23d2e3e6babab082d656fc0859b5d631ba4
2020-05-28 15:59:24 +00:00
Ian Wienand
a52318853b Move wheel build jobs out of project-config
Move the wheel build jobs out of project config so we can run
speculative testing (project-config is a protected repo).
requirements project has these jobs in it's gate.

The only change here is they are renamed to "wheel-cache" from
"wheel-mirror"; both to not conflict but also I think cache describes
what it does a bit better -- we put the cache on the mirror but we're
not mirroring something created externally.

See-Also: https://review.opendev.org/731401

Change-Id: Ib41607c4842dccf27b711947bf356ef5d74a1059
2020-05-28 17:57:59 +02:00
Andreas Jaeger
c52d8430c0 build-pdf-docs: Pass environment
Run pdf-docs with same environment that is used for the --notest run.

Change-Id: I5b6a263e01e574097f03011eaca768730701da97
2020-05-15 15:51:41 +02:00
Andreas Jaeger
537788fde6 Fix build-pdf-docs working directory
We need to use zuul_work_dir to build in the correct directory,
otherwise cross-building won't work. Line 3 in the file checks this
already but then we suddenly build in the source_dir. If those are
different - like in cross-builds, we fail.

Change-Id: I1e51a39133d1b58f3791d19af23d5cdcc194b33e
2020-05-15 14:24:50 +02:00
Sean McGinnis
e19fa5ad47
Don't run pdf-build if no target available
Change-Id: Id1e9f814cb4f4faf55facbaffdb80d3e858fe83f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-14 19:56:04 -05:00
Clark Boylan
edc845f0a6 Put sphinx-build pdf logs in a separate file
Previously we would get very large console logs for the sphinx-build pdf
generation. hboxes being over and under full, problems that are likely
difficult for projects to fix and have minimal impact on the final
product. Unforatuntely, having giant console logs makes tools
like logstash and elasticsearch very unhappy. Address this by logging
this output separately.

Change-Id: I11e1bca2826287ac415683909645f4860822edff
2020-05-14 09:38:09 -07:00
Andreas Jaeger
96680e8177 Fix translation jobs following horizon change
Change I38a1c2c3455a10b4852d25f6c39b3c21f5c450fc in horizon broke the
translation jobs for dashboard projects, example failure:
http://zuul.opendev.org/t/openstack/build/a636f60d416948849d5402de9ae83b1a/

Follow horizon bindep change and install libpcre3-dev to fix it.

Change-Id: Ib9e7627ed7cdc24cd2db423aba8db82099788ce4
2020-03-18 22:41:45 +01:00
Monty Taylor
7b10451785 Remove mirror-info role and tests
This is in opendev/base-jobs, so we don't need a second copy over
here.

Change-Id: I55640615b2c363be38ce7f76ed1795b80854282f
2020-03-11 15:13:14 -05:00
Andreas Jaeger
d519361045 Remove legacy run-jsbuild.sh
codesearch shows that this file is not used anywhere else:
http://codesearch.openstack.org/?q=run-jsbuild.sh&i=nope&files=&repos=

Looking at the history of this repo, it was used in these legacy jobs
legacy-vinz-webclient-js-release-branch
legacy-vinz-webclient-js-release-master
legacy-merlin-js-release-branch
legacy-merlin-js-release-master
legacy-storyboard-js-integration-ubuntu-trusty
legacy-storyboard-js-integration
legacy-storyboard-webclient-js-draft
legacy-storyboard-webclient-js-release-branch
legacy-storyboard-webclient-js-release-master

All these jobs either got replaces by native Zuul v3 jobs or the repos
retired and thus jobs removed completely.

The file should have been removed as part of
Icda2d699c26a03a5a5b0718ce90fe7813f0f0061.

I also grepped through all branches of existing repos and could not find
any occurence of run-jsbuild.sh (assuming my script is correct).

Thus, we can remove the file.

Change-Id: I9480ecb39d9fadbe97b393d03521118a26cd68f5
2020-02-29 17:28:54 +01:00
Zuul
0dba4fbe3b Merge "Update ansible-lint" 2020-01-24 09:33:59 +00:00
Andreas Jaeger
3e151e9818 Increase hacking version
Let's use hacking 2.0 to get newer Python3 pep8.
Fix the one error found (E305: expected two lines...).

Change-Id: Iebe51ede1e80ce256a1f6015746e8ed62a3d946b
2020-01-19 17:03:30 +01:00
Ian Wienand
0bbbe0e9b1 Update ansible-lint
I started looking at this because what I thought was an ansible-lint
bug is an ansible bug (you can't use a apostrophe in shell
comments?!...)

Anyway, the ignore numbers have changed and we don't need to enforce
pipefail.  Otherwise it found some variable typos.

[1] https://github.com/ansible/ansible/issues/28674

Change-Id: I35315c82200686841a9b1d4c87c7f27a71bd8931
2020-01-15 13:24:49 +11:00
Andreas Jaeger
6c1cfd9b8f Require JDK 8 for translations
zanata-cli does not work with openjdk > version 8, so pin down version
8.

Ubuntu has version 8 both in Xenial and Bionic:
https://packages.ubuntu.com/xenial/java/openjdk-8-jre-headless
https://packages.ubuntu.com/bionic/java/openjdk-8-jre-headless

Change-Id: I6a728c85ded138c9098054535d8b61d897d97595
2020-01-14 21:28:05 +01:00
Andreas Jaeger
ffa2095870 Fix get-modulename.py for python3
ConfigParser got renamed in Python 3 to configparser, update
get-modulename.py so that it now runs under python3.

Change-Id: Ic32b22d537ce43fb5382fdad4b550061d9060c6e
2020-01-06 17:35:11 +01:00
Andreas Jaeger
38aabd672c Update Zanata script for Python 3.5
Python 3.5 - unlike Python 3.6 - does not accept byte strings in
json.loads, so convert from bytes to string first.

This is needed now since the script is run under python3.5 and not anymore
using python 2.7.

Change-Id: Ie223ab715d6f1c2d77344a9bcc3d3885c3be85c7
2020-01-06 16:48:29 +01:00
Akihiro Motoki
ff46a36f79 translation job: Fix horizon installation
Horizon was added to upper-constraints.txt in stable branches
recently. This causes a failure in horizon installation in
the translation jobs like [1]. The script tries to install horizon
from the git repo but it conflicts with upper-constraints.

This commit updates the horizon installation to what tox does,
i.e. to install requirements with upper-constraints first and
then install horizon from the git repo.

Note that python modules from test-requirements.txt are now installed.
django-babel (or enmerkar in near future) is what we need here and
it is not a horizon runtime requirement so including test-requirements.txt
allows the horizon team to move django-babel to test-requirements.txt
in future. [2]

[1] https://zuul.opendev.org/t/openstack/build/9d4055098385422580f031c0ee668d2b
[2] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011783.html

Change-Id: Ibdb63d83b368709fdb1a69da3ce4c60d908c0edb
2020-01-06 17:38:14 +09:00
Jens Harbott
629524a047 Add xindy to pdf-docs prereqs
xindy[0] is an index processor used by LaTeX. Instead of disabling it in
all projects, let's add it to prereqs and see how it goes. See also [1].

[0] https://ctan.org/pkg/xindy?lang=en
[1] https://review.opendev.org/684451

Change-Id: Iaef8a36dc24b0aacb11454d112044c80613d5263
2019-09-26 14:47:45 +00:00
Andreas Jaeger
b1d63e2995 Remove auto translate comments from translation strings
Cleanup imports like https://review.opendev.org/684510, we do not need
these auto import information.

Change-Id: I32a8058064e231336455d45badb7bea1283ffce4
2019-09-25 08:50:28 +02:00
Andreas Jaeger
a987857df5 Fix upstream_translation_update
The HORIZON_DIR parameter is the last one, the previous change missed
the number 3. Update to fix.

This script still has Zuul v3 compatitibility, remove it.

Change-Id: Ib89d44225de4bd171594659647408900321f23e9
2019-09-19 09:51:09 +02:00
Andreas Jaeger
403005bb56 Fix horizon translation setup
The previous change left an extra /horizon in, so that we did:
cd /home/zuul/src/opendev.org/openstack/horizon/horizon

Fix this.

Change-Id: I5e42280c64425bce7579470a7bffdefc44442162
2019-09-18 11:48:45 +02:00
Andreas Jaeger
40fb0dfa2e Use on-disk horizon dir for translations
Instead of checking out horizon ourselves, let Zuul do it for us. The
updated playbooks pass in the horizon directory as parameter.

This fixes manila-ui/train translations since we do not have a
stable/train branch for horizon and therefore the git commit fails. If
we use Zuul for it, it will checkout master as fallback for us.

Depends-On: https://review.opendev.org/682808
Change-Id: Ifa96087916d21be9b4a6fcce99088c5fc1565b41
2019-09-18 09:06:35 +02:00
Andreas Jaeger
b49d5c8961 Remove kata jobs
These jobs and roles are not needed anymore here,
kata has its own tenant.

Change-Id: I4d9912864ec549e46fe398baef5265b89e4f4a96
2019-08-28 20:54:26 +02:00
Akihiro Motoki
bba1a617ee Add support for building PDFs
We want PDFs. We don't want to have to go to each project and add a
'pdf' tox target or similar. Get the former and avoid the latter by
adding a step to build our PDFs to the 'openstack-tox-docs' job.

The job assumes a 'pdf-docs' tox env to build PDF docs.
If 'pdf-docs' tox env does not exist, PDF doc build will be skipped.

Note that only the job for master and stein is updated,
we do not build PDFs for older releases.

Do not build PDFs for deploy-guide for now.

Co-Authored-By: Stephen Finucane <stephenfin@redhat.com>
Co-Authored-By: Andreas Jaeger <aj@suse.com>
Depends-On: https://review.opendev.org/678077
Change-Id: Ibc455ba511a877159848f4060a6058800d7d2f9b
2019-08-23 17:29:58 +02:00
Andreas Jaeger
16c5684be3 Install gettext for translation jobs
Add install of gettext to prepare-zanata-client role so that projects
don't need to add it to their bindep.txt file.
gettext is essential for the role to work.

For consistency capitalize all names of tasks in the file.

Change-Id: Ic36534c4edfbb086f5bcc66ab657107552ba92de
2019-07-22 10:53:04 +02:00
Dirk Mueller
9dc9548066 Switch to using TOX_CONSTRAINTS_FILE
To make lower-constraints handling less awkward, there was
the motion to rename the UPPER_CONSTRAINTS_FILE to TOX_CONSTRAINTS_FILE,
as it can be used to specify either upper or lower constraints.

Change-Id: I47fcd93c9b2a5f47fa36c43574f51c86715ac326
2019-05-22 22:28:20 +02:00
OpenDev Sysadmins
cae8e2bafd OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:29:57 +00:00
Clark Boylan
10405004a5 Remove configure-unbound role
This role now lives in opendev/base-jobs so to reduce confusion we clean
it out of ozj.

Depends-On: https://review.openstack.org/635900
Change-Id: I27971ad1bd3ef93ea04aaede76e51afe9d770b5b
2019-02-08 09:49:18 -08:00
Sorin Sbarnea
5777cfe479 Implement workaround for ansible unmount bug
It seems that unmount can falsely report success but removing it from
fstab seems to be a way to avoid that bug.

Example build failure caused by it:
http://logs.openstack.org/13/626613/1/check/tripleo-buildimage-overcloud-full-centos-7/e9d97b2/job-output.txt.gz

See https://github.com/ansible/ansible/issues/48313

Change-Id: I608ba6801f36f834d4e82c0438b41ea1e90620e8
2019-01-08 12:47:31 +00:00
Salvador Fuentes
cd1385507e kata-containers: add /usr/sbin to the PATH
Add `/usr/sbin` to the PATH.
On Fedora zuul machines `/usr/sbin` seems to not be in
the PATH by default, which makes fdisk to not be found
making some docker tests to fail.

Change-Id: I9732e45dee43a53f5ecd4d3fd4853d82f236c288
Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-12-17 12:00:19 -06:00
Jesse Pretorius
6f91f687ea Correct MariaDB proxy suffix
In https://review.openstack.org/577945 the suffix was changed
from 'mariadb' to 'MariaDB', so this env var should also be
changed to match it.

Change-Id: Ic6ee98e0d74eb31b067de398de2739ed7a50d438
2018-11-22 12:03:17 +00:00
Andreas Jaeger
135fb6a31a Handle operations-guide and contributor-guide
Documentation projects are whitelisted. Since operations-guide and
contributor-guide has a normal "doc/source" setup, we can use the
generic handling of this and just extend the whitelist.
Instead of having the whiteliste in two places, move it in a central
shared place.

This will handle operations-guide and contributor-guide the same way
as horizon, openstack-ansible, and openstack-helm.

Change-Id: Ie1065b7d4ad47dee738568c70df28a5cdaa6f6f7
2018-11-15 09:25:32 +01:00
Sorin Sbarnea
1863cb0063 Avoid ignore_errors on Set ephemeral device by label
Avoids confusing output that looks like:
primary | ERROR: Ignoring Errors

Return code different than zero is normal and should not fail
that task, using failed_when does avoid displaying the task
as a failure when the rc is not zero (most builds).

Change-Id: I00e70c65c73880281f13d8bf92c5779814e57837
2018-10-19 14:58:40 +01:00
Ian Wienand
3cdfbc6f67 Move prepare-zanata-client to o-z-j
This role is currently in project-config where it is untested.  Move
it to o-z-j, and rename to prepare-zanata-client for consistency with
other names.  Add integration test.  We will migrate the jobs to the
new name when this merges.

Change-Id: I26ee7057b9ad70507959286acc17d9c7bc1471db
2018-10-03 14:42:17 +02:00
Salvador Fuentes
6d01e66ff0 Add zuul envar for kata tests
In order to know we are running under a zuul slave,
we need to add an environment variable. In this case
it is ZUUL=true. This will allow the kata-containers
scripts to run the tests with the appropiate flags.

Change-Id: Ib040c032ad49beb5e2041e92efe41ce4e8b98799
Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-09-27 09:51:09 -05:00
Clark Boylan
e38bcb4c12 Configure git user details for kata
The kata test jobs apply patches to git repos with git. This creates
commits which requires you have a user configured in git. Set up a
global git config with generic Zuul identify info in it to address this.

More details at https://github.com/kata-containers/ci/issues/65

Change-Id: I08a6a13501fad92cd290f0a9e5559f61b11d7fab
2018-09-14 08:25:37 -07:00
Emilien Macchi
5eb6547460 mirror-info: add variable for CBS proxy
Add info for NODEPOOL_CBS_CENTOS_PROXY for the new proxy created by:
I6337b73703c3cecab714a2fa2de3884c96f1216a

Change-Id: Icb9e50e02c5462e432124e0d1dada02e61734140
2018-08-30 17:45:27 +00:00
Clark Boylan
97b42fefb3 Add $GOPATH/bin to kata job's $PATH
Kata need their built binaries to be in the PATH for tests to work
properly.

Change-Id: I8c05e7a5a02fdeb337c7ba6871dba0a9cebcdf13
2018-07-12 08:55:03 -07:00
Zuul
4ce9d6d1c3 Merge "Cleanup the kata setup role" 2018-07-04 17:52:38 +00:00
Clark Boylan
7bfe80df73 Cleanup the kata setup role
This adds a modprobe for vhost_net to ensure it is present for kata.
Also we will remove the explicit docker install as .ci/setup.sh should
do it for us now.

Allow k8s cluster-cidr range to talk to port 6443 (k8s api server).

Change-Id: Ic1c13e9635e00bcdd096c48c8a7983ccc19bfbda
2018-07-03 20:07:42 -07:00
Mohammed Naser
558dda1c79 Add NODEPOOL_MARIADB_PROXY and drop NODEPOOL_MARIADB_MIRROR
The mirror was having issues and it only was mirorring Ubuntu, with
the proxy, it will be caching all of the different architectures
and operating systems.

Depends-On: I30bd6b658d87abc765b5caee5003e387981850b4
Change-Id: I435b006abe434361014ad8cdc8093e6863c7a109
2018-06-24 20:41:14 -04:00
Clark Boylan
0356342edb Add fedora 28 support to kata runsh job
This refactors the kata runsh job a bit to push pre.yaml setup into a
role so that we can more easily include distro specific tasks. This
allows us to have differing ubuntu and fedora setup in a readable
manner.

Change-Id: I06029f49a81d0a4262c58e8f45a1414efa43d920
2018-06-22 14:42:07 -07:00
Mohammed Naser
5a550cfa67 Add NODEPOOL_PERCONA_PROXY to mirror_info
The way that we consume things, we need to actually have the host with
port and URL to configure the roles downstream.

Depends-On: Ic3c255bd24d1202fec436d4fcc484ce420110518
Change-Id: I19be46fc3ac7e8938178d9d8cb4d1dfeadc6f00b
2018-06-19 09:23:28 -04:00
Colleen Murphy
8cee140416 Revert "Add Zuulv3 jobs for puppet functional testing"
This reverts commit 0474b75e86.

Since the general consensus is to just keep puppet on life support
until we're ready for a new config management system, it's not
worth the effort to convert our beaker-rspec tests to a new thing.

Change-Id: I275443442ced5c76ec016b66b957bee28dea40dc
2018-06-06 21:04:01 +02:00
Colleen Murphy
0474b75e86 Add Zuulv3 jobs for puppet functional testing
We need to abandon rspec as a functional testing framework. It
constantly breaks as ruby gems change their dependencies, no one likes
to deal with it and it did not help bring new people from the puppet
community to help maintain the modules. This patch adds a Zuulv3-native
job that can act as a framework for the future of puppet functional
testing. Puppet modules will need to have the files
tests/scenarios/*/{manifest.pp,tests.sh} in order to use this.
Eventually this should replace the legacy-puppet-beaker jobs.

Change-Id: Ic47e93fb091a9cdc876981674cff6b452a1afe87
2018-05-09 19:35:33 +02:00
Zuul
e64e0fb901 Merge "Fix ansible-lint job" 2018-03-29 13:47:31 +00:00
Colleen Murphy
665704e3dc Fix ansible-lint job
The ansible-linter was only running on playbooks, not on roles. This
patch corrects the tox environment to ensure that roles are linted.

Since roles weren't being linted, some errors had sneaked through. This
patch also corrects an unnamed task and disables linting of a task that
deliberately uses the shell module.

Also fix a comment in the tox file that was referencing a non-existent
zuul job.

Change-Id: I5b1ca3947e80588ab0cc8d3632f1292fc707aac1
2018-03-28 23:24:07 +02:00
Ian Wienand
a282af81ac Use hostvars to get ansible_default_ipv6
I think this is missing picking this up from hostvars, so always
assumes ipv4.

Change-Id: I65929a87f864701ac07966a6b0d9ca9225da5145
2018-03-27 20:22:44 +11:00
Paul Belanger
cad4f10733
Create old jenkins directories
We've deleted these from DIB, and now need them to be created for
legacy jobs.

Change-Id: I9ef48d0b1983671900ca08684db982ee4336adbc
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-05 16:45:18 -05:00
Zuul
285d47bfe6 Merge "Resolve Ansible variable precedence issue with include_vars" 2018-03-04 21:44:57 +00:00