Commit Graph

38 Commits

Author SHA1 Message Date
James E. Blair
12709a1c8b Run a docker registry for CI
Change-Id: If9669bb3286e25bb16ab09373e823b914b645f26
2019-02-01 10:12:51 -08:00
James E. Blair
8062f4c1ec Grab container logs at the end of run-base
So that we automatically get container logs for future jobs
which use containers.

Change-Id: I329c67eefb8c6a2ff9a8ce8ef69cc844cef6012a
2019-02-01 10:12:39 -08:00
James E. Blair
22ad414a86 Use stage-output role in system-config-run
This simplifies log collection.

Change-Id: I754637115f8c7469efbc1856e88bbcb6fb83b4ce
Depends-On: https://review.openstack.org/634293
2019-01-31 11:03:25 -08:00
James E. Blair
d145e86f71 Switch to zuul-jobs docker build jobs
There are upstream jobs in zuul-jobs with the docker build playbooks,
so use them.  The system-config jobs are kept so that we don't have
to duplicate the secret stanza.

Change-Id: Iceee55a3d0e8b243549fa988f134b1ea9bb6dac5
2019-01-23 13:44:04 -08:00
James E. Blair
7a6ae3aadb Add docker image building jobs
This adds the infrastructure for building docker images: the
credential used to upload to Docker Hub as well as the parent jobs
and playbooks to perform the builds.

Change-Id: I7cbbcdd184c4934f1b0ce5905d9760c732b06aa9
Depends-On: https://review.openstack.org/631078
2019-01-16 11:28:14 -08:00
James E. Blair
9e6a4d327e Remove all docker image building jobs
These will be re-added in followup changes based on the new
zuul-jobs roles.

Change-Id: I0c0691afb180903c573cc1a9eb39590fa58e7ca6
2019-01-16 11:28:07 -08:00
Monty Taylor
b1c4a6d36a Build a gerrit image
The gerrit source dir needs three plugins cloned into
the plugins dir and also a few files updated.

Depends-On: https://review.openstack.org/631007
Change-Id: I56037137d43ee1cea0a4c17e48d09102e1599ddc
2019-01-16 11:27:20 -08:00
Monty Taylor
d1a2dfa6f5 Add support for per-image build-arg settings
Change-Id: Ia14397389be0c7178c1879844cd675a3d291a3d0
2019-01-15 09:27:16 -08:00
James E. Blair
a546e25e76 Delete change tags from docker image repos
Whenever we promote an image, delete the change tag for that image
in Docker Hub, and also delete any change tags older than 24 hours
in order to keep the Docker Hub image registry tidy.

Change-Id: Id4654c893963bdb0a364b1132793fe4fb152bf27
2019-01-14 16:51:13 -08:00
Monty Taylor
700f195706
Add support for passing path to docker build
If we clone gerrit to ~/src/gerrit.googlesource.com/gerrit but
want to keep the Dockerfile in system-config, then we need to be
able to run:

  docker build ~/src/gerrit.googlesource.com/gerrit -f Dockerfile

Most of the time the dir will just be '.', so put in a sensible
default.

Change-Id: I235080c05e679d2ac270cd5401b85c655fab3112
2019-01-12 21:34:40 +00:00
James E. Blair
2059b73584 Fix image promote playbook
This job has no nodes; the playbook needs to run on localhost.
The only tasks use the uri module without local files, so should
be safe.

Change-Id: Ic012426a66be3b85efe9af35089addf1316dfa63
2019-01-11 13:39:07 -08:00
James E. Blair
899a2023d2 Use promote pipeline for system-config images
Upload an image to dockerhub with a change-specific tag in every
gate job, and then, if the change lands, re-tag the image in
dockerhub.

Change-Id: Ie57fc342cbe29d261d33845829b77a0c1bae5ff4
2019-01-11 08:22:19 -08:00
James E. Blair
3ce494abab Add gitea dockerfile
Change-Id: Iae2c4cbd174212e97445dd3080e2b85be730c5c4
Story: 2004627
2019-01-10 10:53:48 -08:00
Ian Wienand
f07bf2a507 Import install-docker role
This is a role for installing docker on our control-plane servers.

It is based on install-docker from zuul-jobs.

Basic testinfra tests are added; because docker fiddles the iptables
rules in magic ways, the firewall testing is moved out of the base
tests and modified to partially match our base firewall configuration.

Change-Id: Ia4de5032789ff0f2b07d4f93c0c52cf94aa9c25c
2018-12-14 11:30:47 -08:00
Ian Wienand
860b0f9773 Collect syslogs from nodes in ansible tests
This collects syslogs from nodes running in our ansible gate tests.
The node's logs are grouped under a "hosts" directory (the bridge.o.o
logs are moved there for consistentcy too).

Change-Id: I3869946888f09e189c61be4afb280673aa3a3f2e
2018-12-14 10:33:27 -08:00
Ian Wienand
3bed6e0fd3
Enable ARA reports for system-config bridge CI jobs
This change takes the ARA report from the "inner" run of the base
playbooks on our bridge.o.o node and publishes it into the final log
output.  This is then displayed by the middleware.

Create a new log hierarchy with a "bridge.o.o" to make it clear the
logs here are related to the test running on that node.  Move the
ansible config under there too.

Change-Id: I74122db09f0f712836a0ee820c6fac87c3c9c734
2018-12-04 17:46:47 -05:00
James E. Blair
6368113ec9 Add kube config to nodepool servers
This adds connection information for an experimental kubernetes
cluster hosted in vexxhost-sjc1 to the nodepool servers.

Change-Id: Ie7aad841df1779ddba69315ddd9e0ae96a1c8c53
2018-11-28 16:24:53 -08:00
Zuul
9679f57225 Merge "Add Arm64 CI cloud" 2018-11-06 01:21:48 +00:00
James E. Blair
dae1a0351c Configure opendev nameservers using ansible
Change-Id: Ie6430053159bf5a09b2c002ad6a4f84334a5bca3
2018-11-02 13:49:38 -07:00
James E. Blair
90e6088881 Configure adns1.opendev.org server via ansible
Change-Id: Ib4d3cd7501a276bff62e3bc0998d93c41f3ab185
2018-11-02 13:49:38 -07:00
Gary Perkins
10c8da0e90 Add Arm64 CI cloud
Changes:
  * Set URLs to https://
  * Set new keystone domain
  * Set region to nrt1

Change-Id: Ib96d14008ab3b8b7c12429d7432eaa485c404bb2
2018-11-02 12:22:15 +00:00
Monty Taylor
e998db36f2 Add yamlgroup inventory plugin
The constructed inventory plugin allows expressing additional groups,
but it's too heavy weight for our needs. Additionally, it is a full
inventory plugin that will add hosts to the inventory if they don't
exist.

What we want instead is something that will associate existing hosts
(that would have come from another source) with groups.

This also switches to using emergency.yaml instead of emergency, which
uses the same format.

We add an extra groups file for gate testing to ensure the CI nodes
get puppet installed.

Change-Id: Iea8b2eb2e9c723aca06f75d3d3307893e320cced
2018-11-02 08:19:53 +11:00
Clark Boylan
0e9d830a64 Add job to run playbook on bridge.o.o
This new job is a parent job allowing us to CD from Zuul via
bridge.openstack.org. Using Zuul project ssh keys we add_host bridge.o.o
to our running inventory on the executor then run ansible on bridge.o.o
to run an ansible playbook in
bridge.openstack.org:/opt/system-config/playbooks.

Change-Id: I5cd2dcc53ac480459a22d9e19ef38af78a9e90f7
2018-09-12 11:26:55 -06:00
Ian Wienand
20c0340c3c Add Linaro London cloud
The Linaro London cloud runs as a separate entity to the existing
cloud, so add it separately.

Change-Id: Ia263034de4c8d1ac52f4521d4c82374b06a68e68
2018-09-11 17:04:40 +10:00
Zuul
4db29ef5d3 Merge "Fix variable names in nodepool cloud config" 2018-09-05 18:21:48 +00:00
Ian Wienand
a7e85e75b1 Fix variable names in nodepool cloud config
Deployment of the nodepool cloud.yaml file is currently failing with

 FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'rackspace_username' is undefined"}

This is because the variables in the group_vars on bridge.o.o are all
prefixed with "nodepool_".  Switch to this.

Change-Id: I524cc628138d85e3a31c216d04e4f49bcfaaa4a8
2018-09-05 15:55:34 +10:00
James E. Blair
4477291111 Add testinfra tests for bridge
Change-Id: I4df79669c9daa3eb998ee666be6c53c957467748
2018-09-05 14:24:00 +10:00
Clark Boylan
09288c7c37 Manage clouds.yaml files in ansible
This manages the clouds.yaml files in ansible so that we can get them
updated automatically on bridge.openstack.org (which does not puppet).

Co-Authored-By: James E. Blair <jeblair@redhat.com>
Depends-On: https://review.openstack.org/598378
Change-Id: I2071f2593f57024bc985e18eaf1ffbf6f3d38140
2018-09-04 08:49:00 -07:00
Zuul
017fdf3e34 Merge "run-base: exclude __pycache__ from logs" 2018-08-30 18:57:02 +00:00
Zuul
2a51a493e0 Merge "Add system-config-run-eavesdrop" 2018-08-30 18:38:18 +00:00
Zuul
94ea02f635 Merge "run-base: collect ansible configuration" 2018-08-30 18:29:56 +00:00
James E. Blair
a491f80f6f run-base: exclude __pycache__ from logs
We don't need to copy this back over.

Change-Id: I419c9c80526953177f808ad7b180deba9bf0434e
2018-08-30 11:05:49 -07:00
Zuul
768df606b0 Merge "Generate junit.xml file for testinfra" 2018-08-30 17:58:39 +00:00
James E. Blair
09b1ff4bc3 Add system-config-run-eavesdrop
Add a job which runs testinfra for the eavesdrop server.  When we
have a per-hostgroup playbook, we will add it to this job too.

The puppet group is removed from the run-base job because the
groups.yaml file is now used to construct groups (as it does
in production) and will construct the group correctly.

The testinfra iptables module may throw an error if it's run
multiple times simultaneously on the same host.  To avoid this,
stop using parallel execution.

Change-Id: I1a7bab5c14b0da22393ab568000d0921c28675aa
2018-08-30 10:25:23 -07:00
James E. Blair
6de5aee53b run-base: collect ansible configuration
This is helpful for debugging.

Change-Id: I0423dcede48c98379e58edd245842afe3292c0b1
2018-08-30 10:25:19 -07:00
James E. Blair
800397c3da base-test: iptables: allow zuul console streaming
This adds a group var which should normally be the empty list but
can be overridden by the test framework to inject additional iptables
rules.  It's used to add the zuul console streaming port.  To
accomplish this, the base+extras pattern is adopted for
iptables public tcp/udp ports.  This means all host/group vars should
use the "extra" form of the variable rather than the actual variable
defined by the role.

Change-Id: I33fe2b7de4a4ba79c25c0fb41a00e3437cee5463
2018-08-29 09:20:42 -07:00
Paul Belanger
20286235b6 Generate junit.xml file for testinfra
And collect it on post, it is helpful to see the results.

Change-Id: I0dbecf57bf9182168eb6f99cdf88329fcdeb1bdc
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-29 12:00:43 -04:00
James E. Blair
19aebca3e7 Add system-config-run-base job
This adds a job which creates a bridge-like node and bootstraps it,
and then runs the base playbook against all of the node types we
use in our control plane.  It uses testinfra to validate the results.

Change-Id: Ibdbaf511bbdaee46e1335f2c83b95ba1553a1d94
Depends-On: https://review.openstack.org/595905
2018-08-24 08:50:01 -07:00