323 Commits

Author SHA1 Message Date
Paul Belanger
066bd3b748 Add $site_variables_yaml_file to zuul::executor
Change-Id: I2a5d65b6f43a3866f1f10e47cb545f70397739f0
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-03 14:12:24 -04:00
Paul Belanger
90df637aaa
Use zuul_status_url for /keys endpoint
Change-Id: I908e219cb16c3d6a9554916da805b8722552d940
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-01 13:30:23 -04:00
Paul Belanger
81e1edc0f1
Add /keys to apache vhost
We need to expose our public keys for secrets.

Change-Id: I3d80aaa008b3b0d18f6e8a6bc479352eb9c076d7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-01 12:35:48 -04:00
Monty Taylor
392bc6a954
Add paths bind mount option names to match zuul
We just made this change in zuul v3 itself, update the puppet to expose
it.

Change-Id: I448f6710b54d0e1d0955b7c657a84a0fcc8b405f
2017-07-28 16:39:15 -05:00
James E. Blair
a98e898002 Add support for disk_limit_per_job
Change-Id: Ic267a156ee7eef4eb9e5f431cc22b236c017d0e8
Depends-On: I415e5930cc3ebe2c7e1a84316e78578d6b9ecf30
2017-07-28 13:14:54 -07:00
Jenkins
774915d6f8 Merge "Add PyMySQL python library for SQL Reporter support" 2017-07-27 20:47:34 +00:00
Monty Taylor
98056b2c60
Add PyMySQL python library for SQL Reporter support
It doens't have any other dependencies, but it's an optional depend for
Zuul itself, because plugins. Install it for the scheduler. Also,
provide an option flag to enable it.

Change-Id: I4cad419598ffefcdca5161664c6d073da37fb42f
2017-07-27 13:54:46 -05:00
Jenkins
267330257e Merge "Add support for a parallel set of secrets for connections" 2017-07-26 20:56:12 +00:00
Monty Taylor
afbbf2eca6
Add support for a parallel set of secrets for connections
github and mysql connections both contain secret data that is not
appropriate for public hiera. Create a second list of data that can go
into private hiera. If it's found, it will be added to the connection
config.

Change-Id: I265b445cfbf4b6d4555621e3663e71e9a0f85813
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-26 15:30:47 -04:00
Paul Belanger
04f4d9a55d
Fix stop / restart for zuul-web init script
We don't yet have socket support for zuul-web, revert to default way
of doing this.

Change-Id: I2ec2c928c43def839d0ba8a212135d5414da9a52
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-21 10:29:10 -04:00
Monty Taylor
bdaeb08041
Send websocket traffic to ws:// protocol
The console-stream URL needs to be talked to over ws:// to make sure the
WebSocket upgrade headers are sent.

Change-Id: I38f1f0b0f05403b9a47acdf88d5d8059f9af6295
2017-07-20 05:58:59 +09:00
Monty Taylor
0ef5671c85
Add web proxying to non-SSL vhost too
The previous add only added to the SSL site. Whoops.

Change-Id: I9c4f63b9cb981e9ca68d63c5021dee151769ce3e
2017-07-20 02:48:39 +09:00
Jenkins
ba474aa21c Merge "Run zuul-web as zuul" 2017-07-19 00:45:15 +00:00
James E. Blair
d726212730 Run zuul-web as zuul
It does not need to run as root.

Change-Id: Id06623e42dd6bc963c5a0012fcd7adae78c6e666
2017-07-18 17:06:38 -07:00
Jenkins
c2865118c9 Merge "Clean up zuul-scheduler init script" 2017-07-18 22:46:45 +00:00
Jenkins
4ceab2ffe9 Merge "Add support for zuul-web and scheduler connections" 2017-07-18 16:21:56 +00:00
Paul Belanger
796c154ac3
Send SIGHUP for zuul-scheduler reload
We haven't added socket support yet to zuulv3, revert to the original
way of using SIGHUP for reloading zuul-scheduler.

Change-Id: Id466ccb28ea37a3b7171237e98ea08c8db7199cd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-14 15:43:07 -04:00
Monty Taylor
3f4f1d9820
Add support for zuul-web and scheduler connections
Zuul v3 has a zuul-web service that provides web streaming and
eventually other web content. Add support for running it. Also add it to
the Apache mod_rewrite proxy info.

While we're adding things, add proxy info for plugin connections, which
should allow receiving github webhook events.

Change-Id: Ic79681b287dbd1a44469da70c680060940734f40
2017-07-12 05:26:04 -05:00
James E. Blair
faa53c009c Update zuulv3.conf
Several config options have recently changed, update the conf
file template to match.

Change-Id: I084417b6d622bbb34377c973cc73c4cb0b5e84bb
2017-07-07 10:39:40 -07:00
James E. Blair
efa87a0fa7 Add gerrit IO logger
This reduces the log noise from gerrit -- we won't automatically
log all of the JSON values we get.

Change-Id: Ibfec14211f8c644a9c1e9104a96df5f47d5430ff
2017-06-29 10:06:33 -07:00
Jenkins
daa1e5bd12 Merge "Add support for mapping dirs into bubblewrap" 2017-06-27 23:06:39 +00:00
James E. Blair
d7a484b188 Add support for mapping dirs into bubblewrap
I9a8a74a338a8a837913db5e2effeef1bd949a49c introduced these options.

Change-Id: I2e457e05ccee8da33cddf11deb47144c556a024f
2017-06-27 14:26:02 -07:00
Colleen Murphy
220481fab4 Make services work with systemd
Without this patch, the SysV init scripts defined in this module will
not be recognized by systemd and puppet will fail to enable the
services. This is because between installing the init script and
starting the service, systemd needs to be reloaded. This patch adds a
new class to define the systemd reload. The conditional logic for
whether the reload exec should be in the catalog is defined in the
class. Then the file and service resources that depend on the reload
define their relationships to the new class rather than to the exec
resource so that the conditional logic does not have to be repeated for
every resource.

We also need to correct a bug in the init scripts themselves. The
Provides statement needs to give a unique service name, not just 'zuul'.
If it is not unique, insserv will error with "service zuul already
provided!" when systemd tries to enable the service.

Change-Id: Ica849094c6011806bdd0b205ba6b2b73856b7aa6
2017-06-17 19:53:01 +02:00
Paul Belanger
7028052386
Fix file path to SSL files
The previous patch incorrect add the SSL contents to zuul.conf, we
actually want the file path.

Change-Id: If1f9e9d333d23d22a9f3e2aa320b1b5cd1642d91
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-16 20:34:36 -04:00
Paul Belanger
1a08165eea
Add gearman server / client SSL support
Note this only works for zuulv3 today.

Change-Id: Iecd4ccc230653ef803764d10c626879d9ad3b1d2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-16 19:14:26 -04:00
Colleen Murphy
58e66ed91f Fix beaker on xenial
Add a xenial nodeset and update the spec helper to install puppet 3 from
the Ubuntu repos instead of from puppetlabs.

Change-Id: I2c17666694ab95ea13a636e5939d507bf860df2b
2017-06-11 18:23:33 +02:00
Paul Belanger
df1913b654
Expose zuul_status_url for zuulv3
With zuulv3 we have created per tenant status.json files. Expose this
to allow us to properly configure the 'openstack' tenant for the
webui.

Change-Id: I866e4ffa90930a44b4cdac5a48e7034bb0d6398d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-08 19:13:48 -04:00
Paul Belanger
02f569bda1
Clean up zuul-scheduler init script
We didn't properly create our init script before, rebase from existing
zuul.init script.

We also need to add systemctl daemon-reload for scheduler / executor,
as we run them under xenial currently.

Change-Id: I0df2720750737fa7220c04a6d2d6506a45e2761e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-08 17:37:50 -04:00
Paul Belanger
534fa140b2
Start executor as root
zuul-executor will drop privileges to the zuul user.

Change-Id: I8f717a693e50a7e99c6d906860419161effc4718
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-07 14:09:27 -04:00
Paul Belanger
ca7cda5983
Fix typo with executor logging in zuul.conf
Change-Id: Ifc26afda49ad1d0b84ae88935292c835aa58f4ec
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-07 13:47:50 -04:00
Paul Belanger
219e0d4644
Initial commit of zuul-executor
This is part of our zuulv3 effort. We want the ability to manage
zuul-executor on a server.

Change-Id: Iddac03a7187df47d2c72d957a50610f8b6a1dff5
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-05 17:27:57 -04:00
Paul Belanger
84488919c6
Add zuulv3 configuration file
Rather then updating the existing zuul.conf.erb file, create a new
zuulv3.conf.erb specific to zuulv3. Add new settings for tennant /
zookeeper configurations.

Change-Id: I6d0ec2ef2d1b0c86fdcd06bfca01c02a6fffa7f3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-02 12:27:37 -04:00
Paul Belanger
cd746e236b
Add ansible dependencies for zuulv3
We need to ssl and ffi development libraries to properly compile
ansible. Now a dependency of zuulv3.

Change-Id: If7f64b9e74aaf5f0611343b9d323ae92c42be865
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-01 21:22:17 -04:00
James E. Blair
1c2aca510f
Add initial py3 support (with pip3)
Support using pip3 binary to install python applications under
python3.

Change-Id: I943e5cbcc55a09edb9431298e3f0a17e2aef5acf
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-01 18:22:07 -04:00
James E. Blair
5d5bdb0e42 Allow passing arbitrary connection objects
The fixed 'gerrit' and 'smtp' connection entries are deprecated.
We should allow users to pass in any connection information and
write that to the config file.

Change-Id: I4ebb7cd6b64d1854cd2e8e6e146572330558864c
2017-04-25 07:48:34 -07:00
Jeremy Stanley
6d356334de Bump JJB pin to 1.6.2
In order to address a dependency installation issue around PBR
version capping, update to the stable point release of
jenkins-job-builder which solves that problem.

Change-Id: I999e9490217ef3f03cee46b7843fedf53230d230
2017-04-15 14:50:06 +00:00
Clark Boylan
8015dd90b2 Add zuul-scheduler service
This is a halfway step to adding an explicit zuul::scheduler manifest to
run the zuulv3 scheduler daemon.

Change-Id: I99302505c4055559ee0ee09c42e69673aff57e97
2017-02-20 14:24:59 -05:00
Clark Boylan
890878481c Install ansible 2.1.4 release
This undoes the previous hacks that were used to install the 2.1.4 RC
and installs the new 2.1.4 release from pypi proper. This is just a
saner way to do package installation than the hack needed to install the
RC.

Change-Id: Ib2cecce5a629c866092370a1a0cf6cbf6e4194f4
2017-01-17 23:53:13 -08:00
Clark Boylan
ace0412fb7 Install ansible 2.1.4.0 RC1 on launchers
This installs ansible 2.1.4.0 RC1 which includes some useful bug fixes
on the zuul launchers. Note that we couldn't easily use the pip package
provider for this beacuse ansible does not publish ansible RCs to pypi.
So we replace the package resource with an exec that invokes pip
directly to do what we want.

Change-Id: If49a64540258b3a7a7be4df7e86e23b14e019c52
2017-01-10 17:17:26 -08:00
Paul Belanger
16bfaaa27a
Add keytab support for sites
This is needed for our AFSDocs migration to zuul-launcher.

Change-Id: Ia8f756c8482536a67f12ca5d76f3037d33055182
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-09-16 11:35:26 -04:00
James E. Blair
790106561a Ansible launcher: Pin JJB to 1.6.1
This version of JJB has what we need, and since we're using an
internal API, probably best to stick with it for now.

Change-Id: I733d479539ef86dd315afe0279113a58995c33d6
2016-08-05 10:56:05 -07:00
Paul Belanger
1446f72dde
Upgrade to ansible 2.1.1.0
This includes the fixes we are running in our forked version plus
more!

Change-Id: I95ba59c13ce1a0f6db49017e3561b393ddfd0ee5
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-07-29 18:34:03 -04:00
Spencer Krum
a156f0e421 Use new infra_spec_helper for gem dependencies
Change-Id: I45dd33f14754d3697e86d7ac043f8be476e4b105
2016-06-21 23:27:49 -07:00
Philip Marc Schwartz
b504281e8a Update pip provider to use openstack_pip provider
Change-Id: Ief5a43e23073e24971b9051d936e9c399c21197f
Signed-off-by: Philip Marc Schwartz <philip@progmad.com>
2016-06-16 10:21:04 -04:00
Paul Belanger
ad15caa0e4
Ensure ansible 2.1.0.0 is installed
This is needed now for zuul_log with adds type=raw.

See: Icb4d20e3ef0166eb43b1b45aec049040813b5d37 which exposes this
issue.

  {"changed": false, "failed": true, "msg": "implementation error:
  unknown type raw requested for msg"}

Change-Id: I1ba1fc21d2a7b3f73aca73adf72fe3a1f2903af1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-06-13 14:10:22 -04:00
Jenkins
f1ddc3d131 Merge "Expose gearman_check_job_registration setting" 2016-06-13 17:19:33 +00:00
Paul Belanger
f0522bdcc8
Add lftp to zuul-launcher
We need this for our FTP publisher task.

Change-Id: Ibf0043ac1b15a067c24c35109241f07317909bf6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-06-11 10:30:16 -04:00
Paul Belanger
24b3606d28
Expose gearman_check_job_registration setting
A new zuul.conf setting to check if jobs are registered with gearman
or not.

Change-Id: I745ef2342d8feb1695ce0b0d034caf428d6bef4f
Depends-On: I06f2221770e04f958ce7ac4cfe8d5d92d7164cac
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-06-08 17:31:51 -04:00
James E. Blair
dadcd7905b Remove deprecated hyphens from zuul launcher
Change-Id: I412a38552e89049562490f3810faf03f9eef1752
2016-06-08 14:29:39 -07:00
James E. Blair
16d595990c Add support for launcher accept_nodes parameter
Change-Id: I0086c25d4817a6a3302648bd0560762791a0e572
2016-06-08 14:13:59 -07:00