80 Commits

Author SHA1 Message Date
Jenkins
ef686b72d4 Merge "Fix jquery-visibility.min.js name" 2014-06-04 11:02:22 +00:00
Jenkins
7066a8e06d Merge "Fix zuul's app.js name" 2014-06-04 10:53:22 +00:00
Clark Boylan
5fa31b09da Better apache mem cache tuneable values
By default our apache mod mem cache tuneables are too conservative to
cache the zuul status.json. Bump the numbers to something reasonable for
the status.json file. This means increasing the total cache size and the
max object/stream sizes.

Change-Id: Icf6ad2591e67d5916efbd65ddf93379d2c3ffc69
2014-06-03 20:51:18 -07:00
Jenkins
0aff17e1b3 Merge "Install yappi on zuul nodes." 2014-06-04 02:23:22 +00:00
Clark Boylan
c7ad6909e5 Install yappi on zuul nodes.
Zuul is growing the ability to toggle yappi profiling on and off via
SIGUSR2. This will be an optional requirement so we need to install it
ourselves if we want it. Install it here.

Change-Id: I97086b9301e1801d4af2156e81dff3c385472333
2014-06-03 17:50:39 -07:00
Monty Taylor
4498f9d2ec Cache status.json for 5 seconds in Apache
There's no need to actually serve to-the-second accurate status.json
content. Cache it in Apache.

Closes-bug: 1326170
Change-Id: If3bc44b781ce4e12ac95d3929242ec0c912f0010
2014-06-04 00:57:21 +01:00
Monty Taylor
7fced08a32 Use Apache to compress status.json
Doing this in python is teh suck on large status pages. Apache is
pretty good at it.

Partial-bug: 1326170
Change-Id: Iccee19dd6179f5a5f928fef43068d2450828fa0d
2014-06-04 01:47:15 +02:00
Longgeek
7f4af17b72 Fix jquery-visibility.min.js name
what i meant was that we can just copy the same mechanism we're using to
run the minifier, instead of using the file object with ensure=>link

Fixed bug #1325117

Change-Id: I10cf4b4772e8d3fff869bfd638b2895b6e8980ea
2014-05-31 06:49:02 +08:00
Jenkins
526613897a Merge "Update zuul module to take params for smtp" 2014-05-30 21:04:09 +00:00
James E. Blair
fd66afe4c0 Merge "Rotate zuul logs with logrotate" 2014-05-27 21:46:30 +00:00
Clark Boylan
e376d82672 Rotate zuul logs with logrotate
Python logrotate does not compress. Use logrotate utility to do log
rotation so that older logs can be compressed saving disk space.

Change python logging to use normal file handlers as it will no longer
be doing the rotation for us. Keep all the file names the same.

Change-Id: I7302e4b197e99fd49fd9d5678702333b17a46efa
2014-05-27 14:30:53 -07:00
Clark Boylan
dd33ac2e10 Pass tenant_name to zuul config.
Turns out that while keystoneclient does not require a tenant_name be
set swiftclient does require it so the fix to not pass a tenant_name to
zuul did not work.

Alex_Gaynor pointed out the reason using the normal tenant_name doesn't
work with Rax object store is that the tenant_name for object store in
Rax is different than the normal tenant_name. Use that different and
correct tenant_name instead.

Change-Id: I13d6b8dfc190d35178e7fc979d7feb58e7e1f8dd
2014-05-23 16:00:50 -07:00
Joshua Hesketh
47fd1e7d1b Update zuul module to take params for smtp
Allow setting the smtp configuration options in the zuul puppet module

Change-Id: I1a603e64b87cf05cebaaca88334c19eeb09bec5f
2014-05-23 15:36:42 +10:00
Joshua Hesketh
b741746b2e Fix zuul's app.js name
When I1895c74f73e92bcacde132b71ec5798e8307db01 merges zuul's app.js
will be renamed to jquery.zuul.js.

When I1c6e12629db3e121b3513b0b1054ed1274ceb51f merges zuul requires
another file: zuul.app.js.

Change I2698207391307bc1021330cb123b756cb7f729ea moved inline css
into a styles/ directory.

Update the zuul module for these changes.

Change-Id: I98e788f03bf9e85a51ff38c42d7cba368319bd9e
2014-05-23 15:13:59 +10:00
Jenkins
bab6620b1c Merge "Ensure zuul's status images are available" 2014-05-22 00:47:18 +00:00
Jenkins
a20a288677 Merge "Configure swift credentials for workers to push to" 2014-05-20 20:56:56 +00:00
Joshua Hesketh
564f8ab237 Configure swift credentials for workers to push to
This change adds credentials as set up by fungi (2014-04-05T03:16:46)
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-05.log

This allows zuul to send signed URL's for workers to push files to on
a per-job basis.

This change will require a zuul restart, not just a reload!

Note this is unreverts the revert in
c1b98eaff618b8c739cc02f089ef6fe6c02cbcc3 but is slightly edited hence
the lack of a proper revert commit. The reason for the revert has been
debugged and was due to rax identity service not providing a catalog
entry for swift when a tenant name is provided.

Change-Id: I04d3207002f7422b9851515ee88a74b19dd2f248
2014-05-20 13:36:36 -07:00
Joshua Hesketh
4423fb212d Ensure zuul's status images are available
Lets apache serve up zuul's status images to fix broken image src's
on http://zuul.openstack.org

Change-Id: Ie3b0ecf6b9d6cfa417f866bbaa8714e8b84ff239
2014-05-15 14:28:42 -07:00
Jenkins
477e93f92c Merge "Install bootstrap for zuul module" 2014-05-09 21:56:26 +00:00
Clark Boylan
c1b98eaff6 Revert "Configure swift credentials for workers to push to"
This reverts commit 176f15cd48ceea7e990fc61362875a23524f963a.

Puppet and zuul aren't ready for this. The swift client throws an
exception. We need to restart zuul to fix a more pressing issue hence
this revert. Will debug when able to.

Change-Id: Ie4487d84b7f1495c7ec6fd31e4b5b90d6699a31d
2014-05-05 18:49:15 -07:00
Jenkins
30d8b49457 Merge "Configure swift credentials for workers to push to" 2014-05-05 22:14:52 +00:00
Joshua Hesketh
4e300337b4 Install bootstrap for zuul module
Install twitter bootstrap for the zuul status page to use.

Change-Id: Id99ce1a9c811b91f6381cd10d79816cd7b251dee
2014-04-30 12:00:59 +10:00
Joshua Hesketh
6366ca63a7 Update zuul merger init script
Set the description of the zuul-merger.init to 'Zuul Merger' to
differentiate from the main zuul-server daemon.

Change-Id: I9ee914df0f98157b2710cbc2f9c29f7b41726273
2014-04-30 11:15:44 +10:00
Joshua Hesketh
176f15cd48 Configure swift credentials for workers to push to
This change adds credentials as set up by fungi (2014-04-05T03:16:46)
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-05.log

This allows zuul to send signed URL's for workers to push files to on
a per-job basis.

This change will require a zuul restart, not just a reload!

Change-Id: Ib198112db0493456345fec83b32fabf606d835d2
2014-04-25 12:07:24 -07:00
Jay Pipes
be09d2bcda Adds ! defined() guards around a2mod declarations
If you attempt to include both Zuul and Jenkins master modules in a
single manifest, you will get an error about A2mod[rewrite] resource
already being declared, like so:

Duplicate declaration: A2mod[rewrite] is already declared in file
/home/ubuntu/os-ext-testing/puppet/modules/os_ext_testing/manifests/master.pp
at line 30; cannot redeclare at
/root/config/modules/zuul/manifests/init.pp:236 on node undef

This isn't noticed by upstream, because no server runs both a Jenkins
master and a Zuul server. However, in smaller systems, where both
Jenkins and Zuul can be installed on the same host, the problem becomes
apparent.

This patch fixes that issue by adding guards around the a2mod
declarations in the Zuul init.pp and Jenkins master.pp manifests.

Change-Id: I2a298eb09ec8da33caed9349f465b2d163e2c299
Closes-bug: #1281676
2014-02-21 14:13:14 -05:00
James E. Blair
17e09934db Fix zuul installation
The following items were missing from an automated zuul install:

 * A pre-populated known hosts file
 * The git user/email for creating commits (.gitconfig existed
   on zuul.o.o, but switch to using zuul's config file instead).

Also, make sure that the mergers specify the zuul server as the
gearman server address rather than localhost.

Change-Id: I47a473f60c4b5b2daaa910aa61cdf6e0c6fe1528
2014-02-18 02:33:24 +00:00
James E. Blair
083281f7ee Add Zuul merge servers
Remove obsolete config options from zuul module.

The server and merger classes are constructed so that they
may coexist on a single server.  Also, the init section is
constructed so that it will install everything needed for
both services but without activating them (leaving maximum
flexibility for the operator).

Change-Id: I7b86fbbe4611c5edfb463a0a6944e0717f664188
2014-02-17 11:11:21 -08:00
Clark Boylan
2093dff5be Pack zuul git refs daily.
Zuul was packing git refs weekly, do this daily instead to handle the
number of refs created by gate resets in large gate queues.

Change-Id: I8fc8c78436ebf79b457d392f1ce8de31b15c762c
2014-01-21 11:53:04 -08:00
James E. Blair
0a68468f4d Fix typo in zuul config template
Change-Id: Ica5ae3936b3396ce7d29f69c0c688ec6867bea48
2014-01-08 16:26:21 +08:00
James E. Blair
2866945db2 Have Zuul replicate to git.o.o
Change-Id: I750e8ea75181dacb10275a38fa702ed1edb56d4c
2014-01-08 15:52:00 +08:00
Clark Boylan
7780ef382a Give Zuul file descriptor breathing room.
The Zuul service opens a lot of TCP connections and files resulting in
many open file descriptors. The default limit of 2048 is bumped into
regularly. Bump the limit to 8192 when starting the Zuul service to give
it breathing room to grow into.

Change-Id: I9a3aee664c1c6c6139b012fc3dc0631725f59741
2014-01-01 12:59:11 -08:00
James E. Blair
c3b363c2c2 Add zuul pack-refs cron
Ad a cron job to pack refs in zuul repos.  Zuul creates _a lot_
of refs, packing them should make client git repo access more
efficient.  Up to a point at least -- ultimately we may still
need to delete them later.

Change-Id: Iaf5180069b38833d6e25f6880e4868d5e2319b75
2013-12-04 10:12:29 -08:00
Arx Cruz
5e5023ebae Zuul puppet support to zuul_url option
This adds the zuul_url support in puppet.

Change-Id: Ie37744f11452efc4029c1fe93d880dd3a4ecb831
2013-11-27 09:58:47 -02:00
Monty Taylor
0222d3040e Use pip install instead of setup.py install
The pip vs. easy_install interactions continually get the systems
into a weird state. pip install $path_to_repo, on the other hand,
works like a charm.

Left out two graphite installs because they are doing path manipulation
and I want to do further testing before touching that.

Change-Id: I373b29aca03f1ebd20e533ebaaf5de9ef2db017e
2013-10-27 18:38:38 -04:00
Jenkins
d78e4e6acc Merge "Use cgit server instead of github for everything" 2013-08-23 17:01:32 +00:00
Monty Taylor
258abe1a23 Use cgit server instead of github for everything
We have a cgit server now, which means we should replace
all references to github with references to git.openstack.org.

Change-Id: I68ad1ce514fb4326c7d9940b5a84999af5b58562
2013-08-19 10:19:47 -07:00
James E. Blair
b93eb82cad Add testresult commentlink
Will take effect when Zuul is running this change:
  I74702fd7d37358e6f4caa7e7ac0a3ede73184077

This change also adds that feature to the Zuul config and enables
it for OpenStack.  It also adds the ability to specify HTML in a
commentlink (and uses it).

Change-Id: Idb4ad8e6079165d681271987a92cab5d8b7c81be
2013-08-18 12:43:15 -07:00
Jenkins
72d6967405 Merge "Ensure python pip is installed before zuul" 2013-08-15 01:51:10 +00:00
Jenkins
8d346cd442 Merge "Ensure /var/lib/zuul is owned by zuul" 2013-08-06 20:14:50 +00:00
Dan Bode
12ced38ca7 Ensure python pip is installed before zuul
Previously, installing zuul resulted in the erros:

  /usr/bin/python: No module named pip

This commit ensures that python-pip is always installed
before zuul is installed so that it can install
correctly.

Change-Id: I6d73a67e1fbb9eb077f39f99c45ceb4a00bc652b
2013-08-01 18:48:40 -07:00
Dan Bode
e06560841e Ensure /var/lib/zuul is owned by zuul
Previously, the file resource did not have an
owner specfied, meaning that it defaulted to root:root.

This resulted in the following failure when starting the
zuul service:

  IOError: [Errno 13] Permission denied: '/var/lib/zuul/test'

This commit ensures propoer ownership so that zuul can start.

Change-Id: I3ee9ae49097695632114ed6692c78ebf22a08823
2013-07-31 17:32:35 -07:00
Clark Boylan
663fd2225f Cleanup unneeded Jenkins vars.
Now that Zuul and Jenkins use Gearman we don't need to push Jenkins auth
details down into the zuul manifests. Removes these parameters and vars.

Change-Id: I744977f002eb9fb60c3392f0281ab033ffe2bb12
2013-07-31 16:19:49 -07:00
James E. Blair
b064f95e52 Use Zuul's internal gearman server.
(This is only used by zuul-dev currently.)

Remove co-hosted gearmand server (will require manual cleanup).

Also, listen for feature/gearman events in openstack-infra.

Change-Id: I2dd58e62ff79277f1ef5c79da54bcc7511f2ccee
Reviewed-on: https://review.openstack.org/30175
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-06-04 21:47:33 +00:00
James E. Blair
fff4eb63d2 Install the default zuul status page.
On a zuul server, install the default status page as shipped by
Zuul (and its dependencies).

Remove openstack-specific redirects (hopefully it's been long enough
now).

Change-Id: Idfa2d9a4aa261182d166a36193fe74e1d922bcd9
Reviewed-on: https://review.openstack.org/29262
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-05-17 17:07:20 +00:00
James E. Blair
b081333c0b Add zuul-dev.openstack.org.
And connect it to the firehose.

Change-Id: Id860ff329e33a588d669608e5402bdb3765001dd
Reviewed-on: https://review.openstack.org/28742
Reviewed-by: Khai Do <zaro0508@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-05-13 20:32:34 +00:00
James E. Blair
8368dec679 Add apache license header to zuul module manifest.
Change-Id: I8374f389dc0a470a9e12b2987b3bd45dd07a9b74
Reviewed-on: https://review.openstack.org/27513
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Khai Do <zaro0508@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Antoine Musso <hashar@free.fr>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2013-04-26 14:48:28 +00:00
James E. Blair
35b8274533 Move zuul status to static.o.o.
And update the status page to use common.js.
Redirect old zuul status urls to status.o.o.

Change-Id: I98e86eb78b70d5cbe53410919642f302e9709d19
Reviewed-on: https://review.openstack.org/25664
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-03-29 20:49:54 +00:00
James E. Blair
2b44017725 Stop installing pip packages on zuul.
Let setup.py install handle it instead.

Change-Id: I7f143dcc4c19af267ef820973184c45e2020a3de
Reviewed-on: https://review.openstack.org/25551
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2013-03-27 18:52:25 +00:00
James E. Blair
8250da7911 Add statsd configuration to zuul.
And allow zuul to send to statsd.

Change-Id: Ie9f78268a673523c4ea4025bdbbe40d8d3819398
Reviewed-on: https://review.openstack.org/18658
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-12-26 21:44:31 +00:00
James E. Blair
396cfb1bd4 Add /rechecks alias to zuul.
Since .html extensions are uncool.

Change-Id: I66ed8b6c43e7917d793ec372fac755e02b207c63
Reviewed-on: https://review.openstack.org/18552
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-12-25 03:30:51 +00:00