55 Commits

Author SHA1 Message Date
James E. Blair
17e09934db Fix zuul installation
The following items were missing from an automated zuul install:

 * A pre-populated known hosts file
 * The git user/email for creating commits (.gitconfig existed
   on zuul.o.o, but switch to using zuul's config file instead).

Also, make sure that the mergers specify the zuul server as the
gearman server address rather than localhost.

Change-Id: I47a473f60c4b5b2daaa910aa61cdf6e0c6fe1528
2014-02-18 02:33:24 +00:00
James E. Blair
083281f7ee Add Zuul merge servers
Remove obsolete config options from zuul module.

The server and merger classes are constructed so that they
may coexist on a single server.  Also, the init section is
constructed so that it will install everything needed for
both services but without activating them (leaving maximum
flexibility for the operator).

Change-Id: I7b86fbbe4611c5edfb463a0a6944e0717f664188
2014-02-17 11:11:21 -08:00
Clark Boylan
2093dff5be Pack zuul git refs daily.
Zuul was packing git refs weekly, do this daily instead to handle the
number of refs created by gate resets in large gate queues.

Change-Id: I8fc8c78436ebf79b457d392f1ce8de31b15c762c
2014-01-21 11:53:04 -08:00
James E. Blair
0a68468f4d Fix typo in zuul config template
Change-Id: Ica5ae3936b3396ce7d29f69c0c688ec6867bea48
2014-01-08 16:26:21 +08:00
James E. Blair
2866945db2 Have Zuul replicate to git.o.o
Change-Id: I750e8ea75181dacb10275a38fa702ed1edb56d4c
2014-01-08 15:52:00 +08:00
Clark Boylan
7780ef382a Give Zuul file descriptor breathing room.
The Zuul service opens a lot of TCP connections and files resulting in
many open file descriptors. The default limit of 2048 is bumped into
regularly. Bump the limit to 8192 when starting the Zuul service to give
it breathing room to grow into.

Change-Id: I9a3aee664c1c6c6139b012fc3dc0631725f59741
2014-01-01 12:59:11 -08:00
James E. Blair
c3b363c2c2 Add zuul pack-refs cron
Ad a cron job to pack refs in zuul repos.  Zuul creates _a lot_
of refs, packing them should make client git repo access more
efficient.  Up to a point at least -- ultimately we may still
need to delete them later.

Change-Id: Iaf5180069b38833d6e25f6880e4868d5e2319b75
2013-12-04 10:12:29 -08:00
Arx Cruz
5e5023ebae Zuul puppet support to zuul_url option
This adds the zuul_url support in puppet.

Change-Id: Ie37744f11452efc4029c1fe93d880dd3a4ecb831
2013-11-27 09:58:47 -02:00
Monty Taylor
0222d3040e Use pip install instead of setup.py install
The pip vs. easy_install interactions continually get the systems
into a weird state. pip install $path_to_repo, on the other hand,
works like a charm.

Left out two graphite installs because they are doing path manipulation
and I want to do further testing before touching that.

Change-Id: I373b29aca03f1ebd20e533ebaaf5de9ef2db017e
2013-10-27 18:38:38 -04:00
Jenkins
d78e4e6acc Merge "Use cgit server instead of github for everything" 2013-08-23 17:01:32 +00:00
Monty Taylor
258abe1a23 Use cgit server instead of github for everything
We have a cgit server now, which means we should replace
all references to github with references to git.openstack.org.

Change-Id: I68ad1ce514fb4326c7d9940b5a84999af5b58562
2013-08-19 10:19:47 -07:00
James E. Blair
b93eb82cad Add testresult commentlink
Will take effect when Zuul is running this change:
  I74702fd7d37358e6f4caa7e7ac0a3ede73184077

This change also adds that feature to the Zuul config and enables
it for OpenStack.  It also adds the ability to specify HTML in a
commentlink (and uses it).

Change-Id: Idb4ad8e6079165d681271987a92cab5d8b7c81be
2013-08-18 12:43:15 -07:00
Jenkins
72d6967405 Merge "Ensure python pip is installed before zuul" 2013-08-15 01:51:10 +00:00
Jenkins
8d346cd442 Merge "Ensure /var/lib/zuul is owned by zuul" 2013-08-06 20:14:50 +00:00
Dan Bode
12ced38ca7 Ensure python pip is installed before zuul
Previously, installing zuul resulted in the erros:

  /usr/bin/python: No module named pip

This commit ensures that python-pip is always installed
before zuul is installed so that it can install
correctly.

Change-Id: I6d73a67e1fbb9eb077f39f99c45ceb4a00bc652b
2013-08-01 18:48:40 -07:00
Dan Bode
e06560841e Ensure /var/lib/zuul is owned by zuul
Previously, the file resource did not have an
owner specfied, meaning that it defaulted to root:root.

This resulted in the following failure when starting the
zuul service:

  IOError: [Errno 13] Permission denied: '/var/lib/zuul/test'

This commit ensures propoer ownership so that zuul can start.

Change-Id: I3ee9ae49097695632114ed6692c78ebf22a08823
2013-07-31 17:32:35 -07:00
Clark Boylan
663fd2225f Cleanup unneeded Jenkins vars.
Now that Zuul and Jenkins use Gearman we don't need to push Jenkins auth
details down into the zuul manifests. Removes these parameters and vars.

Change-Id: I744977f002eb9fb60c3392f0281ab033ffe2bb12
2013-07-31 16:19:49 -07:00
James E. Blair
b064f95e52 Use Zuul's internal gearman server.
(This is only used by zuul-dev currently.)

Remove co-hosted gearmand server (will require manual cleanup).

Also, listen for feature/gearman events in openstack-infra.

Change-Id: I2dd58e62ff79277f1ef5c79da54bcc7511f2ccee
Reviewed-on: https://review.openstack.org/30175
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-06-04 21:47:33 +00:00
James E. Blair
fff4eb63d2 Install the default zuul status page.
On a zuul server, install the default status page as shipped by
Zuul (and its dependencies).

Remove openstack-specific redirects (hopefully it's been long enough
now).

Change-Id: Idfa2d9a4aa261182d166a36193fe74e1d922bcd9
Reviewed-on: https://review.openstack.org/29262
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-05-17 17:07:20 +00:00
James E. Blair
b081333c0b Add zuul-dev.openstack.org.
And connect it to the firehose.

Change-Id: Id860ff329e33a588d669608e5402bdb3765001dd
Reviewed-on: https://review.openstack.org/28742
Reviewed-by: Khai Do <zaro0508@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-05-13 20:32:34 +00:00
James E. Blair
8368dec679 Add apache license header to zuul module manifest.
Change-Id: I8374f389dc0a470a9e12b2987b3bd45dd07a9b74
Reviewed-on: https://review.openstack.org/27513
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Khai Do <zaro0508@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Antoine Musso <hashar@free.fr>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2013-04-26 14:48:28 +00:00
James E. Blair
35b8274533 Move zuul status to static.o.o.
And update the status page to use common.js.
Redirect old zuul status urls to status.o.o.

Change-Id: I98e86eb78b70d5cbe53410919642f302e9709d19
Reviewed-on: https://review.openstack.org/25664
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-03-29 20:49:54 +00:00
James E. Blair
2b44017725 Stop installing pip packages on zuul.
Let setup.py install handle it instead.

Change-Id: I7f143dcc4c19af267ef820973184c45e2020a3de
Reviewed-on: https://review.openstack.org/25551
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2013-03-27 18:52:25 +00:00
James E. Blair
8250da7911 Add statsd configuration to zuul.
And allow zuul to send to statsd.

Change-Id: Ie9f78268a673523c4ea4025bdbbe40d8d3819398
Reviewed-on: https://review.openstack.org/18658
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-12-26 21:44:31 +00:00
James E. Blair
396cfb1bd4 Add /rechecks alias to zuul.
Since .html extensions are uncool.

Change-Id: I66ed8b6c43e7917d793ec372fac755e02b207c63
Reviewed-on: https://review.openstack.org/18552
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-12-25 03:30:51 +00:00
James E. Blair
bd4d0794c9 Add OpenStack themed status screen for Zuul.
Requires Zuul support for JSON status endpoints.

Change-Id: I899a54af6d380aa2836a85757492e7590be90e8b
Reviewed-on: https://review.openstack.org/18580
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-12-23 03:29:39 +00:00
James E. Blair
2c5f820e5a Fix rechecks.html URL on zuul.o.o.
Change-Id: Icb504f56499651388c6dd4ba720bd55a93602322
Reviewed-on: https://review.openstack.org/18449
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-12-20 01:05:28 +00:00
James E. Blair
6bf249ee4b Add recheckwatch.
* New puppet class to install the script, init script, and set up
  user and directories.
* Add class to zuul server.
* Install openstack themed scoreboard template.
* Add URL to zuul.openstack.org/rechecks.html.

Change-Id: I9046cd21923aae40107f0d558080c44f65481fd7
Reviewed-on: https://review.openstack.org/18442
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2012-12-20 00:27:37 +00:00
James E. Blair
2b30f04e8c Fix some zuul config errors.
Update the status URL that is left in gerrit.
Don't push zuul refs to gerrit.
Don't proxy git requests to zuul in the apache config.

Change-Id: I578383d8bcec63b8581dc1a8b50405ad7e7070a5
Reviewed-on: https://review.openstack.org/18195
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
2012-12-16 19:42:45 +00:00
Monty Taylor
74460853d1 Rename CI to Infra in MANY MANY places.
Change-Id: I409bd50ae374e0288531f07cfeea34856c5f8067
Reviewed-on: https://review.openstack.org/17319
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
2012-12-16 17:30:47 +00:00
James E. Blair
5b9461b5e1 Move Zuul to its own server.
Remove zuul from jenkins server.
Remove transitional zuul config.
Add an extra argument to gerrit-git-prep for the git site name, so that it
doesn't have to be the same as the review site.
Serve zuul git repos via apache.

Change-Id: I342032bcedf317c0f48ff89b773546eb2ccd08f8
Reviewed-on: https://review.openstack.org/17949
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
2012-12-16 17:25:55 +00:00
Monty Taylor
5c5033cbc7 Protect install of every jeepyb package.
Because puppet sucks at installing packages, wrap the installation of
everything that jeepyb needs in both jeepyb and in other places with
if blocks. *facepalm*

Change-Id: I69c73dabc3ea5d68bd057141b1ad758cdd68eafc
Reviewed-on: https://review.openstack.org/17130
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-11-30 18:25:30 +00:00
Monty Taylor
3a71173aab Install extras on zuul server.
Change-Id: Iaa2c9510bd9707f28715742ad95a64d232e0a8e1
Reviewed-on: https://review.openstack.org/16925
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-11-26 23:12:31 +00:00
Matthew Wagoner
4f433a021f Clean up of minor puppet-lint warnings.
Mostly documentation and parameterised class parameter complaints.

Change-Id: I5b3ffa4ad3f707f385165c2d86c891c6bd4c1ae0
Reviewed-on: https://review.openstack.org/16901
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2012-11-26 20:55:57 +00:00
Antoine Musso
9f424f3968 zuul: learn the ability to set push_change_refs
push_change_refs controls whether zuul should push its reference back in
Gerrit.  This might not be always wanted, for example on very busy
Gerrit installation that never use the refs/zuul/* hierarchy for any
purpose.

Pushing back to Gerrit is now disabled by default and is explicitly
enabled on OpenStack infrastructure (openstack_project::zuul).

Change-Id: I24111ba4b20ab3d5198fdf7deaf902485260d8a5
Reviewed-on: https://review.openstack.org/16475
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-11-23 21:57:31 +00:00
Matthew Wagoner
999d4e70c2 Cleanup zuul manifest init.pp lint errors.
Change-Id: Ic2b8f638d4c328eeb92a86c6ce1b92a55b13d6bb
Reviewed-on: https://review.openstack.org/14981
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-30 16:36:18 +00:00
unicell
c6059ce5d6 Fix restart logic in zuul init script
Change-Id: I4ddd69916db4d9df036d3d674fb58847981d2758
Reviewed-on: https://review.openstack.org/13810
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Antoine Musso <hashar@free.fr>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-09 20:55:13 +00:00
Clark Boylan
a4f1a267ac Fix Jenkins modules.
Previously you could not use the Jenkins module to go from fresh host to
running jenkins. There were two problems.
1. jenkins::master does not explicitly create a jenkins user which is
required by Zuul. Instead the jenkins deb package creates the users.
Zuul File resources that need the jenkins user should require the
jenkins package.
2. jenkins::master did not install Bazaar but attempted to run a service
that depended on bzr.

Correct these two errors (require jenkins package and remove bzr
service).

Change-Id: Ib2d6f58fbcd456781f10cf2ceefc11503d6f92f2
Reviewed-on: https://review.openstack.org/13926
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2012-10-05 16:07:11 +00:00
Clark Boylan
e8606a02d9 Add zuul status url to zuul config.
Zuul can now provide links to the status page if a url for that page is
included in the zuul configuration. Add the status_url option to the
zuul config erb and default it to https://$fqdn/zuul/status.

Change-Id: I160a4e1cd71a13981cb171c71a923cc1af753a80
Reviewed-on: https://review.openstack.org/13859
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-01 17:03:37 +00:00
Clark Boylan
607f653a9e Be conservative requiring python-yaml.
A lot of modules require python-yaml. Be conservative requiring this
package to avoid conflicts when two or more of these modules end up
being used on the same host.

Change-Id: I35bf4f1070e6f248734c593e9277b737166904c1
Reviewed-on: https://review.openstack.org/13688
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-09-26 23:40:42 +00:00
Antoine Musso
f668e7c3a0 zuul dependencies update
This add some Ubuntu packages as Zuul dependencies, although they are
listed in tools/pip-requires already. New dependencies are:

 - python-lockfile
 - python-paramiko
 - python-yaml

I am not sure how well the Ubuntu versions will work out versus the one
from pip.

Change-Id: Ie56da494db7eb4925834824028ed6471b91be77b
Reviewed-on: https://review.openstack.org/13575
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-09-25 15:54:21 +00:00
Antoine Musso
87778caaee zuul can now be installed from a different git repo
$git_source_repo is a new optional parameter to the zuul class, let you
specify from which repository you want to install zuul. The default is
the maintainer version on GitHub.

The reason for this change is that some third party always hosts their
software on their own infrastructure. That is the case of the Wikimedia
Foundation which is reusing this module.

Change-Id: I5ac8852b08d158ff3531f6795855df20a204789e
Reviewed-on: https://review.openstack.org/13576
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-09-24 19:15:04 +00:00
James E. Blair
a2c8473fe4 Add url_pattern to zuul.conf.
Change-Id: Ib1d05a42e98613e746cbd7865cd97ce25f6a26c0
Reviewed-on: https://review.openstack.org/11940
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
2012-08-24 17:47:11 +00:00
James E. Blair
9038098310 Configure zuul to push refs.
Change-Id: I59456356579d8e83b3f7d552ac795c4de55c0817
Reviewed-on: https://review.openstack.org/11590
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-08-17 21:26:10 +00:00
James E. Blair
9e8a187482 Add GitPython to zuul.
Also, set up the git directory.

Change-Id: Ia753c00df0665b3e8dec18083f113a900b168216
Reviewed-on: https://review.openstack.org/11446
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-08-16 17:04:08 +00:00
Monty Taylor
4aa77df2e5 Fix jenkins loop and zuul vcsrepo.
Change-Id: I83cf1303bb07e59b0c3305a2540f048638940372
Reviewed-on: https://review.openstack.org/10979
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-07 17:19:36 +00:00
Monty Taylor
246566d65e Replace manual git work with vcsrepo module.
Change-Id: Ia0f3acd50aab07cc83d13cd12c8f7fc67f1c697d
Reviewed-on: https://review.openstack.org/10832
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-07 07:09:43 +00:00
Monty Taylor
6b20654720 Remove reference to non-existent group.
Change-Id: I2965c548fb51f7235faa02b8808b8b36f31ded7c
Reviewed-on: https://review.openstack.org/10829
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-04 13:15:30 +00:00
Monty Taylor
5fdfa26704 Add zuul.conf.
Add a templated zuul.conf with apikey filled in by hiera.
Made an openstack_project::zuul class to encapsulate installation
and config of zuul with our config files.

Change-Id: Idc09819b27672879b74b90bdbf699b07aa416a2a
Reviewed-on: https://review.openstack.org/10460
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-03 23:14:01 +00:00
Clark Boylan
81e77fc27a Fix the zuul reloading on config changes.
Zuul config files were broken out into a new module which introduced
cross module subscription dependencies. To make this work the module
doing the notify/subscribe must include the module the resource that
the subscription/notify is on. To fix this have config files notify
the zuul_reload exec.

Change-Id: I2d9e2236bc378d9dd92f811830dee6540b4cf4e2
2012-07-27 17:10:46 -07:00