32 Commits

Author SHA1 Message Date
Monty Taylor
74460853d1 Rename CI to Infra in MANY MANY places.
Change-Id: I409bd50ae374e0288531f07cfeea34856c5f8067
Reviewed-on: https://review.openstack.org/17319
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
2012-12-16 17:30:47 +00:00
Jeremy Stanley
25d4e6f4c9 Enable Gerrit CLA and Contact Store on review-dev.
This replaces the previous Echosign+Launchpad+Wiki+approver-based
asynchronous contributor license agreement signing process with a
fully-automated one contained entirely within Gerrit itself.

Note that the CLA features in Gerrit's WebUI depend on a modified
gerrit.war with an earlier patch reverted:

    https://review.openstack.org/12716

* manifests/site.pp(review-dev.openstack.org): Fill contactstore_appsec
and contactstore_pubkey private material from hiera, for use by Gerrit's
contact store feature. Similar entries should be added for
review.openstack.org before going into production.

* modules/gerrit/manifests/init.pp(gerrit): Add contactstore,
contactstore_appsec and contactstore_url variables needed by the
gerrit.config.erb template, and contactstore_pubkey needed by the
contact_information.pub.erb template. Add a conditional block so that if
contactstore is enabled it installs the libbcpg-java package which
Bouncy Castle needs for OpenPGP operations, links the bcpg.jar into
Gerrit's lib directory, and builds contact_information.pub from the
contact_information.pub.erb template.

* modules/gerrit/templates/contact_information.pub.erb: New template
which is effectively an empty file waiting to be filled with the
contents of the contactstore_pubkey variable. The
gerrit_contact_information.pub file built from it gets used to encrypt
contact information filed by users in such a way that it can only be
decrypted by the private key held by the Foundation.

* modules/gerrit/templates/gerrit.config.erb(contactstore): New section,
implemented conditionally for safety. Once enabled, if the
contactstore_appsec and contactstore_url are unset then Gerrit will
refuse to start. If the system referred to by contactstore_url is
unresponsive or contactstore_appsec does not contain the shared secret
it's expecting, contributors will be unable to file initial or updated
contact information through Gerrit's WebUI.

* modules/openstack_project/files/gerrit/cla.html: A stripped-down HTML
copy of http://wiki.openstack.org/CLA retaining all the original
wording. This will probably need updating by OpenStack Foundation staff.

* modules/openstack_project/manifests/gerrit.pp
(openstack_project::gerrit): Add contactstore, contactstore_appsec,
contactstore_pubkey and contactstore_url variables to pass back into the
gerrit module. Also define the cla_description, cla_file, cla_id and
cla_name variables which get used in the gerrit_set_agreements.sh.erb
template. Add an entry to install the cla.html file.

* modules/openstack_project/manifests/review_dev.pp
(openstack_project::review_dev): Add the contactstore_appsec and
contactstore_pubkey variables so they can be filled in by hiera.
Override the war to pull in the g69c8fa6 test build which has the
aforementioned CLA bits restored. Turn on contactstore and set
contactstore_url to point to an existing test CGI on the Internet until
the Foundation has theirs ready. Pass contactstore_appsec and
contactstore_pubkey through up into gerrit.pp. Add an entry for the
set_agreements.sh script built from the gerrit_set_agreements.sh.erb
template and then execute it to add the new CLA to Gerrit's DB and mark
the old one expired. Similar changes should be made in review.pp before
going into production.

* modules/openstack_project/templates/gerrit_set_agreements.sh.erb: New
template used to build a set_agreements.sh script which checks Gerrit's
database and, if necessary, expires the old Echosign CLA and adds the
new local CLA. These conditions are checked and associated operations
performed independently, so subsequent runs become a no-op.
Post-migration, this can probably be neutered further and kept around
for pushing future CLA modifications into the database when needed.

Change-Id: Ib7136fef23dbd5602955649b33a57bc8d7106026
Reviewed-on: https://review.openstack.org/13058
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-09-20 15:15:42 +00:00
Monty Taylor
7d8c838038 Align all web server usage on apache module.
Change-Id: Idd712a8ee5ec81c6b88b7d3e2270dce4da254927
Reviewed-on: https://review.openstack.org/10838
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-12 17:03:46 +00:00
Monty Taylor
e63619e980 Make gitweb a boolean option.
Change-Id: Iff55f35c0d9888f1029115c17d4644a68d4e8b4c
Reviewed-on: https://review.openstack.org/10727
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-03 23:12:38 +00:00
Monty Taylor
892a620737 Have apache honor replicate_local.
Change-Id: I44538a151afda2086469d0309b8113be0a477f93
2012-08-02 10:55:08 -05:00
Monty Taylor
383c023b15 Add support for initial project creation.
If replicate_local is set, this will ensure that /var/lib/git is created,
and that projects listed in the projects.config have repos there.

Additionally, it creates a new config file, projects.config which is a
yaml file listing all of the projects and various operational semantics about
them, such as whether or not they should have pull requests closed and whether
or not they track any remotes. This replaces remotes.config and github.config.

Moving forward, there is no reason to not have this script be able to
do github api calls to create the github repo if it's not there, set the
github project description, gerrit api calls to create the project in gerrit,
and initial project permissions templates.

Change-Id: I1ad803b0aa5f7386206d0c3f4cd858017242fe64
2012-08-02 15:49:02 +00:00
Monty Taylor
0b921968d1 Added an option to toggle replication choices.
New options for github replication and local replication.

Change-Id: I06a6ca5347232ec80e26f6116742ab0007435ffe
2012-07-29 13:04:17 -05:00
Monty Taylor
46282d44c1 Fixed a template typo.
Variable interpolation needs <%= not just <%. :)

Also, while I was in there, I replaced default with "oneiric",
because I don't actually know that the value is a good default value,
and I removed a couple of comments about moving to MySQL and Apache
modules from upstream.

Change-Id: Iec5b10cee2cbd0e0a2573fefa707d34d2a363cb4
2012-07-28 10:04:43 -05:00
Monty Taylor
c510a30e1e Modified gerrit to use MySQL and Apache modules.
Change-Id: I82ff3c46438f8db126fa6a881efa09c90b1906e4
2012-07-27 17:37:37 +00:00
Monty Taylor
8f1adc171f Split github into its own module.
TODO: Add another script that sets the project description. Add the project
description to the config hash.

Change-Id: If4584b2a1e55e6eb912e1f557e31de216d49a516
2012-07-23 11:42:12 -05:00
Monty Taylor
e3e9aaba10 Split gerritbot into its own module.
Change-Id: Ia9034d87321624006ccf447b058ee828a97fcc7a
2012-07-23 10:33:59 -05:00
Monty Taylor
221869cd60 Change database_password to mysql_password
The parameter, which filters straight down, is mysql_password, not
database_password.

Change-Id: Ib6a0dfbb65657367bda51b3ca54905740c13cf6b
2012-07-21 19:21:52 -07:00
Monty Taylor
d025dca604 First pass at parameterizing secret infos.
Change-Id: Iee56a7e65be51ebf19a61eefd60cc93de6a764bf
2012-07-20 14:40:42 -07:00
David Ostrovsky
89c230cb4e reconfigure gitweb to show the whole content of the commit and not only the changed filed
Change-Id: Ie9940a0b8afc0fa85f89395c4fa8bdc84f9be6ff
2012-07-07 22:26:12 +02:00
Monty Taylor
0fbbe34dfc Use apache to serve out http git repos.
Assumes that every project in gerrit has a corresponding repo in
/var/lib/git that can be replicated to. That's probably a one-time offline
creation, followed by an additional step in the adding a project docs.

Change-Id: If9b987717550d5b251366c1408d949c55e64828a
2012-07-06 14:06:21 -05:00
James E. Blair
7558862fde Set gerrit http maxwait.
Set the timeout to 5000 minutes, which, due to a bug in gerrit
really means 5 minutes, which is the documented default value.

Change-Id: I85127cc44ed6f182a0e06083641d2d872f11d8b3
2012-07-02 12:35:51 -07:00
Clark Boylan
17c836ab28 Enable Melody on review-dev.
Review-dev is running a Melody capable WAR, but the Melody service
was not enabled. Enable it.

Change-Id: I68934151a38c2572f37d996e1c923a880ab9f50f
2012-06-25 13:26:46 -07:00
Andrew Hutchings
d42c95b727 Add color values
Gerrit without my patch will ignore these config values.  Review-dev will apply
them.

Change-Id: I5f01229ad6c2e1c3a0cf2de53fffd0c2b1a3626b
2012-06-20 14:12:19 -07:00
James E. Blair
0082fa9c49 More gerrit tuning.
Increase the heap size and dramatically increase the ssh threads.
Add some more recommended parameters (see site manifest for details).

Parameterize tunables in gerrit config file.

Change-Id: Ia6446b29426f56a77425eed93a7f0e448c3cd7b1
2012-05-29 18:15:27 +00:00
Monty Taylor
6fdebcf5af Adds support for tracking an upstream repos.
Change-Id: I3c92c19cf2c2fd6a911fbdbf4aeeb66e53cf07e0
2012-05-10 13:12:31 -04:00
Andrew Hutchings
9069914ebe Change outdated color
To be landed after the gerrit change in https://review.openstack.org/6749/ had been applied.  Lightens the red used for the outdated highlight.

Change-Id: I33a89873968c0c8e6cb8dfa30892d50337b9a124
2012-04-24 15:56:56 +01:00
James E. Blair
16a44c5416 Add ServerName to apache vhost
Change-Id: I8f52bab84dd7ab5713d31e5df6e9a389532a5af2
2012-04-12 16:46:32 -07:00
James E. Blair
c4eaff3957 Install and manage more of Gerrit.
Upgrade gerrit to 2.3.0.

Add management of the apache virtualhost.

Remove gerrit body styling (including the javascript hack) in favor
of using the gerrit theme config options for body styling.  Keep header
and top menu changes.  This should make it easier to keep up with new
gerrit versions without chasing weird GWT changes.

Add management of the gerrit init script.

Add management of MySQL.

Add installation and upgrading of Gerrit.

Change-Id: Idf9e551552d335a2ae82cd27a63edcf6daf94115
2012-04-11 22:38:15 +00:00
James E. Blair
33ef15108a Revert "Move gerrit scripts into puppet module"
This reverts commit 428ec0b42dd0a33eba9752aa0e5f475044be5508
2012-04-05 20:15:19 +00:00
Andrew Hutchings
428ec0b42d Move gerrit scripts into puppet module
Moving the cron executed gerrit scripts into the puppet module.  This is for two reasons:

1. Easier/faster to deploy/maintain than having a cron job for a separate git repo
2. We can add customisations required for review-dev and stackforge

Change-Id: Iaf44e1d57d6ee6ea282575b1b48261f4ccbbaf3f
2012-04-04 15:27:18 +01:00
Devananda van der Veen
8b2ec5b7ae Enable MySQL connection pooling
Connection pool must be explicitly enabled.
See lines 160 - 175 of file
  gerrit:gerrit-server/src/main/java/com/google/gerrit/server/schema/DataSourceProvider.java

Change-Id: I0f2d712cfa622d318e81a327ce18283aa086f894
2012-04-03 20:01:16 -07:00
James E. Blair
6c779a5f83 Increase sshd threads for gerrit.
Based on the advice in this thread:
http://groups.google.com/group/repo-discuss/browse_thread/thread/b91491c185295a71

Size the sshd threads (which are also used to handle https git requests)
better.  Based on current trends, we peak at about 50% cpu usage
and 25% memory usage.  Double the number of threads here to try to
allow us to get closer to 100% cpu.

(Corresponding increase in db connections.)

Change-Id: Icde18233de01466b241ab28d38d2e98735108193
2012-02-02 14:29:12 +13:00
James E. Blair
559b0c86d5 Tune gerrit parameters.
In order to clone 8 keystone repos simultaneously via https in dev,
the following were needed:

httpd.acceptorThreads>=4
sshd.threads>=8

And database.poolLimit is supposed to be higher than http+ssh threads.

Change-Id: I8d011af41e32f7865424d54d5a78a10a3689e708
2012-01-20 15:50:04 +11:00
James E. Blair
ebac164ee0 Add CLA config to gerrit.
Also add the echosign "cla" which just points devs to the wiki page.

Change-Id: I0ef37511d9f7e7495d78f2f3cd9fde47f88aa99c
2012-01-04 10:49:40 -08:00
James E. Blair
ced39692eb Increase gerrit session timeout to 24h.
Set session cookies to SSL-only.

Change-Id: Ie49b0fe90f4c8c9f1c603d21b21e67fbdda6196c
2011-09-14 11:20:47 -07:00
James E. Blair
f252d95d5c Add cron job for closing pull requests.
Also move the launchpad sync cron into puppet.
Create config file for github pull close script.

This change depends on https://review.openstack.org/#change,224

Change-Id: I1b7ad599a6c7542614780ea0ce46a42a8995d15b
Reviewed-on: https://review.openstack.org/225
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2011-08-15 16:06:02 +00:00
James E. Blair
9b06e7fdc8 Add Gerrit configuration to puppet.
Change-Id: I26ebd80adb00ac5bf676533d5dd9359cbbe08075
2011-08-05 16:36:48 -07:00