71 Commits

Author SHA1 Message Date
James E. Blair
3425cb3762 Use the mysql class to get python-mysql.
Change-Id: I6d6addc2bc0e28b289726cddd6626669dbec1e17
Reviewed-on: https://review.openstack.org/17292
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-12-02 16:42:49 +00:00
Paul Belanger
a7ce7fd02e Additional puppet-lint formatting
Change-Id: I6e5fa77a301eec30cff8e16bad33a91bfd95b13f
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/17176
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2012-11-30 20:56:32 +00:00
Monty Taylor
52db16762b Consume jeepyb.
Instead of keeping many of these files directly in the tree, use them
from the out-of-tree jeepyb project, which makes them easier to consume
for other people who are not us.

Change-Id: Id704f2e17dd80709ef63cbbf2c5475a08a835f91
Reviewed-on: https://review.openstack.org/16777
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-11-30 18:02:30 +00:00
Matthew Wagoner
5812510ea7 Clean up of minor puppet-lint warnings.
Mostly documentation and parameterised class parameter complaints.

Change-Id: Idbfd348a5befb041ce6eb36f9c6b195fc0c6799f
Reviewed-on: https://review.openstack.org/16685
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-11-23 21:41:37 +00:00
Monty Taylor
d76f051d22 Attempt to more fully manage project creation.
Manage project creation via yaml files. Also,
Modify the manage_projects scripts to configure Gerrit project ACLs.
This change expects the project yaml to exist. The change will clone the
project for the localhost Gerrit install. It will then checkout the
meta/config ref, copy the ACL config file into the repo, commit, and
push to the origin. The ACL config location should be specified in the
projects.yaml file with the acl_config key.

For this to work the ACLs will need to be copied by Puppet from Puppet
to the Gerrit host. Add the file resource to do this as well.

Change-Id: I15a1ec13b381dce3c115c01c21f404ab79e72cc4
Reviewed-on: https://review.openstack.org/15352
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-11-14 01:18:23 +00:00
Jeremy Stanley
d1f76c1b99 Undo problematic line continuations and ignore.
* Rakefile: Override line length warnings from puppet-lint with its
disable_80chars option.

* modules/gerrit/manifests/cron.pp, modules/gerrit/manifests/init.pp,
modules/gerrit/manifests/remotes.pp, modules/lodgeit/manifests/site.pp,
modules/openstack_project/manifests/cacti.pp: Undo line continuations on
long strings. These were causing particular problems when attempting to
apply crontab entries.

Change-Id: I417788d7953ee0d2b717349564ee9cc78c0c49c2
Reviewed-on: https://review.openstack.org/15822
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2012-11-12 18:30:14 +00:00
Matthew Wagoner
dbf0221db6 Cleanup gerrit manifest lint errors.
Change-Id: I5a52c0fd0f5a35c32aa71c0f93500aa59e495066
Reviewed-on: https://review.openstack.org/14910
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-30 16:35:35 +00:00
Clark Boylan
a14e20e900 Have hiera and puppet manage gerrit ssh:29418 keys
Have hiera and puppet manage gerrits ssh:29418 keys (RSA and DSA). These
keys go in /home/gerrit2/review_site/etc.

Change-Id: If8cb3ec5a2e2c582b7fa6d87c520fc0cb7c2f205
Reviewed-on: https://review.openstack.org/14365
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-11 23:21:01 +00:00
Clark Boylan
16ba76b01d Fix Gerrit MySQL DB.
I hope. The grant parameter appears to want an array so give it one.
Also enforce order by requiring the mysql server and account security
settings in the DB resource.

Change-Id: I2c99c25cb09cb5b68240a5fbd146f47ba8aee410
Reviewed-on: https://review.openstack.org/14320
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-10-11 19:12:40 +00:00
Clark Boylan
3a6437a21b Ensure destination dir for bcpg link is present.
The destination dir for the bcpg link needs to be present before the
link can be made. Add that dir to the gerrit init manifest and require
it in the link file resource.

Change-Id: I462cc96dcd0eafa814e3e3599a96eacc64665bcf
Reviewed-on: https://review.openstack.org/14319
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-11 19:03:09 +00:00
Clark Boylan
09152c2dfd Cleanup gerrit init.pp manifest lint errors.
Change-Id: I68c6cd9b24c93f9f1cc2ba92eceae49b3c38ed36
Reviewed-on: https://review.openstack.org/14176
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-10-10 15:00:14 +00:00
Clark Boylan
0a61d5c434 Set ssl keys group to ssl-cert.
Recent ssl cert management changed the group on the ssl keys to root
from ssl-cert. Change it back.

Change-Id: I6dcbeca364fa9c435aee520248a59f0917cd02a8
Reviewed-on: https://review.openstack.org/14116
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-05 23:24:50 +00:00
Clark Boylan
15e526fb18 Pass review.o.o SSL certs in from Hiera.
Use Hiera to store the review.o.o SSL certs and pass them down to the
gerrit module.

While modifying these files fix indentation and rocket ship alignment
according to puppet lint in the sections touched.

Change-Id: I914b0dea72c77dedb44a4e6f51417985e673b315
Reviewed-on: https://review.openstack.org/13975
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-05 22:50:49 +00:00
Jeremy Stanley
ad56ee88c2 Another Gerrit Contact Store fix for review-dev.
* modules/gerrit/manifests/init.pp: The file block for the bcpg.jar
symlink should require the libbcpg-java package rather than the jarfile
it installs.

Change-Id: Icf4356c51425a816aea523f835e8bc7c62055b28
Reviewed-on: https://review.openstack.org/13392
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-09-20 19:24:53 +00:00
Jeremy Stanley
25d4e6f4c9 Enable Gerrit CLA and Contact Store on review-dev.
This replaces the previous Echosign+Launchpad+Wiki+approver-based
asynchronous contributor license agreement signing process with a
fully-automated one contained entirely within Gerrit itself.

Note that the CLA features in Gerrit's WebUI depend on a modified
gerrit.war with an earlier patch reverted:

    https://review.openstack.org/12716

* manifests/site.pp(review-dev.openstack.org): Fill contactstore_appsec
and contactstore_pubkey private material from hiera, for use by Gerrit's
contact store feature. Similar entries should be added for
review.openstack.org before going into production.

* modules/gerrit/manifests/init.pp(gerrit): Add contactstore,
contactstore_appsec and contactstore_url variables needed by the
gerrit.config.erb template, and contactstore_pubkey needed by the
contact_information.pub.erb template. Add a conditional block so that if
contactstore is enabled it installs the libbcpg-java package which
Bouncy Castle needs for OpenPGP operations, links the bcpg.jar into
Gerrit's lib directory, and builds contact_information.pub from the
contact_information.pub.erb template.

* modules/gerrit/templates/contact_information.pub.erb: New template
which is effectively an empty file waiting to be filled with the
contents of the contactstore_pubkey variable. The
gerrit_contact_information.pub file built from it gets used to encrypt
contact information filed by users in such a way that it can only be
decrypted by the private key held by the Foundation.

* modules/gerrit/templates/gerrit.config.erb(contactstore): New section,
implemented conditionally for safety. Once enabled, if the
contactstore_appsec and contactstore_url are unset then Gerrit will
refuse to start. If the system referred to by contactstore_url is
unresponsive or contactstore_appsec does not contain the shared secret
it's expecting, contributors will be unable to file initial or updated
contact information through Gerrit's WebUI.

* modules/openstack_project/files/gerrit/cla.html: A stripped-down HTML
copy of http://wiki.openstack.org/CLA retaining all the original
wording. This will probably need updating by OpenStack Foundation staff.

* modules/openstack_project/manifests/gerrit.pp
(openstack_project::gerrit): Add contactstore, contactstore_appsec,
contactstore_pubkey and contactstore_url variables to pass back into the
gerrit module. Also define the cla_description, cla_file, cla_id and
cla_name variables which get used in the gerrit_set_agreements.sh.erb
template. Add an entry to install the cla.html file.

* modules/openstack_project/manifests/review_dev.pp
(openstack_project::review_dev): Add the contactstore_appsec and
contactstore_pubkey variables so they can be filled in by hiera.
Override the war to pull in the g69c8fa6 test build which has the
aforementioned CLA bits restored. Turn on contactstore and set
contactstore_url to point to an existing test CGI on the Internet until
the Foundation has theirs ready. Pass contactstore_appsec and
contactstore_pubkey through up into gerrit.pp. Add an entry for the
set_agreements.sh script built from the gerrit_set_agreements.sh.erb
template and then execute it to add the new CLA to Gerrit's DB and mark
the old one expired. Similar changes should be made in review.pp before
going into production.

* modules/openstack_project/templates/gerrit_set_agreements.sh.erb: New
template used to build a set_agreements.sh script which checks Gerrit's
database and, if necessary, expires the old Echosign CLA and adds the
new local CLA. These conditions are checked and associated operations
performed independently, so subsequent runs become a no-op.
Post-migration, this can probably be neutered further and kept around
for pushing future CLA modifications into the database when needed.

Change-Id: Ib7136fef23dbd5602955649b33a57bc8d7106026
Reviewed-on: https://review.openstack.org/13058
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-09-20 15:15:42 +00:00
Monty Taylor
7d8c838038 Align all web server usage on apache module.
Change-Id: Idd712a8ee5ec81c6b88b7d3e2270dce4da254927
Reviewed-on: https://review.openstack.org/10838
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-12 17:03:46 +00:00
Monty Taylor
339f4e115f Use account_security to remove bogus accounts.
Change-Id: I20aa549cf87f24d13d302fc219df84727c9fba10
Reviewed-on: https://review.openstack.org/11041
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-10 14:33:17 +00:00
Monty Taylor
f774c793ba Add in final two steps for initial install.
With these applied, initial install actually works 100% through.

Change-Id: I6587a537beb5703bf11783f3df79278ea1c7aca5
Reviewed-on: https://review.openstack.org/10718
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-03 23:14:26 +00:00
Monty Taylor
e63619e980 Make gitweb a boolean option.
Change-Id: Iff55f35c0d9888f1029115c17d4644a68d4e8b4c
Reviewed-on: https://review.openstack.org/10727
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins
2012-08-03 23:12:38 +00:00
Monty Taylor
383c023b15 Add support for initial project creation.
If replicate_local is set, this will ensure that /var/lib/git is created,
and that projects listed in the projects.config have repos there.

Additionally, it creates a new config file, projects.config which is a
yaml file listing all of the projects and various operational semantics about
them, such as whether or not they should have pull requests closed and whether
or not they track any remotes. This replaces remotes.config and github.config.

Moving forward, there is no reason to not have this script be able to
do github api calls to create the github repo if it's not there, set the
github project description, gerrit api calls to create the project in gerrit,
and initial project permissions templates.

Change-Id: I1ad803b0aa5f7386206d0c3f4cd858017242fe64
2012-08-02 15:49:02 +00:00
Monty Taylor
ef3881d487 Set some parameters to more sensible defaults.
Also, moved depends for launchpad sync script to the launchpad sync
module, and put the ntp stuff into an ntp module.

Change-Id: I2568752493fefa305f9108a23da101d80a311552
2012-07-29 13:04:17 -05:00
Monty Taylor
0b921968d1 Added an option to toggle replication choices.
New options for github replication and local replication.

Change-Id: I06a6ca5347232ec80e26f6116742ab0007435ffe
2012-07-29 13:04:17 -05:00
Monty Taylor
279b406a4b Add natty entry for java_home.
Change-Id: I0c2605c7dfd2ccff3d966ee4e4e8067a91eb1b98
2012-07-28 10:54:35 -05:00
Monty Taylor
46282d44c1 Fixed a template typo.
Variable interpolation needs <%= not just <%. :)

Also, while I was in there, I replaced default with "oneiric",
because I don't actually know that the value is a good default value,
and I removed a couple of comments about moving to MySQL and Apache
modules from upstream.

Change-Id: Iec5b10cee2cbd0e0a2573fefa707d34d2a363cb4
2012-07-28 10:04:43 -05:00
Monty Taylor
c510a30e1e Modified gerrit to use MySQL and Apache modules.
Change-Id: I82ff3c46438f8db126fa6a881efa09c90b1906e4
2012-07-27 17:37:37 +00:00
Monty Taylor
2805fed59f Fix the update_users cron job.
Change-Id: Icb546b9c48f2618e1b2269d82b34cc5588bc2624
2012-07-25 13:43:40 -05:00
Monty Taylor
dbd4da09a7 Fix all of the gerrit module problems.
Change-Id: I46001fd677bc9a3634c9860ec07438c326e908e3
2012-07-24 00:28:04 -05:00
Monty Taylor
5609c50cd5 Move OpenStack branding and launchpad integration.
Launchpad integration and the OpenStack branding files are really
more about the OpenStack specific install of Gerrit than they are
about any installation of gerrit. Both of these are moved to the
openstack_project module.

Change-Id: I8b281aa5cb751a8023c2101c45146a3aca5f90f3
2012-07-23 14:46:55 -05:00
Monty Taylor
1b61f7673b Split gerrit cron jobs out.
Change-Id: I53faafc4d692c3dc62fd3356fd39f6e2ce64a481
2012-07-23 11:42:18 -05:00
Monty Taylor
8f1adc171f Split github into its own module.
TODO: Add another script that sets the project description. Add the project
description to the config hash.

Change-Id: If4584b2a1e55e6eb912e1f557e31de216d49a516
2012-07-23 11:42:12 -05:00
Monty Taylor
6173771627 Move comment link default values to openstack.
Change-Id: I015c8601c73a541007de5d8127e8d9305c320c6c
2012-07-23 10:33:59 -05:00
Monty Taylor
4774c02153 Split gerrit remote fetcher into a module.
Change-Id: I85fd220da7105363471a1e67fec86cd25dac95c1
2012-07-23 10:33:59 -05:00
Monty Taylor
e3e9aaba10 Split gerritbot into its own module.
Change-Id: Ia9034d87321624006ccf447b058ee828a97fcc7a
2012-07-23 10:33:59 -05:00
Monty Taylor
d025dca604 First pass at parameterizing secret infos.
Change-Id: Iee56a7e65be51ebf19a61eefd60cc93de6a764bf
2012-07-20 14:40:42 -07:00
Monty Taylor
09116ced39 Tarballs site has moved.
Change-Id: I9bca1fcb1692d139a397f77edbb11e231057054b
2012-07-06 12:48:38 -05:00
Clark Boylan
bc2448199a Allow GerritBot to talk on multiple channels.
Fixes bug #1020987

Update GerritBot with the ability to talk on multiple channels. This
way a single GerritBot instance can operate in multiple channels for
multiple projects. To make this work this change introduces a new
channel configuration file (yaml) for GerritBot that specifies each
channel that GerritBot should join and the changes that channel is
interested in.

The config should look something like:
channel-foo:
    events:
      - patchset-created
      - change-merged
    projects:
      - test/bar
      - test/foo
    branches:
      - master

Change-Id: I8e278f9be5182611981a3d912cc323bd3d386fc5
2012-07-05 15:02:25 -07:00
James E. Blair
2dd8ebe53b Add httpd_maxwait parameter to gerrit module.
Was missing from earlier change that added it to the invocation.

Change-Id: I71d9b62e1b959fbd9d87f21800d47606be31d95a
2012-07-03 06:39:24 -07:00
Clark Boylan
17c836ab28 Enable Melody on review-dev.
Review-dev is running a Melody capable WAR, but the Melody service
was not enabled. Enable it.

Change-Id: I68934151a38c2572f37d996e1c923a880ab9f50f
2012-06-25 13:26:46 -07:00
Clark Boylan
b9ebb81800 Update Github request closer script to v3 of API.
Fixes bug #1012310

Github disabled v1 and v2 of their API permanently, forcing us to
update the script that closes Github pull requests to v3 of the API.
Update the script using the PyGithub lib.

Change-Id: I90c9faacdb7a72a470b8ad6aaea674edd9b8329e
2012-06-13 10:14:36 -07:00
James E. Blair
33f0963048 Fix gerrit group permissions.
review.openstack.org will restart on every puppet run otherwise.

Change-Id: I999f279dec016f201390d47ab6f2ca3d412c86eb
2012-06-06 10:06:46 -07:00
James E. Blair
ae0f98e0cd Use unattended upgrades.
Stop using latest for packages installed by puppet.  This way,
all system packages get updated, not just some random ones.

The unattended-upgrades config will email root.  It is configured
for openstack servers and jenkins slaves, but not template hosts
so that it doesn't interfere with spin-up.

Also, fix some bits in the gerrit module that were causing
continuous restarts on gerrit-dev.

Install emacs.

Change-Id: I51c9083ccd3669f284fce4b50c36a37a0cac92d8
2012-06-05 22:59:46 +00:00
James E. Blair
0082fa9c49 More gerrit tuning.
Increase the heap size and dramatically increase the ssh threads.
Add some more recommended parameters (see site manifest for details).

Parameterize tunables in gerrit config file.

Change-Id: Ia6446b29426f56a77425eed93a7f0e448c3cd7b1
2012-05-29 18:15:27 +00:00
Monty Taylor
f5c0cbb07f Remove refreshonly - it's only for exec types.
Change-Id: I5340d4bd3d1b2998d67ec1fcbcb48837e47bdac0
2012-05-22 11:52:37 -04:00
Monty Taylor
d2d85fff67 Restart gerritbot if we push new versions.
Change-Id: I767a52f5b25aff0e64acd4b0f75dc522256b7af3
2012-05-21 12:41:06 -04:00
Andrew Hutchings
ab8c2a7ba7 Add parameters for scripts that talk to Gerrit
Not all gerrit installations are configured the same.  This adds parameters
to the gerrit sync script and expire script so that it can work from the
command line and cron on all servers.

Fixes bug #994201

Change-Id: Iac37e18aaf7a10fb5594be233e33b000eabed336
2012-05-17 15:16:29 +01:00
Monty Taylor
6fdebcf5af Adds support for tracking an upstream repos.
Change-Id: I3c92c19cf2c2fd6a911fbdbf4aeeb66e53cf07e0
2012-05-10 13:12:31 -04:00
Monty Taylor
993563afe5 Install gerritbot files from puppet.
Change-Id: Ife98fa24e731bcbbe719f583b1788870433b7f10
2012-05-03 08:25:56 -07:00
David Shrewsbury
2b7496c0ee Compress database backups.
Fixes bug 951600.

The update_gerrit_users.py script creates a database backup
everytime it runs. This change compresses the backups as they
are being made and adds a cron job that will remove backups that
are 30 days or older. Current backups will need to be manually
compressed.

Change-Id: Idb90d4af629892f1bc2e3e98f35f7de2ad58d4be
2012-05-03 09:35:45 -04:00
Monty Taylor
af48c6d986 Move gerrit scripts to puppet.
Take the things from openstack-ci/gerrit and move them directly
in to the puppet module. Install them using the model we're using
for the jenkins slave scripts.

Change-Id: I420b2b895bd57d40232b2cdda437617373a82890
2012-05-01 17:31:05 -04:00
James E. Blair
69bc0c1061 Fix gerrit config file permissions.
Match what gerrit init creates; otherwise, gerrit init will
delete and recreate the secure config file, losing the database
password in the process.

Change-Id: Ic1632fe3b24a0e4498b2415029e8a1db0fd1dfe2
2012-04-24 07:36:24 -07:00