4630 Commits

Author SHA1 Message Date
Clark Boylan
e6d78af997 Add a simple watchdog for logstash-indexer service
The logstash-indexer services sometimes fall out of the elasticsearch
cluster when it is under heavy load and unable to respond to pings.
Logstash doesn't do anything to reconnect :( so restart the service if a
node detects that it has fallen out of the cluster.

Upstream bug submitted at https://logstash.jira.com/browse/LOGSTASH-1951

Change-Id: I2e7767c5fe20cff279366fec2ddadd7710dbb4a9
2014-02-28 13:53:53 -08:00
Jenkins
f45fe600fb Merge "Retrieve ATC project list from governance repo" 2014-02-28 17:20:57 +00:00
Jeremy Stanley
9822dd517e Retrieve ATC project list from governance repo
* tools/atc/email-stats.sh: We now have an official list of projects
in the openstack/governance project, so use it instead of the old
hard-coded list.

Change-Id: If93c71174687ac7e4e46d514642c48edb841adfe
2014-02-28 16:50:29 +00:00
Jenkins
9edce3fd9a Merge "modified stackforge/milk project in zuul/layout.yaml" 2014-02-28 00:29:20 +00:00
Sean Roberts
6a37295a52 modified stackforge/milk project in zuul/layout.yaml
removed python tests from stackforge/milk. we
will add tests after we get some base code in
the repo.
removed the post section as well

Change-Id: If393edd3ccc1a7fe5534e479e81514e7c5358f54
2014-02-27 16:15:56 -08:00
Clark Boylan
688bdd459e Create new version-properties.sh script
Create a new version-properties.sh script that will be used on slave
hosts to determine version information from git. This is mostly a copy
of the maven-properties.sh script. The reason we are not simply renaming
is that this commit must be merged then all slave images updated before
we change our jenkins jobs to use the new script. Renaming doesn't allow
us to do that.

Change-Id: I37c0b7cc3ca034addf32897a5a8c6501dedf00b0
2014-02-27 15:11:42 -08:00
Clark Boylan
db20f1f27a Revert "rename maven-properties.sh script to version-properties.sh"
This reverts commit dea3bdb050a55d56a1913e50e59cf176d70d3eb7.

This revert is necessary because it changed jenkins jobs before the
slave images could be rebuilt. We need to update the slave images first
then update the jenkins jobs in a pair of changes that can be merged one
at a time.

Change-Id: Iae4abf8c2dd8ad67ad008c57bbd17ba1304134c6
2014-02-27 15:09:43 -08:00
Jenkins
880924a621 Merge "Optionally give mysql user all global privs." 2014-02-27 22:13:23 +00:00
Jenkins
3b26f2a3c5 Merge "fix scp reference to variable" 2014-02-27 21:27:43 +00:00
Jenkins
a56c077f2c Merge "rename maven-properties.sh script to version-properties.sh" 2014-02-27 19:31:36 +00:00
Jenkins
bf96db40f3 Merge "Change mysql package for Fedora >= 19 for bare-precise" 2014-02-27 19:24:49 +00:00
Jenkins
25c9ba79cd Merge "Fix logstash indexer upstart conf." 2014-02-27 19:18:10 +00:00
Clark Boylan
1986685e29 Fix logstash indexer upstart conf.
A line which should've been a comment in the upstart config for
logstash-indexers lost its '#' make it not a comment and a syntax error.
Make this line a comment again.

Change-Id: Idc4853db68a3ae25bea6ebee80f047cc087457ac
2014-02-27 11:06:52 -08:00
Jenkins
974819db58 Merge "Incrase heap size for logstash indexers." 2014-02-27 18:41:54 +00:00
Clark Boylan
4454aac98f Incrase heap size for logstash indexers.
The default java heap size is too small for our logstash indexers.
Double it to 2g. Do this by adding an /etc/default/logstash-indexer file
that the upstart configs source if it is present. This required some
tweaks to the upstart configuration to load the defaults properly.

Co-Authored-By: K Jonathan Harker <k.jonathan.harker@hp.com>
Change-Id: I63447f59f3fa6d466a7d275476121fe8339479dc
2014-02-27 10:30:28 -08:00
Jenkins
8e82596ab1 Merge "Move primary elasticsearch discover node" 2014-02-27 18:06:16 +00:00
Jenkins
eb0fd1382f Merge "Update logstash doc for an elasticsearch cluster" 2014-02-27 17:52:54 +00:00
Jenkins
9744529740 Merge "Don't enable oslo-incubator py33 for stable/havana" 2014-02-27 12:11:20 +00:00
Khai Do
1cb96d781c fix scp reference to variable
JJB escapes double brackets only if it's used in a JJB template.  The reference is not
used in a template so changing it to single brackets.

Change-Id: Ibf728a58f3246461c77b57cf2c934b7257cf8ec8
2014-02-26 20:56:43 -05:00
Jenkins
72be2f94c1 Merge "Remove deprecated elasticsearch nodes from kibana" 2014-02-27 00:40:03 +00:00
Jeremy Stanley
9f0d0dfc83 Fix rsyslog duplicate declaration for git.o.o
It's already declared in the template class, which is included by
the git class.

Change-Id: I94b660f13f93a459a9c130804f0d9b3409055138
2014-02-27 00:12:55 +00:00
Jenkins
988cbc1d05 Merge "Add git05 to the git.openstack.org haproxy farm" 2014-02-26 23:41:37 +00:00
Jeremy Stanley
d90a9bc3fb Move primary elasticsearch discover node
Now that the new elasticsearch cluster members are joined, we need
to move the primary discover node from elasticsearch.openstack.org
to one of the new servers before we can remove it from service.

Change-Id: I79fe4e2154def6b0404b6620601f4a02b63fee8a
2014-02-26 22:00:08 +00:00
Jeremy Stanley
ac13b41bed Update logstash doc for an elasticsearch cluster
Change-Id: I57b84c87599f129d9c273e29ff6b1a48c56c115d
2014-02-26 22:00:07 +00:00
Jeremy Stanley
ccc39c3bb8 Remove deprecated elasticsearch nodes from kibana
Kibana seems to be the only module using this list, and by default
it will try to query any nodes in this list even if they've been
pulled from service. Preemptively remove them from its configuration
so that it will continue to perform queries correctly.

Change-Id: Iecdc3414dec7ed2816b5a68a3398b0643fccaf31
2014-02-26 22:00:06 +00:00
James E. Blair
f9c8016fce Revert "Set Zuul gear server logs to debug"
They are too large currently.

This reverts commit ec21ba78f6b3aeb192f9c1d7e8eb15f42d6cc86c.

Change-Id: I02b35765b113d2db746743474ee255b4939e5bb5
2014-02-26 13:15:54 -08:00
Jenkins
0573f6e574 Merge "Add git05 to cacti and gerrit replication" 2014-02-26 21:04:04 +00:00
Jeremy Stanley
cc4bbfcdbc Add git05 to the git.openstack.org haproxy farm
Once the git repositories on git05 are replicated from
review.openstack.org and confirmed in sync, merge this to add it to
the farm.

Change-Id: I6bc87957ff9ba7983c48ce156ab7658a9ab8a5ad
2014-02-26 20:40:29 +00:00
Andreas Jaeger
8e42cf083f Fix success-pattern usage for openstack-manuals
Change Ia8f60c8a4b9d1b18583366d83ddb82dc61bff9f5 added a generic
rule to use a success-pattern to point to docs-draft for built books.
The order here is important, the regex line must come before the
non-regex line so that the regex is applied in all cases.

Change-Id: Ife43544864a1084d82325ced1d4a61a9cb6e5fa1
2014-02-26 21:08:55 +01:00
Jeremy Stanley
701aca626e Add git05 to cacti and gerrit replication
Change-Id: I681fc1409cb61061c9c6d888b8e2ee7393db4b6e
2014-02-26 20:08:34 +00:00
Clark Boylan
54ad9496cd Optionally give mysql user all global privs.
One our single use unittest slaves we want to allow unittests to create
and destroy arbitrary mysql database schemas. The simplest way to do
this is to grant all priveleges to the test user on the mysql server
globally. This is safe because the nodes are thrown away post test.

Change-Id: Ic92ff2f9c752d955e6f82eb4b88330b5448cfa75
2014-02-26 11:48:59 -08:00
Jenkins
5fd7c43342 Merge "Add new elasticsearch cluster members to cacti" 2014-02-26 19:24:58 +00:00
James E. Blair
ec21ba78f6 Set Zuul gear server logs to debug
For real this time.

Change-Id: I86cca02cbe8282b95853dc9a140c1e0acb6e22c7
2014-02-26 09:45:10 -08:00
Jeremy Stanley
2bcf1f00ac Add new elasticsearch cluster members to cacti
Change-Id: I1dc1e746d6701e56c6ba8941e9927b165b1403fc
2014-02-26 16:04:26 +00:00
Jenkins
643032bfd7 Merge "Fix manila-tempest-job to use exported env var" 2014-02-26 15:49:56 +00:00
Jenkins
ebcc474464 Merge "Extract check/gate req jobs to check-requirements" 2014-02-26 15:38:14 +00:00
Sergey Lukjanov
fbe9b983b6 Extract check/gate req jobs to check-requirements
Change-Id: Id34c8d9c1bfb80a2dbd3fccfd850fa9af149a49d
2014-02-26 19:22:25 +04:00
Flavio Percoco
131b72fa7f Don't enable oslo-incubator py33 for stable/havana
Py33 gate was enabled for oslo-incubator as part of Ia38b9ad2d6b2180459653017b308841eb78bda89

This change, however, enabled the gate also for stable branches, which
was not the goal of the patch. This patch intends exclude this job for
stable/grizzly and stable/havana.

Change-Id: I15cfc2d7d567a3c45dff803ebac8eb372ff09379
2014-02-26 09:32:59 +01:00
Jeremy Stanley
c5bd6f0fcb Don't ensure elasticsearch service is running
The package maintainer scripts start elasticsearch when it gets
installed, but it seems to return control before the pidfile is
created which allows puppet to race for it and ultimately launch a
second daemon resulting in an OOM condition. It's configured to
start at boot anyway, so the puppet ensure running isn't buying us
much.

Change-Id: I1b374f90c198b9cb73d412fe63b439623f434d9a
2014-02-26 01:52:46 +00:00
Jenkins
16e42ff1fa Merge "Puppet the /etc/elasticsearch directory" 2014-02-26 00:56:45 +00:00
Jeremy Stanley
f25d388f4b Puppet the /etc/elasticsearch directory
* modules/elasticsearch/manifests/init.pp: Brown bag fix... the
package installation used to make this directory, but now we want to
put files in it prior to package installation.

Change-Id: Id3069999d59f965226bcae8f21ef7a93e4d44c72
2014-02-26 00:26:09 +00:00
Marton Kiss
4b6eecf506 Fix openstackid vhost override
Remove /srv/openstackid/w symlink file override by puppet, and
add deploy dependency on /etc/openstackid/environment.php.

Change-Id: Ib3874f48cad720d667861d90f76c515e014002c2
2014-02-26 01:20:55 +01:00
Jenkins
e4a7180522 Merge "Update some deprecated Puppet variable references" 2014-02-25 23:55:54 +00:00
Jenkins
b96b5bff2d Merge "Correct elasticsearch config dependency ordering" 2014-02-25 23:51:56 +00:00
Jenkins
4d315d62bf Merge "Add logstash job queue graph" 2014-02-25 22:40:16 +00:00
Jeremy Stanley
46c3c63def Correct elasticsearch config dependency ordering
* modules/elasticsearch/manifests/init.pp: The config files need to
be in place before package installation starts the service, and they
don't depend on the package itself anyway.

Change-Id: Iabc57e3091533eb5168c670f5c29d0d2dac9a1a2
2014-02-25 22:05:12 +00:00
Marton Kiss
c7bdb10bbd Openstackid deployment symlink override fix
Override existing config files with symlinks, like
bootstrap/environment.php.

Change-Id: Ibd4c81ec3b03c5c6860e5d9e0931fae7eedd8a1d
2014-02-25 22:22:35 +01:00
Jenkins
9aa0cc776c Merge "Enable puppetdb from puppetmaster" 2014-02-25 20:42:33 +00:00
Jenkins
62dfbdce48 Merge "Set puppetdb server to listen on all interfaces" 2014-02-25 20:11:42 +00:00
Jenkins
a31b9f1c53 Merge "Install and run elasticsearch on new workers" 2014-02-25 19:34:37 +00:00