5bbe7824e0
Logstash performs filtering in a single thread so it does not scale up very well. Work around this by scaling Logstash out to multiple indexer hosts. Current plan is to have a small (2GB) kibana web front end host that does nothing but talk to elasticsearch, three 4GB logstash indexers that will run a single log-pusher.py + logstash indexer with some partition of the logfiles assigned to each indexer, and finally the existing large elasticsearch node. Eventually properly load balancing log processing across the worker nodes would be great, but the current partition method should work well enough with little additional effort. Change-Id: Ifc6396560934314ffd6a7c47eb2acff9e9c2a7af Reviewed-on: https://review.openstack.org/30573 Reviewed-by: James E. Blair <corvus@inaugust.com> Approved: Jeremy Stanley <fungi@yuggoth.org> Reviewed-by: Jeremy Stanley <fungi@yuggoth.org> Tested-by: Jenkins
These are a set of puppet manifests and modules that are currently being used to manage the OpenStack Project infrastructure.
The main entry point is in manifests/site.pp.
In general, most of the modules here are designed to be able to be run either in agent or apply mode.
These puppet modules require puppet 2.7 or greater. Additionally, the site.pp manifest assumes the existence of hiera.
See http://ci.openstack.org for more information.