opendev/system-config
Code Issues Proposed changes
5bbe7824e0
system-config/modules/openstack_project/manifests
History
Clark Boylan 5bbe7824e0 Scale out logstash indexing to multiple hosts. 
Logstash performs filtering in a single thread so it does not scale up
very well. Work around this by scaling Logstash out to multiple indexer
hosts.

Current plan is to have a small (2GB) kibana web front end host that
does nothing but talk to elasticsearch, three 4GB logstash indexers that
will run a single log-pusher.py + logstash indexer with some partition
of the logfiles assigned to each indexer, and finally the existing large
elasticsearch node.

Eventually properly load balancing log processing across the worker
nodes would be great, but the current partition method should work well
enough with little additional effort.

Change-Id: Ifc6396560934314ffd6a7c47eb2acff9e9c2a7af
Reviewed-on: https://review.openstack.org/30573
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2013-05-29 00:33:36 +00:00
..
automatic_upgrades.pp Update automatic_upgrades to support RHEL/Centos. 2013-05-17 17:21:37 +00:00
backup_server.pp Cleanup openstack_project manifest lint errors. 2012-11-21 17:48:11 +00:00
bare_slave.pp Style Guide updates for puppet-lint 2012-09-27 14:59:57 +00:00
base.pp Use facter's osfamily instead of operatingsystem. 2013-04-25 21:37:03 +00:00
cacti_device.pp Make cacti_device a define. 2012-10-30 13:37:58 +00:00
cacti.pp Add graphite.o.o to the list of cacti'd servers. 2013-04-25 20:14:35 +00:00
community.pp Style Guide updates for puppet-lint 2012-09-27 14:59:57 +00:00
dashboard.pp No longer include mysql::server for dashboard. 2013-05-21 23:49:02 +00:00
dev_slave_template.pp Add jenkins dev slaves. 2013-05-20 17:34:27 +00:00
devstack_launch_slave.pp Add jenkins ssh private key to devstack-launch. 2013-03-18 16:08:07 +00:00
eavesdrop.pp Add nicks parameter to statusbot. 2013-03-29 22:46:56 +00:00
elasticsearch.pp Scale out logstash indexing to multiple hosts. 2013-05-29 00:33:36 +00:00
etherpad_dev.pp add ep_headings option to etherpad install 2013-04-08 19:00:46 +00:00
etherpad.pp Cleanup openstack_project manifest lint errors. 2012-11-21 17:48:11 +00:00
gerrit.pp Extend manage-projects exec timeout. 2013-04-08 17:06:39 +00:00
graphite.pp Add graphite. 2012-12-02 20:28:38 +00:00
groups.pp Add groups.openstack.org. 2012-11-20 01:34:55 +00:00
init.pp Add jenkins dev slaves. 2013-05-20 17:34:27 +00:00
jenkins_dev.pp Add jenkins dev slaves. 2013-05-20 17:34:27 +00:00
jenkins.pp Scale out logstash indexing to multiple hosts. 2013-05-29 00:33:36 +00:00
lists.pp Adding the list for OpenStack discussions in Spanish 2013-05-28 15:40:18 +00:00
logstash_worker.pp Scale out logstash indexing to multiple hosts. 2013-05-29 00:33:36 +00:00
logstash.pp Scale out logstash indexing to multiple hosts. 2013-05-29 00:33:36 +00:00
mirror26_slave.pp Add pypi mirror building slaves. 2013-03-25 20:26:10 +00:00
mirror27_slave.pp Add pypi mirror building slaves. 2013-03-25 20:26:10 +00:00
mirror_slave.pp Remove direct mirror_slave depends on python-yaml. 2013-05-09 04:27:21 +00:00
params.pp The facter osfamily of Ubuntu is Debian. 2013-04-25 21:37:00 +00:00
paste.pp Cleanup openstack_project manifest lint errors. 2012-11-21 17:48:11 +00:00
planet.pp Cleanup openstack_project manifest lint errors. 2012-11-21 17:48:11 +00:00
puppet_cron.pp Add update_pkg_list_cmd to openstack_project. 2013-03-05 20:55:27 +00:00
puppetmaster.pp Move puppetmaster git repo. 2012-12-16 17:53:53 +00:00
pypi_slave.pp Cleanup puppet parser warnings. 2012-12-28 16:52:17 +00:00
pypi.pp Update to multi-mirror jeepyb. 2013-03-18 20:50:37 +00:00
review_dev.pp Install gerrit-2.4.2-17 on review-dev. 2013-05-15 05:14:54 +00:00
review.pp Install gerrit-2.4.2-17 on review. 2013-05-15 05:29:03 +00:00
server.pp Remove whoopsie package from all servers. 2013-05-14 18:55:57 +00:00
slave_template.pp Cleanup openstack_project manifest lint errors. 2012-11-21 17:48:11 +00:00
slave.pp Add jenkins dev slaves. 2013-05-20 17:34:27 +00:00
ssl_cert_check.pp Add ssl-cert-check to cacti server 2013-05-26 02:57:11 +00:00
static.pp Expose releasestatus under status.o.o/release.new 2013-05-22 02:29:50 +00:00
summit.pp Open port 80 on summit.o.o and add ttx. 2013-02-08 03:12:00 +00:00
template.pp Switch to puppetlabs-ntp. 2013-03-04 17:59:56 +00:00
tmpcleanup.pp Use facter's osfamily instead of operatingsystem. 2013-04-25 21:37:03 +00:00
translation_slave.pp Cleanup openstack_project manifest lint errors. 2012-11-21 17:48:11 +00:00
users.pp Add rockstar account to puppet 2013-05-23 22:31:15 +00:00
wiki.pp Add backups to wiki server. 2013-01-17 18:28:24 +00:00
zuul_dev.pp Rename zuul manifest to zuul_prod. 2013-05-13 22:08:47 +00:00
zuul_prod.pp Install the default zuul status page. 2013-05-17 17:07:20 +00:00