openstack/openstack-ansible-ops
Code Issues Proposed changes
eb29d99642
openstack-ansible-ops/elk_metrics
History
Kevin Carter 17fb37f075
Update elk 6.x playbooks 
Most of the changes in this PR are for style and to adapt the playbooks
so that the system can operate on a multi-node cloud.

Functional change includes the removal of mainline Java 8 in favor of
OpenJDK 8.

A site playbook was add to allow an operator to just run everything.

Old tools that no longer function within the stack have been removed.

Packetbeat was added to the install list
Auditbeat was added to the install list

All of the config files have been updated for the recent ElasticStack
6.x changes.

Change-Id: I01200ad4772ff200b9c5c93f8f121145dfb88170
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-04-11 03:11:44 -05:00
..
conf.d Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
env.d Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
templates Update elk 6.x playbooks 2018-04-11 03:11:44 -05:00
vars Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
installElastic.yml Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
installKibana.yml Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
installLogstash.yml Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
installTopbeat.yml Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
loadKibana.yml Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
readme.rst Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00
reverseProxyKibana.yml Added playbooks to install ELK stack with topbeat 2017-12-20 13:27:22 +00:00

readme.rst

install Elk stack with topbeat to gather metrics

tags

openstack, ansible

About this repository

This set of playbooks will deploy elk cluster (Elasticsearch, Logstash, Kibana) with topbeat to gather metrics from hosts metrics to the ELK cluster.

Process

Clone the elk-osa repo

cd /opt
git clone https://github.com/openstack/openstack-ansible-ops

Copy the env.d file into place

cd openstack-ansible-ops
cp env.d/elk.yml /etc/openstack_deploy/env.d/

Copy the conf.d file into place

cp conf.d/elk.yml /etc/openstack_deploy/conf.d/

In elk.yml, list your logging hosts under elastic-logstash_hosts to create the elasticsearch cluster in multiple containers and one logging host under kibana_hosts to create the kibana container

vi /etc/openstack_deploy/conf.d/elk.yml

Create the containers

cd /opt/openstack-ansible-playbooks
openstack-ansible lxc-containers-create.yml -e 'container_group=elastic-logstash:kibana'

install master/data elasticsearch nodes on the elastic-logstash containers

cd /opt/openstack-ansible-ops
openstack-ansible installElastic.yml -e elk_hosts=elastic-logstash -e node_master=true -e node_data=true

Install an Elasticsearch client on the kibana container to serve as a loadbalancer for the Kibana backend server

openstack-ansible installElastic.yml -e elk_hosts=kibana -e node_master=false -e node_data=false

Install Logstash on all the elastic containers

openstack-ansible installLogstash.yml

InstallKibana on the kibana container

openstack-ansible installKibana.yml

(Optional) Reverse proxy kibana container to your loadbalancer host

openstack-ansible reverseProxyKibana.yml

load topbeat indices into elastic-search and kibana

openstack-ansible loadKibana.yml

install Topbeat everywhere to start shipping metrics to our logstash instances

openstack-ansible installTopbeat.yml --forks 100