x
/
fuel-plugin-lma-collector
Code Issues Proposed changes
RETIRED, Fuel plugin to collect Logging Monitoring and Alerting metrics
489 Commits 1 Branch 32 Tags 39 MiB
Go to file
Simon Pasquier 821f90b55e Generate Hiera data as soon as possible 
The Hiera data from the LMA collector plugin is used by other plugins
such as the LMA Infrastructure Alerting plugin so it needs to be
available right after checking that the configuration is valid.

Change-Id: Id2902c5091a715f05ab936820991ec96d8b59962
 		2015-10-21 17:01:05 +02:00
contrib Fix ShellCheck warnings 2015-05-07 10:11:02 +02:00
deployment_scripts/puppet Merge "Create a Lua module for table functions" 2015-10-20 14:31:38 +00:00
doc Replaced LMA Collector by Elasticsearch-Kibana 2015-10-21 12:30:41 +02:00
repositories Add a tox environment to build the plugin 2015-09-08 18:28:28 +02:00
specs s/ElasticSearch/Elasticsearch/ where appropriate 2015-04-23 15:50:06 +02:00
.gitignore Add a tox environment to build the plugin 2015-09-08 18:28:28 +02:00
.gitreview Update .gitreview for new namespace 2015-10-17 22:24:08 +00:00
LICENSE Add appendix to LICENSE 2015-06-19 18:45:30 +02:00
README.md Modified README and User Doc 2015-10-20 16:22:10 +02:00
environment_config.yaml Switch to InfluxDB 0.9 mode 2015-08-17 11:12:25 +02:00
functions.sh Add Pacemaker Puppet modules 2015-08-14 17:07:04 +02:00
metadata.yaml Replace 'stackforge' references by 'openstack' 2015-10-20 12:13:32 +02:00
pre_build_hook Replace 'stackforge' references by 'openstack' 2015-10-20 12:13:32 +02:00
tasks.yaml Generate Hiera data as soon as possible 2015-10-21 17:01:05 +02:00
test-requirements.txt Add a tox environment to build the plugin 2015-09-08 18:28:28 +02:00
tox.ini Pass HOME variable to the test environments 2015-09-30 14:48:39 +02:00

README.md

Logging, Monitoring and Alerting (LMA) Collector Plugin for Fuel

The Logging, Monitoring & Alerting (LMA) Collector is a kind of advanced monitoring agent that should be installed on each of the OpenStack nodes you want to monitor. The Collector is a key component of the LMA Toolchain since it is individually responsible for supporting all the sensing, measurement, collection, analysis and computation functions for the node it is running on.

A wealth of operational data are collected from a variety of sources including the log files, collectd and RabbitMQ for the OpenStack notifications. The Collector, which runs on the active controller of the control plane cluster, is called the Aggregator because it performs additional aggregation and multivariate correlation functions to compute service healthiness metrics at the cluster level. An important function of the Collector is to sanitize and transform the ingested raw operational data into internal messages which uses the Heka message structure. This structure is used to match, filter and route certain types of messages to plugins written in Lua which perform the analysis and computation functions of the toolchain.

It’s main building blocks are:

  • collectd which is bundled with a collection of standard and purpose-built plugins for OpenStack.
  • Heka which is the swiss army knife we use for data processing.
  • A collection of Heka plugins written in Lua.

There are three types of Lua plugins running in the LMA Collector / Aggregator:

  • The input plugins to collect, decode, and sanitize the operational data that are transformed into internal messages which in turn are injected into the Heka pipeline.
  • The filter plugins to execute the alarms, the anomaly detection logic and the correlation functions.
  • The output plugins to encode and transmit the messages to external systems like Elasticsearch, InfluxDB or Nagios where the information is persisted or further processed.

The output of the Collector / Aggregator is of four kinds:

  • The logs and notifications that are sent to Elasticsearch for indexing. Elasticsearch combined with Kibana provides an insightful log analytic dashboards.
  • The metrics which are sent to InfluxDB. InfluxDB combined with Grafana provides insightful time-series analytic dashboards.
  • The health status checks that are sent to Nagios (or through SMTP) for all the OpenStack services and clusters of nodes.
  • The annotation messages that are sent to InfluxDB. The annotation messages contain information about what caused a cluster of services or a cluster of nodes to change a state. The annotation messages provide root cause analysis hints whenever possible. The annotation messages are also used to construct the alert notifications sent via SMTP.

Please check the LMA Collector Plugin for Fuel documentation for additional details.

Release Notes

0.8.0

  • Support for alerting in two different modes:
    • Email notifications.
    • Integration with Nagios.
  • Upgrade to InfluxDB 0.9.4.
  • Upgrade to Grafana 2.1
  • Management of the LMA collector service by Pacemaker on the controller nodes for improved reliability.
  • Monitoring of the LMA toolchain components (self-monitoring).
  • Support for configurable alarm rules in the Collector.

0.7.0

  • Initial release of the plugin. This is a beta version.

Requirements

The plugin's requirements are defined in the LMA Collector Plugin Documentation

Known issues

No known issues so far.

Limitations

The plugin is only compatible with OpenStack environments deployed with Neutron for networking.

Installation Guide

Please follow the installation instructions of the LMA Collector Plugin Documentation

User Guide

How to configure and use the plugin is detailed in the The LMA Collector Plugin Documentation

Communication

The OpenStack Development Mailing List is the preferred way to communicate with the members of the project. Emails should be sent to openstack-dev@lists.openstack.org with the subject prefixed by [fuel][plugins][lma].

Reporting Bugs

Bugs should be filled on the Launchpad fuel-plugins project (not GitHub) with the tag lma.

Contributing

If you would like to contribute to the development of this Fuel plugin you must follow the OpenStack development workflow.

Patch reviews take place on the OpenStack gerrit system.

Contributors