Monasca Database Persister
Go to file
Andreas Jaeger a9b647d743 Sync tools/tox_install.sh
Sync tools/tox_install.sh with python-openstackclient. This brings in
the following improvements:

* Do not leave temporary directory around, instead delete temporary
  directory directly after usage (change
  I939eae82dba3287fd4e4086128ebf4609a0e0770).
* Do not set ZUUL_BRANCH explicitely and remove unused if condition
  (change I0077c986a17d6bb92791474e03d1e77776e9382f).

Change-Id: I89b86e20d063f53338c87e76d9a39d600d35f3cd
2016-08-30 20:17:06 +02:00
common Ensure the same branch is used for common build 2016-02-10 15:30:38 -07:00
java Check value meta length before writing to Vertica 2016-08-19 12:27:20 -06:00
monasca_persister Merge "Revival of Cassandra repository" 2016-08-04 13:17:44 +00:00
tools Sync tools/tox_install.sh 2016-08-30 20:17:06 +02:00
.gitignore Add virtenv dir and contents to .gitignore 2015-08-31 08:32:11 -06:00
.gitreview Update .gitreview for new namespace 2015-10-17 22:30:59 +00:00
LICENSE Added copyright header and LICENSE file. 2014-05-01 12:45:08 -06:00
pom.xml Change repositories from stackforge to openstack 2015-10-19 09:25:20 +05:30
README.md Minor readme fixes 2015-12-16 15:52:11 -07:00
requirements.txt Revival of Cassandra repository 2016-08-04 02:01:51 +09:00
run_maven.sh Ensure the same branch is used for common build 2016-02-10 15:30:38 -07:00
setup.cfg Fix setup.cfg console_scripts 2016-07-29 12:54:15 +00:00
setup.py Add files for pypi. 2014-09-30 15:21:50 -06:00
test-requirements.txt Removed H302, H305 and H307 2016-06-30 22:22:23 +03:00
tox.ini Fixes H201: no 'except:' at least use 'except Exception:' 2016-07-29 10:22:02 +03:00

monasca-persister

The Monitoring Persister consumes metrics and alarm state transitions from the Message Queue and stores them in the Metrics and Alarms database.

Although the Persister isn't primarily a Web service it uses DropWizard, https://dropwizard.github.io/dropwizard/, which provides a nice Web application framework to expose an http endpoint that provides an interface through which metrics about the Persister can be queried as well as health status.

The basic design of the Persister is to have one Kafka consumer publish to a Disruptor, https://github.com/LMAX-Exchange/disruptor, that has output processors. The output processors use prepared batch statements to write to the Metrics and Alarms database.

The number of output processors/threads in the Persister can be specified to scale to more messages. To horizontally scale and provide fault-tolerance any number of Persisters can be started as consumers from the the Message Queue.

Build

Requires monasca-common from https://github.com/openstack/monasca-common. Download and build following instructions in its README.md. Then build monasca-persister by:

mvn clean package

Configuration

A sample configuration file is available in java/src/deb/etc/persister-config.yml-sample.

A second configuration file is provided in java/src/main/resources/persister-config.yml for use with the vagrant "mini-mon" development environment.

TODO

  • Purge metrics on shutdown
  • Add more robust offset management in Kafka. Currently, the offset is advanced as each message is read. If the Persister stops after the metric has been read and prior to it being committed to the Metrics and Alarms database, the metric will be lost.
  • Add better handling of SQL exceptions.
  • Complete health check.
  • Specify and document the names of the metrics that are available for monitoring of the Persister.
  • Document the yaml configuration parameters.

License

Copyright (c) 2014 Hewlett-Packard Development Company, L.P.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.