Monasca Database Persister
Go to file
James Gu 169f2c3ca3 Add performance testing in Monasca persister
Leverages the JMeter and KafkaMeter to inject into persister metric
messages with permutation of random metric names and dimensions.
Monitors the metrics flushed to the database in all persister
instances and calculates the throughput. The number of metrics and
the number of unique metic definitions are configurable in the test
plan. The script currently supports only the Java implementation of
the Monasca persister. Monasca python does not provide the same
internal metric api as the Dropwizard in the Java implemation.
Future work is required for the Python implemation.

Change-Id: Id8e6a5b62aa434d9943c7eee4be8991536b1c45f
story: 2001292
task: 5841
2018-02-12 09:53:13 +00:00
common Make legacy java gate working 2017-10-19 04:09:07 +00:00
config-generator Use oslo-config-generator 2017-07-28 13:48:01 +02:00
etc/monasca Use oslo-config-generator 2017-07-28 13:48:01 +02:00
java Java persister config: defaults and robustness 2018-01-10 17:31:19 +01:00
monasca_persister Merge "Allow persister to handle retention policies in influx" 2018-01-09 23:53:04 +00:00
perf Add performance testing in Monasca persister 2018-02-12 09:53:13 +00:00
tools Sync tools/ 2016-08-30 20:17:06 +02:00
.gitignore Add performance testing in Monasca persister 2018-02-12 09:53:13 +00:00
.gitreview Update .gitreview for new namespace 2015-10-17 22:30:59 +00:00
.stestr.conf Add .stestr.conf . 2017-09-22 14:13:18 +02:00
.testr.conf Add unit tests 2017-01-12 09:18:29 +00:00
.zuul.yaml Zuul: Remove project name 2018-01-28 15:25:24 -08:00
LICENSE Added copyright header and LICENSE file. 2014-05-01 12:45:08 -06:00 Optimize the link address 2017-04-11 11:59:32 +05:30
bindep.txt Add bindep.txt 2017-04-12 18:10:55 +02:00
pom.xml Upper pom version to 1.3.0 2017-12-19 13:11:41 -08:00
requirements.txt Updated from global requirements 2018-01-16 04:13:25 +00:00 Ensure the same branch is used for common build 2016-02-10 15:30:38 -07:00
setup.cfg Updated from global requirements 2017-11-16 11:10:56 +00:00 Updated from global requirements 2017-03-13 19:34:04 +00:00
test-requirements.txt Updated from global requirements 2018-01-16 04:13:25 +00:00
tox.ini Add support for persisting events 2017-11-09 08:57:17 +01:00

Team and repository tags

Team and repository tags


The Monitoring Persister consumes metrics and alarm state transitions from the Message Queue and stores them in the Metrics and Alarms database.

Although the Persister isn't primarily a Web service it uses DropWizard,, which provides a nice Web application framework to expose an http endpoint that provides an interface through which metrics about the Persister can be queried as well as health status.

The basic design of the Persister is to have one Kafka consumer publish to a Disruptor,, that has output processors. The output processors use prepared batch statements to write to the Metrics and Alarms database.

The number of output processors/threads in the Persister can be specified to scale to more messages. To horizontally scale and provide fault-tolerance any number of Persisters can be started as consumers from the Message Queue.


Requires monasca-common from Download and build following instructions in its Then build monasca-persister by:

mvn clean package


A sample configuration file is available in java/src/deb/etc/persister-config.yml-sample.

A second configuration file is provided in java/src/main/resources/persister-config.yml for use with the vagrant "mini-mon" development environment.


  • Purge metrics on shutdown
  • Add more robust offset management in Kafka. Currently, the offset is advanced as each message is read. If the Persister stops after the metric has been read and prior to it being committed to the Metrics and Alarms database, the metric will be lost.
  • Add better handling of SQL exceptions.
  • Complete health check.
  • Specify and document the names of the metrics that are available for monitoring of the Persister.
  • Document the yaml configuration parameters.


Copyright (c) 2014 Hewlett-Packard Development Company, L.P.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.