Hervé Beraud 37411e792a Add doc/requirements
We need to specify doc requirements in doc/requirements.txt
to avoid problems with the pip resolver [1] for the release team [2][3].

Removing specific doc requirements from test-requirements.txt.

The problem here is that this repos haven't doc/requirements.txt file
and by default in this case zuul will use the test-requirements.txt file
to pull requirements [4].

This requirements file contains extra requirements like flake8 that
collided with those allowed in our job environment and so the new pip
resolver fails to install these requirements and the job exits in error.

This project meet the conditions leading to the bug.

Notice that I voluntarily added the doc directory even if no docs
are generated here because zuul will try to pull this requirements from
there first and the contained requirements are needed for reno but AFAIK
the releasenotes dir is ignored by zuul. c.f [4] for further details.

Bump a series of lower-constraints and requirements to work with new pip
resolver, testing with steps outlined at [5]

Fix Flake8 E741 [6]


Change-Id: Ic1504a18a780b0f517c5aa3dd3bfb04998d42e74
2021-01-18 10:55:41 +01:00
conf Set legacy_kafka_client_enabled = False on default 2020-07-14 15:51:31 +02:00
hacking Update hacking for Python3 2020-04-21 19:27:10 +02:00
kafka Remove six 2020-11-24 15:10:57 +01:00
repositories Remove six 2020-11-24 15:10:57 +01:00
tests Remove six 2020-11-24 15:10:57 +01:00
tools Add doc/requirements 2021-01-18 10:55:41 +01:00 Replace URLs with URLs 2019-06-03 15:33:15 +02:00 Refactor the python persister 2016-03-10 09:14:24 -07:00 Tool to migrate existing data to db per tenant 2019-09-27 08:31:16 +00:00 Update hacking for Python3 2020-04-21 19:27:10 +02:00 Pass version and description into oslo setup 2017-03-28 08:45:46 +02:00

Monasca Persister

A Monasca Persister written in Python.

Reads alarms and metrics from a Kafka queue and stores them in an InfluxDB database.


Note that this document refers to the Python implementation of the persister. For information regarding the Java implementation, see the in the root of this repository.

Package Installation

Monasca Persister should be installed from PyPI via pip. Use your distribution package manager, or other preferred method, to ensure that you have pip installed, it is typically called python-pip.

e.g. For Debian/Ubuntu:

sudo apt-get install python-pip

Alternately, you may want to follow the official instructions available at:

Now, to install a particular released version:

sudo pip install monasca-persister==<version>

If using InfluxDB, the persister requires the necessary Python library to be installed. This can also be achieved via pip:

sudo pip install influxdb

Alternatively, pip can be used to install the latest development version or a specific revision. This requires you have git installed in addition to pip.

sudo apt-get install git
sudo pip install git+<revision>#egg=monasca-persister

The installation will not cause the persister to run - it should first be configured.


Using the persister requires that the following components of the Monasca system are deployed and available:

  • Kafka
  • Zookeeper (Required by Kafka)
  • InfluxDB

If running the persister as a daemon, it is good practice to create a dedicated system user for the purpose, for example:

sudo groupadd --system monasca
sudo useradd --system --gid monasca mon-persister

Additionally, it is good practice to give the daemon a dedicated working directory, in the event it ever needs to write any files.

sudo mkdir -p /var/lib/monasca-persister
sudo chown mon-persister:monasca /var/lib/monasca-persister

The persister will write a log file in a location which can be changed via the configuration, but by default requires that a suitable directory be created as follows:

sudo mkdir -p /var/log/monasca/persister
sudo chown mon-persister:monasca /var/log/monasca/persister

Make sure to allow the user who will be running the persister to have write access to the log directory.


There is minimum amount of configuration which must be performed before running the persister. A template configuration file is installed in the default location of /etc/monasca/monasca-persister.conf.

Note that the configuration will contain authentication information for the database being used, so depending on your environment it may be desirable to inhibit read access except for the monasca-persister group.

sudo chown root:monasca /etc/monasca/monasca-persister.conf
sudo chmod 640 /etc/monasca/monasca-persister.conf

Most of the configuration options should be left at default, but at a minimum, the following should be changed:

uri = <host1>:<port1>,<host2>:<port2>,...

uri = <host1>:<port1>,<host2>:<port2>,...

uri = <host1>:<port1>,<host2>:<port2>,...

database_name =
ip_address =
port =
user =
password =


The installation does not provide scripts for starting the persister, it is up to the user how the persister is run. To run the persister manually, which may be useful for troubleshooting:

sudo -u mon-persister \
  monasca-persister \
  --config-file /etc/monasca/monasca-persister.conf

Note that it is important to deploy the daemon in a manner such that the daemon will be restarted if it exits (fails). There are a number of situations in which the persister will fail-fast, such as all Kafka endpoints becoming unreachable. For an example of this, see the systemd deployment section below.

Running (systemd)

To run the persister as a daemon, the following systemd unit file can be used and placed in /etc/systemd/system/monasca-persister.service:

Description=OpenStack Monasca Persister

ExecStart=/usr/local/bin/monasca-persister --config-file /etc/monasca/monasca-persister.conf


After creating or modifying the service file, you should run:

sudo systemctl daemon-reload

The service can then be managed as normal, e.g.

sudo systemctl start monasca-persister

For a production deployment, it will almost always be desirable to use the Restart clause in the service file. The RestartSec clause is also important as by default, systemd assumes a delay of 100ms. A number of failures in quick succession will cause the unit to enter a failed state, therefore extending this period is critical.


(C) Copyright 2014-2016 Hewlett Packard Enterprise Development Company LP

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.