At present, all time series are accumulated in the same database in
InfluxDB. Separate database per tenant would make queries faster for
tenants. It would also allow administrators to define retention policy
per tenancy.
This changeset adds a migration tool and a guide to allow users to
migrate existing data to a database per tenant.
Task: 36435
Story: 2006331
Change-Id: I3faf3d96b1121c8b8c383dee2fda27d34af5c853
At present, all time series are accumulated in the same database in
InfluxDB. This makes queries slow for tenants that have less data. This
patch enables the option to use separate database per tenancy.
This task implements the changes on monasca-persister which handles
write requests to InfluxDB database.
Change-Id: I7de6e0faf069b889d3953583b29a876c3d82c62c
Story: 2006331
Task: 36072
InfluxdbClientError has content attribute no message attribute.
Message attribute deprecated in Python 2.6 and it's go away
in Python 3.0
Change-Id: I5fe4bcb018a7c8c99df29d9798aa0e2736f33b6f
Story: 2005900
Task: 33762
The change introduces the possibility to run the persister with
the new confluent-kafka client. It has to be enabled in the
configuration file.
Story: 2003705
Task: 30117
Depends-On: https://review.opendev.org/675297
Change-Id: I05428b8ae9e0ba9af5b81d3b103434ebd5657108
Cassandra Time to Live (TTL) is applied per-column in a
row, and with a default TTL for the table. The Cassandra
driver appears to have been written with a misunderstanding
that the TTL was per-row, and as a result the update
statements were written to avoid writing all cells.
But that results in some cells in the row not being updated,
and after their TTL expires they are removed, so a metric
query may result in partial data (updated_at set but
metric_id and created_at missing).
This fix changes the statement used when a new measurement
is received to use the same insert statement for the metric
row and thus refresh the TTL for all the columns.
A follow on patch should be created to remove the Update
statements for Measurements, and consider refactoring/removing
the caching model.
Change-Id: I6b7636a52e8bdb2ce8ad97f839acb6184cd58a8b
Story: 2005832
Task: 35872
The job has been failing for a long time. Java implementation of
InfluxDB persister is not maintained anymore.
Change-Id: I3756d10198172762dd964d89615c4b7da7b35304
Cassandra client version had to be changed because its earlier version
doesn't contain DCAwareRoundRobinPolicy and TokenAwarePolicy in
cassandra.cluster (those are located in cassandra.policies instead),
needed in persister (specifically cassandra/connection_util.py).
Change-Id: I9f9eb478808ad978815c19a5c4e1baaa35fa6d28
metrics table in cassandra always contain row which has no metric_id.
it is casused by cassandra ttl.cassandra supports TTL column only,
so when update a row,metric_id must update as well.
Change-Id: Ia2d2e1585965dd31f9679e2a63497bf4cc68349e
Story: 2005832
Task: 33590
It was found in some installations that python-cachetools needed
to be present on the system. This can happen if the installation
was configured for the Java mon-persister but the recreate-metric-id
tool is written in Python. Added a bullet item to describe this
dependency and the quick workaround of installing the
python-cachetools package.
Change-Id: I25635c069a5ed21342780a28b516c74b61dae30b
Story: 2005305
Task: 30611
It was discovered that the example paths included in the README.rst
used monasca-persister rather than monasca_persister.
Also added a footnote with two error messages that occur when the
contact_points is not specified correctly.
Story: 2005305
Task: 30611
Change-Id: I0c53af817504e638bd82401ca0b62874cb591f36
In some rare cases it is possible for a row in Cassandra for
metrics to have no value for metric_id or created_at, though
they may still have updated_at and the other required columns.
This tool is for recreating the metric_id from the other required
columns.
An additional 'persister-check-missing-metric-id.py' tool is
provided which can be run to see if there are missing metric-id
values that need to be recreated.
Please see the README.rst for usage directions.
Story: 2005305
Task: 30611
Change-Id: I0593558407c8c773d728bbd035dde91310b59be3
(cherry picked from commit 09af9bff91)
Bharat Kunwar