46 Commits

Author SHA1 Message Date
Kaiyan Sheng
014f14d9e7 Import simport from monasca-common
Change-Id: I6830a12c5cee7339ee02c55bad9ad4cd79338043
2016-06-17 14:32:32 -06:00
Michael James Hoppal
7c77ad3e44 Enable the ability to run multiple consumer processes at once
Change-Id: Ibed328d0b2bc11206d9975e84ce957305f8b973a
2016-03-11 09:34:45 -07:00
Michael James Hoppal
22e18453dd Remove cassandra repository
Change-Id: I914176d60bfce91fbe449702f7e78bb2f78706ce
2016-03-10 10:46:50 -07:00
Michael James Hoppal
5b1ffc3e2b Refactor the python persister
Separate out the persister classes into their own python files
and clean up some unused code for start up.

Change-Id: Id1460baef9e06025ed57030438a0021c253cc99a
2016-03-10 09:14:24 -07:00
Ryan Brandt
cb3ec75b80 Fix message parsing when link or lifecycle state is None
Change-Id: Ibd826ca4b2ddf2c1ae586b1168cbe642a1028609
2016-02-10 11:58:04 -07:00
Ryan Brandt
b544aa5ba8 Fix error if link or lifecycle state doesn't exist in transition
Change-Id: I2ef7b6b0fd9aa4d13c7e4bb23ee9e0e6574a618d
2016-02-05 08:54:47 -07:00
Ryan Brandt
7b4b41ff28 Pass link and lifecycle state in state transitions
Requires changes to monasca-thresh

Change-Id: I93113bc854629d45f34cdff9205ee97f353f999a
2016-01-27 14:27:08 -07:00
Deklan Dieterly
52b38a0dfb Enable Cassandra Database
Allow Cassandra Database to be used as backend database.

100% backward compatible with Influxdb.

Should not require any install or config changes to
use the default Influxdb.

Change-Id: I4cfe50a99eea0f8ae7421dcefbdd2cb2cc008b3b
2016-01-12 08:07:15 -07:00
Joe Keen
023646b3a2 Adjustments to support new oslo libraries
Removed the old openstack directory and replaced it with oslo libraries.

Updated requirements for liberty versions of oslo libraries.

Change-Id: I4d6bfb4251eb23cf86b3173ac9cb0b889ebbf674
2015-12-10 15:38:50 -07:00
Craig Bryant
b20d019a20 Lack of time field caused loss of Alarm History
The time field was name timestamp which caused influxdb to use the
write time for the time field. If multiple Alarm History messages
were written, Influxdb only kept the last because the times were
not unique.

Change-Id: I3808838bb8a26d7a6272ab41c3e586659a037f02
2015-11-19 10:53:09 -07:00
Joe Keen
3964f2aae2 Adapting persister because of changes to monasca common
Change-Id: I1a10e26340dcbd4dd634b32a52f554d073bfb946
2015-11-09 10:50:50 -07:00
Jenkins
b2479ae244 Merge "Modifying Persister to use common kafka interface" 2015-11-04 23:38:30 +00:00
Michael Bielinski
e87fbe3971 force measurement value to type float before writing to db
whole number values (such as 0) are json decoded as type int
If the value field is first written to InfluxDB as type int (example: 0)
and later as type float (example 0.123), InfluxDB throws a
"field type conflict" exception. Forcing all data to type float fixes.

Change-Id: Ifc139bbc67077137501ffe9045a988b8afc393e4
2015-11-02 12:30:43 -06:00
Joe Keen
0b461f047c Modifying Persister to use common kafka interface
Adding six to the requirements file to handle version conflict between the
influxdb and Kazoo libraries.

Added monasca-common to requirements and removed kafka and kazoo libraries

Change-Id: I1b390b52db6bc58f6eb9850ad799970c50e23695
2015-10-30 12:24:36 -06:00
Jenkins
d9436392e9 Merge "Store {} when value_meta is empty" 2015-10-14 16:01:45 +00:00
Deklan Dieterly
f88f71c51f Store {} when value_meta is empty
Always store {} when value_meta is empty for Influxdb.

Needed because the Java Influxdb driver is
capable of storing an empty string, but the
Python Influxdb driver is not.

Change-Id: Id6b7b795454afc1a4ab2e6230c6a9d3bd94790f7
2015-10-13 17:06:11 -06:00
Deklan Dieterly
9b1f5b60cc Change camelCase to snake_case in JSON
The Java Persister converts camelCase to snake_case for
the alarm state transition sub_alarms JSON element.

The python persister needs to do the same.

The following element names in the alarm state transition
sub_alarm element need to be converted to snake_case:

subAlarmExpression -> sub_alarm_expression
metricDefinition -> metric_definition
alarmState -> alarm_state

These elements originate from the Monasca Threshold Engine.
Ideally, the Threshold Engine should be modified to send
the JSON in snake_case.

Change-Id: I33c779e4741c797fbeeaf179441a65504444b0bd
2015-10-13 14:30:11 -06:00
Deklan Dieterly
075002df2f Fix problem where value_meta is null in JSON
Change-Id: Ie6d88227a9c2618e909e5ff5ead15fe997824215
2015-10-08 14:05:03 -06:00
Deklan Dieterly
fefff77a07 Convert empty string to empty JSON for sub alarms
Change 'name' to 'measurement' for alarm state transition.
Store empty JSON for reason data.

Change-Id: I66b0c05618734a978d2f8ef9c8123815ec7b1d85
2015-06-25 14:57:53 -06:00
Deklan Dieterly
906dee94b1 Fixes for Influxd 0.9 latest
change 'name' to 'measurement'.
change 'timestamp' to 'time'.
store value_meta as "" instead of "\"\"" when empty.

Change-Id: I391ab269897e6575abd9027b6b4252083a015536
2015-06-19 14:03:34 -06:00
Deklan Dieterly
417feb37f5 Include dimensions in measurement tags
Change-Id: I0073790397957dec9424a88b4dccdbbb1fbfe68e
2015-05-15 14:25:46 -06:00
Deklan Dieterly
77abfc4168 Migrate to Influxdb 0.9.0
Change-Id: I0119301dfd9544b68358eb04576b040f1f40ffb2
2015-05-13 10:34:46 -06:00
Roland Hochmuth
960529b171 Conversion to milliseconds
Change-Id: I9044899c212ff2ca93090e6d7b552456f616cfcf
2015-03-02 18:18:19 -07:00
Joe Keen
fc49e2aad3 Python persister batches data points with common name
Removed some checked in .pyc files

Change-Id: I39896e4669ca98378d1d6c5825405fe24e667093
2015-02-10 12:42:12 -07:00
David Scott
c081e3f65c Use standard Openstack config options for daemon
Bring monasca-persister daemon startup into sync with
other Openstack daemons with respect to config
management

Change-Id: I7a6b4367783a36fd2cefd0cb86a0abff15927811
Closes-Bug: 1414806
2015-01-26 17:16:38 -08:00
Deklan Dieterly
150dbb190f Move tenant id and region to front of serie name
Make serie name tenantId?region&metricName&dimName1=dimValue1&dimName2=dimValue2...

Change-Id: I1f8efbabe34548c080ccb932fc58e4f1e517042c
2014-12-09 15:04:56 -07:00
Deklan Dieterly
331ac31551 Add logging at info level when persister is waiting to acquire locks
Change-Id: Ia889f518df3f1ada15819cd7c82dfcd57555e1b4
2014-12-08 08:43:22 -07:00
Deklan Dieterly
f4231d41f4 Add high-availability
Allow persister instances to come and go. Automatically rebalance
reading the partitions amongst the existing persisters.

Change-Id: Ifd47459b0a29e91680e155d22b82086e04426215
2014-12-03 09:24:37 -07:00
Deklan Dieterly
a133bf4f59 Fix comment
Change-Id: Ie34c67b8f041536a2d8cbaa8490a08ef479f00ee
2014-12-01 13:16:11 -07:00
Deklan Dieterly
b20b721af9 Add kafka configuration values
Change-Id: I303c89449bb65c52178719bfdababb135a344fa7
2014-12-01 13:03:12 -07:00
Deklan Dieterly
a6e509cb81 Make info statement easier to read
Change-Id: I12cec09b7901ce1d20beef5fca6f04b8292d2a49
2014-11-25 10:12:02 -07:00
Deklan Dieterly
e5ed3afcac Fix issue with restarts rereading kafka messages
Change-Id: I54265a4fcfb70dcf53f8d7db14290e89f009aec1
2014-11-24 19:50:28 -07:00
Deklan Dieterly
55e433ad7c Add logging of message count info when messages sent to influxdb
Change-Id: I8e0082c86a88d8f3c6efe184d113d8c0cbcecbce
2014-11-21 09:08:01 -07:00
Deklan Dieterly
b4c0394d76 Add banner
Change-Id: I15310dac47460b28b745951f71870ca1703e4141
2014-11-20 10:43:03 -07:00
Deklan Dieterly
fac4f6c236 Set logging level to info
Change-Id: I991786de4a64d37c0061804f0df25fbeef5f28cf
2014-11-20 09:27:58 -07:00
Deklan Dieterly
ec24b5aee4 Fix shutdown problem
Make the python interpreter exit when any thread encounters an error.

Change-Id: I583f4afc67564390e3b2c74b85d82e77d8105312
2014-11-19 15:41:18 -07:00
Deklan Dieterly
c4bd85d6c4 Add more error handling
Change-Id: I353b9712eb9f5055eda9c20921c2554bd6427ee1
2014-11-19 10:50:42 -07:00
Deklan Dieterly
d919090a88 Fix bug JAH-865
Call sys.exit on exception from main loop.

Change-Id: Ie03acbd3b6f493aeb2288c4fbe0062738cc1b75e
2014-11-18 10:03:37 -07:00
Deklan Dieterly
ddf3f1b4a1 Make code more python-like
Remove uneeded parens.
Use only one underscore.
Remove repetitive statements.

Change-Id: Ifbb600bc61a978eb729bf97302b6f850625ecb0d
2014-10-08 17:10:24 -06:00
Deklan Dieterly
4ed35ac9ac Fix flush logic to remove unnecessary variable assignment
Change-Id: Ic5512018256e5307bbebcbd9d57d3c8d2a0db937
2014-10-06 08:55:24 -06:00
Deklan Dieterly
885112b605 Refactor code to remove duplication
Create an abstract super class to handle the logic in the run method.

Change-Id: I8f4f1598d724cdbffa08235995f055f42674941a
2014-10-03 14:52:15 -06:00
Deklan Dieterly
0482d1c1a0 Allow unicode in metrics column
Change-Id: I4df0e03d78eb5413ed74245da471cb010ca6e4cc
2014-10-02 14:27:43 -06:00
Deklan Dieterly
d3acf12034 Increase throughput of persister
Add batching to InfluxDB writes to improve throughput.

Change-Id: Ia173f55726cb11245f0bcf4580f1af8129c23aa3
2014-10-02 11:40:18 -06:00
Deklan Dieterly
a58acce9b4 Fix unicode error when logging.
Change-Id: I2400bc4e63934f118a4b1b32326585269677c085
2014-09-30 16:14:53 -06:00
Deklan Dieterly
d191f7d49b Add servicerunner to run the persister using the Openstack service infrastructure.
Change-Id: I579d712d6e6b8990caf579bc19b8d7e2f89e74c8
2014-09-29 09:34:48 -06:00
Deklan Dieterly
06822e6131 Initial Python Persister
Alarms and metrics persisted.  Simple first-pass approach.

Change-Id: Iea3e6b5506563a2954989cd3170212ad7b296c69
2014-09-19 08:35:53 -06:00