|Jenkins 20479d1c1d Merge "Fix slack notification"||1 year ago|
|monasca_notification||1 year ago|
|tests||1 year ago|
|tools||2 years ago|
|.coveragerc||2 years ago|
|.gitignore||2 years ago|
|.gitreview||3 years ago|
|.testr.conf||2 years ago|
|HACKING.rst||5 years ago|
|LICENSE||5 years ago|
|README.md||2 years ago|
|notification.yaml||2 years ago|
|requirements.txt||2 years ago|
|setup.cfg||2 years ago|
|setup.py||2 years ago|
|test-requirements.txt||2 years ago|
|tox.ini||2 years ago|
This engine reads alarms from Kafka and then notifies the customer using their configured notification method. Multiple notification and retry engines can run in parallel up to one per available Kafka partition. Zookeeper is used to negotiate access to the Kafka partitions whenever a new process joins or leaves the working set.
The notification engine generates notifications using the following steps:
The notification engine uses three Kafka topics:
A retry engine runs in parallel with the notification engine and gives any failed notification a configurable number of extra chances at succeess.
The retry engine generates notifications using the following steps:
The retry engine uses two Kafka topics:
When reading from the alarm topic no committing is done. The committing is done only after processing. This allows the processing to continue even though some notifications can be slow. In the event of a catastrophic failure some notifications could be sent but the alarms not yet acknowledged. This is an acceptable failure mode, better to send a notification twice than not at all.
The general process when a major error is encountered is to exit the daemon which should allow the other processes to renegotiate access to the Kafka partitions. It is also assumed the notification engine will be run by a process supervisor which will restart it in case of a failure. This way any errors which are not easy to recover from are automatically handled by the service restarting and the active daemon switching to another instance.
Though this should cover all errors there is risk that an alarm or set of alarms can be processed and notifications sent out multiple times. To minimize this risk a number of techniques are used:
Yaml config file by default is in ‘/etc/monasca/notification.yaml’, a sample is in this project.
statsd is incorporated into the daemon and will send all stats to statsd server launched by monasca-agent. Default host and port points at localhost:8125.
Copyright © 2014 Hewlett-Packard Development Company, L.P.
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.