vexxhost
/
openstack-operator
Code Issues Proposed changes

Add monasca-notification 

Change-Id: Ia43d545713d8f5eb4ca3a835ae8aeddc54ece285
This commit is contained in:
okozachenko 2020-09-09 09:10:43 +03:00
parent bd6855fa35
commit 61a6735081
8 changed files with 601 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
images/monasca-nontification/Dockerfile Normal file
View File

@ -0,0 +1,32 @@
ARG COMMON_REPO=https://review.opendev.org/openstack/monasca-common
ARG COMMON_VERSION=3.1.0
ARG REPO_VERSION=stable/ussuri
ARG CONSTRAINTS_BRANCH=master
ARG CONSTRAINTS_FILE=https://opendev.org/openstack/requirements/raw/branch/stable/ussuri/upper-constraints.txt
ARG DOCKER_IMAGE=monasca/notification
ARG APP_REPO=https://review.opendev.org/openstack/monasca-notification
ARG EXTRA_DEPS="netaddr gevent==1.3.5 greenlet"
FROM monasca/base:ussuri as monasca-notification
ENV \
KAFKA_URI=kafka:9092 \
KAFKA_WAIT_FOR_TOPICS=retry-notifications,alarm-state-transitions,alarm-notifications,60-seconds-notifications \
ZOOKEEPER_URL=zookeeper:2181 \
ALARM_PROCESSORS=2 \
NOTIFICATION_PROCESSORS=2 \
RETRY_INTERVAL=30 \
RETRY_MAX_ATTEMPTS=5 \
MYSQL_HOST=mysql \
MYSQL_PORT=3306 \
MYSQL_USER=notification \
MYSQL_PASSWORD=password \
MYSQL_DB=mon \
STATSD_ENABLE=true \
STATSD_HOST=monasca-statsd \
STATSD_PORT=8125 \
STAY_ALIVE_ON_FAILURE="false"
COPY notification.yaml.j2 /etc/monasca/
CMD ["/start.sh"]

153
images/monasca-nontification/README.rst Normal file
View File

@ -0,0 +1,153 @@
=====================================
Docker image for Monasca notification
=====================================
The Monasca notification image is based on the monasca-base image.
Building monasca-base image
===========================
See https://github.com/openstack/monasca-common/tree/master/docker/README.rst
Building Monasca notification image
===================================
Example:
$ ./build_image.sh <repository_version> <upper_constrains_branch> <common_version>
Everything after ``./build_image.sh`` is optional and by default configured
to get versions from ``Dockerfile``. ``./build_image.sh`` also contain more
detailed build description.
Environment variables
~~~~~~~~~~~~~~~~~~~~~
========================== ================= ================================================
Variable Default Description
========================== ================= ================================================
KAFKA_URI kafka:9092 The host and port for kafka
ZOOKEEPER_URL zookeeper:2181 URL to Zookeeper
ALARM_PROCESSORS 2 Number of alarm processing threads
NOTIFICATION_PROCESSORS 2 Number of notification processing threads
RETRY_INTERVAL 30 Retry interval in seconds
RETRY_MAX_ATTEMPTS 5 Max number of notification retries
LOG_LEVEL WARN Logging level
STATSD_ENABLE true Monasca agent StatsD enable or disable
STATSD_HOST monasca-statsd Monasca agent StatsD host for self-monitoring
STATSD_PORT 8125 Monasca agent StatsD port for self-monitoring
NF_PLUGINS <not set> See below "Notification Plugins"
MYSQL_HOST mysql The host for MySQL
MYSQL_PORT 3306 The port for MySQL
MYSQL_USER notification The MySQL username
MYSQL_PASSWORD password The MySQL password
MYSQL_DB mon The MySQL database name
STAY_ALIVE_ON_FAILURE false If true, container runs 2 hours even start fails
========================== ================= ================================================
Wait scripts environment variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
======================== ========================== ==========================================
Variable Default Description
======================== ========================== ==========================================
KAFKA_URI kafka:9092 URI to Apache Kafka
KAFKA_WAIT_FOR_TOPICS retry-notifications, The topics where metric-api streams
alarm-state-transitions, the metric messages and alarm-states
alarm-notifications,
60-seconds-notifications
KAFKA_WAIT_RETRIES 24 Number of kafka connect attempts
KAFKA_WAIT_DELAY 5 Seconds to wait between attempts
MYSQL_HOST mysql The host for MySQL
MYSQL_PORT 3306 The port for MySQL
MYSQL_USER notification The MySQL username
MYSQL_PASSWORD password The MySQL password
MYSQL_DB mon The MySQL database name
MYSQL_WAIT_RETRIES 24 Number of MySQL connection attempts
MYSQL_WAIT_DELAY 5 Seconds to wait between attempts
======================== ========================== ==========================================
Scripts
~~~~~~~
start.sh
In this starting script provide all steps that lead to the proper service
start. Including usage of wait scripts and templating of configuration
files. You also could provide the ability to allow running container after
service died for easier debugging.
health_check.py
This file will be used for checking the status of the application.
Notification Plugins
--------------------
A list of notification plugins can be provided by setting NF_PLUGINS to a comma-separated list of plugin names
e.g. email,webhook,hipchat.
Email
-----
Name: email
This plugin sends email notifications when an alarm is triggered.
Options:
* NF_EMAIL_SERVER: SMTP server address, required, unset by default
* NF_EMAIL_PORT: SMTP server port, default: 25
* NF_EMAIL_USER: SMTP username, optional, unset by default
* NF_EMAIL_PASSWORD, SMTP password, required only if NF_EMAIL_USER is set
* NF_EMAIL_FROM_ADDR: "from" field for emails sent, e.g. "Name" <name@example.com>
* NF_EMAIL_GRAFANA_URL: grafana url, required, unset by default
Webhook
-------
Name: webhook
This plugin calls a webhook when an alarm is triggered. Specific parameters, like the URL to load, are part of the notification rather than the notification plugin.
Options:
* NF_WEBHOOK_TIMEOUT: timeout in seconds, default: 5
PagerDuty
---------
Name: pagerduty
Creates a PagerDuty event for the given alarm.
Options:
* NF_PAGERDUTY_TIMEOUT: timeout in seconds, default: 5
* NF_PAGERDUTY_URL: PagerDuty Event API endpoint, defaults to official URL
HipChat
-------
Name: hipchat
Notifies via a HipChat message to some room. Authentication and destination details are configured with the notification.
Options:
* NF_HIPCHAT_TIMEOUT: timeout in seconds, default: 5
* NF_HIPCHAT_SSL_CERTS: path to SSL certs, default: system certs
* NF_HIPCHAT_INSECURE: if true, don't verify SSL
* NF_HIPCHAT_PROXY: if set, use the given HTTP(S) proxy server to send notifications
Slack
-----
Name: slack
Notifies via a Slack message.
Options:
* NF_SLACK_TIMEOUT: timeout in seconds, default: 5
* NF_SLACK_CERTS: path to SSL certs, default: system certs
* NF_SLACK_INSECURE: if true, don't verify SSL
* NF_SLACK_PROXY: if set, use the given HTTP(S) proxy server to send notifications
Provide Configuration templates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* notification.yaml.j2
Links
~~~~~
https://github.com/openstack/monasca-notification/blob/master/README.rst

150
images/monasca-nontification/build_image.sh Normal file
View File

@ -0,0 +1,150 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# TODO(Dobroslaw): move this script to monasca-common/docker folder
# and leave here small script to download it and execute using env variables
# to minimize code duplication.
set -x # Print each script step.
set -eo pipefail # Exit the script if any statement returns error.
# This script is used for building Docker image with proper labels
# and proper version of monasca-common.
#
# Example usage:
# $ ./build_image.sh <repository_version> <upper_constains_branch> <common_version>
#
# Everything after `./build_image.sh` is optional and by default configured
# to get versions from `Dockerfile`.
#
# To build from master branch (default):
# $ ./build_image.sh
# To build specific version run this script in the following way:
# $ ./build_image.sh stable/queens
# Building from specific commit:
# $ ./build_image.sh cb7f226
# When building from a tag monasca-common will be used in version available
# in upper constraint file:
# $ ./build_image.sh 2.5.0
# To build image from Gerrit patch sets that is targeting branch stable/queens:
# $ ./build_image.sh refs/changes/51/558751/1 stable/queens
#
# If you want to build image with custom monasca-common version you need
# to provide it as in the following example:
# $ ./build_image.sh master master refs/changes/19/595719/3
# Go to folder with Docker files.
REAL_PATH=$(python -c "import os,sys; print(os.path.realpath('$0'))")
cd "$(dirname "$REAL_PATH")/../docker/"
[ -z "$DOCKER_IMAGE" ] && \
DOCKER_IMAGE=$(\grep DOCKER_IMAGE Dockerfile | cut -f2 -d"=")
: "${REPO_VERSION:=$1}"
[ -z "$REPO_VERSION" ] && \
REPO_VERSION=$(\grep REPO_VERSION Dockerfile | cut -f2 -d"=")
# Let's stick to more readable version and disable SC2001 here.
# shellcheck disable=SC2001
REPO_VERSION_CLEAN=$(echo "$REPO_VERSION" | sed 's|/|-|g')
[ -z "$APP_REPO" ] && APP_REPO=$(\grep APP_REPO Dockerfile | cut -f2 -d"=")
GITHUB_REPO=$(echo "$APP_REPO" | sed 's/review.opendev.org/github.com/' | \
sed 's/ssh:/https:/')
if [ -z "$CONSTRAINTS_FILE" ]; then
CONSTRAINTS_FILE=$(\grep CONSTRAINTS_FILE Dockerfile | cut -f2 -d"=") || true
: "${CONSTRAINTS_FILE:=https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt}"
fi
: "${CONSTRAINTS_BRANCH:=$2}"
[ -z "$CONSTRAINTS_BRANCH" ] && \
CONSTRAINTS_BRANCH=$(\grep CONSTRAINTS_BRANCH Dockerfile | cut -f2 -d"=")
# When using stable version of repository use same stable constraints file.
case "$REPO_VERSION" in
*stable*)
CONSTRAINTS_BRANCH_CLEAN="$REPO_VERSION"
CONSTRAINTS_FILE=${CONSTRAINTS_FILE/master/$CONSTRAINTS_BRANCH_CLEAN}
# Get monasca-common version from stable upper constraints file.
CONSTRAINTS_TMP_FILE=$(mktemp)
wget --output-document "$CONSTRAINTS_TMP_FILE" \
$CONSTRAINTS_FILE
UPPER_COMMON=$(\grep 'monasca-common' "$CONSTRAINTS_TMP_FILE")
# Get only version part from monasca-common.
UPPER_COMMON_VERSION="${UPPER_COMMON##*===}"
rm -rf "$CONSTRAINTS_TMP_FILE"
;;
*)
CONSTRAINTS_BRANCH_CLEAN="$CONSTRAINTS_BRANCH"
;;
esac
# Monasca-common variables.
if [ -z "$COMMON_REPO" ]; then
COMMON_REPO=$(\grep COMMON_REPO Dockerfile | cut -f2 -d"=") || true
: "${COMMON_REPO:=https://review.opendev.org/openstack/monasca-common}"
fi
: "${COMMON_VERSION:=$3}"
if [ -z "$COMMON_VERSION" ]; then
COMMON_VERSION=$(\grep COMMON_VERSION Dockerfile | cut -f2 -d"=") || true
if [ "$UPPER_COMMON_VERSION" ]; then
# Common from upper constraints file.
COMMON_VERSION="$UPPER_COMMON_VERSION"
fi
fi
# Clone project to temporary directory for getting proper commit number from
# branches and tags. We need this for setting proper image labels.
# Docker does not allow to get any data from inside of system when building
# image.
TMP_DIR=$(mktemp -d)
(
cd "$TMP_DIR"
# This many steps are needed to support gerrit patch sets.
git init
git remote add origin "$APP_REPO"
git fetch origin "$REPO_VERSION"
git reset --hard FETCH_HEAD
)
GIT_COMMIT=$(git -C "$TMP_DIR" rev-parse HEAD)
[ -z "${GIT_COMMIT}" ] && echo "No git commit hash found" && exit 1
rm -rf "$TMP_DIR"
# Do the same for monasca-common.
COMMON_TMP_DIR=$(mktemp -d)
(
cd "$COMMON_TMP_DIR"
# This many steps are needed to support gerrit patch sets.
git init
git remote add origin "$COMMON_REPO"
git fetch origin "$COMMON_VERSION"
git reset --hard FETCH_HEAD
)
COMMON_GIT_COMMIT=$(git -C "$COMMON_TMP_DIR" rev-parse HEAD)
[ -z "${COMMON_GIT_COMMIT}" ] && echo "No git commit hash found" && exit 1
rm -rf "$COMMON_TMP_DIR"
CREATION_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
docker build --no-cache \
--build-arg CREATION_TIME="$CREATION_TIME" \
--build-arg GITHUB_REPO="$GITHUB_REPO" \
--build-arg APP_REPO="$APP_REPO" \
--build-arg REPO_VERSION="$REPO_VERSION" \
--build-arg GIT_COMMIT="$GIT_COMMIT" \
--build-arg CONSTRAINTS_FILE="$CONSTRAINTS_FILE" \
--build-arg COMMON_REPO="$COMMON_REPO" \
--build-arg COMMON_VERSION="$COMMON_VERSION" \
--build-arg COMMON_GIT_COMMIT="$COMMON_GIT_COMMIT" \
--tag "$DOCKER_IMAGE":"$REPO_VERSION_CLEAN" .

27
images/monasca-nontification/health_check.py Normal file
View File

@ -0,0 +1,27 @@
#!/usr/bin/env python
# coding=utf-8
# (C) Copyright 2018 FUJITSU LIMITED
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Health check will returns 0 when service is working properly."""
def main():
"""health check for Monasca-notification"""
#TODO wait for health check endpoint ...
return 0
if __name__ == '__main__':
main()

153
images/monasca-nontification/notification.yaml.j2 Normal file
View File

@ -0,0 +1,153 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
# Copyright 2018 Fujitsu LIMITED
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
kafka:
url: {{ KAFKA_URI | default('kafka:9092') }}
group: "monasca-notification"
alarm_topic: "alarm-state-transitions"
notification_topic: "alarm-notifications"
notification_retry_topic: "retry-notifications"
periodic:
60: 60-seconds-notifications
max_offset_lag: 600 # In seconds, undefined for none
mysql:
host: "{{ MYSQL_HOST | default('mysql') }}"
port: {{ MYSQL_PORT | default('3306') }}
user: "{{ MYSQL_USER | default('notification') }}"
passwd: "{{ MYSQL_PASSWORD | default('password') }}"
db: "{{ MYSQL_DB | default('mon') }}"
notification_types:
{% if NF_PLUGINS %}
{% set plugins = NF_PLUGINS.split(',')|map('trim')|list %}
plugins:
{% if 'email' in plugins %}
- monasca_notification.plugins.email_notifier:EmailNotifier
{% endif %}
{% if 'webhook' in plugins %}
- monasca_notification.plugins.webhook_notifier:WebhookNotifier
{% endif %}
{% if 'pagerduty' in plugins %}
- monasca_notification.plugins.pagerduty_notifier:PagerdutyNotifier
{% endif %}
{% if 'hipchat' in plugins %}
- monasca_notification.plugins.hipchat_notifier:HipChatNotifier
{% endif %}
{% if 'slack' in plugins %}
- monasca_notification.plugins.slack_notifier:SlackNotifier
{% endif %}
{% else %}
plugins: []
{% set plugins = [] %}
{% endif %}
{% if 'email' in plugins %}
email:
server: "{{ NF_EMAIL_SERVER }}"
port: {{ NF_EMAIL_PORT | default(25) }}
user: "{{ NF_EMAIL_USER }}"
password: "{{ NF_EMAIL_PASSWORD }}"
timeout: {{ NF_EMAIL_TIMEOUT | default(15) }}
from_addr: "{{ NF_EMAIL_FROM_ADDR }}"
grafana_url: "{{ NF_EMAIL_GRAFANA_URL }}"
{% endif -%}
{% if 'webhook' in plugins %}
webhook:
timeout: {{ NF_WEBHOOK_TIMEOUT | default(5) }}
{% endif -%}
{% if 'pagerduty' in plugins %}
pagerduty:
timeout: {{ NF_PAGERDUTY_TIMEOUT | default(5) }}
url: "{{ NF_PAGERDUTY_URL | default('https://events.pagerduty.com/generic/2010-04-15/create_event.json') }}"
{% endif -%}
{% if 'hipchat' in plugins %}
hipchat:
timeout: {{ NF_HIPCHAT_TIMEOUT | default(5) }}
ca_certs: "{{ NF_HIPCHAT_SSL_CERTS | default('/etc/ssl/certs/ca-certificates.crt') }}"
insecure: {{ NF_HIPCHAT_INSECURE | default('false') }}
{% if NF_HIPCHAT_PROXY %}
proxy: {{ NF_HIPCHAT_PROXY }}
{% endif %}
{% if NF_HIPCHAT_TEMPLATE %}
template: "{{ NF_HIPCHAT_TEMPLATE }}"
{% endif %}
{% endif -%}
{% if 'slack' in plugins %}
slack:
timeout: {{ NF_SLACK_TIMEOUT | default(5) }}
ca_certs: "{{ NF_SLACK_CA_CERTS | default('/etc/ssl/certs/ca-certificates.crt') }}"
insecure: {{ NF_SLACK_INSECURE | default('false') }}
{% if NF_SLACK_PROXY %}
proxy: {{ NF_SLACK_PROXY }}
{% endif %}
{% endif %}
processors:
alarm:
number: {{ ALARM_PROCESSORS | default(2) }}
# In seconds, undefined for none. Alarms older than this are not processed
ttl: 14400
notification:
number: {{ NOTIFICATION_PROCESSORS | default(2) }}
retry:
interval: {{ RETRY_INTERVAL | default(30) }}
max_attempts: {{ RETRY_MAX_ATTEMPTS | default(5) }}
queues:
alarms_size: 256
finished_size: 256
notifications_size: 256
sent_notifications_size: 50
zookeeper:
url: "{{ ZOOKEEPER_URL | default('zookeeper:2181') }}"
notification_path: "/notification/alarms"
notification_retry_path: "/notification/retry"
periodic_path:
60: /notification/60_seconds
logging: # Used in logging.dictConfig
version: 1
disable_existing_loggers: False
formatters:
default:
format: "%(asctime)s %(levelname)s %(name)s %(message)s"
handlers:
console:
class: logging.StreamHandler
formatter: default
loggers:
kazoo:
level: WARN
kafka:
level: WARN
statsd:
level: WARN
root:
handlers:
- console
level: {{ LOG_LEVEL | default('WARN') }}
statsd:
enable: {{ STATSD_ENABLE }}
host: "{{ STATSD_HOST }}"
port: {{ STATSD_PORT }}

41
images/monasca-nontification/start.sh Normal file
View File

@ -0,0 +1,41 @@
#!/bin/sh
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Starting script.
# All checks and configuration templating you need to do before service
# could be safely started should be added in this file.
set -eo pipefail # Exit the script if any statement returns error.
# Test services we need before starting our service.
echo "Start script: waiting for needed services"
python3 /kafka_wait_for_topics.py
python3 /mysql_check.py
# Template all config files before start, it will use env variables.
# Read usage examples: https://pypi.org/project/Templer/
echo "Start script: creating config files from templates"
templer -v -f /etc/monasca/notification.yaml.j2 /etc/monasca/notification.yaml
# Start our service.
echo "Start script: starting container"
monasca-notification /etc/monasca/notification.yaml
# Allow server to stay alive in case of failure for 2 hours for debugging.
EXIT_CODE=$?
if [ $EXIT_CODE != 0 ] && [ "$STAY_ALIVE_ON_FAILURE" = "true" ]; then
echo "Service died, waiting 120 min before exiting"
sleep 7200
fi
exit $EXIT_CODE

4
zuul.d/functional-jobs.yaml
View File

@ -83,6 +83,8 @@
soft: true
- name: openstack-operator:images:build:glance
soft: true
- name: openstack-operator:images:build:monasca-nontification
soft: true
- name: openstack-operator:images:build:rabbitmq
soft: true
- name: openstack-operator:images:build:ceilometer
@ -122,6 +124,8 @@
soft: true
- name: openstack-operator:images:upload:glance
soft: true
- name: openstack-operator:images:upload:monasca-nontification
soft: true
- name: openstack-operator:images:upload:rabbitmq
soft: true
- name: openstack-operator:images:upload:ceilometer

41
zuul.d/monasca-nontification-jobs.yaml Normal file
View File

@ -0,0 +1,41 @@
- job:
name: openstack-operator:images:build:monasca-nontification
parent: vexxhost-build-docker-image
provides: openstack-operator:image:monasca-nontification
nodeset: &id001
nodes:
- name: ubuntu-bionic
label: ubuntu-bionic-vexxhost
vars: &id002
docker_images:
- context: images/monasca-nontification
repository: vexxhost/monasca-nontification
dependencies:
- openstack-operator:images:build:openstack-operator
files: &id003
- ^images/monasca-nontification/.*
- job:
name: openstack-operator:images:upload:monasca-nontification
parent: vexxhost-upload-docker-image
provides: openstack-operator:image:monasca-nontification
nodeset: *id001
vars: *id002
dependencies:
- openstack-operator:images:upload:openstack-operator
files: *id003
- job:
name: openstack-operator:images:promote:monasca-nontification
parent: vexxhost-promote-docker-image
nodeset: *id001
vars: *id002
files: *id003
- project:
check:
jobs:
- openstack-operator:images:build:monasca-nontification
gate:
jobs:
- openstack-operator:images:upload:monasca-nontification
promote:
jobs:
- openstack-operator:images:promote:monasca-nontification