openstack
/
monasca-agent
Code Issues Proposed changes

Add tooling for building Docker images 

Change-Id: Id87eb1f9b0023a24d695b0faf01230c9cffe88c0
Story: 2001694
Task: 28222
This commit is contained in:
Dobroslaw Zybort 2018-11-28 15:56:22 +01:00
parent 9ee96263e9
commit 1f91144fc9
28 changed files with 1342 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

252
docker/README.rst Normal file
View File

@ -0,0 +1,252 @@
===============================
Docker images for Monasca Agent
===============================
The Monasca Agent images are based on the monasca-base image.
Building monasca-base image
===========================
See https://github.com/openstack/monasca-common/tree/master/docker/README.rst
Building Monasca Agent images
=============================
Example:
$ ./build_image.sh <repository_version> <upper_constains_branch> <common_version>
Requirements from monasca-base image
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
health_check.py
This file will be used for checking the status of the Monasca Agent
(unfortunately monasca agent doesn't have any endpoint for checking health).
Scripts
~~~~~~~
start.sh
In this starting script provide all steps that lead to the proper service
start. Including usage of wait scripts and templating of configuration
files. You also could provide the ability to allow running container after
service died for easier debugging.
build_image.sh
Please read detailed build description inside the script.
Environment variables
~~~~~~~~~~~~~~~~~~~~~
============================== =========================== ====================================================
Variable Default Description
============================== =========================== ====================================================
LOG_LEVEL WARN Python logging level
MONASCA_URL http://monasca:8070/v2.0 Versioned Monasca API URL
FORWARDER_URL http://localhost:17123 Monasca Agent Collector URL
KEYSTONE_DEFAULTS_ENABLED true Use all OS defaults
OS_AUTH_URL http://keystone:35357/v3/ Versioned Keystone URL
OS_USERNAME monasca-agent Agent Keystone username
OS_PASSWORD password Agent Keystone password
OS_USER_DOMAIN_NAME Default Agent Keystone user domain
OS_PROJECT_NAME mini-mon Agent Keystone project name
OS_PROJECT_DOMAIN_NAME Default Agent Keystone project domain
HOSTNAME_FROM_KUBERNETES false Determine node hostname from Kubernetes
AUTORESTART false Auto Restart Monasca Agent Collector
COLLECTOR_RESTART_INTERVAL 24 Interval in hours to restart Monasca Agent Collector
STAY_ALIVE_ON_FAILURE false If true, container runs 2 hours after tests fail
============================== =========================== ====================================================
Note that additional variables can be specified as well, see the
[config template][8] for a definitive list.
Note that the auto restart feature can be enabled if the agent collector
has unchecked memory growth. The proper restart behavior must be enabled
in Docker or Kubernetes if this feature is turned on.
Environment variables for self monitoring
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Provide them to monasca agent collector container.
============================== =========== =====================================
Variable Default Description
============================== =========== =====================================
DOCKER false Monitor Docker
CADVISOR false Monitor Cadvisor
KUBERNETES false Monitor Kubernetes
KUBERNETES_API false Monitor Kubernetes API
PROMETHEUS false Monitor Prometheus
MONASCA_MONITORING false Monitor services for metrics pipeline
MONASCA_LOG_MONITORING false Monitor services for logs pipeline
============================== =========== =====================================
Docker Plugin
-------------
This plugin is enabled when ``DOCKER=true``. It has the following options:
* ``DOCKER_ROOT``: The mounted host rootfs volume. Default: ``/host``
* ``DOCKER_SOCKET``: The mounted Docker socket. Default: ``/var/run/docker.sock``
This plugin monitors Docker containers directly. It should only be used in a
bare Docker environment (i.e. not Kubernetes), and requires two mounted volumes
from the host:
* Host ``/`` mounted to ``/host`` (path configurable with ``DOCKER_ROOT``)
* Host ``/var/run/docker.sock`` mounted to ``/var/run/docker.sock`` (path
configurable with ``DOCKER_SOCKET``)
Kubernetes Plugin
-----------------
This plugin is enabled when ``KUBERNETES=true``. It has the following options:
* ``KUBERNETES_TIMEOUT``: The K8s API connection timeout. Default: ``3``
* ``KUBERNETES_NAMESPACE_ANNOTATIONS``: If set, will grab annotations from
namespaces to include as dimensions for metrics that are under that
namespace. Should be passed in as 'annotation1,annotation2,annotation3'.
Default: unset
* ``KUBERNETES_MINIMUM_WHITELIST``: Sets whitelist on kubernetes plugin for
the following metrics pod.cpu.total_time_sec, pod.mem.cache_bytes,
pod.mem.swap_bytes, pod.mem.used_bytes, pod.mem.working_set_bytes. This
will alleviate the amount of load on Monasca.
Default: unset
The Kubernetes plugin is intended to be run as a DaemonSet on each Kubernetes
node. In order for API endpoints to be detected correctly, ``AGENT_POD_NAME`` and
`AGENT_POD_NAMESPACE`` must be set using the [Downward API][7] as described
above.
Kubernetes API Plugin
---------------------
This plugin is enabled when ``KUBERNETES_API=true``. It has the following options:
* ``KUBERNETES_API_HOST``: If set, manually sets the location of the Kubernetes
API host. Default: unset
* ``KUBERNETES_API_PORT``: If set, manually sets the port for of the Kubernetes
API host. Only used if ``KUBERNETES_API_HOST`` is also set. Default: 8080
* ``KUBERNETES_API_CUSTOM_LABELS``: If set, provides a list of Kubernetes label
keys to include as dimensions from gathered metrics. Labels should be comma
separated strings, such as ``label1,label2,label3`. The ``app`` label is always
included regardless of this value. Default: unset
* ``KUBERNETES_NAMESPACE_ANNOTATIONS``: If set, will grab annotations from
namespaces to include as dimensions for metrics that are under that
namespace. Should be passed in as 'annotation1,annotation2,annotation3'.
Default: unset
* ``REPORT_PERSISTENT_STORAGE``: If set, will gather bound pvc per a storage
class. Will be reported by namespace and cluster wide. Default: true
* ``STORAGE_PARAMETERS_DIMENSIONS``: If set and report_persistent_storage is
set, will grab storage class parameters as dimensions when reporting
persistent storage. Should be passed in as 'parameter1,parameter2". Default:
unset
The Kubernetes API plugin is intended to be run as a standalone deployment and
will collect cluster-level metrics.
Prometheus Plugin
-----------------
This plugin is enabled when ``PROMETHEUS=true``. It has the following options:
* ``PROMETHEUS_TIMEOUT``: The connection timeout. Default: ``3``
* ``PROMETHEUS_ENDPOINTS``: A list of endpoints to scrape. If unset,
they will be determined automatically via the Kubernetes API. See below for
syntax. Default: unset
* ``PROMETHEUS_DETECT_METHOD``: When endpoints are determined automatically,
this specifies the resource type to scan, one of: ``pod``, ``service``.
Default: ``pod``
* ``PROMETHEUS_KUBERNETES_LABELS``: When endpoints are determined automatically,
this comma-separated list of labels will be included as dimensions (by name).
Default: ``app``
If desired, a static list of Prometheus endpoints can be provided by setting
`PROMETHEUS_ENDPOINTS`. Entries in this list should be comma-separated.
Additionally, each entry can specify a set of dimensions like so:
``http://host-a/metrics,http://host-b/metrics|prop=value&prop2=value2,http://host-c``
Note that setting ``PROMETHEUS_ENDPOINTS`` disables auto-detection.
When autodetection is enabled, this plugin will automatically scrape all
annotated Prometheus endpoints on the node the agent is running on. Ideally, it
should be run alongside the Kubernetes plugin as a DaemonSet on each node.
cAdvisor_host Plugin
--------------------
This plugin is enabled when ``CADVISOR=true``. It has the following options:
* ``CADVISOR_TIMEOUT``: The connection timeout for the cAdvisor API. Default: ``3``
* ``CADVISOR_URL``: If set, sets the URL at which to access cAdvisor. If unset,
(default) the cAdvisor host will be determined automatically via the
Kubernetes API.
* ``CADVISOR_MINIMUM_WHITELIST``: Sets whitelist on cadvisor host plugin for
the following metrics cpu.total_time_sec, mem.cache_bytes,
mem.swap_bytes, mem.used_bytes, mem.working_set_bytes. This
will alleviate the amount of load on Monasca.
Default: unset
This plugin collects host-level metrics from a running cAdvisor instance.
cAdvisor is included in ``kubelet`` when in Kubernetes environments and is
necessary to retrieve host-level metrics. As with the Kubernetes plugin,
`AGENT_POD_NAME`` and ``AGENT_POD_NAMESPACE`` must be set to determine the URL
automatically.
cAdvisor can be easily run in [standard Docker environments][9] or directly on
host systems. In these cases, the URL must be manually provided via
``CADVISOR_URL``.
Monasca-monitoring
------------------
Metrics pipeline
^^^^^^^^^^^^^^^^
The monasca-monitoring enables plugins for HTTP endpoint check and processes.
Additionally enables plugins for detailed metrics for the following components:
Kafka, MySQL, and Zookeeper. This is enabled when ``MONASCA_MONITORING=true``.
The components use the default configuration. A user can specify own settings
for them in docker-compose file. To customize those settings you can adjust the
configuration of the components by passing environment variables:
Kafka
+++++
* ``KAFKA_CONNECT_STR``: The kafka connection string. Default: ``kafka:9092``
Zookeeper
+++++++++
* ``ZOOKEEPER_HOST``: The zookeeper host name. Default: ``zookeeper``
* ``ZOOKEEPER_PORT``: The port to listen for client connections. Default: ``2181``
MySQL
+++++
* ``MYSQL_SERVER``: The MySQL server name. Default: ``mysql``
* ``MYSQL_USER``, ``MYSQL_PASSWORD``: These variables are used in conjunction to specify user and password for this user. Default: ``root`` and ``secretmysql``
* ``MYSQL_PORT``: The port to listen for client connections. Default: ``3306``
Logs pipeline
^^^^^^^^^^^^^
For logs pipeline you can enable HTTP endpoint check, process and
`Elasticsearch`` plugins. This is enabled when ``MONASCA_LOG_MONITORING=true``.
You can adjust the configuration of the components by passing environment
variables:
Elasticsearch
+++++++++++++
* ``ELASTIC_URL``: The Elasticsearch connection string. Default: ``http://elasticsearch:9200``
Monasca-statsd
^^^^^^^^^^^^^^
To monitor ``monasca-notifcation`` and ``monasca-log-api`` use ``statsd``. Enable the
statsd monitoring by setting up ``STATSD_HOST`` and ``STATSD_PORT`` environment
variables in those projects.
Custom plugins
~~~~~~~~~~~~~~
Custom plugin configuration files can be provided by mounting them to
``/plugins.d/*.yaml`` inside the container of monasca agent collector.
Plugins should have ``yaml`` extension when you don't need any templating.
When they have ``yaml.j2`` extension, they will be processed as Jinja2
templates with access to all environment variables.
Links
~~~~~
https://github.com/openstack/monasca-agent/blob/master/README.rst

35
docker/collector/Dockerfile Normal file
View File

@ -0,0 +1,35 @@
ARG DOCKER_IMAGE=monasca/agent-collector
ARG APP_REPO=https://git.openstack.org/openstack/monasca-agent
# Branch, tag or git hash to build from.
ARG REPO_VERSION=master
ARG CONSTRAINTS_BRANCH=master
# Extra Python3 dependencies.
ARG EXTRA_DEPS="Jinja2 prometheus_client docker-py"
# Always start from `monasca-base` image and use specific tag of it.
ARG BASE_TAG=master
FROM monasca/base:$BASE_TAG
# Environment variables used for our service or wait scripts.
ENV \
KEYSTONE_DEFAULTS_ENABLED=true \
MONASCA_URL=http://monasca:8070/v2.0 \
LOG_LEVEL=WARN \
HOSTNAME_FROM_KUBERNETES=false \
STAY_ALIVE_ON_FAILURE="false"
# Copy all neccessary files to proper locations.
COPY templates/ /templates
COPY agent.yaml.j2 /etc/monasca/agent/agent.yaml.j2
COPY kubernetes_get_host.py /
# Run here all additionals steps your service need post installation.
# Stay with only one `RUN` and use `&& \` for next steps to don't create
# unnecessary image layers. Clean at the end to conserve space.
RUN \
apk add --no-cache util-linux libxml2 py2-psutil libffi-dev openssl-dev
# Implement start script in `start.sh` file.
CMD ["/start.sh"]

49
docker/collector/agent.yaml.j2 Normal file
View File

@ -0,0 +1,49 @@
Api:
url: "{{ MONASCA_URL | default('') }}"
service_type: "{{ SERVICE_TYPE | default('') }}"
endpoint_type: "{{ ENDPOINT_TYPE | default('') }}"
region_name: "{{ REGION_NAME | default('') }}"
username: "{{ OS_USERNAME }}"
password: "{{ OS_PASSWORD }}"
keystone_url: "{{ OS_AUTH_URL }}"
user_domain_id: "{{ OS_USER_DOMAIN_ID | default('') }}"
user_domain_name: "{{ OS_USER_DOMAIN_NAME | default('') }}"
project_name: "{{ OS_PROJECT_NAME | default('') }}"
project_domain_id: "{{ OS_PROJECT_DOMAIN_ID | default('') }}"
project_domain_name: "{{ OS_PROJECT_DOMAIN_NAME | default('') }}"
project_id: "{{ OS_PROJECT_ID | default('') }}"
insecure: {{ INSECURE | default(False)}}
ca_file: "{{ CA_FILE | default('') }}"
max_buffer_size: {{ MAX_BUFFER_SIZE | default(1000) }}
max_batch_size: {{ MAX_BATCH_SIZE | default(0) }}
max_measurement_buffer_size: {{ MAX_MEASUREMENT_BUFFER_SIZE | default(-1)}}
backlog_send_rate: {{ BACKLOG_SEND_RATE | default(5) }}
Main:
hostname: {{ '"%s"' % AGENT_HOSTNAME if AGENT_HOSTNAME else 'null' }}
{% if DIMENSIONS %}
dimensions:
{% for dimension in DIMENSIONS.split(',') %}
{% set k, v = dimension.split('=', 1) %}
{{k.strip()}}: "{{v}}"
{% endfor %}
{% else %}
dimensions: {}
{% endif %}
check_freq: {{ CHECK_FREQ | default(30) }}
num_collector_threads: {{ NUM_COLLECTOR_THREADS | default(1) }}
pool_full_max_retries: {{ POOL_FULL_MAX_TRIES | default(4) }}
sub_collection_warn: {{ SUB_COLLECTION_WARN | default(6) }}
autorestart: {{ AUTORESTART | default(False) }}
collector_restart_interval: {{ COLLECTOR_RESTART_INTERVAL | default(24) }}
non_local_traffic: {{ NON_LOCAL_TRAFFIC | default(False) }}
forwarder_url: {{ FORWARDER_URL | default("http://localhost:17123") }}
Statsd:
monasca_statsd_port : {{ STATSD_PORT | default(8125) }}
Logging:
log_level: {{ LOG_LEVEL | default('WARN') }}
disable_file_logging: True

147
docker/collector/build_image.sh Executable file
View File

@ -0,0 +1,147 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# TODO(Dobroslaw): move this script to monasca-common/docker folder
# and leave here small script to download it and execute using env variables
# to minimize code duplication.
set -x # Print each script step.
set -eo pipefail # Exit the script if any statement returns error.
# This script is used for building Docker image with proper labels
# and proper version of monasca-common.
#
# Example usage:
# $ ./build_image.sh <repository_version> <upper_constains_branch> <common_version>
#
# Everything after `./build_image.sh` is optional and by default configured
# to get versions from `Dockerfile`.
#
# To build from master branch (default):
# $ ./build_image.sh
# To build specific version run this script in the following way:
# $ ./build_image.sh stable/queens
# Building from specific commit:
# $ ./build_image.sh cb7f226
# When building from a tag monasca-common will be used in version available
# in upper constraint file:
# $ ./build_image.sh 2.5.0
# To build image from Gerrit patch sets that is targeting branch stable/queens:
# $ ./build_image.sh refs/changes/51/558751/1 stable/queens
#
# If you want to build image with custom monasca-common version you need
# to provide it as in the following example:
# $ ./build_image.sh master master refs/changes/19/595719/3
[ -z "$DOCKER_IMAGE" ] && \
DOCKER_IMAGE=$(\grep DOCKER_IMAGE Dockerfile | cut -f2 -d"=")
: "${REPO_VERSION:=$1}"
[ -z "$REPO_VERSION" ] && \
REPO_VERSION=$(\grep REPO_VERSION Dockerfile | cut -f2 -d"=")
# Let's stick to more readable version and disable SC2001 here.
# shellcheck disable=SC2001
REPO_VERSION_CLEAN=$(echo "$REPO_VERSION" | sed 's|/|-|g')
[ -z "$APP_REPO" ] && APP_REPO=$(\grep APP_REPO Dockerfile | cut -f2 -d"=")
GITHUB_REPO=$(echo "$APP_REPO" | sed 's/git.openstack.org/github.com/' | \
sed 's/ssh:/https:/')
if [ -z "$CONSTRAINTS_FILE" ]; then
CONSTRAINTS_FILE=$(\grep CONSTRAINTS_FILE Dockerfile | cut -f2 -d"=") || true
: "${CONSTRAINTS_FILE:=http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt}"
fi
: "${CONSTRAINTS_BRANCH:=$2}"
[ -z "$CONSTRAINTS_BRANCH" ] && \
CONSTRAINTS_BRANCH=$(\grep CONSTRAINTS_BRANCH Dockerfile | cut -f2 -d"=")
# When using stable version of repository use same stable constraints file.
case "$REPO_VERSION" in
*stable*)
CONSTRAINTS_BRANCH_CLEAN="$REPO_VERSION"
# Get monasca-common version from stable upper constraints file.
CONSTRAINTS_TMP_FILE=$(mktemp)
wget --output-document "$CONSTRAINTS_TMP_FILE" \
"$CONSTRAINTS_FILE"?h="$CONSTRAINTS_BRANCH_CLEAN"
UPPER_COMMON=$(\grep 'monasca-common' "$CONSTRAINTS_TMP_FILE")
# Get only version part from monasca-common.
UPPER_COMMON_VERSION="${UPPER_COMMON##*===}"
rm -rf "$CONSTRAINTS_TMP_FILE"
;;
*)
CONSTRAINTS_BRANCH_CLEAN="$CONSTRAINTS_BRANCH"
;;
esac
# Monasca-common variables.
if [ -z "$COMMON_REPO" ]; then
COMMON_REPO=$(\grep COMMON_REPO Dockerfile | cut -f2 -d"=") || true
: "${COMMON_REPO:=https://git.openstack.org/openstack/monasca-common}"
fi
: "${COMMON_VERSION:=$3}"
if [ -z "$COMMON_VERSION" ]; then
COMMON_VERSION=$(\grep COMMON_VERSION Dockerfile | cut -f2 -d"=") || true
if [ "$UPPER_COMMON_VERSION" ]; then
# Common from upper constraints file.
COMMON_VERSION="$UPPER_COMMON_VERSION"
fi
fi
# Clone project to temporary directory for getting proper commit number from
# branches and tags. We need this for setting proper image labels.
# Docker does not allow to get any data from inside of system when building
# image.
TMP_DIR=$(mktemp -d)
(
cd "$TMP_DIR"
# This many steps are needed to support gerrit patch sets.
git init
git remote add origin "$APP_REPO"
git fetch origin "$REPO_VERSION"
git reset --hard FETCH_HEAD
)
GIT_COMMIT=$(git -C "$TMP_DIR" rev-parse HEAD)
[ -z "${GIT_COMMIT}" ] && echo "No git commit hash found" && exit 1
rm -rf "$TMP_DIR"
# Do the same for monasca-common.
COMMON_TMP_DIR=$(mktemp -d)
(
cd "$COMMON_TMP_DIR"
# This many steps are needed to support gerrit patch sets.
git init
git remote add origin "$COMMON_REPO"
git fetch origin "$COMMON_VERSION"
git reset --hard FETCH_HEAD
)
COMMON_GIT_COMMIT=$(git -C "$COMMON_TMP_DIR" rev-parse HEAD)
[ -z "${COMMON_GIT_COMMIT}" ] && echo "No git commit hash found" && exit 1
rm -rf "$COMMON_TMP_DIR"
CREATION_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
docker build --no-cache \
--build-arg CREATION_TIME="$CREATION_TIME" \
--build-arg GITHUB_REPO="$GITHUB_REPO" \
--build-arg APP_REPO="$APP_REPO" \
--build-arg REPO_VERSION="$REPO_VERSION" \
--build-arg GIT_COMMIT="$GIT_COMMIT" \
--build-arg CONSTRAINTS_FILE="$CONSTRAINTS_FILE" \
--build-arg CONSTRAINTS_BRANCH="$CONSTRAINTS_BRANCH_CLEAN" \
--build-arg COMMON_REPO="$COMMON_REPO" \
--build-arg COMMON_VERSION="$COMMON_VERSION" \
--build-arg COMMON_GIT_COMMIT="$COMMON_GIT_COMMIT" \
--tag "$DOCKER_IMAGE":"$REPO_VERSION_CLEAN" .

28
docker/collector/health_check.py Executable file
View File

@ -0,0 +1,28 @@
#!/usr/bin/env python
# coding=utf-8
# (C) Copyright 2018 FUJITSU LIMITED
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Health check will returns 0 when service is working properly."""
def main():
"""Health check for Monasca-agent collector."""
# TODO wait for health check endpoint ...
return 0
if __name__ == '__main__':
main()

7
docker/collector/kubernetes_get_host.py Normal file
View File

@ -0,0 +1,7 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
# coding=utf-8
from monasca_agent.collector.checks import utils
kubernetes_connector = utils.KubernetesConnector(3)
print(kubernetes_connector.get_agent_pod_host(return_host_name=True))

129
docker/collector/start.sh Normal file
View File

@ -0,0 +1,129 @@
#!/bin/sh
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Starting script.
# All checks and configuration templating you need to do before service
# could be safely started should be added in this file.
set -eo pipefail # Exit the script if any statement returns error.
PLUGIN_TEMPLATES="/templates"
USER_PLUGINS="/plugins.d"
AGENT_CONF="/etc/monasca/agent"
AGENT_PLUGINS="$AGENT_CONF/conf.d"
if [ "$KEYSTONE_DEFAULTS_ENABLED" = "true" ]; then
export OS_AUTH_URL=${OS_AUTH_URL:-"http://keystone:35357/v3/"}
export OS_USERNAME=${OS_USERNAME:-"monasca-agent"}
export OS_PASSWORD=${OS_PASSWORD:-"password"}
export OS_USER_DOMAIN_NAME=${OS_USER_DOMAIN_NAME:-"Default"}
export OS_PROJECT_NAME=${OS_PROJECT_NAME:-"mini-mon"}
export OS_PROJECT_DOMAIN_NAME=${OS_PROJECT_DOMAIN_NAME:-"Default"}
fi
mkdir -p "$AGENT_PLUGINS"
# Test services we need before starting our service.
#echo "Start script: waiting for needed services"
# Template all config files before start, it will use env variables.
# Read usage examples: https://pypi.org/project/Templer/
echo "Start script: creating config files from templates"
alias template="templer --ignore-undefined-variables --force --verbose"
if [ "$HOSTNAME_FROM_KUBERNETES" = "true" ]; then
if ! AGENT_HOSTNAME=$(python /kubernetes_get_host.py); then
echo "Error getting hostname from Kubernetes"
return 1
fi
export AGENT_HOSTNAME
fi
if [ "$DOCKER" = "true" ]; then
template $PLUGIN_TEMPLATES/docker.yaml.j2 $AGENT_PLUGINS/docker.yaml
fi
# Cadvisor.
if [ "$CADVISOR" = "true" ]; then
template $PLUGIN_TEMPLATES/cadvisor_host.yaml.j2 $AGENT_PLUGINS/cadvisor_host.yaml
fi
# Kubernetes.
if [ "$KUBERNETES" = "true" ]; then
template $PLUGIN_TEMPLATES/kubernetes.yaml.j2 $AGENT_PLUGINS/kubernetes.yaml
fi
# Kubernetes_api.
if [ "$KUBERNETES_API" = "true" ]; then
template $PLUGIN_TEMPLATES/kubernetes_api.yaml.j2 $AGENT_PLUGINS/kubernetes_api.yaml
fi
# Prometheus scraping.
if [ "$PROMETHEUS" = "true" ]; then
template $PLUGIN_TEMPLATES/prometheus.yaml.j2 $AGENT_PLUGINS/prometheus.yaml
fi
# Monasca monitoring.
if [ "$MONASCA_MONITORING" = "true" ]; then
template $PLUGIN_TEMPLATES/zk.yaml.j2 $AGENT_PLUGINS/zk.yaml
template $PLUGIN_TEMPLATES/kafka_consumer.yaml.j2 $AGENT_PLUGINS/kafka_consumer.yaml
template $PLUGIN_TEMPLATES/mysql.yaml.j2 $AGENT_PLUGINS/mysql.yaml
fi
# Monasca-log-monitoring.
if [ "$MONASCA_LOG_MONITORING" = "true" ]; then
template $PLUGIN_TEMPLATES/elastic.yaml.j2 $AGENT_PLUGINS/elastic.yaml
fi
# Common for monasca-monitoring and monasca-log-monitoring.
if [ "$MONASCA_MONITORING" = "true" ] || [ "$MONASCA_LOG_MONITORING" = "true" ]; then
template $PLUGIN_TEMPLATES/http_check.yaml.j2 $AGENT_PLUGINS/http_check.yaml
template $PLUGIN_TEMPLATES/process.yaml.j2 $AGENT_PLUGINS/process.yaml
fi
# System.
template $PLUGIN_TEMPLATES/cpu.yaml.j2 $AGENT_PLUGINS/cpu.yaml
template $PLUGIN_TEMPLATES/disk.yaml.j2 $AGENT_PLUGINS/disk.yaml
template $PLUGIN_TEMPLATES/load.yaml.j2 $AGENT_PLUGINS/load.yaml
template $PLUGIN_TEMPLATES/memory.yaml.j2 $AGENT_PLUGINS/memory.yaml
# Apply user templates.
for f in $USER_PLUGINS/*.yaml.j2; do
if [ -e "$f" ]; then
template "$f" "$AGENT_PLUGINS/$(basename "$f" .j2)"
fi
done
# Copy plain user plugins.
for f in $USER_PLUGINS/*.yaml; do
if [ -e "$f" ]; then
cp "$f" "$AGENT_PLUGINS/$(basename "$f")"
fi
done
template $AGENT_CONF/agent.yaml.j2 $AGENT_CONF/agent.yaml
# Start our service.
echo "Start script: starting container"
monasca-collector foreground
# Allow server to stay alive in case of failure for 2 hours for debugging.
RESULT=$?
if [ $RESULT != 0 ] && [ "$STAY_ALIVE_ON_FAILURE" = "true" ]; then
echo "Service died, waiting 120 min before exiting"
sleep 7200
fi
exit $RESULT

19
docker/collector/templates/cadvisor_host.yaml.j2 Normal file
View File

@ -0,0 +1,19 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
init_config:
connection_timeout: {{ CADVISOR_TIMEOUT | default(3) }}
{% if CADVISOR_MINIMUM_WHITELIST %}
white_list:
metrics:
cpu.total_time_sec:
mem.cache_bytes:
mem.swap_bytes:
mem.used_bytes:
mem.working_set_bytes:
{% endif %}
instances:
{% if CADVISOR_URL %}
- cadvisor_url: "{{ CADVISOR_URL }}"
{% else %}
- kubernetes_detect_cadvisor: True
{% endif %}

5
docker/collector/templates/cpu.yaml.j2 Normal file
View File

@ -0,0 +1,5 @@
init_config:
process_fs_path: /rootfs/proc
instances:
# Cpu check only supports one configured instance
- name: cpu_stats

6
docker/collector/templates/disk.yaml.j2 Normal file
View File

@ -0,0 +1,6 @@
init_config:
process_fs_path: /rootfs/proc
instances:
# Disk check only supports one configured instance
- name: disk_stats
ignore_filesystem_types: iso9660,tmpfs,nsfs

7
docker/collector/templates/docker.yaml.j2 Normal file
View File

@ -0,0 +1,7 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
init_config:
docker_root: "{{ DOCKER_ROOT | default('/host') }}"
instances:
- url: "{{ DOCKER_SOCKET | default('unix://var/run/docker.sock') }}"

3
docker/collector/templates/elastic.yaml.j2 Normal file
View File

@ -0,0 +1,3 @@
init_config:
instances:
- url: "{{ ELASTIC_URL | default('http://elasticsearch:9200') }}"

46
docker/collector/templates/http_check.yaml.j2 Normal file
View File

@ -0,0 +1,46 @@
init_config: null
instances:
{% if MONASCA_MONITORING %}
- name: keystone
dimensions:
service: keystone
timeout: 3
url: http://keystone:5000
- name: mysql
dimensions:
service: mysql
timeout: 3
url: http://mysql:3306
- name: cadvisor
dimensions:
service: cadvisor
timeout: 3
url: http://cadvisor:8080/healthz
- name: influxdb
dimensions:
service: influxdb
timeout: 3
url: http://influxdb:8086/ping
- name: cadvisor
dimensions:
service: cadvisor
timeout: 3
url: http://cadvisor:8080/healthz
{% endif %}
{% if MONASCA_LOG_MONITORING %}
- name: log-api
dimensions:
service: log-api
timeout: 3
url: http://log-api:5607/healthcheck
- name: elasticsearch
dimensions:
service: elasticsearch
timeout: 3
url: http://elasticsearch:9200/_cat/health
- name: kibana
dimensions:
service: kibana
timeout: 3
url: http://kibana:5601/status
{% endif %}

14
docker/collector/templates/kafka_consumer.yaml.j2 Normal file
View File

@ -0,0 +1,14 @@
init_config:
instances:
- built_by: Kafka
consumer_groups:
1_metrics:
metrics: []
thresh-event:
events: []
thresh-metric:
metrics: []
kafka_connect_str: "{{ KAFKA_CONNECT_STR | default('kafka:9092') }}"
name: "{{ KAFKA_CONNECT_STR | default('kafka:9092') }}"
per_partition: false

24
docker/collector/templates/kubernetes.yaml.j2 Normal file
View File

@ -0,0 +1,24 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
init_config:
connection_timeout: {{ KUBERNETES_TIMEOUT | default(3) }}
{% if KUBERNETES_MINIMUM_WHITELIST %}
white_list:
metrics:
pod.cpu.total_time_sec:
pod.mem.cache_bytes:
pod.mem.swap_bytes:
pod.mem.used_bytes:
pod.mem.working_set_bytes:
pod.mem.rss_bytes:
container.mem.usage_percent:
{% endif %}
report_container_mem_percent: True
instances:
- derive_host: True
{% if KUBERNETES_NAMESPACE_ANNOTATIONS %}
namespace_annotations:
{% for annotation in KUBERNETES_NAMESPACE_ANNOTATIONS.split(',') %}
- "{{ annotation }}"
{% endfor %}
{% endif %}

31
docker/collector/templates/kubernetes_api.yaml.j2 Normal file
View File

@ -0,0 +1,31 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
init_config:
connection_timeout: {{ KUBERNETES_API_TIMEOUT | default(3) }}
instances:
{% if KUBERNETES_API_HOST %}
- host: "{{ KUBERNETES_API_HOST }}"
kubernetes_master_port: {{ KUBERNETES_API_PORT | default(8080) }}
{% else %}
- derive_api_url: true
{% endif %}
{% if KUBERNETES_API_CUSTOM_LABELS %}
custom_kubernetes_labels:
{% for label in KUBERNETES_API_CUSTOM_LABELS.split(',') %}
- "{{ label }}"
{% endfor %}
{% endif %}
{% if KUBERNETES_NAMESPACE_ANNOTATIONS %}
namespace_annotations:
{% for annotation in KUBERNETES_NAMESPACE_ANNOTATIONS.split(',') %}
- "{{ annotation }}"
{% endfor %}
{% endif %}
{% if STORAGE_PARAMETERS_DIMENSIONS %}
storage_parameters_dimensions:
{% for parameter in STORAGE_PARAMETERS_DIMENSIONS.split(',') %}
- "{{ parameter }}"
{% endfor %}
{% endif %}
report_persistent_storage: {{ REPORT_PERSISTENT_STORAGE | default(true) }}

5
docker/collector/templates/load.yaml.j2 Normal file
View File

@ -0,0 +1,5 @@
init_config:
process_fs_path: /rootfs/proc
instances:
# Load check only supports one configured instance
- name: load_stats

5
docker/collector/templates/memory.yaml.j2 Normal file
View File

@ -0,0 +1,5 @@
init_config:
process_fs_path: /rootfs/proc
instances:
# Memory check only supports one configured instance
- name: memory_stats

8
docker/collector/templates/mysql.yaml.j2 Normal file
View File

@ -0,0 +1,8 @@
init_config:
instances:
- built_by: MySQL
name: "{{ MYSQL_SERVER | default('mysql') }}"
server: "{{ MYSQL_SERVER | default('mysql') }}"
port: "{{ MYSQL_PORT | default('3306') }}"
user: "{{ MYSQL_USER | default('root') }}"
pass: "{{ MYSQL_PASSWORD | default('secretmysql') }}"

154
docker/collector/templates/process.yaml.j2 Normal file
View File

@ -0,0 +1,154 @@
init_config:
process_fs_path: /rootfs/proc
instances:
{% if MONASCA_MONITORING %}
- name: influxd
detailed: true
dimensions:
service: influxd
exact_match: false
search_string:
- influxd
- name: monasca-statsd
detailed: true
dimensions:
service: monasca-statsd
exact_match: false
search_string:
- monasca-statsd
- name: monasca-notification
detailed: true
dimensions:
service: monasca-notification
exact_match: false
search_string:
- monasca-notification
- name: persister
detailed: true
dimensions:
service: persister
exact_match: false
search_string:
- persister
- name: monasca-thresh
detailed: true
dimensions:
service: monasca-thresh
exact_match: false
search_string:
- monasca-thresh
- name: monasca-api
detailed: true
dimensions:
service: monasca-api
exact_match: false
search_string:
- monasca-api
- name: monasca-collector
detailed: true
dimensions:
service: monasca-collector
exact_match: false
search_string:
- monasca-collector
- name: memcached
detailed: true
dimensions:
service: memcached
exact_match: false
search_string:
- memcached
- name: cadvisor
detailed: true
dimensions:
service: cadvisor
exact_match: false
search_string:
- cadvisor
- name: monasca-forwarder
detailed: true
dimensions:
service: monasca-forwarder
exact_match: false
search_string:
- monasca-forwarder
- name: zookeeper
detailed: true
dimensions:
service: zookeeper
exact_match: false
search_string:
- zookeeper
- name: kafka
detailed: true
dimensions:
service: kafka
exact_match: false
search_string:
- kafka
- name: mysqld
detailed: true
dimensions:
service: mysqld
exact_match: false
search_string:
- mysqld
{% endif %}
{% if MONASCA_LOG_MONITORING %}
- name: logspout
detailed: true
dimensions:
service: logspout
exact_match: false
search_string:
- logspout
- name: log-agent
detailed: true
dimensions:
service: log-agent
exact_match: false
search_string:
- log-agent
- name: log-api
detailed: true
dimensions:
service: log-api
exact_match: false
search_string:
- log-api
- name: kibana
detailed: true
dimensions:
service: kibana
exact_match: false
search_string:
- kibana
- name: elasticsearch
detailed: true
dimensions:
service: elasticsearch
exact_match: false
search_string:
- elasticsearch
- name: log-transformer
detailed: true
dimensions:
service: log-transformer
exact_match: false
search_string:
- log-transformer
- name: log-persister
detailed: true
dimensions:
service: log-persister
exact_match: false
search_string:
- log-persister
- name: log-metrics
detailed: true
dimensions:
service: log-metrics
exact_match: false
search_string:
- log-metrics
{% endif %}

37
docker/collector/templates/prometheus.yaml.j2 Normal file
View File

@ -0,0 +1,37 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
init_config:
timeout: {{ PROMETHEUS_TIMEOUT | default(3) }}
{% if not PROMETHEUS_ENDPOINTS %}
auto_detect_endpoints: true
detect_method: {{ PROMETHEUS_DETECT_METHOD | default('pod') }}
{% endif %}
instances:
{% if PROMETHEUS_ENDPOINTS %}
{% for endpoint in PROMETHEUS_ENDPOINTS.split(',') %}
{% if '|' in endpoint %}
{% set endpoint, dimensions = endpoint.split('|', 2) %}
{% set dimensions = dimensions.split('&') %}
{% else %}
{% set dimensions = [] %}
{% endif %}
- metric_endpoint: "{{ endpoint }}"
{% if dimensions %}
default_dimensions:
{% for dimension in dimensions %}
{% set k, v = dimension.split('=', 1) %}
{{k}}: {{v}}
{% endfor %}
{% endif %}
{% endfor %}
{% else %}
{% if PROMETHEUS_KUBERNETES_LABELS %}
- kubernetes_labels:
{% for label in PROMETHEUS_KUBERNETES_LABELS.split(',') %}
- {{ label }}
{% endfor %}
{% else %}
- kubernetes_labels: ['app']
{% endif %}
{% endif %}

6
docker/collector/templates/zk.yaml.j2 Normal file
View File

@ -0,0 +1,6 @@
init_config:
instances:
- host: "{{ ZOOKEEPER_HOST | default('zookeeper') }}"
port: "{{ ZOOKEEPER_PORT | default('2181') }}"
timeout: 3

33
docker/forwarder/Dockerfile Normal file
View File

@ -0,0 +1,33 @@
ARG DOCKER_IMAGE=monasca/agent-forwarder
ARG APP_REPO=https://git.openstack.org/openstack/monasca-agent
# Branch, tag or git hash to build from.
ARG REPO_VERSION=master
ARG CONSTRAINTS_BRANCH=master
# Extra Python3 dependencies.
#ARG EXTRA_DEPS=""
# Always start from `monasca-base` image and use specific tag of it.
ARG BASE_TAG=master
FROM monasca/base:$BASE_TAG
# Environment variables used for our service or wait scripts.
ENV \
KEYSTONE_DEFAULTS_ENABLED=true \
MONASCA_URL=http://monasca:8070/v2.0 \
LOG_LEVEL=WARN \
HOSTNAME_FROM_KUBERNETES=false \
STAY_ALIVE_ON_FAILURE="false"
# Copy all neccessary files to proper locations.
COPY agent.yaml.j2 /etc/monasca/agent/agent.yaml.j2
COPY kubernetes_get_host.py /
# Run here all additionals steps your service need post installation.
# Stay with only one `RUN` and use `&& \` for next steps to don't create
# unnecessary image layers. Clean at the end to conserve space.
#RUN
# Implement start script in `start.sh` file.
CMD ["/start.sh"]

49
docker/forwarder/agent.yaml.j2 Normal file
View File

@ -0,0 +1,49 @@
Api:
url: "{{ MONASCA_URL | default('') }}"
service_type: "{{ SERVICE_TYPE | default('') }}"
endpoint_type: "{{ ENDPOINT_TYPE | default('') }}"
region_name: "{{ REGION_NAME | default('') }}"
username: "{{ OS_USERNAME }}"
password: "{{ OS_PASSWORD }}"
keystone_url: "{{ OS_AUTH_URL }}"
user_domain_id: "{{ OS_USER_DOMAIN_ID | default('') }}"
user_domain_name: "{{ OS_USER_DOMAIN_NAME | default('') }}"
project_name: "{{ OS_PROJECT_NAME | default('') }}"
project_domain_id: "{{ OS_PROJECT_DOMAIN_ID | default('') }}"
project_domain_name: "{{ OS_PROJECT_DOMAIN_NAME | default('') }}"
project_id: "{{ OS_PROJECT_ID | default('') }}"
insecure: {{ INSECURE | default(False)}}
ca_file: "{{ CA_FILE | default('') }}"
max_buffer_size: {{ MAX_BUFFER_SIZE | default(1000) }}
max_batch_size: {{ MAX_BATCH_SIZE | default(0) }}
max_measurement_buffer_size: {{ MAX_MEASUREMENT_BUFFER_SIZE | default(-1)}}
backlog_send_rate: {{ BACKLOG_SEND_RATE | default(5) }}
Main:
hostname: {{ '"%s"' % AGENT_HOSTNAME if AGENT_HOSTNAME else 'null' }}
{% if DIMENSIONS %}
dimensions:
{% for dimension in DIMENSIONS.split(',') %}
{% set k, v = dimension.split('=', 1) %}
{{k.strip()}}: "{{v}}"
{% endfor %}
{% else %}
dimensions: {}
{% endif %}
check_freq: {{ CHECK_FREQ | default(30) }}
num_collector_threads: {{ NUM_COLLECTOR_THREADS | default(1) }}
pool_full_max_retries: {{ POOL_FULL_MAX_TRIES | default(4) }}
sub_collection_warn: {{ SUB_COLLECTION_WARN | default(6) }}
autorestart: {{ AUTORESTART | default(False) }}
collector_restart_interval: {{ COLLECTOR_RESTART_INTERVAL | default(24) }}
non_local_traffic: {{ NON_LOCAL_TRAFFIC | default(False) }}
forwarder_url: {{ FORWARDER_URL | default("http://localhost:17123") }}
Statsd:
monasca_statsd_port : {{ STATSD_PORT | default(8125) }}
Logging:
log_level: {{ LOG_LEVEL | default('WARN') }}
disable_file_logging: True

147
docker/forwarder/build_image.sh Executable file
View File

@ -0,0 +1,147 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# TODO(Dobroslaw): move this script to monasca-common/docker folder
# and leave here small script to download it and execute using env variables
# to minimize code duplication.
set -x # Print each script step.
set -eo pipefail # Exit the script if any statement returns error.
# This script is used for building Docker image with proper labels
# and proper version of monasca-common.
#
# Example usage:
# $ ./build_image.sh <repository_version> <upper_constains_branch> <common_version>
#
# Everything after `./build_image.sh` is optional and by default configured
# to get versions from `Dockerfile`.
#
# To build from master branch (default):
# $ ./build_image.sh
# To build specific version run this script in the following way:
# $ ./build_image.sh stable/queens
# Building from specific commit:
# $ ./build_image.sh cb7f226
# When building from a tag monasca-common will be used in version available
# in upper constraint file:
# $ ./build_image.sh 2.5.0
# To build image from Gerrit patch sets that is targeting branch stable/queens:
# $ ./build_image.sh refs/changes/51/558751/1 stable/queens
#
# If you want to build image with custom monasca-common version you need
# to provide it as in the following example:
# $ ./build_image.sh master master refs/changes/19/595719/3
[ -z "$DOCKER_IMAGE" ] && \
DOCKER_IMAGE=$(\grep DOCKER_IMAGE Dockerfile | cut -f2 -d"=")
: "${REPO_VERSION:=$1}"
[ -z "$REPO_VERSION" ] && \
REPO_VERSION=$(\grep REPO_VERSION Dockerfile | cut -f2 -d"=")
# Let's stick to more readable version and disable SC2001 here.
# shellcheck disable=SC2001
REPO_VERSION_CLEAN=$(echo "$REPO_VERSION" | sed 's|/|-|g')
[ -z "$APP_REPO" ] && APP_REPO=$(\grep APP_REPO Dockerfile | cut -f2 -d"=")
GITHUB_REPO=$(echo "$APP_REPO" | sed 's/git.openstack.org/github.com/' | \
sed 's/ssh:/https:/')
if [ -z "$CONSTRAINTS_FILE" ]; then
CONSTRAINTS_FILE=$(\grep CONSTRAINTS_FILE Dockerfile | cut -f2 -d"=") || true
: "${CONSTRAINTS_FILE:=http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt}"
fi
: "${CONSTRAINTS_BRANCH:=$2}"
[ -z "$CONSTRAINTS_BRANCH" ] && \
CONSTRAINTS_BRANCH=$(\grep CONSTRAINTS_BRANCH Dockerfile | cut -f2 -d"=")
# When using stable version of repository use same stable constraints file.
case "$REPO_VERSION" in
*stable*)
CONSTRAINTS_BRANCH_CLEAN="$REPO_VERSION"
# Get monasca-common version from stable upper constraints file.
CONSTRAINTS_TMP_FILE=$(mktemp)
wget --output-document "$CONSTRAINTS_TMP_FILE" \
"$CONSTRAINTS_FILE"?h="$CONSTRAINTS_BRANCH_CLEAN"
UPPER_COMMON=$(\grep 'monasca-common' "$CONSTRAINTS_TMP_FILE")
# Get only version part from monasca-common.
UPPER_COMMON_VERSION="${UPPER_COMMON##*===}"
rm -rf "$CONSTRAINTS_TMP_FILE"
;;
*)
CONSTRAINTS_BRANCH_CLEAN="$CONSTRAINTS_BRANCH"
;;
esac
# Monasca-common variables.
if [ -z "$COMMON_REPO" ]; then
COMMON_REPO=$(\grep COMMON_REPO Dockerfile | cut -f2 -d"=") || true
: "${COMMON_REPO:=https://git.openstack.org/openstack/monasca-common}"
fi
: "${COMMON_VERSION:=$3}"
if [ -z "$COMMON_VERSION" ]; then
COMMON_VERSION=$(\grep COMMON_VERSION Dockerfile | cut -f2 -d"=") || true
if [ "$UPPER_COMMON_VERSION" ]; then
# Common from upper constraints file.
COMMON_VERSION="$UPPER_COMMON_VERSION"
fi
fi
# Clone project to temporary directory for getting proper commit number from
# branches and tags. We need this for setting proper image labels.
# Docker does not allow to get any data from inside of system when building
# image.
TMP_DIR=$(mktemp -d)
(
cd "$TMP_DIR"
# This many steps are needed to support gerrit patch sets.
git init
git remote add origin "$APP_REPO"
git fetch origin "$REPO_VERSION"
git reset --hard FETCH_HEAD
)
GIT_COMMIT=$(git -C "$TMP_DIR" rev-parse HEAD)
[ -z "${GIT_COMMIT}" ] && echo "No git commit hash found" && exit 1
rm -rf "$TMP_DIR"
# Do the same for monasca-common.
COMMON_TMP_DIR=$(mktemp -d)
(
cd "$COMMON_TMP_DIR"
# This many steps are needed to support gerrit patch sets.
git init
git remote add origin "$COMMON_REPO"
git fetch origin "$COMMON_VERSION"
git reset --hard FETCH_HEAD
)
COMMON_GIT_COMMIT=$(git -C "$COMMON_TMP_DIR" rev-parse HEAD)
[ -z "${COMMON_GIT_COMMIT}" ] && echo "No git commit hash found" && exit 1
rm -rf "$COMMON_TMP_DIR"
CREATION_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
docker build --no-cache \
--build-arg CREATION_TIME="$CREATION_TIME" \
--build-arg GITHUB_REPO="$GITHUB_REPO" \
--build-arg APP_REPO="$APP_REPO" \
--build-arg REPO_VERSION="$REPO_VERSION" \
--build-arg GIT_COMMIT="$GIT_COMMIT" \
--build-arg CONSTRAINTS_FILE="$CONSTRAINTS_FILE" \
--build-arg CONSTRAINTS_BRANCH="$CONSTRAINTS_BRANCH_CLEAN" \
--build-arg COMMON_REPO="$COMMON_REPO" \
--build-arg COMMON_VERSION="$COMMON_VERSION" \
--build-arg COMMON_GIT_COMMIT="$COMMON_GIT_COMMIT" \
--tag "$DOCKER_IMAGE":"$REPO_VERSION_CLEAN" .

28
docker/forwarder/health_check.py Normal file
View File

@ -0,0 +1,28 @@
#!/usr/bin/env python
# coding=utf-8
# (C) Copyright 2018 FUJITSU LIMITED
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Health check will returns 0 when service is working properly."""
def main():
"""Health check for Monasca-agent forwarder."""
# TODO wait for health check endpoint ...
return 0
if __name__ == '__main__':
main()

7
docker/forwarder/kubernetes_get_host.py Normal file
View File

@ -0,0 +1,7 @@
# (C) Copyright 2017 Hewlett Packard Enterprise Development LP
# coding=utf-8
from monasca_agent.collector.checks import utils
kubernetes_connector = utils.KubernetesConnector(3)
print(kubernetes_connector.get_agent_pod_host(return_host_name=True))

61
docker/forwarder/start.sh Normal file
View File

@ -0,0 +1,61 @@
#!/bin/sh
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Starting script.
# All checks and configuration templating you need to do before service
# could be safely started should be added in this file.
set -eo pipefail # Exit the script if any statement returns error.
AGENT_CONF="/etc/monasca/agent"
if [ "$KEYSTONE_DEFAULTS_ENABLED" = "true" ]; then
export OS_AUTH_URL=${OS_AUTH_URL:-"http://keystone:35357/v3/"}
export OS_USERNAME=${OS_USERNAME:-"monasca-agent"}
export OS_PASSWORD=${OS_PASSWORD:-"password"}
export OS_USER_DOMAIN_NAME=${OS_USER_DOMAIN_NAME:-"Default"}
export OS_PROJECT_NAME=${OS_PROJECT_NAME:-"mini-mon"}
export OS_PROJECT_DOMAIN_NAME=${OS_PROJECT_DOMAIN_NAME:-"Default"}
fi
# Test services we need before starting our service.
#echo "Start script: waiting for needed services"
# Template all config files before start, it will use env variables.
# Read usage examples: https://pypi.org/project/Templer/
echo "Start script: creating config files from templates"
alias template="templer --ignore-undefined-variables --force --verbose"
if [ "$HOSTNAME_FROM_KUBERNETES" = "true" ]; then
if ! AGENT_HOSTNAME=$(python /kubernetes_get_host.py); then
echo "Error getting hostname from Kubernetes"
return 1
fi
export AGENT_HOSTNAME
fi
template $AGENT_CONF/agent.yaml.j2 $AGENT_CONF/agent.yaml
# Start our service.
echo "Start script: starting container"
monasca-forwarder
# Allow server to stay alive in case of failure for 2 hours for debugging.
RESULT=$?
if [ $RESULT != 0 ] && [ "$STAY_ALIVE_ON_FAILURE" = "true" ]; then
echo "Service died, waiting 120 min before exiting"
sleep 7200
fi
exit $RESULT