diff --git a/.zuul.yaml b/.zuul.yaml index 868595e2..23bcdc2f 100644 --- a/.zuul.yaml +++ b/.zuul.yaml @@ -6,6 +6,7 @@ - openstack-python37-jobs check: jobs: + - docker-build-monasca-service - monasca-tempest-python2-influxdb: voting: false - monasca-tempest-python3-influxdb: diff --git a/docker/README.rst b/docker/README.rst index cc93e612..2036f44c 100644 --- a/docker/README.rst +++ b/docker/README.rst @@ -1,7 +1,10 @@ =============================== Docker images for Monasca Agent =============================== -The Monasca Agent images are based on the monasca-base image. +There are two separate images for monasca-agent services: collector +and forwarder. Collector is working best with services that allow remote access +to them and to gather host level metrics collector will need to work together +with cAdvisor service. Building monasca-base image @@ -12,26 +15,16 @@ See https://github.com/openstack/monasca-common/tree/master/docker/README.rst Building Monasca Agent images ============================= +``build_image.sh`` script in top level folder (``docker/build_image.sh``) is +dummy script that will build both collector and forwarder images at once. + Example: - $ ./build_image.sh + $ ./build_image.sh +Everything after ``./build_image.sh`` is optional and by default configured +to get versions from ``Dockerfile``. ``./build_image.sh`` also contain more +detailed build description. -Requirements from monasca-base image -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -health_check.py - This file will be used for checking the status of the Monasca Agent - (unfortunately monasca agent doesn't have any endpoint for checking health). - -Scripts -~~~~~~~ -start.sh - In this starting script provide all steps that lead to the proper service - start. Including usage of wait scripts and templating of configuration - files. You also could provide the ability to allow running container after - service died for easier debugging. - -build_image.sh - Please read detailed build description inside the script. Environment variables ~~~~~~~~~~~~~~~~~~~~~ @@ -55,7 +48,7 @@ STAY_ALIVE_ON_FAILURE false If true, container ru ============================== =========================== ==================================================== Note that additional variables can be specified as well, see the -[config template][8] for a definitive list. +``agent.yaml.j2`` for a definitive list in every image folder. Note that the auto restart feature can be enabled if the agent collector has unchecked memory growth. The proper restart behavior must be enabled @@ -77,6 +70,18 @@ MONASCA_MONITORING false Monitor services for metrics pipeline MONASCA_LOG_MONITORING false Monitor services for logs pipeline ============================== =========== ===================================== +Scripts +~~~~~~~ +start.sh + In this starting script provide all steps that lead to the proper service + start. Including usage of wait scripts and templating of configuration + files. You also could provide the ability to allow running container after + service died for easier debugging. + +health_check.py + This file will be used for checking the status of the application. + + Docker Plugin ------------- @@ -111,7 +116,7 @@ This plugin is enabled when ``KUBERNETES=true``. It has the following options: The Kubernetes plugin is intended to be run as a DaemonSet on each Kubernetes node. In order for API endpoints to be detected correctly, ``AGENT_POD_NAME`` and -`AGENT_POD_NAMESPACE`` must be set using the [Downward API][7] as described +``AGENT_POD_NAMESPACE`` must be set using the `Downward API`_ as described above. Kubernetes API Plugin @@ -187,10 +192,10 @@ This plugin is enabled when ``CADVISOR=true``. It has the following options: This plugin collects host-level metrics from a running cAdvisor instance. cAdvisor is included in ``kubelet`` when in Kubernetes environments and is necessary to retrieve host-level metrics. As with the Kubernetes plugin, -`AGENT_POD_NAME`` and ``AGENT_POD_NAMESPACE`` must be set to determine the URL +``AGENT_POD_NAME`` and ``AGENT_POD_NAMESPACE`` must be set to determine the URL automatically. -cAdvisor can be easily run in [standard Docker environments][9] or directly on +cAdvisor can be easily run in `standard Docker environments`_ or directly on host systems. In these cases, the URL must be manually provided via ``CADVISOR_URL``. @@ -224,7 +229,7 @@ MySQL Logs pipeline ^^^^^^^^^^^^^ For logs pipeline you can enable HTTP endpoint check, process and -`Elasticsearch`` plugins. This is enabled when ``MONASCA_LOG_MONITORING=true``. +``Elasticsearch`` plugins. This is enabled when ``MONASCA_LOG_MONITORING=true``. You can adjust the configuration of the components by passing environment variables: @@ -250,3 +255,6 @@ templates with access to all environment variables. Links ~~~~~ https://github.com/openstack/monasca-agent/blob/master/README.rst + +.. _`Downward API`: https://kubernetes.io/docs/user-guide/downward-api/ +.. _`standard Docker environments`: https://github.com/google/cadvisor#quick-start-running-cadvisor-in-a-docker-container diff --git a/docker/build_image.sh b/docker/build_image.sh new file mode 100755 index 00000000..53168377 --- /dev/null +++ b/docker/build_image.sh @@ -0,0 +1,58 @@ +#!/bin/bash + +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +# TODO(Dobroslaw): move this script to monasca-common/docker folder +# and leave here small script to download it and execute using env variables +# to minimize code duplication. + +set -x # Print each script step. +set -eo pipefail # Exit the script if any statement returns error. + +# Dummy script for building both images for monasca-agent: collector and +# forwarder. It will relay all arguments to every image build script. + +# This script is used for building Docker image with proper labels +# and proper version of monasca-common. +# +# Example usage: +# $ ./build_image.sh +# +# Everything after `./build_image.sh` is optional and by default configured +# to get versions from `Dockerfile`. +# +# To build from master branch (default): +# $ ./build_image.sh +# To build specific version run this script in the following way: +# $ ./build_image.sh stable/queens +# Building from specific commit: +# $ ./build_image.sh cb7f226 +# When building from a tag monasca-common will be used in version available +# in upper constraint file: +# $ ./build_image.sh 2.5.0 +# To build image from Gerrit patch sets that is targeting branch stable/queens: +# $ ./build_image.sh refs/changes/51/558751/1 stable/queens +# +# If you want to build image with custom monasca-common version you need +# to provide it as in the following example: +# $ ./build_image.sh master master refs/changes/19/595719/3 + +# Go to folder with Docker files. +REAL_PATH=$(python -c "import os,sys; print(os.path.realpath('$0'))") +cd "$(dirname "$REAL_PATH")/../docker/" + +./collector/build_image.sh "$@" + +printf "\n\n\n" + +./forwarder/build_image.sh "$@" diff --git a/docker/collector/build_image.sh b/docker/collector/build_image.sh index 0f54b839..3cefab62 100755 --- a/docker/collector/build_image.sh +++ b/docker/collector/build_image.sh @@ -33,7 +33,7 @@ set -eo pipefail # Exit the script if any statement returns error. # To build specific version run this script in the following way: # $ ./build_image.sh stable/queens # Building from specific commit: -# $ ./build_image.sh cb7f226 +# $ ./build_image.sh cb7f226 # When building from a tag monasca-common will be used in version available # in upper constraint file: # $ ./build_image.sh 2.5.0 @@ -44,6 +44,10 @@ set -eo pipefail # Exit the script if any statement returns error. # to provide it as in the following example: # $ ./build_image.sh master master refs/changes/19/595719/3 +# Go to folder with Docker files. +REAL_PATH=$(python -c "import os,sys; print(os.path.realpath('$0'))") +cd "$(dirname "$REAL_PATH")/../collector/" + [ -z "$DOCKER_IMAGE" ] && \ DOCKER_IMAGE=$(\grep DOCKER_IMAGE Dockerfile | cut -f2 -d"=") @@ -90,7 +94,6 @@ if [ -z "$COMMON_REPO" ]; then COMMON_REPO=$(\grep COMMON_REPO Dockerfile | cut -f2 -d"=") || true : "${COMMON_REPO:=https://git.openstack.org/openstack/monasca-common}" fi - : "${COMMON_VERSION:=$3}" if [ -z "$COMMON_VERSION" ]; then COMMON_VERSION=$(\grep COMMON_VERSION Dockerfile | cut -f2 -d"=") || true diff --git a/docker/forwarder/build_image.sh b/docker/forwarder/build_image.sh index 0f54b839..aa1f8f48 100755 --- a/docker/forwarder/build_image.sh +++ b/docker/forwarder/build_image.sh @@ -33,7 +33,7 @@ set -eo pipefail # Exit the script if any statement returns error. # To build specific version run this script in the following way: # $ ./build_image.sh stable/queens # Building from specific commit: -# $ ./build_image.sh cb7f226 +# $ ./build_image.sh cb7f226 # When building from a tag monasca-common will be used in version available # in upper constraint file: # $ ./build_image.sh 2.5.0 @@ -44,6 +44,10 @@ set -eo pipefail # Exit the script if any statement returns error. # to provide it as in the following example: # $ ./build_image.sh master master refs/changes/19/595719/3 +# Go to folder with Docker files. +REAL_PATH=$(python -c "import os,sys; print(os.path.realpath('$0'))") +cd "$(dirname "$REAL_PATH")/../forwarder/" + [ -z "$DOCKER_IMAGE" ] && \ DOCKER_IMAGE=$(\grep DOCKER_IMAGE Dockerfile | cut -f2 -d"=") @@ -90,7 +94,6 @@ if [ -z "$COMMON_REPO" ]; then COMMON_REPO=$(\grep COMMON_REPO Dockerfile | cut -f2 -d"=") || true : "${COMMON_REPO:=https://git.openstack.org/openstack/monasca-common}" fi - : "${COMMON_VERSION:=$3}" if [ -z "$COMMON_VERSION" ]; then COMMON_VERSION=$(\grep COMMON_VERSION Dockerfile | cut -f2 -d"=") || true