Browse Source

Add prometheus-adapter

This will install the prometheus-adapter stable
helm chart. Requires monitoring_enabled=true.

The chart version can be configured using
prometheus_adapter_chart_tag and an option is
available to overwrite the default configuration
rules for a user defined ConfigMap referenced
by using prometheus_adapter_configmap label.

story: 2006765
task: 37278

Change-Id: I5b86f4455f88c8dbeac6e56942e1ca55f1d1726c
Signed-off-by: Diogo Guerra <diogo.filipe.tomas.guerra@cern.ch>
changes/46/691646/6
Diogo Guerra 3 years ago committed by Diogo Guerra
parent
commit
354575804f
  1. 22
      doc/source/user/index.rst
  2. 3
      magnum/drivers/common/templates/kubernetes/fragments/write-heat-params-master.sh
  3. 122
      magnum/drivers/common/templates/kubernetes/helm/prometheus-adapter.sh
  4. 3
      magnum/drivers/heat/k8s_fedora_template_def.py
  5. 19
      magnum/drivers/k8s_fedora_atomic_v1/templates/kubecluster.yaml
  6. 15
      magnum/drivers/k8s_fedora_atomic_v1/templates/kubemaster.yaml
  7. 19
      magnum/drivers/k8s_fedora_coreos_v1/templates/kubecluster.yaml
  8. 15
      magnum/drivers/k8s_fedora_coreos_v1/templates/kubemaster.yaml
  9. 18
      magnum/tests/unit/drivers/test_template_definition.py
  10. 10
      releasenotes/notes/prometheus-adapter-15fba9d739676e70.yaml

22
doc/source/user/index.rst

@ -311,6 +311,13 @@ the table are linked to more details elsewhere in the user guide.
+---------------------------------------+--------------------+---------------+
| `prometheus_operator_chart_tag`_ | see below | see below |
+---------------------------------------+--------------------+---------------+
| `prometheus_adapter_enabled`_ | - true | true |
| | - false | |
+---------------------------------------+--------------------+---------------+
| `prometheus_adapter_chart_tag`_ | see below | see below |
+---------------------------------------+--------------------+---------------+
| `prometheus_adapter_configmap`_ | (rules CM name) | "" |
+---------------------------------------+--------------------+---------------+
| `swarm_strategy`_ | - spread | spread |
| | - binpack | |
| | - random | |
@ -1312,6 +1319,21 @@ _`monitoring_enabled`
stable/prometheus-operator helm chart.
Default: false
_`prometheus_adapter_enabled`
Enable installation of cluster custom metrics provided by the
stable/prometheus-adapter helm chart. This service depends on
monitoring_enabled.
Default: true
_`prometheus_adapter_chart_tag`
The stable/prometheus-adapter helm chart version to use.
Train-default: 1.4.0
_`prometheus_adapter_configmap`
The name of the prometheus-adapter rules ConfigMap to use. Using this label
will overwrite the default rules.
Default: ""
_`prometheus_operator_chart_tag`
Add prometheus_operator_chart_tag to select version of the
stable/prometheus-operator chart to install. When installing the chart,

3
magnum/drivers/common/templates/kubernetes/fragments/write-heat-params-master.sh

@ -44,6 +44,9 @@ HEAT_PARAMS=/etc/sysconfig/heat-params
MAGNUM_URL="$MAGNUM_URL"
MONITORING_ENABLED="$MONITORING_ENABLED"
PROMETHEUS_OPERATOR_CHART_TAG="$PROMETHEUS_OPERATOR_CHART_TAG"
PROMETHEUS_ADAPTER_ENABLED="$PROMETHEUS_ADAPTER_ENABLED"
PROMETHEUS_ADAPTER_CHART_TAG="$PROMETHEUS_ADAPTER_CHART_TAG"
PROMETHEUS_ADAPTER_CONFIGMAP="$PROMETHEUS_ADAPTER_CONFIGMAP"
VOLUME_DRIVER="$VOLUME_DRIVER"
REGION_NAME="$REGION_NAME"
HTTP_PROXY="$HTTP_PROXY"

122
magnum/drivers/common/templates/kubernetes/helm/prometheus-adapter.sh

@ -0,0 +1,122 @@
#!/bin/bash
. /etc/sysconfig/heat-params
set -ex
step="prometheus-adapter"
printf "Starting to run ${step}\n"
### Configuration
###############################################################################
CHART_NAME="prometheus-adapter"
if [ "$(echo ${MONITORING_ENABLED} | tr '[:upper:]' '[:lower:]')" = "true" ] && \
[ "$(echo ${PROMETHEUS_ADAPTER_ENABLED} | tr '[:upper:]' '[:lower:]')" = "true" ]; then
HELM_MODULE_CONFIG_FILE="/srv/magnum/kubernetes/helm/${CHART_NAME}.yaml"
[ -f ${HELM_MODULE_CONFIG_FILE} ] || {
echo "Writing File: ${HELM_MODULE_CONFIG_FILE}"
mkdir -p $(dirname ${HELM_MODULE_CONFIG_FILE})
cat << EOF > ${HELM_MODULE_CONFIG_FILE}
---
kind: ConfigMap
apiVersion: v1
metadata:
name: ${CHART_NAME}-config
namespace: magnum-tiller
labels:
app: helm
data:
install-${CHART_NAME}.sh: |
#!/bin/bash
set -ex
mkdir -p \${HELM_HOME}
cp /etc/helm/* \${HELM_HOME}
# HACK - Force wait because of bug https://github.com/helm/helm/issues/5170
until helm init --client-only --wait
do
sleep 5s
done
helm repo update
if [[ \$(helm history ${CHART_NAME} | grep ${CHART_NAME}) ]]; then
echo "${CHART_NAME} already installed on server. Continue..."
exit 0
else
# TODO: Set namespace to monitoring. This is needed as the Kubernetes default priorityClass can only be used in NS kube-system
helm install stable/${CHART_NAME} --namespace kube-system --name ${CHART_NAME} --version ${PROMETHEUS_ADAPTER_CHART_TAG} --values /opt/magnum/install-${CHART_NAME}-values.yaml
fi
install-${CHART_NAME}-values.yaml: |
image:
repository: ${CONTAINER_INFRA_PREFIX:-docker.io/directxman12/}k8s-prometheus-adapter-amd64
priorityClassName: "system-cluster-critical"
prometheus:
url: http://web.tcp.prometheus-prometheus.kube-system.svc.cluster.local
resources:
requests:
cpu: 150m
memory: 400Mi
rules:
existing: ${PROMETHEUS_ADAPTER_CONFIGMAP}
# tls:
# enable: true
# ca: |-
# # Public CA file that signed the APIService
# key: |-
# # Private key of the APIService
# certificate: |-
# # Public key of the APIService
---
apiVersion: batch/v1
kind: Job
metadata:
name: install-${CHART_NAME}-job
namespace: magnum-tiller
spec:
backoffLimit: 5
template:
spec:
serviceAccountName: tiller
containers:
- name: config-helm
image: ${CONTAINER_INFRA_PREFIX:-docker.io/openstackmagnum/}helm-client:dev
command:
- bash
args:
- /opt/magnum/install-${CHART_NAME}.sh
env:
- name: HELM_HOME
value: /helm_home
- name: TILLER_NAMESPACE
value: magnum-tiller
- name: HELM_TLS_ENABLE
value: "true"
volumeMounts:
- name: install-${CHART_NAME}-config
mountPath: /opt/magnum/
- mountPath: /etc/helm
name: helm-client-certs
restartPolicy: Never
volumes:
- name: install-${CHART_NAME}-config
configMap:
name: ${CHART_NAME}-config
- name: helm-client-certs
secret:
secretName: helm-client-secret
EOF
}
fi
printf "Finished running ${step}\n"

3
magnum/drivers/heat/k8s_fedora_template_def.py

@ -87,6 +87,9 @@ class K8sFedoraTemplateDefinition(k8s_template_def.K8sTemplateDefinition):
'keystone_auth_enabled', 'k8s_keystone_auth_tag',
'monitoring_enabled',
'prometheus_operator_chart_tag',
'prometheus_adapter_enabled',
'prometheus_adapter_chart_tag',
'prometheus_adapter_configmap',
'tiller_enabled',
'tiller_tag',
'tiller_namespace',

19
magnum/drivers/k8s_fedora_atomic_v1/templates/kubecluster.yaml

@ -666,6 +666,21 @@ parameters:
description: The stable/prometheus-operator chart version to use.
default: 5.12.3
prometheus_adapter_enabled:
type: boolean
description: Enable or disable prometheus-adapter custom metrics.
default: true
prometheus_adapter_chart_tag:
type: string
description: The stable/prometheus-adapter chart version to use.
default: 1.4.0
prometheus_adapter_configmap:
type: string
description: The prometheus adapter rules ConfigMap name to use as overwrite.
default: ""
project_id:
type: string
description: >
@ -1076,6 +1091,9 @@ resources:
k8s_keystone_auth_tag: {get_param: k8s_keystone_auth_tag}
monitoring_enabled: {get_param: monitoring_enabled}
prometheus_operator_chart_tag: {get_param: prometheus_operator_chart_tag}
prometheus_adapter_enabled: {get_param: prometheus_adapter_enabled}
prometheus_adapter_chart_tag: {get_param: prometheus_adapter_chart_tag}
prometheus_adapter_configmap: {get_param: prometheus_adapter_configmap}
project_id: {get_param: project_id}
tiller_enabled: {get_param: tiller_enabled}
tiller_tag: {get_param: tiller_tag}
@ -1136,6 +1154,7 @@ resources:
template: {get_file: ../../common/templates/kubernetes/helm/prometheus-operator.sh}
params:
"${ADMIN_PASSWD}": {get_param: grafana_admin_passwd}
- get_file: ../../common/templates/kubernetes/helm/prometheus-adapter.sh
- get_file: ../../common/templates/kubernetes/helm/ingress-nginx.sh
- get_file: ../../common/templates/kubernetes/fragments/install-helm-modules.sh

15
magnum/drivers/k8s_fedora_atomic_v1/templates/kubemaster.yaml

@ -459,6 +459,18 @@ parameters:
description: The stable/prometheus-operator chart version to use.
default: 5.12.3
prometheus_adapter_enabled:
type: boolean
description: Enable or disable prometheus-adapter custom metrics.
prometheus_adapter_chart_tag:
type: string
description: The stable/prometheus-adapter chart version to use.
prometheus_adapter_configmap:
type: string
description: The prometheus adapter rules ConfigMap name to use as overwrite.
project_id:
type: string
description: >
@ -691,6 +703,9 @@ resources:
"$K8S_KEYSTONE_AUTH_TAG": {get_param: k8s_keystone_auth_tag}
"$MONITORING_ENABLED": {get_param: monitoring_enabled}
"$PROMETHEUS_OPERATOR_CHART_TAG": {get_param: prometheus_operator_chart_tag}
"$PROMETHEUS_ADAPTER_ENABLED": {get_param: prometheus_adapter_enabled}
"$PROMETHEUS_ADAPTER_CHART_TAG": {get_param: prometheus_adapter_chart_tag}
"$PROMETHEUS_ADAPTER_CONFIGMAP": {get_param: prometheus_adapter_configmap}
"$PROJECT_ID": {get_param: project_id}
"$EXTERNAL_NETWORK_ID": {get_param: external_network}
"$TILLER_ENABLED": {get_param: tiller_enabled}

19
magnum/drivers/k8s_fedora_coreos_v1/templates/kubecluster.yaml

@ -666,6 +666,21 @@ parameters:
description: The stable/prometheus-operator chart version to use.
default: 5.12.3
prometheus_adapter_enabled:
type: boolean
description: Enable or disable prometheus-adapter custom metrics.
default: true
prometheus_adapter_chart_tag:
type: string
description: The stable/prometheus-adapter chart version to use.
default: 1.4.0
prometheus_adapter_configmap:
type: string
description: The prometheus adapter rules ConfigMap name to use as overwrite.
default: ""
project_id:
type: string
description: >
@ -1079,6 +1094,9 @@ resources:
k8s_keystone_auth_tag: {get_param: k8s_keystone_auth_tag}
monitoring_enabled: {get_param: monitoring_enabled}
prometheus_operator_chart_tag: {get_param: prometheus_operator_chart_tag}
prometheus_adapter_enabled: {get_param: prometheus_adapter_enabled}
prometheus_adapter_chart_tag: {get_param: prometheus_adapter_chart_tag}
prometheus_adapter_configmap: {get_param: prometheus_adapter_configmap}
project_id: {get_param: project_id}
tiller_enabled: {get_param: tiller_enabled}
tiller_tag: {get_param: tiller_tag}
@ -1139,6 +1157,7 @@ resources:
template: {get_file: ../../common/templates/kubernetes/helm/prometheus-operator.sh}
params:
"${ADMIN_PASSWD}": {get_param: grafana_admin_passwd}
- get_file: ../../common/templates/kubernetes/helm/prometheus-adapter.sh
- get_file: ../../common/templates/kubernetes/helm/ingress-nginx.sh
- get_file: ../../common/templates/kubernetes/fragments/install-helm-modules.sh

15
magnum/drivers/k8s_fedora_coreos_v1/templates/kubemaster.yaml

@ -463,6 +463,18 @@ parameters:
description: The stable/prometheus-operator chart version to use.
default: 5.12.3
prometheus_adapter_enabled:
type: boolean
description: Enable or disable prometheus-adapter custom metrics.
prometheus_adapter_chart_tag:
type: string
description: The stable/prometheus-adapter chart version to use.
prometheus_adapter_configmap:
type: string
description: The prometheus adapter rules ConfigMap name to use as overwrite.
project_id:
type: string
description: >
@ -697,6 +709,9 @@ resources:
"$K8S_KEYSTONE_AUTH_TAG": {get_param: k8s_keystone_auth_tag}
"$MONITORING_ENABLED": {get_param: monitoring_enabled}
"$PROMETHEUS_OPERATOR_CHART_TAG": {get_param: prometheus_operator_chart_tag}
"$PROMETHEUS_ADAPTER_ENABLED": {get_param: prometheus_adapter_enabled}
"$PROMETHEUS_ADAPTER_CHART_TAG": {get_param: prometheus_adapter_chart_tag}
"$PROMETHEUS_ADAPTER_CONFIGMAP": {get_param: prometheus_adapter_configmap}
"$PROJECT_ID": {get_param: project_id}
"$EXTERNAL_NETWORK_ID": {get_param: external_network}
"$TILLER_ENABLED": {get_param: tiller_enabled}

18
magnum/tests/unit/drivers/test_template_definition.py

@ -546,6 +546,12 @@ class AtomicK8sTemplateDefinitionTestCase(BaseK8sTemplateDefinitionTestCase):
'monitoring_enabled')
prometheus_operator_chart_tag = mock_cluster.labels.get(
'prometheus_operator_chart_tag')
prometheus_adapter_enabled = mock_cluster.labels.get(
'prometheus_adapter_enabled')
prometheus_adapter_chart_tag = mock_cluster.labels.get(
'prometheus_adapter_chart_tag')
prometheus_adapter_configmap = mock_cluster.labels.get(
'prometheus_adapter_configmap')
project_id = mock_cluster.project_id
tiller_enabled = mock_cluster.labels.get(
'tiller_enabled')
@ -639,6 +645,9 @@ class AtomicK8sTemplateDefinitionTestCase(BaseK8sTemplateDefinitionTestCase):
'k8s_keystone_auth_tag': k8s_keystone_auth_tag,
'monitoring_enabled': monitoring_enabled,
'prometheus_operator_chart_tag': prometheus_operator_chart_tag,
'prometheus_adapter_enabled': prometheus_adapter_enabled,
'prometheus_adapter_chart_tag': prometheus_adapter_chart_tag,
'prometheus_adapter_configmap': prometheus_adapter_configmap,
'project_id': project_id,
'external_network': external_network_id,
'tiller_enabled': tiller_enabled,
@ -992,6 +1001,12 @@ class AtomicK8sTemplateDefinitionTestCase(BaseK8sTemplateDefinitionTestCase):
'monitoring_enabled')
prometheus_operator_chart_tag = mock_cluster.labels.get(
'prometheus_operator_chart_tag')
prometheus_adapter_enabled = mock_cluster.labels.get(
'prometheus_adapter_enabled')
prometheus_adapter_chart_tag = mock_cluster.labels.get(
'prometheus_adapter_chart_tag')
prometheus_adapter_configmap = mock_cluster.labels.get(
'prometheus_adapter_configmap')
project_id = mock_cluster.project_id
tiller_enabled = mock_cluster.labels.get(
'tiller_enabled')
@ -1087,6 +1102,9 @@ class AtomicK8sTemplateDefinitionTestCase(BaseK8sTemplateDefinitionTestCase):
'k8s_keystone_auth_tag': k8s_keystone_auth_tag,
'monitoring_enabled': monitoring_enabled,
'prometheus_operator_chart_tag': prometheus_operator_chart_tag,
'prometheus_adapter_enabled': prometheus_adapter_enabled,
'prometheus_adapter_chart_tag': prometheus_adapter_chart_tag,
'prometheus_adapter_configmap': prometheus_adapter_configmap,
'project_id': project_id,
'external_network': external_network_id,
'tiller_enabled': tiller_enabled,

10
releasenotes/notes/prometheus-adapter-15fba9d739676e70.yaml

@ -0,0 +1,10 @@
---
features:
- |
Added custom.metrics.k8s.io API installer by means of stable/prometheus-adapter
helm chart. The label prometheus_adapter_enabled (default: true) controls
configuration. You can also use prometheus_adapter_chart_tag to select helm
chart version, and prometheus_adapter_configmap if you would like to setup
your own metrics (specifying this other than default overwrites default
configurations).
This feature requires the usage of label monitoring_enabled=true.
Loading…
Cancel
Save