install graphite and grafana as docker containers

This adds additional playbooks to deploy:
   - carbon-cache
   - graphite-web
   - grafana

as containers.  The data should map to the host running dockerd
for data persistence across reboots.
Ensure:
   - persistent data if containers are restarted
   - systemd scripts to ensure containers startup
   - new ports defined in the group_vars/all.yml
   - fixed mismatch between templates and variables
   - minor space issues to keep folks happy
   - Use a help script for docker ps to avoid weirdness with
     '{{.Names}}' in the command line when using the command plugin.

These have been tested on RHEL7 hosts, as well as undercloud
deployed via RDO quickstart.  The following variables are
of note in the group_vars/all.yml and how to use them:

Location on the target docker host where whisper data is kept:

     persistent_carbon_data_path: /data/carbon/whisper

Location on the target docker host where grafana runs to ensure
persistence of grafana related data across reboots (e.g. dashboards, etc):

     persistent_grafana_data_path: /data/grafana

This port is mapped from the docker host to the standard carbon-cache port
on the launched container (basically makes it easy to ensure ports don't
clash in case there is anothre service listening here):

     docker_carbon_cache_port: 2003

This is the web port on the docker host that is proxied back into the
container (useful when there are a number of other web services running,
e.g. if deploying on an undercloud host):

     docker_graphite_port: 8888

Same with the following.  It's not likely there will be clash with this
port but just in case you are running an instance of grafana, this
allows mapping an alternate port to the docker container:

     docker_grafana_port: 3000

This is the published docker image for carbon-cache.  This is based on
the centos7 image using python-carbon rpms.  This is published into
hub.docker.com.  See:  https://hub.docker.com/r/kambiz/carbon-cache/

     carbon_cache_docker_image: kambiz/carbon-cache:0.9.15

This is the published docker image for graphite-web.  This is based on
the centos7 image using httpd and graphite-web rpms.  This is published
into hub.docker.com.  See:  https://hub.docker.com/r/kambiz/graphite-web/

     graphite_web_docker_image: kambiz/graphite-web:0.9.15

This is the official grafana project docker image.  The image will never
change since we are using 2.6.0, rather than "latest".  See the images
here:  https://hub.docker.com/r/grafana/grafana/

     grafana_docker_image: grafana/grafana:2.6.0

Change-Id: Icf63d0b044cded56cbecc7d306847dedd84bec87

ansible/README.md

@@ -42,12 +42,12 @@ Then run gen_hosts.sh script to generate your overcloud's hosts file for ansible
 ##### Install Browbeat
 Image upload requires Ansible 2.0
 ```
-# vi install/group_vars/all  # Edit ansible vars file (Installation parameters)
+# vi install/group_vars/all.yml  # Edit ansible vars file (Installation parameters)
 # ansible-playbook -i hosts install/browbeat.yml
 ```
 
 ##### Install Collectd Agent (Requires a Graphite Server)
-Prior to installing the agent, please review install/group_vars/all file to ensure the correct parameters are passed.
+Prior to installing the agent, please review install/group_vars/all.yml file to ensure the correct parameters are passed.
 ```
 # ansible-playbook -i hosts install/collectd-openstack.yml
 ```
@@ -71,15 +71,27 @@ ansible-playbook -i hosts install/elk-openstack-client.yml --extra-vars 'elk_ser
 ```
 # ansible-playbook -i hosts install/graphite.yml
 ```
+##### Install graphite service as a docker container
+Prior to installing graphite as a docker container, please review install/group_vars/all.yml file and ensure
+the docker related settings will work with your target host.
+```
+# ansible-playbook -i hosts install/graphite-docker.yml
+```
 
 ##### Install grafana service
-Prior to installing grafana, please review install/group_vars/all file and your ansible inventory file
+Prior to installing grafana, please review install/group_vars/all.yml file and your ansible inventory file
 ```
 # ansible-playbook -i hosts install/grafana.yml
 ```
+##### Install grafana service as a docker container
+Prior to installing graphite as a docker container, please review install/group_vars/all.yml file and ensure
+the docker related settings will work with your target host.
+```
+# ansible-playbook -i hosts install/grafana-docker.yml
+```
 
 ##### Install Grafana Dashboards (Requires a Grafana Server)
-Review install/group_vars/all before deploying the grafana dashboards
+Review install/group_vars/all.yml before deploying the grafana dashboards
 ```
 # ansible-playbook -i hosts install/dashboards-openstack.yml
 ```

ansible/install/grafana-docker.yml

@@ -0,0 +1,9 @@
+---
+#
+# Playbook to install grafana
+#
+
+- hosts: grafana
+  remote_user: root
+  roles:
+    - { role: grafana_docker }

ansible/install/graphite-docker.yml

@@ -0,0 +1,9 @@
+---
+#
+# Playbook to install graphite-web
+#
+
+- hosts: graphite
+  remote_user: root
+  roles:
+    - { role: graphite_docker }

ansible/install/group_vars/all.yml

@@ -50,6 +50,19 @@ collectd_controller: true
 collectd_ceph: true
 collectd_compute: false
 
+########################################
+# Docker related
+# (use these if deploying graphite/carbon/grafana as containers)
+########################################
+persistent_carbon_data_path: /data/carbon/whisper
+persistent_grafana_data_path: /data/grafana
+docker_carbon_cache_port: 2003
+docker_graphite_port: 8888
+docker_grafana_port: 3000
+carbon_cache_docker_image: kambiz/carbon-cache:0.9.15
+graphite_web_docker_image: kambiz/graphite-web:0.9.15
+grafana_docker_image: grafana/grafana:2.6.0
+
 ########################################
 # Graphite Configuration
 ########################################

ansible/install/roles/grafana_docker/files/docker-ps-names.sh

@@ -0,0 +1,3 @@
+#!/bin/sh
+
+/usr/bin/docker ps -a  --format '{{ .Names }}'

ansible/install/roles/grafana_docker/files/docker.repo

@@ -0,0 +1,6 @@
+[dockerrepo]
+name=Docker Repository
+baseurl=https://yum.dockerproject.org/repo/main/centos/7
+enabled=1
+gpgcheck=1
+gpgkey=https://yum.dockerproject.org/gpg

ansible/install/roles/grafana_docker/tasks/main.yml

@@ -0,0 +1,87 @@
+---
+#
+# Install/run grafana-server for browbeat
+#
+
+- name: Import EPEL GPG Key
+  rpm_key: key=https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7
+    state=present
+
+- name: Check for EPEL repo
+  yum: name=https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
+    state=present
+
+- name: disable firewalld
+  service: name=firewalld state=stopped enabled=false
+  become: true
+  ignore_errors: true
+
+- name: Install repo file for docker
+  copy:
+    src=docker.repo
+    dest=/etc/yum.repos.d/docker.repo
+    owner=root
+    group=root
+    mode=0644
+  become: true
+
+- name: Install docker rpm
+  yum: name={{ item }} state=present
+  become: true
+  with_items:
+    - docker-engine
+
+# Start docker service
+
+- name: Setup docker service
+  service: name=docker state=started enabled=true
+  become: true
+
+- name: ensure data directory exists
+  file: path={{ persistent_grafana_data_path }} state=directory mode=0777
+
+- name: ensure docker overrides for carbon-cache
+  file: path=/etc/docker/grafana state=directory mode=0755
+
+- name: Install docker-ps script
+  copy:
+    src=docker-ps-names.sh
+    dest=/usr/local/bin/docker-ps-names.sh
+    owner=root
+    group=root
+    mode=0755
+  become: true
+
+- name: check active containers
+  command: /usr/local/bin/docker-ps-names.sh
+  register: docker_ps_a
+
+- name: start grafana docker container
+  command: "{{item}}"
+  ignore_errors: true
+  with_items:
+    - docker kill grafana
+    - docker rm grafana
+    - docker run -d --name grafana -p {{ docker_grafana_port }}:3000 -v {{ persistent_grafana_data_path }}:/var/lib/grafana {{ grafana_docker_image }}
+  when: '"grafana" not in docker_ps_a.stdout'
+
+- name: Setup grafana systemd config
+  template:
+    src=grafana-server.service.j2
+    dest=/etc/systemd/system/grafana-server.service
+    owner=root
+    group=root
+    mode=0644
+  become: true
+  register: systemd_grafana_needs_restart
+
+- name: bounce systemd and grafana-server container
+  shell: /usr/bin/systemctl daemon-reload && /usr/bin/systemctl enable grafana-server && /usr/bin/systemctl restart grafana-server
+  become: true
+  when: systemd_grafana_needs_restart.changed
+
+- name: Disable EPEL Repo
+  ini_file: dest=/etc/yum.repos.d/epel.repo
+    section=epel
+    option=enabled
+    value=0

ansible/install/roles/grafana_docker/templates/grafana-server.service.j2

@@ -0,0 +1,37 @@
+# cat /etc/systemd/system/grafana-server.service
+# This is a systemd file to run this docker container under systemd.
+# To make this work:
+#  * place this file in /etc/systemd/system and run the commands:
+#
+#  systemctl daemon-reload
+#  systemctl enable grafana-server
+#  systemctl start grafana-server
+#
+[Unit]
+Description=grafana-server
+Requires=docker.service
+After=docker.service
+
+[Service]
+Type=simple
+TimeoutStartSec=5m
+# systemd syntax '=-' ignore errors from return codes.
+ExecStartPre=-/usr/bin/docker kill "grafana"
+ExecStartPre=-/usr/bin/docker rm "grafana"
+ExecStartPre=-/usr/bin/mkdir -p {{ persistent_grafana_data_path }}
+ExecStartPrs=-/usr/bin/chmod 777 {{ persistent_grafana_data_path }}
+
+ExecStart=/usr/bin/docker run --name grafana -p {{ docker_grafana_port }}:3000 -v {{ persistent_grafana_data_path }}:/var/lib/grafana {{ grafana_docker_image }}
+
+ExecReload=-/usr/bin/docker stop "grafana"
+ExecReload=-/usr/bin/docker rm "grafana"
+ExecReload=/usr/bin/docker run --name grafana -p {{ docker_grafana_port }}:3000 -v {{ persistent_grafana_data_path }}:/var/lib/grafana {{ grafana_docker_image }}
+
+ExecStop=-/usr/bin/docker stop "grafana"
+ExecStop=-/usr/bin/docker rm "grafana"
+
+Restart=always
+RestartSec=30
+
+[Install]
+WantedBy=default.target

ansible/install/roles/graphite_docker/files/carbon.conf

@@ -0,0 +1,370 @@
+[cache]
+# Configure carbon directories.
+#
+# OS environment variables can be used to tell carbon where graphite is
+# installed, where to read configuration from and where to write data.
+#
+#   GRAPHITE_ROOT        - Root directory of the graphite installation.
+#                          Defaults to ../
+#   GRAPHITE_CONF_DIR    - Configuration directory (where this file lives).
+#                          Defaults to $GRAPHITE_ROOT/conf/
+#   GRAPHITE_STORAGE_DIR - Storage directory for whipser/rrd/log/pid files.
+#                          Defaults to $GRAPHITE_ROOT/storage/
+#
+# To change other directory paths, add settings to this file. The following
+# configuration variables are available with these default values:
+#
+#   STORAGE_DIR    = $GRAPHITE_STORAGE_DIR
+#   LOCAL_DATA_DIR = STORAGE_DIR/whisper/
+#   WHITELISTS_DIR = STORAGE_DIR/lists/
+#   CONF_DIR       = STORAGE_DIR/conf/
+#   LOG_DIR        = STORAGE_DIR/log/
+#   PID_DIR        = STORAGE_DIR/
+#
+# For FHS style directory structures, use:
+#
+#   STORAGE_DIR    = /var/lib/carbon/
+#   CONF_DIR       = /etc/carbon/
+#   LOG_DIR        = /var/log/carbon/
+#   PID_DIR        = /var/run/
+#
+#LOCAL_DATA_DIR = /opt/graphite/storage/whisper/
+
+STORAGE_DIR    = /var/lib/carbon/
+LOCAL_DATA_DIR = /var/lib/carbon/whisper/
+WHITELISTS_DIR = /var/lib/carbon/lists/
+CONF_DIR       = /etc/carbon/
+LOG_DIR        = /var/log/carbon/
+PID_DIR        = /var/run/
+
+# Enable daily log rotation. If disabled, a kill -HUP can be used after a manual rotate
+
+ENABLE_LOGROTATION = True
+
+# Specify the user to drop privileges to
+# If this is blank carbon runs as the user that invokes it
+# This user must have write access to the local data directory
+USER = carbon
+
+#
+# NOTE: The above settings must be set under [relay] and [aggregator]
+#       to take effect for those daemons as well
+
+# Limit the size of the cache to avoid swapping or becoming CPU bound.
+# Sorts and serving cache queries gets more expensive as the cache grows.
+# Use the value "inf" (infinity) for an unlimited cache size.
+MAX_CACHE_SIZE = inf
+
+# Limits the number of whisper update_many() calls per second, which effectively
+# means the number of write requests sent to the disk. This is intended to
+# prevent over-utilizing the disk and thus starving the rest of the system.
+# When the rate of required updates exceeds this, then carbon's caching will
+# take effect and increase the overall throughput accordingly.
+MAX_UPDATES_PER_SECOND = 500
+
+# If defined, this changes the MAX_UPDATES_PER_SECOND in Carbon when a
+# stop/shutdown is initiated.  This helps when MAX_UPDATES_PER_SECOND is
+# relatively low and carbon has cached a lot of updates; it enables the carbon
+# daemon to shutdown more quickly.
+# MAX_UPDATES_PER_SECOND_ON_SHUTDOWN = 1000
+
+# Softly limits the number of whisper files that get created each minute.
+# Setting this value low (like at 50) is a good way to ensure your graphite
+# system will not be adversely impacted when a bunch of new metrics are
+# sent to it. The trade off is that it will take much longer for those metrics'
+# database files to all get created and thus longer until the data becomes usable.
+# Setting this value high (like "inf" for infinity) will cause graphite to create
+# the files quickly but at the risk of slowing I/O down considerably for a while.
+MAX_CREATES_PER_MINUTE = 2000
+
+LINE_RECEIVER_INTERFACE = 0.0.0.0
+LINE_RECEIVER_PORT = 2003
+
+# Set this to True to enable the UDP listener. By default this is off
+# because it is very common to run multiple carbon daemons and managing
+# another (rarely used) port for every carbon instance is not fun.
+ENABLE_UDP_LISTENER = False
+UDP_RECEIVER_INTERFACE = 0.0.0.0
+UDP_RECEIVER_PORT = 2003
+
+PICKLE_RECEIVER_INTERFACE = 0.0.0.0
+PICKLE_RECEIVER_PORT = 2004
+
+# Set to false to disable logging of successful connections
+LOG_LISTENER_CONNECTIONS = True
+
+# Per security concerns outlined in Bug #817247 the pickle receiver
+# will use a more secure and slightly less efficient unpickler.
+# Set this to True to revert to the old-fashioned insecure unpickler.
+USE_INSECURE_UNPICKLER = False
+
+CACHE_QUERY_INTERFACE = 0.0.0.0
+CACHE_QUERY_PORT = 7002
+
+# Set this to False to drop datapoints received after the cache
+# reaches MAX_CACHE_SIZE. If this is True (the default) then sockets
+# over which metrics are received will temporarily stop accepting
+# data until the cache size falls below 95% MAX_CACHE_SIZE.
+USE_FLOW_CONTROL = True
+
+# By default, carbon-cache will log every whisper update and cache hit. This can be excessive and
+# degrade performance if logging on the same volume as the whisper data is stored.
+LOG_UPDATES = False
+LOG_CACHE_HITS = False
+LOG_CACHE_QUEUE_SORTS = True
+
+# The thread that writes metrics to disk can use on of the following strategies
+# determining the order in which metrics are removed from cache and flushed to
+# disk. The default option preserves the same behavior as has been historically
+# available in version 0.9.10.
+#
+# sorted - All metrics in the cache will be counted and an ordered list of
+# them will be sorted according to the number of datapoints in the cache at the
+# moment of the list's creation. Metrics will then be flushed from the cache to
+# disk in that order.
+#
+# max - The writer thread will always pop and flush the metric from cache
+# that has the most datapoints. This will give a strong flush preference to
+# frequently updated metrics and will also reduce random file-io. Infrequently
+# updated metrics may only ever be persisted to disk at daemon shutdown if
+# there are a large number of metrics which receive very frequent updates OR if
+# disk i/o is very slow.
+#
+# naive - Metrics will be flushed from the cache to disk in an unordered
+# fashion. This strategy may be desirable in situations where the storage for
+# whisper files is solid state, CPU resources are very limited or deference to
+# the OS's i/o scheduler is expected to compensate for the random write
+# pattern.
+#
+CACHE_WRITE_STRATEGY = sorted
+
+# On some systems it is desirable for whisper to write synchronously.
+# Set this option to True if you'd like to try this. Basically it will
+# shift the onus of buffering writes from the kernel into carbon's cache.
+WHISPER_AUTOFLUSH = False
+
+# By default new Whisper files are created pre-allocated with the data region
+# filled with zeros to prevent fragmentation and speed up contiguous reads and
+# writes (which are common). Enabling this option will cause Whisper to create
+# the file sparsely instead. Enabling this option may allow a large increase of
+# MAX_CREATES_PER_MINUTE but may have longer term performance implications
+# depending on the underlying storage configuration.
+# WHISPER_SPARSE_CREATE = False
+
+# Only beneficial on linux filesystems that support the fallocate system call.
+# It maintains the benefits of contiguous reads/writes, but with a potentially
+# much faster creation speed, by allowing the kernel to handle the block
+# allocation and zero-ing. Enabling this option may allow a large increase of
+# MAX_CREATES_PER_MINUTE. If enabled on an OS or filesystem that is unsupported
+# this option will gracefully fallback to standard POSIX file access methods.
+WHISPER_FALLOCATE_CREATE = True
+
+# Enabling this option will cause Whisper to lock each Whisper file it writes
+# to with an exclusive lock (LOCK_EX, see: man 2 flock). This is useful when
+# multiple carbon-cache daemons are writing to the same files
+# WHISPER_LOCK_WRITES = False
+
+# Set this to True to enable whitelisting and blacklisting of metrics in
+# CONF_DIR/whitelist and CONF_DIR/blacklist. If the whitelist is missing or
+# empty, all metrics will pass through
+# USE_WHITELIST = False
+
+# By default, carbon itself will log statistics (such as a count,
+# metricsReceived) with the top level prefix of 'carbon' at an interval of 60
+# seconds. Set CARBON_METRIC_INTERVAL to 0 to disable instrumentation
+# CARBON_METRIC_PREFIX = carbon
+# CARBON_METRIC_INTERVAL = 60
+
+# Enable AMQP if you want to receve metrics using an amqp broker
+# ENABLE_AMQP = False
+
+# Verbose means a line will be logged for every metric received
+# useful for testing
+# AMQP_VERBOSE = False
+
+# AMQP_HOST = localhost
+# AMQP_PORT = 5672
+# AMQP_VHOST = /
+# AMQP_USER = guest
+# AMQP_PASSWORD = guest
+# AMQP_EXCHANGE = graphite
+# AMQP_METRIC_NAME_IN_BODY = False
+
+# The manhole interface allows you to SSH into the carbon daemon
+# and get a python interpreter. BE CAREFUL WITH THIS! If you do
+# something like time.sleep() in the interpreter, the whole process
+# will sleep! This is *extremely* helpful in debugging, assuming
+# you are familiar with the code. If you are not, please don't
+# mess with this, you are asking for trouble :)
+#
+# ENABLE_MANHOLE = False
+# MANHOLE_INTERFACE = 127.0.0.1
+# MANHOLE_PORT = 7222
+# MANHOLE_USER = admin
+# MANHOLE_PUBLIC_KEY = ssh-rsa AAAAB3NzaC1yc2EAAAABiwAaAIEAoxN0sv/e4eZCPpi3N3KYvyzRaBaMeS2RsOQ/cDuKv11dlNzVeiyc3RFmCv5Rjwn/lQ79y0zyHxw67qLyhQ/kDzINc4cY41ivuQXm2tPmgvexdrBv5nsfEpjs3gLZfJnyvlcVyWK/lId8WUvEWSWHTzsbtmXAF2raJMdgLTbQ8wE=
+
+# Patterns for all of the metrics this machine will store. Read more at
+# http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol#Bindings
+#
+# Example: store all sales, linux servers, and utilization metrics
+# BIND_PATTERNS = sales.#, servers.linux.#, #.utilization
+#
+# Example: store everything
+# BIND_PATTERNS = #
+
+# To configure special settings for the carbon-cache instance 'b', uncomment this:
+#[cache:b]
+#LINE_RECEIVER_PORT = 2103
+#PICKLE_RECEIVER_PORT = 2104
+#CACHE_QUERY_PORT = 7102
+# and any other settings you want to customize, defaults are inherited
+# from [carbon] section.
+# You can then specify the --instance=b option to manage this instance
+
+
+
+[relay]
+USER = carbon
+LINE_RECEIVER_INTERFACE = 0.0.0.0
+LINE_RECEIVER_PORT = 2013
+PICKLE_RECEIVER_INTERFACE = 0.0.0.0
+PICKLE_RECEIVER_PORT = 2014
+
+# Set to false to disable logging of successful connections
+LOG_LISTENER_CONNECTIONS = True
+
+# Carbon-relay has several options for metric routing controlled by RELAY_METHOD
+#
+# Use relay-rules.conf to route metrics to destinations based on pattern rules
+#RELAY_METHOD = rules
+#
+# Use consistent-hashing for even distribution of metrics between destinations
+#RELAY_METHOD = consistent-hashing
+#
+# Use consistent-hashing but take into account an aggregation-rules.conf shared
+# by downstream carbon-aggregator daemons. This will ensure that all metrics
+# that map to a given aggregation rule are sent to the same carbon-aggregator
+# instance.
+# Enable this for carbon-relays that send to a group of carbon-aggregators
+#RELAY_METHOD = aggregated-consistent-hashing
+RELAY_METHOD = rules
+
+# If you use consistent-hashing you can add redundancy by replicating every
+# datapoint to more than one machine.
+REPLICATION_FACTOR = 1
+
+# This is a list of carbon daemons we will send any relayed or
+# generated metrics to. The default provided would send to a single
+# carbon-cache instance on the default port. However if you
+# use multiple carbon-cache instances then it would look like this:
+#
+# DESTINATIONS = 127.0.0.1:2004:a, 127.0.0.1:2104:b
+#
+# The general form is IP:PORT:INSTANCE where the :INSTANCE part is
+# optional and refers to the "None" instance if omitted.
+#
+# Note that if the destinations are all carbon-caches then this should
+# exactly match the webapp's CARBONLINK_HOSTS setting in terms of
+# instances listed (order matters!).
+#
+# If using RELAY_METHOD = rules, all destinations used in relay-rules.conf
+# must be defined in this list
+DESTINATIONS = 127.0.0.1:2004
+
+# This defines the maximum "message size" between carbon daemons.
+# You shouldn't need to tune this unless you really know what you're doing.
+MAX_DATAPOINTS_PER_MESSAGE = 2000
+MAX_QUEUE_SIZE = 400000
+
+# Set this to False to drop datapoints when any send queue (sending datapoints
+# to a downstream carbon daemon) hits MAX_QUEUE_SIZE. If this is True (the
+# default) then sockets over which metrics are received will temporarily stop accepting
+# data until the send queues fall below 80% MAX_QUEUE_SIZE.
+USE_FLOW_CONTROL = True
+
+# Set this to True to enable whitelisting and blacklisting of metrics in
+# CONF_DIR/whitelist and CONF_DIR/blacklist. If the whitelist is missing or
+# empty, all metrics will pass through
+# USE_WHITELIST = False
+
+# By default, carbon itself will log statistics (such as a count,
+# metricsReceived) with the top level prefix of 'carbon' at an interval of 60
+# seconds. Set CARBON_METRIC_INTERVAL to 0 to disable instrumentation
+# CARBON_METRIC_PREFIX = carbon
+# CARBON_METRIC_INTERVAL = 60
+
+
+[aggregator]
+USER = carbon
+LINE_RECEIVER_INTERFACE = 0.0.0.0
+LINE_RECEIVER_PORT = 2023
+
+PICKLE_RECEIVER_INTERFACE = 0.0.0.0
+PICKLE_RECEIVER_PORT = 2024
+
+# Set to false to disable logging of successful connections
+LOG_LISTENER_CONNECTIONS = True
+
+# If set true, metric received will be forwarded to DESTINATIONS in addition to
+# the output of the aggregation rules. If set false the carbon-aggregator will
+# only ever send the output of aggregation.
+FORWARD_ALL = True
+
+# This is a list of carbon daemons we will send any relayed or
+# generated metrics to. The default provided would send to a single
+# carbon-cache instance on the default port. However if you
+# use multiple carbon-cache instances then it would look like this:
+#
+# DESTINATIONS = 127.0.0.1:2004:a, 127.0.0.1:2104:b
+#
+# The format is comma-delimited IP:PORT:INSTANCE where the :INSTANCE part is
+# optional and refers to the "None" instance if omitted.
+#
+# Note that if the destinations are all carbon-caches then this should
+# exactly match the webapp's CARBONLINK_HOSTS setting in terms of
+# instances listed (order matters!).
+DESTINATIONS = 127.0.0.1:2004
+
+# If you want to add redundancy to your data by replicating every
+# datapoint to more than one machine, increase this.
+REPLICATION_FACTOR = 1
+
+# This is the maximum number of datapoints that can be queued up
+# for a single destination. Once this limit is hit, we will
+# stop accepting new data if USE_FLOW_CONTROL is True, otherwise
+# we will drop any subsequently received datapoints.
+MAX_QUEUE_SIZE = 10000
+
+# Set this to False to drop datapoints when any send queue (sending datapoints
+# to a downstream carbon daemon) hits MAX_QUEUE_SIZE. If this is True (the
+# default) then sockets over which metrics are received will temporarily stop accepting
+# data until the send queues fall below 80% MAX_QUEUE_SIZE.
+USE_FLOW_CONTROL = True
+
+# This defines the maximum "message size" between carbon daemons.
+# You shouldn't need to tune this unless you really know what you're doing.
+MAX_DATAPOINTS_PER_MESSAGE = 500
+
+# This defines how many datapoints the aggregator remembers for
+# each metric. Aggregation only happens for datapoints that fall in
+# the past MAX_AGGREGATION_INTERVALS * intervalSize seconds.
+MAX_AGGREGATION_INTERVALS = 5
+
+# By default (WRITE_BACK_FREQUENCY = 0), carbon-aggregator will write back
+# aggregated data points once every rule.frequency seconds, on a per-rule basis.
+# Set this (WRITE_BACK_FREQUENCY = N) to write back all aggregated data points
+# every N seconds, independent of rule frequency. This is useful, for example,
+# to be able to query partially aggregated metrics from carbon-cache without
+# having to first wait rule.frequency seconds.
+# WRITE_BACK_FREQUENCY = 0
+
+# Set this to True to enable whitelisting and blacklisting of metrics in
+# CONF_DIR/whitelist and CONF_DIR/blacklist. If the whitelist is missing or
+# empty, all metrics will pass through
+# USE_WHITELIST = False
+
+# By default, carbon itself will log statistics (such as a count,
+# metricsReceived) with the top level prefix of 'carbon' at an interval of 60
+# seconds. Set CARBON_METRIC_INTERVAL to 0 to disable instrumentation
+# CARBON_METRIC_PREFIX = carbon
+# CARBON_METRIC_INTERVAL = 60

ansible/install/roles/graphite_docker/files/docker.repo

@@ -0,0 +1,6 @@
+[dockerrepo]
+name=Docker Repository
+baseurl=https://yum.dockerproject.org/repo/main/centos/7
+enabled=1
+gpgcheck=1
+gpgkey=https://yum.dockerproject.org/gpg

ansible/install/roles/graphite_docker/files/storage-schemas.conf

@@ -0,0 +1,17 @@
+# Schema definitions for Whisper files. Entries are scanned in order,
+# and first match wins. This file is scanned for changes every 60 seconds.
+#
+#  [name]
+#  pattern = regex
+#  retentions = timePerPoint:timeToStore, timePerPoint:timeToStore, ...
+#
+# Carbon's internal metrics. This entry should match what is specified in
+# CARBON_METRIC_PREFIX and CARBON_METRIC_INTERVAL settings
+[carbon]
+pattern = ^carbon\.
+retentions = 60:90d
+
+[default]
+pattern = .*
+retentions = 10s:7d,60s:90d,1h:180d
+

ansible/install/roles/graphite_docker/tasks/main.yml

@@ -0,0 +1,125 @@
+---
+#
+# Install/run graphite-web for browbeat
+#
+
+- name: Import EPEL GPG Key
+  rpm_key: key=https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7
+    state=present
+
+- name: Check for EPEL repo
+  yum: name=https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
+    state=present
+
+- name: Install repo file for docker
+  copy:
+    src=docker.repo
+    dest=/etc/yum.repos.d/docker.repo
+    owner=root
+    group=root
+    mode=0644
+  become: true
+
+# disable firewalld (might need to create specific firewall rules or leave it to admin to do via iptables)
+
+- name: disable firewalld
+  service: name=firewalld state=stopped enabled=false
+  become: true
+  ignore_errors: true
+
+- name: Install docker rpm
+  yum: name={{ item }} state=present
+  become: true
+  with_items:
+    - docker-engine
+
+# Start docker service
+
+- name: Setup docker service
+  service: name=docker state=started enabled=true
+  become: true
+
+- name: ensure data directory exists
+  file: path={{ persistent_carbon_data_path }} state=directory mode=0777
+
+- name: ensure docker overrides for carbon-cache
+  file: path=/etc/docker/carbon-cache state=directory mode=0755
+
+- name: Install carbon.conf
+  copy:
+    src=carbon.conf
+    dest=/etc/docker/carbon-cache/carbon.conf
+    owner=root
+    group=root
+    mode=0644
+  become: true
+
+- name: Install storage-schemas.conf
+  copy:
+    src=storage-schemas.conf
+    dest=/etc/docker/carbon-cache/storage-schemas.conf
+    owner=root
+    group=root
+    mode=0644
+  become: true
+
+- name: check active containers
+  command: /usr/bin/docker ps -a  --format '{{ "{{" }}.Names{{ "}}" }}'
+  register: docker_ps_a
+
+- name: start carbon-cache docker container
+  command: "{{item}}"
+  ignore_errors: true
+  with_items:
+    - docker kill carbon-cache
+    - docker rm carbon-cache
+    - sleep 5
+    - docker run -d --name carbon-cache -p {{ docker_carbon_cache_port }}:2003 -v /etc/docker/carbon-cache/carbon.conf:/etc/carbon/carbon.conf -v /etc/docker/carbon-cache/storage-schemas.conf:/etc/carbon/storage-schemas.conf -v {{ persistent_carbon_data_path }}:/var/lib/carbon/whisper {{ carbon_cache_docker_image }}
+  when: '"carbon-cache" not in docker_ps_a.stdout'
+
+- name: start graphite-web docker container
+  command: "{{item}}"
+  ignore_errors: true
+  with_items:
+    - docker kill graphite-web
+    - docker rm graphite-web
+    - sleep 5
+    - docker run -d --name graphite-web -p {{ docker_graphite_port }}:80 -v {{ persistent_carbon_data_path }}:/var/lib/carbon/whisper {{ graphite_web_docker_image }}
+  when: '"graphite-web" not in docker_ps_a.stdout'
+
+- name: Setup graphite-web systemd config
+  template:
+    src=graphite-web.service.j2
+    dest=/etc/systemd/system/graphite-web.service
+    owner=root
+    group=root
+    mode=0644
+  become: true
+  register: systemd_graphite_needs_restart
+
+- name: bounce systemd and graphite-web container
+  shell: /usr/bin/systemctl daemon-reload && /usr/bin/systemctl enable graphite-web && /usr/bin/systemctl restart graphite-web
+  become: true
+  when: systemd_graphite_needs_restart.changed
+
+- name: Setup carbon-cache systemd config
+  template:
+    src=carbon-cache.service.j2
+    dest=/etc/systemd/system/carbon-cache.service
+    owner=root
+    group=root
+    mode=0644
+  become: true
+  register: systemd_carbon_needs_restart
+
+- name: bounce systemd and carbon-cache container
+  shell: /usr/bin/systemctl daemon-reload && /usr/bin/systemctl enable carbon-cache && /usr/bin/systemctl restart carbon-cache
+  become: true
+  when: systemd_carbon_needs_restart.changed
+
+
+- name: Disable EPEL Repo
+  ini_file: dest=/etc/yum.repos.d/epel.repo
+    section=epel
+    option=enabled
+    value=0

ansible/install/roles/graphite_docker/templates/carbon-cache.service.j2

@@ -0,0 +1,37 @@
+# cat /etc/systemd/system/carbon-cache.service
+# This is a systemd file to run this docker container under systemd.
+# To make this work:
+#  * place this file in /etc/systemd/system and run the commands:
+#
+#  systemctl daemon-reload
+#  systemctl enable carbon-cache
+#  systemctl start carbon-cache
+#
+[Unit]
+Description=carbon-cache
+Requires=docker.service
+After=docker.service
+
+[Service]
+Type=simple
+TimeoutStartSec=5m
+# systemd syntax '=-' ignore errors from return codes.
+ExecStartPre=-/usr/bin/docker kill "carbon-cache"
+ExecStartPre=-/usr/bin/docker rm "carbon-cache"
+ExecStartPre=-/usr/bin/mkdir -p {{ persistent_carbon_data_path }}
+ExecStartPre=-/usr/bin/chmod 777 {{ persistent_carbon_data_path }}
+
+ExecStart=/usr/bin/docker run --name carbon-cache -p {{ docker_carbon_cache_port }}:2003 -v {{ persistent_carbon_data_path }}:/var/lib/carbon/whisper {{ carbon_cache_docker_image }}
+
+ExecReload=-/usr/bin/docker stop "carbon-cache"
+ExecReload=-/usr/bin/docker rm "carbon-cache"
+ExecReload=/usr/bin/docker run --name carbon-cache -p {{ docker_carbon_cache_port }}:2003 -v {{ persistent_carbon_data_path }}:/var/lib/carbon/whisper {{ carbon_cache_docker_image }}
+
+ExecStop=-/usr/bin/docker stop "carbon-cache"
+ExecStop=-/usr/bin/docker rm "carbon-cache"
+
+Restart=always
+RestartSec=30
+
+[Install]
+WantedBy=default.target

ansible/install/roles/graphite_docker/templates/graphite-web.service.j2

@@ -0,0 +1,37 @@
+# cat /etc/systemd/system/graphite-web.service
+# This is a systemd file to run this docker container under systemd.
+# To make this work:
+#  * place this file in /etc/systemd/system and run the commands:
+#
+#  systemctl daemon-reload
+#  systemctl enable graphite-web
+#  systemctl start graphite-web
+#
+[Unit]
+Description=graphite-web
+Requires=docker.service
+After=docker.service
+
+[Service]
+Type=simple
+TimeoutStartSec=5m
+# systemd syntax '=-' ignore errors from return codes.
+ExecStartPre=-/usr/bin/docker kill "graphite-web"
+ExecStartPre=-/usr/bin/docker rm "graphite-web"
+ExecStartPre=-/usr/bin/mkdir -p {{ persistent_carbon_data_path }}
+ExecStartPre=-/usr/bin/chmod 777 {{ persistent_carbon_data_path }}
+
+ExecStart=/usr/bin/docker run --name graphite-web -p {{ docker_graphite_port }}:80 -v {{ persistent_carbon_data_path }}:/var/lib/carbon/whisper {{ graphite_web_docker_image }}
+
+ExecReload=-/usr/bin/docker stop "graphite-web"
+ExecReload=-/usr/bin/docker rm "graphite-web"
+ExecReload=/usr/bin/docker run --name graphite-web -p {{ docker_graphite_port }}:80 -v {{ persistent_carbon_data_path }}:/var/lib/carbon/whisper {{ graphite_web_docker_image }}
+
+ExecStop=-/usr/bin/docker stop "graphite-web"
+ExecStop=-/usr/bin/docker rm "graphite-web"
+
+Restart=always
+RestartSec=30
+
+[Install]
+WantedBy=default.target