openstack/monasca-agent
Code Issues Proposed changes
Agent for Monasca
1,575 Commits 8 Branches 49 Tags 26 MiB
Python 94.9%
Shell 3.6%
Jinja 0.7%
Dockerfile 0.2%
Makefile 0.2%
Other 0.4%
833e5946fe
Go to file
Matthew Oliver 833e5946fe Add swift_handoffs check plugin to monasca 
A powerful metric to watch for a swift cluster is the
number of handoff partitions on a drive on a storage node.

A build up of handoff nodes on a particular server could
indicate a disk problem somewhere in the cluster. A bottleneck
somewhere. Or better, when would be a good time to rebalance
the ring (as you'd want to do it when existing backend data
movement is at a minimum.

So it turns out to be a great visualisation of the health of
a cluster.

That's what this check plugin does. Each instance check takes
the following values:

  ring: <path to a Swift ring file>
  devices: <path to the directory of mountpoints>
  granularity: <either server or device>

To be able to determine primary vs handoff partitions on a drive
the swift ring needs to be consulted. If a storage node stores
more then 1 ring, and an instance would be defined for each.

You give swift a bunch of disks. These disks are placed in what
swift calls the 'devices' location. That is a directory where a
mount point for each mounted swift drive is located.

Finally, you can decide on the granularity, which defaults to
`server` if not defined. Only 2 metrics are created from this
check:

  swift.partitions.primary_count
  swift.partitions.handoff_count

But with the hostname dimension a ring dimension will also be set.
Allowing the graphing of the handoff vs partitions of each ring.
When the granularity is set to device, then an additional
dimension to the metric is added, the device name (the name of
the devices mount point). This allows the graphing and monitoring
of each device in a server if a finer granularity is required.

Because we need to consult the Swift ring there is a runtime
requirement on the Python Swift module being installed. But
this isn't required for the unit tests. Making it a runtime
dependency means when the check is loaded it'll log an error
and then exit if it can't import the swift module.

This is the second of two Swift check plugins I've been working on.
For more details see my blog post[1]

[1] - https://oliver.net.au/?p=358

Change-Id: Ie91add9af39f2ab0e5b575390c0c6355563c0bfc
2019-10-18 17:16:14 +02:00
conf.d Add swift_handoffs check plugin to monasca 2019-10-18 17:16:14 +02:00
docker Create Docker image from Monasca repos 2019-09-24 08:45:45 +00:00
docs Add swift_handoffs check plugin to monasca 2019-10-18 17:16:14 +02:00
monasca_agent Add swift_handoffs check plugin to monasca 2019-10-18 17:16:14 +02:00
monasca_setup Update hacking version to 1.1.x 2019-06-26 12:49:13 +05:30
packaging Removed dependency on supervisor 2018-06-27 11:28:43 +02:00
playbooks Push Docker image to Docker Hub from Zuul 2019-09-25 12:30:41 +02:00
tests Add swift_handoffs check plugin to monasca 2019-10-18 17:16:14 +02:00
tests_to_fix Add python36 support 2018-10-25 09:09:53 -04:00
.gitignore Removed dependency on supervisor 2018-06-27 11:28:43 +02:00
.gitreview OpenDev Migration Patch 2019-04-19 19:40:33 +00:00
.stestr.conf Switch to using stestr 2018-08-13 11:19:12 -04:00
.testr.conf Extend CI for monasca-agent 2017-03-30 06:33:59 +02:00
.zuul.yaml Remove monasca-tempest-java-influxdb job 2019-07-10 18:11:09 +02:00
agent.yaml.template Support setting statsd metric aggregation interval 2019-02-21 14:23:21 +00:00
bindep.txt Move other-requirements.txt to bindep.txt 2016-08-25 06:43:35 +00:00
LICENSE Add HPE copyright 2016-02-02 10:34:10 -07:00
mkdocs.yml Add HPE copyright 2016-02-02 10:34:10 -07:00
README.rst Merge "Add information about Prometheus plugin to README" 2019-05-30 13:15:49 +00:00
requirements.txt Removed dependency on supervisor 2018-06-27 11:28:43 +02:00
setup.cfg Add swift_handoffs check plugin to monasca 2019-10-18 17:16:14 +02:00
setup.py Update from global requirements 2017-11-15 16:54:00 +01:00
test-requirements.txt Update hacking version to 1.1.x 2019-06-26 12:49:13 +05:30
tox.ini Update the constraints url 2019-09-26 14:15:58 +08:00

README.rst

Team and repository tags

Team and repository tags

Introduction

The Monasca Agent is a modern Python monitoring agent for gathering metrics and sending them to the Monasca API. The Agent supports collecting metrics from a variety of sources as follows:

System metrics

such as cpu and memory utilization.

Prometheus

The Monasca Agent supports scraping metrics from endpoints provided by Prometheus exporters or Prometheus instrumented applications.

Statsd

The Monasca Agent supports an integrated StatsD daemon which can be used by applications via a statsd client library.

OpenStack metrics

The agent can perform checks on OpenStack processes.

Host alive

The Monasca Agent can perform active checks on a host to determine if it is alive using ping (ICMP) or SSH.

Process checks

The Monasca Agent can check a process and return several metrics on the process such as a number of instances, memory, io and threads.

Http Endpoint checks

The Monasca Agent can perform active checks on http endpoints by sending an HTTP request to an API.

Service checks

The Monasca Agent can check services such as MySQL, RabbitMQ, and many more.

Nagios plugins

The Monasca Agent can run Nagios plugins and send the status code returned by the plugin as a metric to the Monasca API.

The Agent can automatically detect and setup checks on certain processes and resources.

The Agent is extensible through the configuration of additional plugins, written in Python.

Detailed Documentation

For an introduction to the Monasca Agent, including a complete list of the metrics that the Agent supports, see the "Agent" detailed documentation.

The Agent is extensible through the configuration of additional check and setup plugins, written in Python. See the "Agent Customizations" detailed documentation.

Agent

https://opendev.org/openstack/monasca-agent/src/branch/master/docs/Agent.md

Agent Customizations

https://opendev.org/openstack/monasca-agent/src/branch/master/docs/Customizations.md

Monasca Metrics

https://opendev.org/openstack/monasca-agent/src/branch/master/docs/MonascaMetrics.md

Agent Plugin details

https://opendev.org/openstack/monasca-agent/src/branch/master/docs/Plugins.md