a88a7560c7
This updates the "architecture" section of the documentation. Architecture schemas have been added and information about the different parts has been updated and extended. Story: 2004179 Task: 28515 Change-Id: Ib20409079a1b5aadabeac28a12ec2baa7009ddf6
141 lines
4.9 KiB
ReStructuredText
141 lines
4.9 KiB
ReStructuredText
=========================
|
|
CloudKitty's Architecture
|
|
=========================
|
|
|
|
CloudKitty can be cut into four big parts:
|
|
|
|
* Data retrieval (API)
|
|
* Data collection (``cloudkitty-processor``)
|
|
* Data rating
|
|
* Data storage
|
|
|
|
These parts are handled by two processes: ``cloudkitty-api`` and
|
|
``cloudkitty-processor``. The data retrieval part is handled by the
|
|
``cloudkitty-api`` process, the other ones are handled by
|
|
``cloudkitty-processor``.
|
|
|
|
The following is an overview of CloudKitty's architecture:
|
|
|
|
.. image:: ../images/cloudkitty_architecture.png
|
|
:scale: 70%
|
|
|
|
For details about the API, see the `api reference`_
|
|
|
|
The processor falls into the following parts:
|
|
|
|
* The **fetcher** retrieves a list of **scopes** to rate. A scope
|
|
distinguishes and isolates data. It also allows to split the workload
|
|
between several cloudkitty-processor workers. It can be anything
|
|
that makes sense in a given context, like an OpenStack project or a
|
|
Kubernetes namespace.
|
|
|
|
* The **collector** collects data from a source for a given scope and
|
|
metric.
|
|
|
|
* The collected data is then passed to the **rating modules** (several modules
|
|
can be enabled at the same time). These will apply user-defined rating rules
|
|
to the collected data.
|
|
|
|
* Once the data has been rated, it is passed to the **storage driver**, which
|
|
will store it in a given storage backend. This data will then be available
|
|
through the API.
|
|
|
|
.. _api reference: ../api-reference/index.html
|
|
|
|
Module loading and extensions
|
|
=============================
|
|
|
|
Nearly every part of CloudKitty makes use of stevedore_ to load extensions
|
|
dynamically. The following schema shows the modular parts:
|
|
|
|
.. image:: ../images/cloudkitty_modules.png
|
|
:scale: 70%
|
|
|
|
Every rating module is loaded at runtime and can be enabled/disabled directly
|
|
via CloudKitty's API. The module is responsible of its own API to ease the
|
|
management of its configuration.
|
|
|
|
Collectors, fetchers and the storage backend are loaded at runtime but must be
|
|
configured in CloudKitty's configuration file.
|
|
|
|
.. _stevedore: https://docs.openstack.org/stevedore/latest/
|
|
|
|
Fetcher
|
|
=======
|
|
|
|
Four fetchers are available in cloudkitty:
|
|
|
|
* The ``keystone`` fetcher retrieves a list of projects on which the
|
|
cloudkitty user has the ``rating`` role from Keystone.
|
|
|
|
* The ``gnocchi`` fetcher retrieves a list of attributes from `Gnocchi`_ for a
|
|
given resource type. This is used for standalone Gnocchi deployments or to
|
|
discover new projects from Gnocchi when it is used with OpenStack. It can be
|
|
used in an OpenStack context or with a standalone Gnocchi deployment.
|
|
|
|
* The ``prometheus`` fetcher works in a similar way to the Gnocchi fetcher,
|
|
which allows to discover scopes from `Prometheus`_.
|
|
|
|
* The ``source`` fetcher is the simplest one: it reads a list of scopes from
|
|
the configuation file and provides it to the collector.
|
|
|
|
Details about the configuration of each fetcher are available in the
|
|
`fetcher configuration guide`_ .
|
|
|
|
.. _fetcher configuration guide: configuration/fetcher.html
|
|
|
|
Collector
|
|
=========
|
|
|
|
There are three collectors available in CloudKitty:
|
|
|
|
* The ``gnocchi`` collector retrieves data from `Gnocchi`_. It can be used in
|
|
an OpenStack context or with a standalone Gnocchi deployment.
|
|
|
|
* The ``monasca`` collector retrieves data from `Monasca`_. Keystone
|
|
authentication is required for this collector.
|
|
|
|
* The ``prometheus`` collector retrieves data from `Prometheus`_.
|
|
|
|
Details about the configuration of each collector are available in the
|
|
`collector configuration guide`_.
|
|
|
|
For information about how to write a custom collector, see
|
|
the `developer documentation`_.
|
|
|
|
.. _developer documentation: ../developer/collector.html
|
|
.. _collector configuration guide: configuration/collector.html
|
|
.. _Gnocchi: https://gnocchi.xyz/
|
|
.. _Monasca: https://docs.openstack.org/monasca-api/latest/
|
|
.. _Prometheus: https://prometheus.io/docs/introduction/overview/
|
|
|
|
Rating
|
|
======
|
|
|
|
Two rating modules are available in cloudkitty (``noop`` is not considered a
|
|
real module, as it does nothing). Several rating modules can be enabled at the
|
|
same time. Data will be passed to the enabled modules consecutively. The
|
|
module priority can be set through the API, and it determines the order in
|
|
which they will process the data (modules with the highest priority first).
|
|
|
|
* The ``hashmap`` rating module is the most used one. It allows to create
|
|
rating rules based on metric metadatas.
|
|
|
|
* The ``pyscripts`` rating module allows to rate data with custom python
|
|
scripts.
|
|
|
|
For information about the usage and configuration of rating modules, see the
|
|
`rating modules documentation`_.
|
|
|
|
.. _rating modules documentation: ../user/rating/index.html
|
|
|
|
Storage
|
|
=======
|
|
|
|
The storage module is responsible for storing and retrieving data from a
|
|
backend. It implements two interfaces (v1 and v2), each providing one or more
|
|
drivers. For more information about the storage backend, see the
|
|
`configuration section`_.
|
|
|
|
.. _configuration section: configuration/storage.html
|