openstack/ceilometer
Code Issues Proposed changes

Merge "Deprecating transformers and pipeline partitioning"

Browse Source
This commit is contained in:
Zuul 2018-04-20 11:44:24 +00:00 committed by Gerrit Code Review
commit d516c1923d
11 changed files with 32 additions and 323 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
ceilometer/notification.py
View File

@ -37,6 +37,7 @@ LOG = log.getLogger(__name__)
OPTS = [
cfg.IntOpt('pipeline_processing_queues',
deprecated_for_removal=True,
default=10,
min=1,
help='Number of queues to parallelize workload across. This '
@ -47,6 +48,7 @@ OPTS = [
default=True,
help='Acknowledge message when event persistence fails.'),
cfg.BoolOpt('workload_partitioning',
deprecated_for_removal=True,
default=False,
help='Enable workload partitioning, allowing multiple '
'notification agents to be run simultaneously.'),

1
ceilometer/pipeline/data/event_pipeline.yaml
View File

@ -7,6 +7,5 @@ sources:
- event_sink
sinks:
- name: event_sink
transformers:
publishers:
- gnocchi://

9
ceilometer/pipeline/data/pipeline.yaml
View File

@ -33,9 +33,10 @@ sources:
- network_sink
sinks:
- name: meter_sink
transformers:
publishers:
- gnocchi://
# All these transformers are deprecated, and will be removed in the future, don't use them.
- name: cpu_sink
transformers:
- name: "rate_of_change"
@ -48,6 +49,8 @@ sinks:
scale: "100.0 / (10**9 * (resource_metadata.cpu_number or 1))"
publishers:
- gnocchi://
# All these transformers are deprecated, and will be removed in the future, don't use them.
- name: cpu_delta_sink
transformers:
- name: "delta"
@ -57,6 +60,8 @@ sinks:
growth_only: True
publishers:
- gnocchi://
# All these transformers are deprecated, and will be removed in the future, don't use them.
- name: disk_sink
transformers:
- name: "rate_of_change"
@ -72,6 +77,8 @@ sinks:
type: "gauge"
publishers:
- gnocchi://
# All these transformers are deprecated, and will be removed in the future, don't use them.
- name: network_sink
transformers:
- name: "rate_of_change"

6
doc/source/admin/telemetry-best-practices.rst
View File

@ -27,9 +27,3 @@ Data collection
central and compute agents as necessary. The agents are designed to scale
horizontally. For more information refer to the `high availability guide
<https://docs.openstack.org/ha-guide/controller-ha-telemetry.html>`_.
#. `workload_partitioning` of notification agents is only required if
the pipeline configuration leverages transformers. It may also be enabled if
batching is required to minimize load on the defined publisher targets. If
transformers are not enabled, multiple agents may still be deployed without
`workload_partitioning` and processing will be done greedily.

7
doc/source/admin/telemetry-data-collection.rst
View File

@ -39,10 +39,9 @@ By default, the notification agent is configured to build both events and
samples. To enable selective data models, set the required pipelines using
`pipelines` option under the `[notification]` section.
Additionally, the notification agent is responsible for all data processing
such as transformations and publishing. After processing, the data is sent
to any supported publisher target such as gnocchi or panko. These services
persist the data in configured databases.
Additionally, the notification agent is responsible to send to any supported
publisher target such as gnocchi or panko. These services persist the data in
configured databases.
The different OpenStack services emit several notifications about the
various types of events that happen in the system during normal

288
doc/source/admin/telemetry-data-pipelines.rst
View File

@ -6,7 +6,7 @@ Data processing and pipelines
The mechanism by which data is processed is called a pipeline. Pipelines,
at the configuration level, describe a coupling between sources of data and
the corresponding sinks for transformation and publication of data. This
the corresponding sinks for publication of data. This
functionality is handled by the notification agents.
A source is a producer of data: ``samples`` or ``events``. In effect, it is a
@ -17,13 +17,9 @@ Each source configuration encapsulates name matching and mapping
to one or more sinks for publication.
A sink, on the other hand, is a consumer of data, providing logic for
the transformation and publication of data emitted from related sources.
the publication of data emitted from related sources.
In effect, a sink describes a chain of handlers. The chain starts with
zero or more transformers and ends with one or more publishers. The
first transformer in the chain is passed data from the corresponding
source, takes some action such as deriving rate of change, performing
unit conversion, or aggregating, before publishing_.
In effect, a sink describes a list of one or more publishers.
.. _telemetry-pipeline-configuration:
@ -52,7 +48,6 @@ The meter pipeline definition looks like:
- 'sink name'
sinks:
- name: 'sink name'
transformers: 'definition of transformers'
publishers:
- 'list of publishers'
@ -97,30 +92,8 @@ The above definition methods can be used in the following combinations:
same pipeline. Wildcard and included meters cannot co-exist in the
same pipeline definition section.
The transformers section of a pipeline sink provides the possibility to
add a list of transformer definitions. The available transformers are:
.. list-table::
:widths: 50 50
:header-rows: 1
* - Name of transformer
- Reference name for configuration
* - Accumulator
- accumulator
* - Aggregator
- aggregator
* - Arithmetic
- arithmetic
* - Rate of change
- rate\_of\_change
* - Unit conversion
- unit\_conversion
* - Delta
- delta
The publishers section contains the list of publishers, where the
samples data should be sent after the possible transformations.
samples data should be sent.
Similarly, the event pipeline definition looks like:
@ -140,229 +113,6 @@ Similarly, the event pipeline definition looks like:
The event filter uses the same filtering logic as the meter pipeline.
.. _telemetry-transformers:
Transformers
------------
.. note::
Transformers maintain data in memory and therefore do not guarantee
durability in certain scenarios. A more durable and efficient solution
may be achieved post-storage using solutions like Gnocchi.
The definition of transformers can contain the following fields:
name
Name of the transformer.
parameters
Parameters of the transformer.
The parameters section can contain transformer specific fields, like
source and target fields with different subfields in case of the rate of
change, which depends on the implementation of the transformer.
The following are supported transformers:
Rate of change transformer
``````````````````````````
Transformer that computes the change in value between two data points in time.
In the case of the transformer that creates the ``cpu_util`` meter, the
definition looks like:
.. code-block:: yaml
transformers:
- name: "rate_of_change"
parameters:
target:
name: "cpu_util"
unit: "%"
type: "gauge"
scale: "100.0 / (10**9 * (resource_metadata.cpu_number or 1))"
The rate of change transformer generates the ``cpu_util`` meter
from the sample values of the ``cpu`` counter, which represents
cumulative CPU time in nanoseconds. The transformer definition above
defines a scale factor (for nanoseconds and multiple CPUs), which is
applied before the transformation derives a sequence of gauge samples
with unit ``%``, from sequential values of the ``cpu`` meter.
The definition for the disk I/O rate, which is also generated by the
rate of change transformer:
.. code-block:: yaml
transformers:
- name: "rate_of_change"
parameters:
source:
map_from:
name: "disk\\.(read|write)\\.(bytes|requests)"
unit: "(B|request)"
target:
map_to:
name: "disk.\\1.\\2.rate"
unit: "\\1/s"
type: "gauge"
Unit conversion transformer
```````````````````````````
Transformer to apply a unit conversion. It takes the volume of the meter
and multiplies it with the given ``scale`` expression. Also supports
``map_from`` and ``map_to`` like the rate of change transformer.
Sample configuration:
.. code-block:: yaml
transformers:
- name: "unit_conversion"
parameters:
target:
name: "disk.kilobytes"
unit: "KB"
scale: "volume * 1.0 / 1024.0"
With ``map_from`` and ``map_to``:
.. code-block:: yaml
transformers:
- name: "unit_conversion"
parameters:
source:
map_from:
name: "disk\\.(read|write)\\.bytes"
target:
map_to:
name: "disk.\\1.kilobytes"
scale: "volume * 1.0 / 1024.0"
unit: "KB"
Aggregator transformer
``````````````````````
A transformer that sums up the incoming samples until enough samples
have come in or a timeout has been reached.
Timeout can be specified with the ``retention_time`` option. If you want
to flush the aggregation, after a set number of samples have been
aggregated, specify the size parameter.
The volume of the created sample is the sum of the volumes of samples
that came into the transformer. Samples can be aggregated by the
attributes ``project_id``, ``user_id`` and ``resource_metadata``. To aggregate
by the chosen attributes, specify them in the configuration and set which
value of the attribute to take for the new sample (first to take the
first sample's attribute, last to take the last sample's attribute, and
drop to discard the attribute).
To aggregate 60s worth of samples by ``resource_metadata`` and keep the
``resource_metadata`` of the latest received sample:
.. code-block:: yaml
transformers:
- name: "aggregator"
parameters:
retention_time: 60
resource_metadata: last
To aggregate each 15 samples by ``user_id`` and ``resource_metadata`` and keep
the ``user_id`` of the first received sample and drop the
``resource_metadata``:
.. code-block:: yaml
transformers:
- name: "aggregator"
parameters:
size: 15
user_id: first
resource_metadata: drop
Accumulator transformer
```````````````````````
This transformer simply caches the samples until enough samples have
arrived and then flushes them all down the pipeline at once:
.. code-block:: yaml
transformers:
- name: "accumulator"
parameters:
size: 15
Multi meter arithmetic transformer
``````````````````````````````````
This transformer enables us to perform arithmetic calculations over one
or more meters and/or their metadata, for example::
memory_util = 100 * memory.usage / memory
A new sample is created with the properties described in the ``target``
section of the transformer's configuration. The sample's
volume is the result of the provided expression. The calculation is
performed on samples from the same resource.
.. note::
The calculation is limited to meters with the same interval.
Example configuration:
.. code-block:: yaml
transformers:
- name: "arithmetic"
parameters:
target:
name: "memory_util"
unit: "%"
type: "gauge"
expr: "100 * $(memory.usage) / $(memory)"
To demonstrate the use of metadata, the following implementation of a
novel meter shows average CPU time per core:
.. code-block:: yaml
transformers:
- name: "arithmetic"
parameters:
target:
name: "avg_cpu_per_core"
unit: "ns"
type: "cumulative"
expr: "$(cpu) / ($(cpu).resource_metadata.cpu_number or 1)"
.. note::
Expression evaluation gracefully handles NaNs and exceptions. In
such a case it does not create a new sample but only logs a warning.
Delta transformer
`````````````````
This transformer calculates the change between two sample datapoints of a
resource. It can be configured to capture only the positive growth deltas.
Example configuration:
.. code-block:: yaml
transformers:
- name: "delta"
parameters:
target:
name: "cpu.delta"
growth_only: True
.. _publishing:
Publishers
@ -533,33 +283,3 @@ specified. A sample ``publishers`` section in the
- panko://
- udp://10.0.0.2:1234
- notifier://?policy=drop&max_queue_length=512&topic=custom_target
Pipeline Partitioning
~~~~~~~~~~~~~~~~~~~~~
.. note::
Partitioning is only required if pipelines contain transformations. It has
secondary benefit of supporting batching in certain publishers.
On large workloads, multiple notification agents can be deployed to handle the
flood of incoming messages from monitored services. If transformations are
enabled in the pipeline, the notification agents must be coordinated to ensure
related messages are routed to the same agent. To enable coordination, set the
``workload_partitioning`` value in ``notification`` section.
To distribute messages across agents, ``pipeline_processing_queues`` option
should be set. This value defines how many pipeline queues to create which will
then be distributed to the active notification agents. It is recommended that
the number of processing queues, at the very least, match the number of agents.
Increasing the number of processing queues will improve the distribution of
messages across the agents. It will also help batching which minimises the
requests to Gnocchi storage backend. It will also increase the load the on
message queue as it uses the queue to shard data.
.. warning::
Decreasing the number of processing queues may result in lost data as any
previously created queues may no longer be assigned to active agents. It
is only recommended that you **increase** processing queues.

11
doc/source/admin/telemetry-measurements.rst
View File

@ -356,12 +356,11 @@ The following meters are collected for OpenStack Compute.
To enable libvirt ``disk.*`` support when running on RBD-backed shared
storage, you need to install libvirt version 1.2.16+.
The Telemetry service supports creating new meters by using
transformers. For more details about transformers see
:ref:`telemetry-transformers`. Among the meters gathered from libvirt and
Hyper-V, there are a few which are derived from other meters. The list of
meters that are created by using the ``rate_of_change`` transformer from the
above table is the following:
The Telemetry service supports creating new meters by using transformers, but
this is deprecated and discouraged to use. Among the meters gathered from
libvirt and Hyper-V, there are a few which are derived from other meters. The
list of meters that are created by using the ``rate_of_change`` transformer
from the above table is the following:
- cpu_util

BIN
doc/source/contributor/3-Pipeline.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 25 KiB

BIN
doc/source/contributor/4-Transformer.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 41 KiB

21
doc/source/contributor/architecture.rst
View File

@ -154,27 +154,6 @@ Ceilometer offers the ability to take data gathered by the agents, manipulate
it, and publish it in various combinations via multiple pipelines. This
functionality is handled by the notification agents.
Transforming the data
---------------------
.. figure:: ./4-Transformer.png
:width: 100%
:align: center
:alt: Transformer example
Example of aggregation of multiple cpu time usage samples in a single
cpu percentage sample.
The data gathered from the polling and notifications agents contains a wealth
of data and if combined with historical or temporal context, can be used to
derive even more data. Ceilometer offers various transformers which can be used
to manipulate data in the pipeline.
.. note::
The equivalent functionality can be handled more stably by storage
drivers such as Gnocchi.
Publishing the data
-------------------

10
releasenotes/notes/transformer-ed4b1ea7d1752576.yaml Normal file
View File

@ -0,0 +1,10 @@
---
deprecations:
- |
Usage of transformers in Ceilometer pipelines is deprecated. Transformers in Ceilometer
have never computed samples correctly when you have multiple workers. This functionality can
be done by the storage backend easily without all issues that Ceilometer has. For example, the
rating is already computed in Gnocchi today.
- |
Pipeline Partitioning is also deprecated. This was only useful to
workaround of some issues that tranformers has.