openstack/openstack-manuals
Code Issues Proposed changes

add page on Events

Browse Source 
this adds a page on events: what events are, how to configure.

Change-Id: I99c709902c3b56eaed6c530f820ef1f0dd721881
This commit is contained in:
gordon chung 2015-03-03 17:57:20 -05:00
parent 0504afcb37
commit 2e82eda3f0
3 changed files with 185 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
doc/admin-guide-cloud/ch_telemetry.xml
View File

@ -22,6 +22,10 @@
collects information about the system and stores it in the form of
samples in order to provide data about anything that can be billed.
</para>
<para>In addition to system measurements, the Telemetry module also
captures event notifications triggered when various actions are
executed in the OpenStack system. This data is captured as Events
and stored alongside metering data.</para>
<para>The list of meters is continuously growing, which makes it
possible to use the data collected by Telemetry for different
purposes, other than billing. For example, the autoscaling feature
@ -45,5 +49,6 @@
<xi:include href="telemetry/section_telemetry-data-retrieval.xml"/>
<xi:include href="telemetry/section_telemetry-alarms.xml"/>
<xi:include href="telemetry/section_telemetry-measurements.xml"/>
<xi:include href="telemetry/section_telemetry-events.xml"/>
<xi:include href="telemetry/section_telemetry-troubleshooting-guide.xml"/>
</chapter>

173
doc/admin-guide-cloud/telemetry/section_telemetry-events.xml Normal file
View File

@ -0,0 +1,173 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="section_telemetry-events">
<title>Events</title>
<para>In addition to metrics, the Telemetry module collects events triggered
within an OpenStack environment. This section provides a brief summary
of the events format in the Telemetry module.</para>
<para>While a sample represents a single, numeric datapoint within a
time-series, an event is a broader concept that represents the state of a
resource at a point in time. The state may be described using various data
types including non-numeric data such as an instance's flavor. In general,
events represent any action made in the OpenStack system.</para>
<section xml:id="section_telemetry-event-configuration">
<title>Event configuration</title>
<para>To enable the creation and storage of events in the Telemetry module
<option>store_events</option> option needs to be set to
<literal>True</literal>. For further configuration options, see the event
section in the
<link xlink:href="http://docs.openstack.org/juno/config-reference/content/ch_configuring-openstack-telemetry.html">
<citetitle>OpenStack Configuration Reference</citetitle></link>.</para>
<note><para>It is advisable to set <option>disable_non_metric_meters</option>
to <literal>True</literal> when enabling events in the Telemetry module.
The Telemetry module historically represented events as metering data,
which may create duplication of data if both events and non-metric meters
are enabled.</para></note>
</section>
<section xml:id="section_telemetry-event-structure">
<title>Event structure</title>
<para>Events captured by the Telemetry module are represented by five
key attributes:</para>
<variablelist>
<varlistentry>
<term>event_type</term>
<listitem>
<para>A dotted string defining what event occurred such as
<literal>compute.instance.resize.start"</literal>.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>message_id</term>
<listitem>
<para>A UUID for the event.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>generated</term>
<listitem>
<para>A timestamp of when the event occurred in the system.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>traits</term>
<listitem>
<para>A flat mapping of key-value pairs which describe the event. The
event's Traits contain most of the details of the event. Traits are
typed, and can be strings, integers, floats, or datetimes.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>raw</term>
<listitem>
<para>Mainly for auditing purpose, the full event message can be
stored (unindexed) for future evaluation.</para>
</listitem>
</varlistentry>
</variablelist>
</section>
<section xml:id="section_telemetry-event-indexing">
<title>Event indexing</title>
<para>The general philosophy of notifications in OpenStack is to emit any
and all data someone might need, and let the consumer filter out what
they are not interested in. In order to make processing simpler and more
efficient, the notifications are stored and processed within Ceilometer
as events. The notification payload, which can be an arbitrarily complex
JSON data structure, is converted to a flat set of key-value pairs. This
conversion is specified by a config file.</para>
<note><para>The event format is meant for efficient processing and querying.
Storage of complete notifications for auditing purposes can be enabled by
configuring <option>store_raw</option> option.</para></note>
<simplesect>
<title>Event conversion</title>
<para>The conversion from notifications to events is driven by a
configuration file defined by the <option>definitions_cfg_file</option>
in the <filename>ceilometer.conf</filename> configuration file.</para>
<para>This includes descriptions of how to map fields in the
notification body to Traits, and optional plugins for doing any
programmatic translations (splitting a string, forcing case).</para>
<para>The mapping of notifications to events is defined per event_type,
which can be wildcarded. Traits are added to events if the corresponding
fields in the notification exist and are non-null.</para>
<note><para>The default definition file included with the Telemetry module
contains a list of known notifications and useful traits. The mappings
provided can be modified to include more or less data according to
user requirements.</para></note>
<para>If the definitions file is not present, a warning will be logged,
but an empty set of definitions will be assumed. By default, any
notifications that do not have a corresponding event definition in the
definitions file will be converted to events with a set of minimal
traits. This can be changed by setting the option
<option>drop_unmatched_notifications</option> in the
<filename>ceilometer.conf</filename> file. If this is set to True, any unmapped
notifications will be dropped.</para>
<para>The basic set of traits (all are TEXT type) that will be added to
all events if the notification has the relevant data are: service
(notification's publisher), tenant_id, and request_id. These do not
have to be specified in the event definition, they are automatically
added, but their definitions can be overridden for a given event_type.
</para>
</simplesect>
<simplesect>
<title>Event definitions format</title>
<para>The event definitions file is in YAML format. It consists of a
list of event definitions, which are mappings. Order is significant,
the list of definitions is scanned in reverse order to find a
definition which matches the notification's event_type. That definition
will be used to generate the event. The reverse ordering is done
because it is common to want to have a more general wildcarded
definition (such as <literal>compute.instance.*</literal>) with a set of traits common
to all of those events, with a few more specific event definitions
afterwards that have all of the above traits, plus a few more.</para>
<para>Each event definition is a mapping with two keys:</para>
<variablelist>
<varlistentry>
<term>event_type</term>
<listitem><para>This is a list (or a string, which will be taken as a 1
element list) of event_types this definition will handle. These
can be wildcarded with unix shell glob syntax. An exclusion
listing (starting with a <literal>!</literal>) will exclude any types listed from
matching. If only exclusions are listed, the definition will match
anything not matching the exclusions.</para></listitem>
</varlistentry>
<varlistentry>
<term>traits</term>
<listitem><para>This is a mapping, the keys are the trait names, and the
values are trait definitions.</para></listitem>
</varlistentry>
</variablelist>
<para>Each trait definition is a mapping with the following keys:</para>
<variablelist>
<varlistentry>
<term>fields</term>
<listitem><para>A path specification for the field(s) in the notification
you wish to extract for this trait. Specifications can be written
to match multiple possible fields. By default the value will be
the first such field. The paths can be specified with a dot syntax
(<literal>payload.host</literal>). Square bracket syntax (<literal>payload[host]</literal>)
is also supported. In either case, if the key for the field you are
looking for contains special characters, like <literal>.</literal>, it will need to
be quoted (with double or single quotes):
<literal>payload.image_meta.’org.openstack__1__architecture’</literal>.
The syntax used for the field specification is a variant of
<link xlink:href="https://github.com/kennknowles/python-jsonpath-rw">JSONPath
</link></para></listitem>
</varlistentry>
<varlistentry>
<term>type</term>
<listitem><para>(Optional) The data type for this trait. Valid options are:
<literal>text</literal>, <literal>int</literal>, <literal>float</literal>,
and <literal>datetime</literal>. Defaults to <literal>text</literal> if not specified.
</para></listitem>
</varlistentry>
<varlistentry>
<term>plugin</term>
<listitem><para>(Optional) Used to execute simple programmatic conversions
on the value in a notification field.</para></listitem>
</varlistentry>
</variablelist>
</simplesect>
</section>
</section>

10
doc/config-reference/telemetry/section_telemetry-sample-configuration-files.xml
View File

@ -20,8 +20,10 @@
<title>event_definitions.yaml</title>
<para>The <filename>event_definitions.yaml</filename> file defines how
events received from other OpenStack components should be translated
to Telemetry samples.</para>
<para>You should not need to modify this file.</para>
to Telemetry events.</para>
<para>This file provides a standard set of events and corresponding traits
that may be of interest. This file can be modified to add and drop traits
that operators may find useful.</para>
<programlisting language="ini"><xi:include parse="text" href="http://git.openstack.org/cgit/openstack/ceilometer/plain/etc/ceilometer/event_definitions.yaml?h=stable/juno"/></programlisting>
</section>
<section xml:id="section_pipeline.yaml">
@ -30,9 +32,11 @@
corresponding sinks for transformation and publication of the
data. They are defined in the <filename>pipeline.yaml</filename>
file.</para>
<para>You should not need to modify this file.</para>
<para>This file can be modified to adjust polling intervals and the
samples generated by the Telemetry module</para>
<programlisting language="ini"><xi:include parse="text" href="http://git.openstack.org/cgit/openstack/ceilometer/plain/etc/ceilometer/pipeline.yaml?h=stable/juno"/></programlisting>
</section>
<!-- TODO(gordc): add event_pipeline.yaml for kilo -->
<section xml:id="section_ceilometer-policy.json">
<title>policy.json</title>
<para>The <filename>policy.json</filename> file defines additional