61c57c7d4c
Add documentation for Stackdistiller's config file format. Change-Id: I59031f62c5f7021b3ea6a89f622716da935f6100
84 lines
3.8 KiB
Markdown
84 lines
3.8 KiB
Markdown
# stackdistiller
|
|
|
|
A data extraction and transformation library for OpenStack notifications.
|
|
|
|
Stackdistiller is designed to extract data from openstack notifications
|
|
and convert it into a form relevant to the application consuming the
|
|
notification. It consists of two components, the Distiller, which extracts
|
|
data from notifications according to a YAML [config file](doc/event_definitions_config.md),
|
|
and the Condenser, which receives the data extracted by the Distiller,
|
|
and formats it into an application-specific object, referred to as an Event.
|
|
This could be a simple python dictionary, an XML document tree, or a set of
|
|
ORM model objects.
|
|
|
|
## Distiller
|
|
|
|
The Distiller reads a YAML config file to determine what data to extract
|
|
from each notification, according to it's event type. event types can be
|
|
wildcarded using shell glob syntax. The distiller will extract two types of
|
|
data from each notification:
|
|
|
|
* Metadata from the notifications envelope, including the event type,
|
|
message id (uuid of the notification) and the timestamp showing when
|
|
notification was generated by the source system.
|
|
* A series of data items extracted from the notification's body. These
|
|
are called Traits. Traits are basically just typed name-value pairs.
|
|
|
|
The distiller can also do some basic data massaging on traits extracted
|
|
from the notification, such as splitting a value from a string. This is
|
|
handled by trait plugins. These are just classes that implement the
|
|
TraitPluginBase interface. They are referred to by name in the config, and
|
|
are looked up in a trait\_plugin\_map passed to the distiller on init.
|
|
The plugin map is just a dictionary, or dictionary-like object (such as a
|
|
plugin manager) that maps names to plugin classes. If no map is passed to
|
|
the distiller, it will use a default that just contains the builtin plugins
|
|
bundled with stackdistiller.
|
|
|
|
If a notification does not match any event definition in the distiller's
|
|
config file, the distiller's to\_event method will return None, indicating
|
|
it cannot extract that notification. This may be what you want (i.e. your
|
|
application may only be interested in certain notifications.), but if you
|
|
want to record basic informaton from *any* event type, you can pass
|
|
"catchall=True" to the distiller, and it will generate a minimal event from
|
|
any notification.
|
|
|
|
## Condenser
|
|
|
|
The Condenser receives the data extracted from the notification by the
|
|
Distiller and formats it into an appropriate type of Event object.
|
|
An instance of a Condenser class is passed, along with the raw,
|
|
deserialized notification, to the distiller object's to\_event method.
|
|
To create your own type of Event from the data extracted by the distiller,
|
|
you just need to create a Condenser class to receive the data.
|
|
Condenser classes don't have to subclass any particular class, as long as
|
|
they implement the methods defined in stackdistiller.condenser.CondenserBase.
|
|
If you do not pass a condenser to the distiller when you call to\_event,
|
|
it will create an instance of the default DictionaryCondenser for you.
|
|
This just formats the event as a plain python dictionary.
|
|
|
|
## Example:
|
|
|
|
import json
|
|
from stackdistiller import distiller
|
|
from stackdistiller import condenser
|
|
|
|
config_file_name = "events_i_want.yaml"
|
|
notification_string = open('a_notification_here.json', 'r').read()
|
|
|
|
notification = json.loads(notification_string)
|
|
config = distiller.load_config(config_file_name)
|
|
|
|
dist = distiller.Distiller(config, catchall=False)
|
|
|
|
#this is the default condenser.
|
|
cond = condenser.DictionaryCondenser()
|
|
|
|
if dist.to_event(notification, cond):
|
|
# What get_event() returns is up to the condenser class. In this
|
|
# case, it's a dictionary.
|
|
event = cond.get_event()
|
|
print "Yay! An Event: %s" % str(event)
|
|
else:
|
|
print "Not something we are interested in. Ignoring."
|
|
|