aae3529160
This will ensure future build are not broken. Change-Id: Ibc644db35fafbe7aeb09c1c9f14e8818527ef2d2
587 lines
21 KiB
Django/Jinja
587 lines
21 KiB
Django/Jinja
================
|
||
REST API Usage
|
||
================
|
||
|
||
Authentication
|
||
==============
|
||
|
||
By default, the authentication is configured to the "basic" mode. You need to
|
||
provide an `Authorization` header in your HTTP requests with a valid username
|
||
(the password is not used). The "admin" password is granted all privileges,
|
||
whereas any other username is recognize as having standard permissions.
|
||
|
||
You can customize permissions by specifying a different `policy_file` than the
|
||
default one.
|
||
|
||
If you set the `api.auth_mode` value to `keystone`, the OpenStack Keystone
|
||
middleware will be enabled for authentication. It is then needed to
|
||
authenticate against Keystone and provide a `X-Auth-Token` header with a valid
|
||
token for each request sent to Gnocchi's API.
|
||
|
||
Metrics
|
||
=======
|
||
|
||
Gnocchi provides an object type that is called *metric*. A metric designates
|
||
any thing that can be measured: the CPU usage of a server, the temperature of a
|
||
room or the number of bytes sent by a network interface.
|
||
|
||
A metric only has a few properties: a UUID to identify it, a name, the archive
|
||
policy that will be used to store and aggregate the measures.
|
||
|
||
To create a metric, the following API request should be used:
|
||
|
||
{{ scenarios['create-metric']['doc'] }}
|
||
|
||
Once created, you can retrieve the metric information:
|
||
|
||
{{ scenarios['get-metric']['doc'] }}
|
||
|
||
To retrieve the list of all the metrics created, use the following request:
|
||
|
||
{{ scenarios['list-metric']['doc'] }}
|
||
|
||
.. note::
|
||
|
||
Considering the large volume of metrics Gnocchi will store, query results are
|
||
limited to `max_limit` value set in the configuration file. Returned results
|
||
are ordered by metrics' id values. To retrieve the next page of results, the
|
||
id of a metric should be given as `marker` for the beginning of the next page
|
||
of results.
|
||
|
||
Default ordering and limits as well as page start can be modified
|
||
using query parameters:
|
||
|
||
{{ scenarios['list-metric-pagination']['doc'] }}
|
||
|
||
It is possible to send measures to the metric:
|
||
|
||
{{ scenarios['post-measures']['doc'] }}
|
||
|
||
If there are no errors, Gnocchi does not return a response body, only a simple
|
||
status code. It is possible to provide any number of measures.
|
||
|
||
.. IMPORTANT::
|
||
|
||
While it is possible to send any number of (timestamp, value), it is still
|
||
needed to honor constraints defined by the archive policy used by the metric,
|
||
such as the maximum timespan.
|
||
|
||
|
||
Once measures are sent, it is possible to retrieve them using *GET* on the same
|
||
endpoint:
|
||
|
||
{{ scenarios['get-measures']['doc'] }}
|
||
|
||
Depending on the driver, there may be some lag after POSTing measures before
|
||
they are processed and queryable. To ensure your query returns all measures
|
||
that have been POSTed, you can force any unprocessed measures to be handled:
|
||
|
||
{{ scenarios['get-measures-refresh']['doc'] }}
|
||
|
||
.. note::
|
||
|
||
Depending on the amount of data that is unprocessed, `refresh` may add
|
||
some overhead to your query.
|
||
|
||
The list of points returned is composed of tuples with (timestamp, granularity,
|
||
value) sorted by timestamp. The granularity is the timespan covered by
|
||
aggregation for this point.
|
||
|
||
It is possible to filter the measures over a time range by specifying the
|
||
*start* and/or *stop* parameters to the query with timestamp. The timestamp
|
||
format can be either a floating number (UNIX epoch) or an ISO8601 formated
|
||
timestamp:
|
||
|
||
{{ scenarios['get-measures-from']['doc'] }}
|
||
|
||
By default, the aggregated values that are returned use the *mean* aggregation
|
||
method. It is possible to request for any other method by specifying the
|
||
*aggregation* query parameter:
|
||
|
||
{{ scenarios['get-measures-max']['doc'] }}
|
||
|
||
The list of aggregation method available is: *mean*, *sum*, *last*, *max*,
|
||
*min*, *std*, *median*, *first*, *count* and *Npct* (with 0 < N < 100).
|
||
|
||
It's possible to provide the `granularity` argument to specify the granularity
|
||
to retrieve, rather than all the granularities available:
|
||
|
||
{{ scenarios['get-measures-granularity']['doc'] }}
|
||
|
||
In addition to granularities defined by the archive policy, measures can be
|
||
resampled to a new granularity.
|
||
|
||
{{ scenarios['get-measures-resample']['doc'] }}
|
||
|
||
.. note::
|
||
|
||
Depending on the aggregation method and frequency of measures, resampled
|
||
data may lack accuracy as it is working against previously aggregated data.
|
||
|
||
Measures batching
|
||
=================
|
||
It is also possible to batch measures sending, i.e. send several measures for
|
||
different metrics in a simple call:
|
||
|
||
{{ scenarios['post-measures-batch']['doc'] }}
|
||
|
||
Or using named metrics of resources:
|
||
|
||
{{ scenarios['post-measures-batch-named']['doc'] }}
|
||
|
||
If some named metrics specified in the batch request do not exist, Gnocchi can
|
||
try to create them as long as an archive policy rule matches:
|
||
|
||
{{ scenarios['post-measures-batch-named-create']['doc'] }}
|
||
|
||
|
||
Archive Policy
|
||
==============
|
||
|
||
When sending measures for a metric to Gnocchi, the values are dynamically
|
||
aggregated. That means that Gnocchi does not store all sent measures, but
|
||
aggregates them over a certain period of time. Gnocchi provides several
|
||
aggregation methods (mean, min, max, sum…) that are builtin.
|
||
|
||
An archive policy is defined by a list of items in the `definition` field. Each
|
||
item is composed of the timespan and the level of precision that must be kept
|
||
when aggregating data, determined using at least 2 of the `points`,
|
||
`granularity` and `timespan` fields. For example, an item might be defined
|
||
as 12 points over 1 hour (one point every 5 minutes), or 1 point every 1 hour
|
||
over 1 day (24 points).
|
||
|
||
By default, new measures can only be processed if they have timestamps in the
|
||
future or part of the last aggregation period. The last aggregation period size
|
||
is based on the largest granularity defined in the archive policy definition.
|
||
To allow processing measures that are older than the period, the `back_window`
|
||
parameter can be used to set the number of coarsest periods to keep. That way
|
||
it is possible to process measures that are older than the last timestamp
|
||
period boundary.
|
||
|
||
For example, if an archive policy is defined with coarsest aggregation of 1
|
||
hour, and the last point processed has a timestamp of 14:34, it's possible to
|
||
process measures back to 14:00 with a `back_window` of 0. If the `back_window`
|
||
is set to 2, it will be possible to send measures with timestamp back to 12:00
|
||
(14:00 minus 2 times 1 hour).
|
||
|
||
The REST API allows to create archive policies in this way:
|
||
|
||
{{ scenarios['create-archive-policy']['doc'] }}
|
||
|
||
By default, the aggregation methods computed and stored are the ones defined
|
||
with `default_aggregation_methods` in the configuration file. It is possible to
|
||
change the aggregation methods used in an archive policy by specifying the list
|
||
of aggregation method to use in the `aggregation_methods` attribute of an
|
||
archive policy.
|
||
|
||
{{ scenarios['create-archive-policy-without-max']['doc'] }}
|
||
|
||
The list of aggregation methods can either be:
|
||
|
||
- a list of aggregation methods to use, e.g. `["mean", "max"]`
|
||
|
||
- a list of methods to remove (prefixed by `-`) and/or to add (prefixed by `+`)
|
||
to the default list (e.g. `["+mean", "-last"]`)
|
||
|
||
If `*` is included in the list, it's substituted by the list of all supported
|
||
aggregation methods.
|
||
|
||
Once the archive policy is created, the complete set of properties is computed
|
||
and returned, with the URL of the archive policy. This URL can be used to
|
||
retrieve the details of the archive policy later:
|
||
|
||
{{ scenarios['get-archive-policy']['doc'] }}
|
||
|
||
It is also possible to list archive policies:
|
||
|
||
{{ scenarios['list-archive-policy']['doc'] }}
|
||
|
||
Existing archive policies can be modified to retain more or less data depending
|
||
on requirements. If the policy coverage is expanded, measures are not
|
||
retroactively calculated as backfill to accommodate the new timespan:
|
||
|
||
{{ scenarios['update-archive-policy']['doc'] }}
|
||
|
||
.. note::
|
||
|
||
Granularities cannot be changed to a different rate. Also, granularities
|
||
cannot be added or dropped from a policy.
|
||
|
||
It is possible to delete an archive policy if it is not used by any metric:
|
||
|
||
{{ scenarios['delete-archive-policy']['doc'] }}
|
||
|
||
.. note::
|
||
|
||
An archive policy cannot be deleted until all metrics associated with it
|
||
are removed by a metricd daemon.
|
||
|
||
|
||
Archive Policy Rule
|
||
===================
|
||
|
||
Gnocchi provides the ability to define a mapping called `archive_policy_rule`.
|
||
An archive policy rule defines a mapping between a metric and an archive policy.
|
||
This gives users the ability to pre-define rules so an archive policy is assigned to
|
||
metrics based on a matched pattern.
|
||
|
||
An archive policy rule has a few properties: a name to identify it, an archive
|
||
policy name that will be used to store the policy name and metric pattern to
|
||
match metric names.
|
||
|
||
An archive policy rule for example could be a mapping to default a medium archive
|
||
policy for any volume metric with a pattern matching `volume.*`. When a sample metric
|
||
is posted with a name of `volume.size`, that would match the pattern and the
|
||
rule applies and sets the archive policy to medium. If multiple rules match,
|
||
the longest matching rule is taken. For example, if two rules exists which
|
||
match `*` and `disk.*`, a `disk.io.rate` metric would match the `disk.*` rule
|
||
rather than `*` rule.
|
||
|
||
To create a rule, the following API request should be used:
|
||
|
||
{{ scenarios['create-archive-policy-rule']['doc'] }}
|
||
|
||
The `metric_pattern` is used to pattern match so as some examples,
|
||
|
||
- `*` matches anything
|
||
- `disk.*` matches disk.io
|
||
- `disk.io.*` matches disk.io.rate
|
||
|
||
Once created, you can retrieve the rule information:
|
||
|
||
{{ scenarios['get-archive-policy-rule']['doc'] }}
|
||
|
||
It is also possible to list archive policy rules. The result set is ordered by
|
||
the `metric_pattern`, in reverse alphabetical order:
|
||
|
||
{{ scenarios['list-archive-policy-rule']['doc'] }}
|
||
|
||
It is possible to delete an archive policy rule:
|
||
|
||
{{ scenarios['delete-archive-policy-rule']['doc'] }}
|
||
|
||
Resources
|
||
=========
|
||
|
||
Gnocchi provides the ability to store and index resources. Each resource has a
|
||
type. The basic type of resources is *generic*, but more specialized subtypes
|
||
also exist, especially to describe OpenStack resources.
|
||
|
||
The REST API allows to manipulate resources. To create a generic resource:
|
||
|
||
{{ scenarios['create-resource-generic']['doc'] }}
|
||
|
||
The *id*, *user_id* and *project_id* attributes must be UUID. The timestamp
|
||
describing the lifespan of the resource are optional, and *started_at* is by
|
||
default set to the current timestamp.
|
||
|
||
It's possible to retrieve the resource by the URL provided in the `Location`
|
||
header.
|
||
|
||
More specialized resources can be created. For example, the *instance* is used
|
||
to describe an OpenStack instance as managed by Nova_.
|
||
|
||
{{ scenarios['create-resource-instance']['doc'] }}
|
||
|
||
All specialized types have their own optional and mandatory attributes,
|
||
but they all include attributes from the generic type as well.
|
||
|
||
It is possible to create metrics at the same time you create a resource to save
|
||
some requests:
|
||
|
||
{{ scenarios['create-resource-with-new-metrics']['doc'] }}
|
||
|
||
To retrieve a resource by its URL provided by the `Location` header at creation
|
||
time:
|
||
|
||
{{ scenarios['get-resource-generic']['doc'] }}
|
||
|
||
It's possible to modify a resource by re-uploading it partially with the
|
||
modified fields:
|
||
|
||
{{ scenarios['patch-resource']['doc'] }}
|
||
|
||
And to retrieve its modification history:
|
||
|
||
{{ scenarios['get-patched-instance-history']['doc'] }}
|
||
|
||
It is possible to delete a resource altogether:
|
||
|
||
{{ scenarios['delete-resource-generic']['doc'] }}
|
||
|
||
It is also possible to delete a batch of resources based on attribute values, and
|
||
returns a number of deleted resources.
|
||
|
||
To delete resources based on ids:
|
||
|
||
{{ scenarios['delete-resources-by-ids']['doc'] }}
|
||
|
||
or delete resources based on time:
|
||
|
||
{{ scenarios['delete-resources-by-time']['doc']}}
|
||
|
||
.. IMPORTANT::
|
||
|
||
When a resource is deleted, all its associated metrics are deleted at the
|
||
same time.
|
||
|
||
When a batch of resources are deleted, an attribute filter is required to
|
||
avoid deletion of the entire database.
|
||
|
||
|
||
All resources can be listed, either by using the `generic` type that will list
|
||
all types of resources, or by filtering on their resource type:
|
||
|
||
{{ scenarios['list-resource-generic']['doc'] }}
|
||
|
||
No attributes specific to the resource type are retrieved when using the
|
||
`generic` endpoint. To retrieve the details, either list using the specific
|
||
resource type endpoint:
|
||
|
||
{{ scenarios['list-resource-instance']['doc'] }}
|
||
|
||
or using `details=true` in the query parameter:
|
||
|
||
{{ scenarios['list-resource-generic-details']['doc'] }}
|
||
|
||
.. note::
|
||
|
||
Similar to metric list, query results are limited to `max_limit` value set in
|
||
the configuration file.
|
||
|
||
Returned results represent a single page of data and are ordered by resouces'
|
||
revision_start time and started_at values:
|
||
|
||
{{ scenarios['list-resource-generic-pagination']['doc'] }}
|
||
|
||
Each resource can be linked to any number of metrics. The `metrics` attributes
|
||
is a key/value field where the key is the name of the relationship and
|
||
the value is a metric:
|
||
|
||
{{ scenarios['create-resource-instance-with-metrics']['doc'] }}
|
||
|
||
It's also possible to create metrics dynamically while creating a resource:
|
||
|
||
{{ scenarios['create-resource-instance-with-dynamic-metrics']['doc'] }}
|
||
|
||
The metric associated with a resource can be accessed and manipulated using the
|
||
usual `/v1/metric` endpoint or using the named relationship with the resource:
|
||
|
||
{{ scenarios['get-resource-named-metrics-measures']['doc'] }}
|
||
|
||
The same endpoint can be used to append metrics to a resource:
|
||
|
||
{{ scenarios['append-metrics-to-resource']['doc'] }}
|
||
|
||
.. _Nova: http://launchpad.net/nova
|
||
|
||
Resource Types
|
||
==============
|
||
|
||
Gnocchi is able to manage resource types with custom attributes.
|
||
|
||
To create a new resource type:
|
||
|
||
{{ scenarios['create-resource-type']['doc'] }}
|
||
|
||
Then to retrieve its description:
|
||
|
||
{{ scenarios['get-resource-type']['doc'] }}
|
||
|
||
All resource types can be listed like this:
|
||
|
||
{{ scenarios['list-resource-type']['doc'] }}
|
||
|
||
It can also be deleted if no more resources are associated to it:
|
||
|
||
{{ scenarios['delete-resource-type']['doc'] }}
|
||
|
||
Attributes can be added or removed:
|
||
|
||
{{ scenarios['patch-resource-type']['doc'] }}
|
||
|
||
Creating resource type means creation of new tables on the indexer backend.
|
||
This is heavy operation that will lock some tables for a short amount of times.
|
||
When the resource type is created, its initial `state` is `creating`. When the
|
||
new tables have been created, the state switches to `active` and the new
|
||
resource type is ready to be used. If something unexpected occurs during this
|
||
step, the state switches to `creation_error`.
|
||
|
||
The same behavior occurs when the resource type is deleted. The state starts to
|
||
switch to `deleting`, the resource type is no more usable. Then the tables are
|
||
removed and the finally the resource_type is really deleted from the database.
|
||
If some unexpected occurs the state switches to `deletion_error`.
|
||
|
||
Searching for resources
|
||
=======================
|
||
|
||
It's possible to search for resources using a query mechanism, using the
|
||
`POST` method and uploading a JSON formatted query.
|
||
|
||
When listing resources, it is possible to filter resources based on attributes
|
||
values:
|
||
|
||
{{ scenarios['search-resource-for-user']['doc'] }}
|
||
|
||
Or even:
|
||
|
||
{{ scenarios['search-resource-for-host-like']['doc'] }}
|
||
|
||
Complex operators such as `and` and `or` are also available:
|
||
|
||
{{ scenarios['search-resource-for-user-after-timestamp']['doc'] }}
|
||
|
||
Details about the resource can also be retrieved at the same time:
|
||
|
||
{{ scenarios['search-resource-for-user-details']['doc'] }}
|
||
|
||
It's possible to search for old revisions of resources in the same ways:
|
||
|
||
{{ scenarios['search-resource-history']['doc'] }}
|
||
|
||
It is also possible to send the *history* parameter in the *Accept* header:
|
||
|
||
{{ scenarios['search-resource-history-in-accept']['doc'] }}
|
||
|
||
The timerange of the history can be set, too:
|
||
|
||
{{ scenarios['search-resource-history-partial']['doc'] }}
|
||
|
||
The supported operators are: equal to (`=`, `==` or `eq`), less than (`<` or
|
||
`lt`), greater than (`>` or `gt`), less than or equal to (`<=`, `le` or `≤`)
|
||
greater than or equal to (`>=`, `ge` or `≥`) not equal to (`!=`, `ne` or `≠`),
|
||
value is in (`in`), value is like (`like`), or (`or` or `∨`), and (`and` or
|
||
`∧`) and negation (`not`).
|
||
|
||
The special attribute `lifespan` which is equivalent to `ended_at - started_at`
|
||
is also available in the filtering queries.
|
||
|
||
{{ scenarios['search-resource-lifespan']['doc'] }}
|
||
|
||
|
||
Searching for values in metrics
|
||
===============================
|
||
|
||
It is possible to search for values in metrics. For example, this will look for
|
||
all values that are greater than or equal to 50 if we add 23 to them and that
|
||
are not equal to 55. You have to specify the list of metrics to look into by
|
||
using the `metric_id` query parameter several times.
|
||
|
||
{{ scenarios['search-value-in-metric']['doc'] }}
|
||
|
||
And it is possible to search for values in metrics by using one or multiple
|
||
granularities:
|
||
|
||
{{ scenarios['search-value-in-metrics-by-granularity']['doc'] }}
|
||
|
||
You can specify a time range to look for by specifying the `start` and/or
|
||
`stop` query parameter, and the aggregation method to use by specifying the
|
||
`aggregation` query parameter.
|
||
|
||
The supported operators are: equal to (`=`, `==` or `eq`), lesser than (`<` or
|
||
`lt`), greater than (`>` or `gt`), less than or equal to (`<=`, `le` or `≤`)
|
||
greater than or equal to (`>=`, `ge` or `≥`) not equal to (`!=`, `ne` or `≠`),
|
||
addition (`+` or `add`), substraction (`-` or `sub`), multiplication (`*`,
|
||
`mul` or `×`), division (`/`, `div` or `÷`). These operations take either one
|
||
argument, and in this case the second argument passed is the value, or it.
|
||
|
||
The operators or (`or` or `∨`), and (`and` or `∧`) and `not` are also
|
||
supported, and take a list of arguments as parameters.
|
||
|
||
Aggregation across metrics
|
||
==========================
|
||
|
||
Gnocchi allows to do on-the-fly aggregation of already aggregated data of
|
||
metrics.
|
||
|
||
It can also be done by providing the list of metrics to aggregate:
|
||
|
||
{{ scenarios['get-across-metrics-measures-by-metric-ids']['doc'] }}
|
||
|
||
.. Note::
|
||
|
||
This aggregation is done against the aggregates built and updated for
|
||
a metric when new measurements are posted in Gnocchi. Therefore, the aggregate
|
||
of this already aggregated data may not have sense for certain kind of
|
||
aggregation method (e.g. stdev).
|
||
|
||
By default, the measures are aggregated using the aggregation method provided,
|
||
e.g. you'll get a mean of means, or a max of maxs. You can specify what method
|
||
to use over the retrieved aggregation by using the `reaggregation` parameter:
|
||
|
||
{{ scenarios['get-across-metrics-measures-by-metric-ids-reaggregate']['doc'] }}
|
||
|
||
It's also possible to do that aggregation on metrics linked to resources. In
|
||
order to select these resources, the following endpoint accepts a query such as
|
||
the one described in `Searching for resources`_.
|
||
|
||
{{ scenarios['get-across-metrics-measures-by-attributes-lookup']['doc'] }}
|
||
|
||
It is possible to group the resource search results by any attribute of the
|
||
requested resource type, and the compute the aggregation:
|
||
|
||
{{ scenarios['get-across-metrics-measures-by-attributes-lookup-groupby']['doc'] }}
|
||
|
||
Similar to retrieving measures for a single metric, the `refresh` parameter
|
||
can be provided to force all POSTed measures to be processed across all
|
||
metrics before computing the result. The `resample` parameter may be used as
|
||
well.
|
||
|
||
.. note::
|
||
|
||
Resampling is done prior to any reaggregation if both parameters are
|
||
specified.
|
||
|
||
Also, aggregation across metrics have different behavior depending
|
||
on whether boundary values are set ('start' and 'stop') and if 'needed_overlap'
|
||
is set.
|
||
|
||
If boundaries are not set, Gnocchi makes the aggregation only with points
|
||
at timestamp present in all timeseries. When boundaries are set, Gnocchi
|
||
expects that we have certain percent of timestamps common between timeseries,
|
||
this percent is controlled by needed_overlap (defaulted with 100%). If this
|
||
percent is not reached an error is returned.
|
||
|
||
The ability to fill in points missing from a subset of timeseries is supported
|
||
by specifying a `fill` value. Valid fill values include any valid float or
|
||
`null` which will compute aggregation with only the points that exist. The
|
||
`fill` parameter will not backfill timestamps which contain no points in any
|
||
of the timeseries. Only timestamps which have datapoints in at least one of
|
||
the timeseries is returned.
|
||
|
||
.. note::
|
||
|
||
A granularity must be specified when using the `fill` parameter.
|
||
|
||
{{ scenarios['get-across-metrics-measures-by-metric-ids-fill']['doc'] }}
|
||
|
||
|
||
Capabilities
|
||
============
|
||
|
||
The list aggregation methods that can be used in Gnocchi are extendable and
|
||
can differ between deployments. It is possible to get the supported list of
|
||
aggregation methods from the API server:
|
||
|
||
{{ scenarios['get-capabilities']['doc'] }}
|
||
|
||
Status
|
||
======
|
||
The overall status of the Gnocchi installation can be retrieved via an API call
|
||
reporting values such as the number of new measures to process for each metric:
|
||
|
||
{{ scenarios['get-status']['doc'] }}
|
||
|
||
|
||
Timestamp format
|
||
================
|
||
|
||
Timestamps used in Gnocchi are always returned using the ISO 8601 format.
|
||
Gnocchi is able to understand a few formats of timestamp when querying or
|
||
creating resources, for example
|
||
|
||
- "2014-01-01 12:12:34" or "2014-05-20T10:00:45.856219", ISO 8601 timestamps.
|
||
- "10 minutes", which means "10 minutes from now".
|
||
- "-2 days", which means "2 days ago".
|
||
- 1421767030, a Unix epoch based timestamp.
|