================ REST API Usage ================ Authentication ============== By default, no authentication is configured in Gnocchi. You need to provides these headers in your HTTP requests: * X-User-Id * X-Project-Id The `X-Roles` header can also be provided in order to match role based ACL specified in `policy.json`. If you enable the OpenStack Keystone middleware, you only need to authenticate against Keystone and provide `X-Auth-Token` header with a valid token for each request sent to Gnocchi. The headers mentionned above will be filled automatically based on your Keystone authorizations. Metrics ======= Gnocchi provides an object type that is called *metric*. A metric designates any thing that can be measured: the CPU usage of a server, the temperature of a room or the number of bytes sent by a network interface. A metric only has a few properties: a UUID to identify it, a name, the archive policy that will be used to store and aggregate the measures. To create a metric, the following API request should be used: {{ scenarios['create-metric']['doc'] }} Once created, you can retrieve the metric information: {{ scenarios['get-metric']['doc'] }} To retrieve the list of all the metrics created, use the following request: {{ scenarios['list-metric']['doc'] }} It is possible to send measures to the metric: {{ scenarios['post-measures']['doc'] }} If there are no errors, Gnocchi does not return a response body, only a simple status code. It is possible to provide any number of measures. .. IMPORTANT:: While it is possible to send any number of (timestamp, value), it is still needed to honor constraints defined by the archive policy used by the metric, such as the maximum timespan. Once measures are sent, it is possible to retrieve them using *GET* on the same endpoint: {{ scenarios['get-measures']['doc'] }} The list of points returned is composed of tuples with (timestamp, granularity, value) sorted by timestamp. The granularity is the timespan covered by aggregation for this point. It is possible to filter the measures over a time range by specifying the *start* and/or *stop* parameters to the query with timestamp. The timestamp format can be either a floating number (UNIX epoch) or an ISO8601 formated timestamp: {{ scenarios['get-measures-from']['doc'] }} By default, the aggregated values that are returned use the *mean* aggregation method. It is possible to request for any other method by specifying the *aggregation* query parameter: {{ scenarios['get-measures-max']['doc'] }} The list of aggregation method available is: *mean*, *sum*, *last*, *max*, *min*, *std*, *median*, *first*, *count* and *Npct* (with 0 < N < 100). It's possible to provide the `granularity` argument to specify the granularity to retrieve, rather than all the granularities available: {{ scenarios['get-measures-granularity']['doc'] }} Measures batching ================= It is also possible to batch measures sending, i.e. send several measures for different metrics in a simple call: {{ scenarios['post-measures-batch']['doc'] }} Or using named metrics of resources: {{ scenarios['post-measures-batch-named']['doc'] }} Archive Policy ============== When sending measures for a metric to Gnocchi, the values are dynamically aggregated. That means that Gnocchi does not store all sent measures, but aggregates them over a certain period of time. Gnocchi provides several aggregation methods (mean, min, max, sum…) that are builtin. An archive policy is defined by a list of items in the `definition` field. Each item is composed of the timespan and the level of precision that must be kept when aggregating data, determined using at least 2 of the `points`, `granularity` and `timespan` fields. For example, an item might be defined as 12 points over 1 hour (one point every 5 minutes), or 1 point every 1 hour over 1 day (24 points). By default, new measures can only be processed if they have timestamps in the future or part of the last aggregation period. The last aggregation period size is based on the largest granularity defined in the archive policy definition. To allow processing measures that are older than the period, the `back_window` parameter can be used to set the number of coarsest periods to keep. That way it is possible to process measures that are older than the last timestamp period boundary. For example, if an archive policy is defined with coarsest aggregation of 1 hour, and the last point processed has a timestamp of 14:34, it's possible to process measures back to 14:00 with a `back_window` of 0. If the `back_window` is set to 2, it will be possible to send measures with timestamp back to 12:00 (14:00 minus 2 times 1 hour). The REST API allows to create archive policies in this way: {{ scenarios['create-archive-policy']['doc'] }} By default, the aggregation methods computed and stored are the ones defined with `default_aggregation_methods` in the configuration file. It is possible to change the aggregation methods used in an archive policy by specifying the list of aggregation method to use in the `aggregation_methods` attribute of an archive policy. {{ scenarios['create-archive-policy-without-max']['doc'] }} The list of aggregation methods can either be: - a list of aggregation methods to use, e.g. `["mean", "max"]` - a list of methods to remove (prefixed by `-`) and/or to add (prefixed by `+`) to the default list (e.g. `["+mean", "-last"]`) If `*` is included in the list, it's substituted by the list of all supported aggregation methods. Once the archive policy is created, the complete set of properties is computed and returned, with the URL of the archive policy. This URL can be used to retrieve the details of the archive policy later: {{ scenarios['get-archive-policy']['doc'] }} It is also possible to list archive policies: {{ scenarios['list-archive-policy']['doc'] }} Existing archive policies can be modified to retain more or less data depending on requirements. If the policy coverage is expanded, measures are not retroactively calculated as backfill to accommodate the new timespan: {{ scenarios['update-archive-policy']['doc'] }} .. note:: Granularities cannot be changed to a different rate. Also, granularities cannot be added or dropped from a policy. It is possible to delete an archive policy if it is not used by any metric: {{ scenarios['delete-archive-policy']['doc'] }} .. note:: An archive policy cannot be deleted until all metrics associated with it are removed by a metricd daemon. Archive Policy Rule =================== Gnocchi provides the ability to define a mapping called `archive_policy_rule`. An archive policy rule defines a mapping between a metric and an archive policy. This gives users the ability to pre-define rules so an archive policy is assigned to metrics based on a matched pattern. An archive policy rule has a few properties: a name to identify it, an archive policy name that will be used to store the policy name and metric pattern to match metric names. An archive policy rule for example could be a mapping to default a medium archive policy for any volume metric with a pattern matching `volume.*`. When a sample metric is posted with a name of `volume.size`, that would match the pattern and the rule applies and sets the archive policy to medium. If multiple rules match, the longest matching rule is taken. For example, if two rules exists which match `*` and `disk.*`, a `disk.io.rate` metric would match the `disk.*` rule rather than `*` rule. To create a rule, the following API request should be used: {{ scenarios['create-archive-policy-rule']['doc'] }} The `metric_pattern` is used to pattern match so as some examples, - `*` matches anything - `disk.*` matches disk.io - `disk.io.*` matches disk.io.rate Once created, you can retrieve the rule information: {{ scenarios['get-archive-policy-rule']['doc'] }} It is also possible to list archive policy rules. The result set is ordered by the `metric_pattern`, in reverse alphabetical order: {{ scenarios['list-archive-policy-rule']['doc'] }} It is possible to delete an archive policy rule: {{ scenarios['delete-archive-policy-rule']['doc'] }} Resources ========= Gnocchi provides the ability to store and index resources. Each resource has a type. The basic type of resources is *generic*, but more specialized subtypes also exist, especially to describe OpenStack resources. The REST API allows to manipulate resources. To create a generic resource: {{ scenarios['create-resource-generic']['doc'] }} The *id*, *user_id* and *project_id* attributes must be UUID. The timestamp describing the lifespan of the resource are optional, and *started_at* is by default set to the current timestamp. It's possible to retrieve the resource by the URL provided in the `Location` header. More specialized resources can be created. For example, the *instance* is used to describe an OpenStack instance as managed by Nova_. {{ scenarios['create-resource-instance']['doc'] }} All specialized types have their own optional and mandatory attributes, but they all include attributes from the generic type as well. It is possible to create metrics at the same time you create a resource to save some requests: {{ scenarios['create-resource-with-new-metrics']['doc'] }} To retrieve a resource by its URL provided by the `Location` header at creation time: {{ scenarios['get-resource-generic']['doc'] }} It's possible to modify a resource by re-uploading it partially with the modified fields: {{ scenarios['patch-resource']['doc'] }} And to retrieve its modification history: {{ scenarios['get-patched-instance-history']['doc'] }} It possible to delete a resource altogether: {{ scenarios['delete-resource-generic']['doc'] }} .. IMPORTANT:: When a resource is deleted, all its associated metrics are deleted at the same time. All resources can be listed, either by using the `generic` type that will list all types of resources, or by filtering on their resource type: {{ scenarios['list-resource-generic']['doc'] }} No attributes specific to the resource type are retrieved when using the `generic` endpoint. To retrieve the details, either list using the specific resource type endpoint: {{ scenarios['list-resource-instance']['doc'] }} or using `details=true` in the query parameter: {{ scenarios['list-resource-generic-details']['doc'] }} Each resource can be linked to any number of metrics. The `metrics` attributes is a key/value field where the key is the name of the relationship and the value is a metric: {{ scenarios['create-resource-instance-with-metrics']['doc'] }} It's also possible to create metrics dynamically while creating a resource: {{ scenarios['create-resource-instance-with-dynamic-metrics']['doc'] }} The metric associated with a resource can be accessed and manipulated using the usual `/v1/metric` endpoint or using the named relationship with the resource: {{ scenarios['get-resource-named-metrics-measures']['doc'] }} The same endpoint can be used to append metrics to a resource: {{ scenarios['append-metrics-to-resource']['doc'] }} .. _Nova: http://launchpad.net/nova Resource Types ============== Gnocchi is able to manage resource types with custom attributes. To create a new resource type: {{ scenarios['create-resource-type']['doc'] }} Then to retrieve its description: {{ scenarios['get-resource-type']['doc'] }} All resource types can be listed like this: {{ scenarios['list-resource-type']['doc'] }} It can also be deleted if no more resources are associated to it: {{ scenarios['delete-resource-type']['doc'] }} Searching for resources ======================= It's possible to search for resources using a query mechanism, using the `POST` method and uploading a JSON formatted query. When listing resources, it is possible to filter resources based on attributes values: {{ scenarios['search-resource-for-user']['doc'] }} Complex operators such as `and` and `or` are also available: {{ scenarios['search-resource-for-user-after-timestamp']['doc'] }} Details about the resource can also be retrieved at the same time: {{ scenarios['search-resource-for-user-details']['doc'] }} It's possible to search for old revisions of resources in the same ways: {{ scenarios['search-resource-history']['doc'] }} It is also possible to send the *history* parameter in the *Accept* header: {{ scenarios['search-resource-history-in-accept']['doc'] }} The timerange of the history can be set, too: {{ scenarios['search-resource-history-partial']['doc'] }} The supported operators are: equal to (`=`, `==` or `eq`), less than (`<` or `lt`), greater than (`>` or `gt`), less than or equal to (`<=`, `le` or `≤`) greater than or equal to (`>=`, `ge` or `≥`) not equal to (`!=`, `ne` or `≠`), value is in (`in`), value is like (`like`), or (`or` or `∨`), and (`and` or `∧`) and negation (`not`). The special attribute `lifespan` which is equivalent to `ended_at - started_at` is also available in the filtering queries. {{ scenarios['search-resource-lifespan']['doc'] }} Searching for values in metrics =============================== It's possible to search for values in metrics. For example, this will look for all values that are greater than or equal to 50 if we add 23 to them and that are not equal to 55. You have to specify the list of metrics to look into by using the `metric_id` query parameter several times. {{ scenarios['search-value-in-metric']['doc'] }} You can specify a time range to look for by specifying the `start` and/or `stop` query parameter, and the aggregation method to use by specifying the `aggregation` query parameter. The supported operators are: equal to (`=`, `==` or `eq`), lesser than (`<` or `lt`), greater than (`>` or `gt`), less than or equal to (`<=`, `le` or `≤`) greater than or equal to (`>=`, `ge` or `≥`) not equal to (`!=`, `ne` or `≠`), addition (`+` or `add`), substraction (`-` or `sub`), multiplication (`*`, `mul` or `×`), division (`/`, `div` or `÷`). These operations take either one argument, and in this case the second argument passed is the value, or it. The operators or (`or` or `∨`), and (`and` or `∧`) and `not` are also supported, and take a list of arguments as parameters. Aggregation across metrics ========================== Gnocchi allows to do on-the-fly aggregation of already aggregated data of metrics. It can also be done by providing the list of metrics to aggregate: {{ scenarios['get-across-metrics-measures-by-metric-ids']['doc'] }} .. Note:: This aggregation is done against the aggregates built and updated for a metric when new measurements are posted in Gnocchi. Therefore, the aggregate of this already aggregated data may not have sense for certain kind of aggregation method (e.g. stdev). It's also possible to do that aggregation on metrics linked to resources. In order to select these resources, the following endpoint accepts a query such as the one described in `Searching for resources`_. {{ scenarios['get-across-metrics-measures-by-attributes-lookup']['doc'] }} It is possible to group the resource search results by any attribute of the requested resource type, and the compute the aggregation: {{ scenarios['get-across-metrics-measures-by-attributes-lookup-groupby']['doc'] }} Also aggregation across metrics have different behavior depending on if boundary are set ('start' and 'stop') and if 'needed_overlap' is set. If boundaries are not set, Gnocchi makes the aggregation only with points at timestamp present in all timeseries. But when boundaries are set, Gnocchi expects that we have certain percent of timestamps common between timeseries, this percent is controlled by needed_overlap (defaulted with 100%). If this percent is not reached an error is returned. Capabilities ============ The list aggregation methods that can be used in Gnocchi are extendable and can differ between deployments. It is possible to get the supported list of aggregation methods from the API server: {{ scenarios['get-capabilities']['doc'] }} Status ====== The overall status of the Gnocchi installation can be retrieved via an API call reporting values such as the number of new measures to process for each metric: {{ scenarios['get-status']['doc'] }} Timestamp format ================ Timestamps used in Gnocchi are always returned using the ISO 8601 format. Gnocchi is able to understand a few formats of timestamp when querying or creating resources, for example - "2014-01-01 12:12:34" or "2014-05-20T10:00:45.856219", ISO 8601 timestamps. - "10 minutes", which means "10 minutes from now". - "-2 days", which means "2 days ago". - 1421767030, a Unix epoch based timestamp.