openstack
/
deb-gnocchi
Code Issues Proposed changes

Add granularity in searching for values in metrics 

Added parameter granularity in 'search_value()' method to allow
search values by granularities.

The http reqeust is like this:
    method: POST
    uri: /v1/search/metric?metric_id=
         <a metric id>&granularity=1second&granularity=2s
    data body: {...}

Change-Id: Ifed526e56a052fec29f9b8b16a3979d959aa814b
Closes-Bug: #1626409
This commit is contained in:
shengping zhang 2016-09-22 16:14:22 +08:00 committed by Julien Danjou
parent 97a0bddc7b
commit b5ad7170a0
6 changed files with 186 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
doc/source/rest.j2
View File

@ -449,13 +449,18 @@ is also available in the filtering queries.
Searching for values in metrics
===============================
It's possible to search for values in metrics. For example, this will look for
It is possible to search for values in metrics. For example, this will look for
all values that are greater than or equal to 50 if we add 23 to them and that
are not equal to 55. You have to specify the list of metrics to look into by
using the `metric_id` query parameter several times.
{{ scenarios['search-value-in-metric']['doc'] }}
And it is possible to search for values in metrics by using one or multiple
granularities:
{{ scenarios['search-value-in-metrics-by-granularity']['doc'] }}
You can specify a time range to look for by specifying the `start` and/or
`stop` query parameter, and the aggregation method to use by specifying the
`aggregation` query parameter.

52
doc/source/rest.yaml
View File

@ -202,6 +202,58 @@
{"and": [{">=": [{"+": 23}, 50]}, {"!=": 55}]}
- name: create-metric-a
request: |
POST /v1/metric HTTP/1.1
Content-Type: application/json
{
"archive_policy_name": "short"
}
- name: post-measures-for-granularity-search
request: |
POST /v1/metric/{{ scenarios['create-metric-a']['response'].json['id'] }}/measures HTTP/1.1
Content-Type: application/json
[
{
"timestamp": "2014-10-06T14:34:12",
"value": 12
},
{
"timestamp": "2014-10-06T14:34:14",
"value": 12
},
{
"timestamp": "2014-10-06T14:34:16",
"value": 12
},
{
"timestamp": "2014-10-06T14:34:18",
"value": 12
},
{
"timestamp": "2014-10-06T14:34:20",
"value": 12
},
{
"timestamp": "2014-10-06T14:34:22",
"value": 12
},
{
"timestamp": "2014-10-06T14:34:24",
"value": 12
}
]
- name: search-value-in-metrics-by-granularity
request: |
POST /v1/search/metric?metric_id={{ scenarios['create-metric-a']['response'].json['id'] }}&granularity=1second&granularity=1800s HTTP/1.1
Content-Type: application/json
{"=": 12}
- name: get-measures
request: GET /v1/metric/{{ scenarios['create-metric']['response'].json['id'] }}/measures HTTP/1.1

12
gnocchi/rest/__init__.py
View File

@ -1245,7 +1245,11 @@ class SearchMetricController(rest.RestController):
)
@pecan.expose('json')
def post(self, metric_id, start=None, stop=None, aggregation='mean'):
def post(self, metric_id, start=None, stop=None, aggregation='mean',
granularity=[]):
granularity = [Timespan(g)
for g in arg_to_list(granularity)]
metrics = pecan.request.indexer.list_metrics(
ids=arg_to_list(metric_id))
@ -1274,11 +1278,15 @@ class SearchMetricController(rest.RestController):
str(metric.id): values
for metric, values in six.iteritems(
pecan.request.storage.search_value(
metrics, query, start, stop, aggregation)
metrics, query, start, stop, aggregation,
granularity
)
)
}
except storage.InvalidQuery as e:
abort(400, e)
except storage.GranularityDoesNotExist as e:
abort(400, e)
class ResourcesMetricsMeasuresBatchController(rest.RestController):

12
gnocchi/storage/_carbonara.py
View File

@ -645,18 +645,20 @@ class CarbonaraBasedStorage(storage.StorageDriver):
for timestamp, g, value in values
if predicate(value)]}
# TODO(jd) Add granularity parameter here and in the REST API
# rather than fetching all granularities
def search_value(self, metrics, query, from_timestamp=None,
to_timestamp=None, aggregation='mean'):
to_timestamp=None, aggregation='mean',
granularity=[]):
predicate = storage.MeasureQuery(query)
results = self._map_in_thread(
self._find_measure,
[(metric, aggregation,
ap.granularity, predicate,
gran, predicate,
from_timestamp, to_timestamp)
for metric in metrics
for ap in metric.archive_policy.definition])
for gran in granularity or
(defin.granularity
for defin in metric.archive_policy.definition)])
result = collections.defaultdict(list)
for r in results:
for metric, metric_result in six.iteritems(r):

110
gnocchi/tests/gabbi/gabbits/search-metric.yaml
View File

@ -16,7 +16,12 @@ tests:
data:
name: high
definition:
- granularity: 1 second
- granularity: 1 second
timespan: 1 hour
- granularity: 2 second
timespan: 1 hour
response_headers:
location: $SCHEME://$NETLOC/v1/archive_policy/high
status: 201
- name: create metric
@ -24,11 +29,110 @@ tests:
request_headers:
content-type: application/json
data:
archive_policy_name: "high"
archive_policy_name: high
status: 201
- name: post measures
desc: for later use
POST: /v1/batch/metrics/measures
request_headers:
content-type: application/json
data:
$RESPONSE['$.id']:
- timestamp: "2014-10-06T14:34:12"
value: 12
- timestamp: "2014-10-06T14:34:14"
value: 12
- timestamp: "2014-10-06T14:34:16"
value: 12
- timestamp: "2014-10-06T14:34:18"
value: 12
- timestamp: "2014-10-06T14:34:20"
value: 12
- timestamp: "2014-10-06T14:34:22"
value: 12
- timestamp: "2014-10-06T14:34:24"
value: 12
- timestamp: "2014-10-06T14:34:26"
value: 12
- timestamp: "2014-10-06T14:34:28"
value: 12
- timestamp: "2014-10-06T14:34:30"
value: 12
- timestamp: "2014-10-06T14:34:32"
value: 12
- timestamp: "2014-10-06T14:34:34"
value: 12
status: 202
- name: get metric id for search one
GET: /v1/metric
status: 200
response_json_paths:
$[0].archive_policy.name: high
- name: search with one correct granularity
POST: /v1/search/metric?metric_id=$RESPONSE['$[0].id']&granularity=1s
request_headers:
content-type: application/json
data:
"=": 12
status: 200
- name: get metric id for search two
GET: /v1/metric
status: 200
response_json_paths:
$[0].archive_policy.name: high
- name: search with multiple correct granularities
POST: /v1/search/metric?metric_id=$RESPONSE['$[0].id']&granularity=1second&granularity=2s
request_headers:
content-type: application/json
data:
"=": 12
status: 200
- name: get metric id for search three
GET: /v1/metric
status: 200
response_json_paths:
$[0].archive_policy.name: high
- name: search with correct and incorrect granularities
POST: /v1/search/metric?metric_id=$RESPONSE['$[0].id']&granularity=1s&granularity=300
request_headers:
content-type: application/json
data:
"=": 12
status: 400
response_strings:
- Granularity '300.0' for metric $RESPONSE['$[0].id'] does not exist
- name: get metric id for search four
GET: /v1/metric
status: 200
response_json_paths:
$[0].archive_policy.name: high
- name: search with incorrect granularity
POST: /v1/search/metric?metric_id=$RESPONSE['$[0].id']&granularity=300
request_headers:
content-type: application/json
data:
"=": 12
status: 400
response_strings:
- Granularity '300.0' for metric $RESPONSE['$[0].id'] does not exist
- name: get metric id for search five
GET: /v1/metric
status: 200
response_json_paths:
$[0].archive_policy.name: high
- name: search measure with wrong start
POST: /v1/search/metric?metric_id=$RESPONSE['$.id']&start=foobar
POST: /v1/search/metric?metric_id=$RESPONSE['$[0].id']&start=foobar
request_headers:
content-type: application/json
data:

4
releasenotes/notes/add-paramter-granularity-7f22c677dc1b1238.yaml Normal file
View File

@ -0,0 +1,4 @@
---
features:
- Allow to search for values in metrics by using
one or more granularities.