Allow operations on Ceilometer dynamic pollster to reference the sample itself

This issue is to allow operators to create expressions that can reference the
sample itself. Currently, the expression is allowed only for fields in the
sample, and not the sample itself.

We discovered this requirement when developing pollsters for Cinder;
the attribute `volume_image_metadata` is not returned in the API response when
it does not exist, therefore, we need a method to check if it exist before
trying to retrieve it.

Change-Id: I5d3eb04534afaeb556ddab4ef603a205745883b8

ceilometer/polling/dynamic_pollster.py

@@ -138,8 +138,12 @@ class PollsterSampleExtractor(object):
                   attribute_key, json_object)
         keys_and_operations = attribute_key.split("|")
         attribute_key = keys_and_operations[0].strip()
-        nested_keys = attribute_key.split(".")
-        value = reduce(operator.getitem, nested_keys, json_object)
+
+        if attribute_key == ".":
+            value = json_object
+        else:
+            nested_keys = attribute_key.split(".")
+            value = reduce(operator.getitem, nested_keys, json_object)
 
         return self.operate_value(keys_and_operations, value)
 

ceilometer/tests/unit/polling/test_dynamic_pollster.py

@@ -759,3 +759,24 @@ class TestDynamicPollster(base.BaseTestCase):
         finally:
             dynamic_pollster.PollsterSampleGatherer. \
                 internal_execute_request_get_samples = original_method
+
+    def test_retrieve_attribute_self_reference_sample(self):
+        key = " . | value['key1']['subKey1'][0]['d'] if 'key1' in value else 0"
+
+        sub_value1 = [{"d": 2}, {"g": {"h": "val"}}]
+        sub_value2 = [{"r": 245}, {"h": {"yu": "yu"}}]
+
+        json_object = {"key1": {"subKey1": sub_value1},
+                       "key2": {"subkey2": sub_value2}}
+
+        pollster = dynamic_pollster.DynamicPollster(
+            self.pollster_definition_only_required_fields)
+
+        returned_value = pollster.definitions.sample_extractor.\
+            retrieve_attribute_nested_value(json_object, key)
+        self.assertEqual(2, returned_value)
+
+        del json_object['key1']
+        returned_value = pollster.definitions.sample_extractor.\
+            retrieve_attribute_nested_value(json_object, key)
+        self.assertEqual(0, returned_value)

doc/source/admin/telemetry-dynamic-pollster.rst

@@ -40,8 +40,13 @@ attributes to define a dynamic pollster:
 *  ``value_attribute``: mandatory attribute; defines the attribute in the
    JSON response from the URL of the component being polled. We also accept
    nested values dictionaries. To use a nested value one can simply use
-   ``attribute1.attribute2.<asMuchAsNeeded>.lastattribute``. In our magnum
-   example, we can use ``status`` as the value attribute;
+   ``attribute1.attribute2.<asMuchAsNeeded>.lastattribute``. It is also
+   possible to reference the sample itself using ``.``; the self reference
+   of the sample is interesting in cases when the attribute might not exist.
+   Therefore, together with the operations one can first check if it exist
+   before retrieving it (e.g.
+   `` .  | value['some_field'] if 'some_field' in value else ''``).
+   In our magnum example, we can use ``status`` as the value attribute;
 
 *  ``endpoint_type``: mandatory field; defines the endpoint type that is
    used to discover the base URL of the component to be monitored; for