From 51f478febe42f857779c685f3720be8a3d3a46a5 Mon Sep 17 00:00:00 2001 From: gordon chung Date: Wed, 27 Jan 2016 08:43:41 -0500 Subject: [PATCH] rewriting history start to rewrite the purpose of 'ceilometer' and how it fits with 'telemetry'. this touches index and overview sections only. updates - fix gnocchi links - correct goal of ceilometer (data collection only) - make ceilometer api optional - highlight purpose of ceilometer db vs gnocchi - point out gnocchi is more responsive if not requiring full fidelity Change-Id: Idce892227281d2cb1fe3c4c0cfdf048b487a3b7f --- doc/source/architecture.rst | 25 +++++++++++++---- doc/source/index.rst | 22 ++++----------- doc/source/install/dbreco.rst | 16 +++++++++-- doc/source/measurements.rst | 12 -------- doc/source/webapi/index.rst | 1 - doc/source/webapi/v1.rst | 8 ------ doc/source/webapi/v2.rst | 52 ++++++++++++++++++++++++++++------- 7 files changed, 79 insertions(+), 57 deletions(-) delete mode 100644 doc/source/webapi/v1.rst diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index d31ea327..27dbd1a2 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -26,15 +26,16 @@ High-Level Architecture Each of Ceilometer's services are designed to scale horizontally. Additional workers and nodes can be added depending on the expected load. Ceilometer -offers five core services, the data agents designed to work independently from +offers three core services, the data agents designed to work independently from collection, but also designed to work together as a complete solution: 1. polling agent - daemon designed to poll OpenStack services and build Meters. 2. notification agent - daemon designed to listen to notifications on message queue, convert them to Events and Samples, and apply pipeline actions. -3. collector - daemon designed to gather and record event and metering data - created by notification and polling agents. -4. api - service to query and view data recorded by collector service. +3. (optional) collector - daemon designed to gather and record event and metering data + created by notification and polling agents (if using Gnocchi or full-fidelity storage). +4. (optional) api - service to query and view data recorded by collector + in internal full-fidelity database (if enabled). As Ceilometer has grown to capture more data, it became apparent that data storage would need to be optimised. To address this, Gnocchi_ (resource metering @@ -42,7 +43,7 @@ as a service) was developed to capture the data in a time series database to optimise storage and querying. Gnocchi is intended to replace the existing metering database interface. -.. _Gnocchi: https://wiki.openstack.org/wiki/Gnocchi +.. _Gnocchi: http://docs.openstack.org/developer/gnocchi/ .. figure:: ./ceilo-gnocchi-arch.png :width: 100% @@ -255,12 +256,24 @@ divide the data into their own databases, tailored for its purpose. For example, a deployer could choose to store alarms in an SQL backend while storing events and metering data in a NoSQL backend. +Ceilometer's storage service is designed to handle use cases where full-fidelity +of the data is required (e.g. auditing). To handle responsive, long-term data +queries, solutions that strip away some of the data's resolution, such as +Gnocchi, are recommended. + +.. note:: + + As of Liberty, alarming support, and subsequently its database, is handled + by Aodh_. + .. note:: We do not guarantee that we won't change the DB schema, so it is highly recommended to access the database through the API and not use direct queries. +_Aodh: http://docs.openstack.org/developer/aodh/ + Accessing the data ================== @@ -268,7 +281,7 @@ Accessing the data API Service ----------- -If the collected data from polling and notification agents are stored in supported +If the collected data from polling and notification agents are stored in Ceilometer's database(s) (see the section :ref:`which-db`), it is possible that the schema of these database(s) may evolve over time. For this reasons, we offer a REST API and recommend that you access the collected data via the API rather than by diff --git a/doc/source/index.rst b/doc/source/index.rst index 91a87389..c3589ad0 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -17,25 +17,13 @@ Welcome to the Ceilometer developer documentation! ================================================== -The :term:`Ceilometer` project aims to deliver a unique point of -contact for billing systems to acquire all of the measurements they -need to establish customer billing, across all current OpenStack core +The :term:`Ceilometer` project is a data collection service that provides the +ability to normalise and transform data across all current OpenStack core components with work underway to support future OpenStack components. -What is the purpose of the project and vision for it? -===================================================== - -* Provide efficient collection of metering data, in terms of CPU and - network costs. -* Allow deployers to integrate with the metering system directly or by - replacing components. -* Data may be collected by monitoring notifications sent from existing - services or by polling the infrastructure. -* Allow deployers to configure the type of data collected to meet - their operating requirements. -* The data collected by the metering system is made visible to some - users through a REST API. -* The metering messages are signed and :term:`non-repudiable`. +Ceilometer is a component of the Telemetry project. Its data can be used to +provide customer billing, resource tracking, and alarming capabilities +across all OpenStack core components. This documentation offers information on how Ceilometer works and how to contribute to the project. diff --git a/doc/source/install/dbreco.rst b/doc/source/install/dbreco.rst index 5df93724..b21db913 100644 --- a/doc/source/install/dbreco.rst +++ b/doc/source/install/dbreco.rst @@ -15,9 +15,17 @@ .. _choosing_db_backend: -============================= - Choosing a database backend -============================= +===================================== + Choosing a database backend - Legacy +===================================== + +.. note: + + Ceilometer's existing database capabilities is intended for post processing + and auditing purposes where responsiveness is not a requirement. It + captures the full fidelity of each datapoint and thus is not designed + for low latency use cases. For more responsive use cases, it's recommended + to store data in an alternative source such as Gnocchi_ Selecting a database backend for Ceilometer should not be done lightly for numerous reasons: @@ -49,3 +57,5 @@ MySQL Yes Yes Yes PostgreSQL Yes Yes Yes HBase Yes Yes, except groupby Yes ================== ============================= =================== ====== + +.. _Gnocchi: http://gnocchi.xyz diff --git a/doc/source/measurements.rst b/doc/source/measurements.rst index 5a1ad18b..bbb2c93f 100644 --- a/doc/source/measurements.rst +++ b/doc/source/measurements.rst @@ -33,15 +33,3 @@ Adding new meters If you would like to add new meters please check the :ref:`add_new_meters` page under in the Contributing section. - -OSprofiler data -=============== - -All messages with event type "profiler.*" will be collected as profiling data. -Using notification plugin profiler/notifications.py. - -.. note:: - - Be sparing with heavy usage of OSprofiler, especially in case of complex - operations like booting and deleting instance that may create over 100kb of - sample data per each request. diff --git a/doc/source/webapi/index.rst b/doc/source/webapi/index.rst index 0e0b9fc8..f8891188 100644 --- a/doc/source/webapi/index.rst +++ b/doc/source/webapi/index.rst @@ -5,7 +5,6 @@ .. toctree:: :maxdepth: 2 - v1 v2 You can get API version list via request to endpoint root path. For example:: diff --git a/doc/source/webapi/v1.rst b/doc/source/webapi/v1.rst deleted file mode 100644 index afab03a1..00000000 --- a/doc/source/webapi/v1.rst +++ /dev/null @@ -1,8 +0,0 @@ -============ - V1 Web API -============ - -.. warning:: - - The V1 API is being deprecated during the Havana development - cycle and no longer exists since Juno. diff --git a/doc/source/webapi/v2.rst b/doc/source/webapi/v2.rst index 6c2f6a9b..18816af6 100644 --- a/doc/source/webapi/v2.rst +++ b/doc/source/webapi/v2.rst @@ -40,14 +40,27 @@ Samples and Statistics .. autotype:: ceilometer.api.controllers.v2.meters.Statistics :members: -When a simple statistics request is invoked (using GET /v2/meters//statistics), it will return the standard set of *Statistics*: *avg*, *sum*, -*min*, *max*, and *count*. +When a simple statistics request is invoked (using GET /v2/meters//statistics), +it will return the standard set of *Statistics*: *avg*, *sum*, *min*, *max*, and *count*. + +.. note:: + + If using Ceilometer data for statistics, it's recommended to use a backend + such as Gnocchi_ rather than Ceilometer's interface. Gnocchi is designed + specifically for this use case by providing a light-weight, aggregated model. + As they manage data differently, the API models returned by Ceilometer and Gnocchi + are different. The Gnocchi API can be found here_. + +.. _Gnocchi: http://docs.openstack.org/developer/gnocchi/ +.. _here: http://docs.openstack.org/developer/gnocchi/rest.html Selectable Aggregates +++++++++++++++++++++ -The Statistics API has been extended to include the aggregate functions *stddev* and *cardinality*. You can explicitly select these functions or any from the -standard set by specifying an aggregate function in the statistics query:: +The Statistics API has been extended to include the aggregate functions +*stddev* and *cardinality*. You can explicitly select these functions or any +from the standard set by specifying an aggregate function in the statistics +query:: GET /v2/meters//statistics?aggregate.func=&aggregate.param= @@ -59,7 +72,9 @@ Duplicate aggregate function and parameter pairs are silently discarded from the are, on the other hand, both allowed by the API and supported by the storage drivers. See the :ref:`functional-examples` section for more detail. -.. note:: Currently only *cardinality* needs aggregate.param to be specified. +.. note:: + + Currently only *cardinality* needs aggregate.param to be specified. .. autotype:: ceilometer.api.controllers.v2.meters.Aggregate :members: @@ -116,7 +131,9 @@ nature of *resource_metadata* field, there is no practical or consistent way to validate the query fields under *metadata* domain like it is done for all other fields. -.. note:: The API call will return HTTP 200 OK status for both of the +.. note:: + + The API call will return HTTP 200 OK status for both of the following cases: when a query with *metadata.* does not match its value, and when ** itself does not exist in any of the records being queried. @@ -156,7 +173,9 @@ supported: *=*, *!=*, *<*, *<=*, *>*, *>=* and *in*; and the following logical operators can be used: *and* *or* and *not*. The field names are validated against the database models. See :ref:`api-queries` for how to query the API. -.. note:: The *not* operator has different meaning in MongoDB and in SQL DB engine. +.. note:: + + The *not* operator has different meaning in MongoDB and in SQL DB engine. If the *not* operator is applied on a non existent metadata field then the result depends on the DB engine. For example if {"not": {"metadata.nonexistent_field" : "some value"}} filter is used in a query @@ -195,8 +214,11 @@ Ceilometer CLI Commands:: $ ceilometer --debug --os-username --os-password --os-auth-url http://localhost:5000/v2.0/ --os-tenant-name admin meter-list -.. note:: The *username*, *password*, and *tenant-name* options are required to be present in these commands or specified via environment variables. Note that the in-line commands will override the environment variables. +.. note:: + The *username*, *password*, and *tenant-name* options are required to be + present in these arguments or specified via environment variables. Note that + the in-line arguments will override the environment variables. .. _api-queries: @@ -205,7 +227,13 @@ API Queries Ceilometer API calls: -.. note:: To successfully query the Ceilometer you must first get a project-specific token from the Keystone service and add it to any API calls that you execute against that project. See the `OpenStack credentials documentation `_ for additional details. +.. note:: + + To successfully query Ceilometer you must first get a project-specific + token from the Keystone service and add it to any API calls that you + execute against that project. See the + `OpenStack credentials documentation `_ + for additional details. A simple query to return a list of available meters:: @@ -251,7 +279,11 @@ the CPU utilization for a given instance (identified by *resource_id*):: "http://localhost:8777/v2/meters/cpu_util/statistics?aggregate.func=max&aggregate.func=stddev"\ "&q.field=resource_id&q.op=eq&q.value=64da755c-9120-4236-bee1-54acafe24980" -.. note:: If any of the requested aggregates are not supported by the storage driver, a HTTP 400 error code will be returned along with an appropriate error message. +.. note:: + + If any of the requested aggregates are not supported by the storage driver, + a HTTP 400 error code will be returned along with an appropriate error + message. JSON based example::