From 05158adafc0fa9df1d7d7e55396650e88fb4582a Mon Sep 17 00:00:00 2001 From: Dina Belova Date: Mon, 6 Oct 2014 15:16:20 +0400 Subject: [PATCH] Fix docs to suit merged compute/central agents concept DocImpact: Fix agents-related docs to describe polling agent Related-Blueprint: merge-compute-central-agents Change-Id: If377ac49bf75c0698f4af6be021e8354240f4e42 --- doc/source/architecture.rst | 29 +++++----- doc/source/contributing/plugins.rst | 82 +++++++++++++++++++---------- doc/source/glossary.rst | 13 ++--- doc/source/install/development.rst | 15 ++++-- 4 files changed, 86 insertions(+), 53 deletions(-) diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index 7fac5701..f7224b8e 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -94,8 +94,7 @@ Each of Ceilometer's services are designed to scale horizontally. Additional workers and nodes can added depending on the expected load. Ceilometer offers five core services: -1. polling agents - compute and central agent daemons designed to poll - OpenStack services. +1. polling agent - daemon designed to poll OpenStack services. 2. notification agent - daemon designed to listen to message queue. 3. collector - daemon designed to gather and record event and metering data created by notification and polling agents. @@ -133,14 +132,17 @@ methods to collect data: prefer this compared to a polling agent method as resilience (high availability) will not be a problem with this method. 3. :term:`Polling agents` which is the least preferred method, that will poll - some API or other tool to collect information at a regular interval. + some API or other tool to collect information at a regular interval. This method is least preferred due to the inherent difficulty in making such a component resilient. The first method is supported by the ceilometer-notification agent, which monitors the message queues for notifications and for metering data coming -from the "push" agents. Methods 2 and 3 rely on the ceilometer-compute-agent -and ceilometer-central-agent respectively. +from the "push" agents. Methods 2 and 3 rely on the ceilometer-polling agent, +which behaves differently depending on the resources it's collecting data +about. Polling agent can be configured either to poll local hypervisor or +remote APIs (public REST APIs exposed by services and/or host-level SNMP +daemons). How to access collected data? ----------------------------- @@ -366,16 +368,15 @@ Polling Metering data comes from two sources: through notifications built into the existing OpenStack components and by polling the infrastructure -(such as via libvirt). Polling for compute resources is handled by an -agent running on the compute node (where communication with the -hypervisor is more efficient). The compute agent daemon is configured -to run one or more *pollster* plugins using the -``ceilometer.poll.compute`` namespace. Polling for resources not tied -to the compute node is handled by the central agent. The central -agent daemon is configured to run one or more *pollster* plugins using -the ``ceilometer.poll.central`` namespace. +(such as via libvirt). Polling for compute resources is handled by a +polling agent running on the compute node (where communication with the +hypervisor is more efficient). Polling for resources not tied +to the compute node is handled by the agent running on the cloud +controller node via services APIs. The polling agent daemon is configured +to run one or more *pollster* plugins using either the +``ceilometer.poll.compute`` and/or ``ceilometer.poll.central`` namespaces. -The agents periodically asks each pollster for instances of +The agents periodically ask each pollster for instances of ``Sample`` objects. The agent framework then publishes the Samples using the publishers defined in the pipeline configuration. For example, the ``notifier`` publisher converts the Sample to metering messages, which it diff --git a/doc/source/contributing/plugins.rst b/doc/source/contributing/plugins.rst index 780ff822..fc27e47d 100644 --- a/doc/source/contributing/plugins.rst +++ b/doc/source/contributing/plugins.rst @@ -24,34 +24,41 @@ has not yet been covered by an existing plugin. Agents ====== -The compute agent runs on each compute node to poll for resource +Polling agent might be run either on central cloud management nodes or on the +compute nodes (where direct hypervisor polling is quite logical). + +The agent running on each compute node polls for compute resources usage. Each metric collected is tagged with the resource ID (such as an instance) and the owner, including tenant and user IDs. The metrics are then reported to the collector via the message bus. More detailed information follows. -The compute agent is implemented in ``ceilometer/compute/manager.py``. As -you will see in the manager, the computeagent loads all plugins defined in -the namespace ``ceilometer.poll.compute``, then periodically calls their -:func:`get_samples` method. +The agent running on the cloud central management node polls other types of +resources from a management server (usually using OpenStack services API to +collect this data). -The central agent polls other types of resources from a management server. -The central agent is defined in ``ceilometer/central/manager.py``. It loads -plugins from the ``ceilometer.poll.central`` namespace and polls them by -calling their :func:`get_samples` method. +The polling agent is implemented in ``ceilometer/agent/manager.py``. As +you will see in the manager, the agent loads all plugins defined in +the namespace ``ceilometer.poll.agent``, then periodically calls their +:func:`get_samples` method. Plugins ======= -An agent can support multiple plugins to retrieve different +A polling agent can support multiple plugins to retrieve different information and send them to the collector. As stated above, an agent -will automatically activate all plugins of a given class. For example, -the compute agent will load all plugins of class -``ceilometer.poll.compute``. This will load, among others, the -:class:`ceilometer.compute.pollsters.CPUPollster`, which is defined in -the file ``ceilometer/compute/pollsters.py`` as well as the -:class:`ceilometer.compute.notifications.InstanceNotifications` plugin -which is defined in the file ``ceilometer/compute/notifications.py`` +will automatically activate all possible plugins if no additional information +about what to poll was passed. Previously we had separated compute and +central agents with different namespaces with plugins (pollsters) defined +within. Currently we keep separated namespaces - ``ceilometer.poll.compute`` +and ``ceilometer.poll.central`` for quick separation of what to poll depending +on where is polling agent running. This will load, among others, the +:class:`ceilometer.compute.pollsters.cpu.CPUPollster`, which is defined in +the folder ``ceilometer/compute/pollsters``. + +Notifications mechanism uses plugins as well, for instance +:class:`ceilometer.compute.notifications.instance.InstanceNotifications` plugin +which is defined in the ``ceilometer/compute/notifications`` folder. We are using these two existing plugins as examples as the first one provides an example of how to interact when you need to retrieve information from an @@ -62,24 +69,45 @@ Pollster -------- Compute plugins are defined as subclasses of the -:class:`ceilometer.compute.plugin.ComputePollster` class as defined in -the ``ceilometer/compute/plugin.py`` file. Pollsters must implement one +:class:`ceilometer.compute.BaseComputePollster` class as defined in +the ``ceilometer/compute/__init__.py`` file. Pollsters must implement one method: ``get_samples(self, manager, context)``, which returns a sequence of ``Sample`` objects as defined in the ``ceilometer/sample.py`` file. -In the ``CPUPollster`` plugin, the ``get_samples`` method is implemented as a loop -which, for each instances running on the local host, retrieves the cpu_time -from the hypervisor and sends back two ``Sample`` objects. The first one, named -"cpu", is of type "cumulative", meaning that between two polls, its value is -not reset, or in other word that the cpu value is always provided as a duration -that continuously increases since the creation of the instance. The second one, -named "cpu_util", is of type "gauge", meaning that its value is the percentage -of cpu utilization. +In the ``CPUPollster`` plugin, the ``get_samples`` method is implemented as a +loop which, for each instances running on the local host, retrieves the +cpu_time from the hypervisor and sends back two ``Sample`` objects. The first +one, named "cpu", is of type "cumulative", meaning that between two polls, its +value is not reset while the instance remains active, or in other words that +the CPU value is always provided as a duration that continuously increases +since the creation of the instance. The second one, named "cpu_util", is of +type "gauge", meaning that its value is the percentage of cpu utilization. Note that the ``LOG`` method is only used as a debugging tool and does not participate in the actual metering activity. +There is the way to specify either namespace(s) with pollsters or just +list of concrete pollsters to use, or even both of these parameters on the +polling agent start via CLI parameter: + + ceilometer-polling --polling-namespaces central compute + +This command will basically make polling agent to load all plugins from the +central and compute namespaces and poll everything it can. If you need to load +only some of the pollsters, you can use ``pollster-list`` option: + + ceilometer-polling --pollster-list image image.size storage.* + +If both of these options are passed, the polling agent will load only those +pollsters specified in the pollster list, that can be loaded from the selected +namespaces. + +.. note:: + + Agents coordination cannot be used in case of pollster-list option usage. + This allows to avoid both samples duplication and their lost. + Notifications ------------- diff --git a/doc/source/glossary.rst b/doc/source/glossary.rst index f630d91f..ab69d660 100644 --- a/doc/source/glossary.rst +++ b/doc/source/glossary.rst @@ -45,10 +45,10 @@ A ceilometer is a device that uses a laser or other light source to determine the height of a cloud base. - central agent - Software service running on a central management node within the - OpenStack infrastructure measuring usage and sending the results - to the :term:`collector`. + polling agent + Software service running either on a central management node within the + OpenStack infrastructure or compute node measuring usage and sending the + results to the :term:`collector`. collector Software service running on the OpenStack infrastructure @@ -56,11 +56,6 @@ samples from the ceilometer agent and recording the results in the database. - compute agent - Software service running on a compute node within the OpenStack - infrastructure measuring usage and sending the results to the - :term:`collector`. - notification agent The different OpenStack services emit several notifications about the various types of events. The notification agent consumes them from diff --git a/doc/source/install/development.rst b/doc/source/install/development.rst index 7bbf9528..6cf1157e 100644 --- a/doc/source/install/development.rst +++ b/doc/source/install/development.rst @@ -18,13 +18,22 @@ Installing and Running the Development Version ================================================ -Ceilometer has several daemons. The basic are: :term:`compute agent` runs on -the Nova compute node(s) while the :term:`central agent`, :term:`collector` -and :term:`notification agent` run on the cloud's management node(s). +Ceilometer has several daemons. The basic are: :term:`polling agent` running +either on the Nova compute node(s) or :term:`polling agent` running on the +central management node(s), :term:`collector` +and :term:`notification agent` running on the cloud's management node(s). In a development environment created by devstack_, these services are typically running on the same server. They do not have to be, though, so some of the instructions below are duplicated. Skip the steps you have already done. +.. note:: + + In fact, previously Ceilometer had separated compute and central agents, and + their support is implemented in devstack_ right now, not one agent variant. + For now we do have deprecated cmd sripts emulating old compute/central + behaviour using namespaces option passed to polling agent, which will be + maintained for a transitional period. + .. _devstack: http://www.devstack.org/ Configuring Devstack