diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index d04b4c47f..de465b701 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -176,30 +176,42 @@ associated :ref:`Audit Template ` and knows the :ref:`Goal ` to achieve. It then selects the most appropriate :ref:`Strategy ` -depending on how Watcher was configured for this :ref:`Goal `. +from the list of available strategies achieving this goal. The :ref:`Strategy ` is then dynamically loaded (via -`stevedore `_). The -:ref:`Watcher Decision Engine ` calls the -**execute()** method of the :ref:`Strategy ` class which -generates a solution composed of a set of :ref:`Actions `. +`stevedore `_). The +:ref:`Watcher Decision Engine ` executes +the strategy. + +In order to compute the potential :ref:`Solution ` for the +Audit, the :ref:`Strategy ` relies on different sets of +data: + +- :ref:`Cluster data models ` that are + periodically synchronized through pluggable cluster data model collectors. + These models contain the current state of various + :ref:`Managed resources ` (e.g., the data stored + in the Nova database). These models gives a strategy the ability to reason on + the current state of a given :ref:`cluster `. +- The data stored in the :ref:`Cluster History Database + ` which provides information about the past of + the :ref:`Cluster `. + +Here below is a sequence diagram showing how the Decision Engine builds and +maintains the :ref:`cluster data models ` that +are used by the strategies. + +.. image:: ./images/sequence_architecture_cdmc_sync.png + :width: 100% + +The execution of a strategy then yields a solution composed of a set of +:ref:`Actions ` as well as a set of :ref:`efficacy +indicators `. These :ref:`Actions ` are scheduled in time by the :ref:`Watcher Planner ` (i.e., it generates an :ref:`Action Plan `). -In order to compute the potential :ref:`Solution ` for the -Audit, the :ref:`Strategy ` relies on two sets of data: - -- the current state of the - :ref:`Managed resources ` - (e.g., the data stored in the Nova database) -- the data stored in the - :ref:`Cluster History Database ` - which provides information about the past of the - :ref:`Cluster ` - - .. _data_model: Data model @@ -216,8 +228,6 @@ Here below is a diagram representing the main objects in Watcher from a database perspective: .. image:: ./images/watcher_db_schema_diagram.png - :width: 100% - .. _sequence_diagrams: diff --git a/doc/source/dev/plugin/action-plugin.rst b/doc/source/dev/plugin/action-plugin.rst index 7969c2239..3e07639bd 100644 --- a/doc/source/dev/plugin/action-plugin.rst +++ b/doc/source/dev/plugin/action-plugin.rst @@ -114,12 +114,15 @@ tune the action to its needs. To do so, you can implement the def execute(self): assert self.config.test_opt == 0 - def get_config_opts(self): - return [ + @classmethod + def get_config_opts(cls): + return super( + DummyAction, cls).get_config_opts() + [ cfg.StrOpt('test_opt', help="Demo Option.", default=0), # Some more options ... ] + The configuration options defined within this class method will be included within the global ``watcher.conf`` configuration file under a section named by convention: ``{namespace}.{plugin_name}``. In our case, the ``watcher.conf`` diff --git a/doc/source/dev/plugin/cdmc-plugin.rst b/doc/source/dev/plugin/cdmc-plugin.rst index 3d229801d..12c8a36f7 100644 --- a/doc/source/dev/plugin/cdmc-plugin.rst +++ b/doc/source/dev/plugin/cdmc-plugin.rst @@ -54,6 +54,10 @@ Define configuration parameters =============================== At this point, you have a fully functional cluster data model collector. +By default, cluster data model collectors define an ``period`` option (see +:py:meth:`~.BaseClusterDataModelCollector.get_config_opts`) that corresponds +to the interval of time between each synchronization of the in-memory model. + However, in more complex implementation, you may want to define some configuration options so one can tune the cluster data model collector to its needs. To do so, you can implement the :py:meth:`~.Loadable.get_config_opts` @@ -73,8 +77,10 @@ class method as followed: # Do something here... return model - def get_config_opts(self): - return [ + @classmethod + def get_config_opts(cls): + return super( + DummyClusterDataModelCollector, cls).get_config_opts() + [ cfg.StrOpt('test_opt', help="Demo Option.", default=0), # Some more options ... ] diff --git a/doc/source/dev/plugin/planner-plugin.rst b/doc/source/dev/plugin/planner-plugin.rst index d85f25fc0..3f539ee1c 100644 --- a/doc/source/dev/plugin/planner-plugin.rst +++ b/doc/source/dev/plugin/planner-plugin.rst @@ -91,8 +91,10 @@ tune the planner to its needs. To do so, you can implement the assert self.config.test_opt == 0 # [...] - def get_config_opts(self): - return [ + @classmethod + def get_config_opts(cls): + return super( + DummyPlanner, cls).get_config_opts() + [ cfg.StrOpt('test_opt', help="Demo Option.", default=0), # Some more options ... ] diff --git a/doc/source/image_src/plantuml/sequence_architecture_cdmc_sync.txt b/doc/source/image_src/plantuml/sequence_architecture_cdmc_sync.txt new file mode 100644 index 000000000..6792b1cfc --- /dev/null +++ b/doc/source/image_src/plantuml/sequence_architecture_cdmc_sync.txt @@ -0,0 +1,41 @@ +@startuml +skinparam maxMessageSize 100 + +actor "Administrator" + +== Initialization == + +"Administrator" -> "Decision Engine" : Start all services +"Decision Engine" -> "Background Task Scheduler" : Start + +activate "Background Task Scheduler" +"Background Task Scheduler" -> "Cluster Model Collector Loader"\ +: List available cluster data models +"Cluster Model Collector Loader" --> "Background Task Scheduler"\ +: list of BaseClusterModelCollector instances + +loop for every available cluster data model collector + "Background Task Scheduler" -> "Background Task Scheduler"\ + : add periodic synchronization job + create "Jobs Pool" + "Background Task Scheduler" -> "Jobs Pool" : Create sync job +end +deactivate "Background Task Scheduler" + +hnote over "Background Task Scheduler" : Idle + +== Job workflow == + +"Background Task Scheduler" -> "Jobs Pool" : Trigger synchronization job +"Jobs Pool" -> "Nova Cluster Data Model Collector" : synchronize + +activate "Nova Cluster Data Model Collector" + "Nova Cluster Data Model Collector" -> "Nova API"\ + : Fetch needed data to build the cluster data model + "Nova API" --> "Nova Cluster Data Model Collector" : Needed data + "Nova Cluster Data Model Collector" -> "Nova Cluster Data Model Collector"\ + : Build an in-memory cluster data model + ]o<-- "Nova Cluster Data Model Collector" : Done +deactivate "Nova Cluster Data Model Collector" + +@enduml diff --git a/doc/source/image_src/plantuml/sequence_trigger_audit_in_decision_engine.txt b/doc/source/image_src/plantuml/sequence_trigger_audit_in_decision_engine.txt index 69796d869..3bfc815e3 100644 --- a/doc/source/image_src/plantuml/sequence_trigger_audit_in_decision_engine.txt +++ b/doc/source/image_src/plantuml/sequence_trigger_audit_in_decision_engine.txt @@ -10,45 +10,41 @@ activate "Decision Engine" "AMQP Bus" <[#blue]- "Decision Engine" : notify new audit state = ONGOING "Decision Engine" -> "Database" : get audit parameters (goal, strategy, ...) "Decision Engine" <-- "Database" : audit parameters (goal, strategy, ...) -"Decision Engine" --> "Decision Engine": select appropriate \ -optimization strategy (via the Strategy Selector) +"Decision Engine" --> "Decision Engine"\ +: select appropriate optimization strategy (via the Strategy Selector) create Strategy -"Decision Engine" -> "Strategy" : execute() +"Decision Engine" -> "Strategy" : execute strategy activate "Strategy" -create "Cluster Data Model Collector" -"Strategy" -> "Cluster Data Model Collector" : get cluster data model - -activate "Cluster Data Model Collector" - loop while enough data to build cluster data model - "Cluster Data Model Collector" -> "Nova API" : get resource state (\ -host, instance, ...) - "Cluster Data Model Collector" <-- "Nova API" : resource state + "Strategy" -> "Cluster Data Model Collector" : get cluster data model + "Cluster Data Model Collector" --> "Strategy"\ + : copy of the in-memory cluster data model + loop while enough history data for the strategy + "Strategy" -> "Ceilometer API" : get necessary metrics + "Strategy" <-- "Ceilometer API" : aggregated metrics end -"Cluster Data Model Collector" -> "Strategy" : cluster data model -deactivate "Cluster Data Model Collector" - -loop while enough history data for the strategy - "Strategy" -> "Ceilometer API": get necessary metrics - "Strategy" <-- "Ceilometer API": aggregated metrics -end -"Strategy" -> "Strategy" : compute/set needed actions for the solution \ -so it achieves its goal -"Strategy" -> "Strategy" : compute/set efficacy indicators for the solution -"Strategy" -> "Strategy" : compute/set the solution global efficacy -"Decision Engine" <-- "Strategy" : solution (contains a list of unordered \ -actions alongside its efficacy indicators as well as its global efficacy) + "Strategy" -> "Strategy"\ + : compute/set needed actions for the solution so it achieves its goal + "Strategy" -> "Strategy" : compute/set efficacy indicators for the solution + "Strategy" -> "Strategy" : compute/set the solution global efficacy + "Decision Engine" <-- "Strategy"\ + : solution (unordered actions, efficacy indicators and global efficacy) deactivate "Strategy" -"Decision Engine" --> "Planner": load actions scheduler (i.e. Planner plugin) create "Planner" -"Decision Engine" -> "Planner": schedule() -"Planner" -> "Planner": schedule actions according to \ -scheduling rules/policies -"Decision Engine" <-- "Planner": new action plan +"Decision Engine" -> "Planner" : load actions scheduler +"Planner" --> "Decision Engine" : planner plugin +"Decision Engine" -> "Planner" : schedule actions +activate "Planner" + "Planner" -> "Planner"\ + : schedule actions according to scheduling rules/policies + "Decision Engine" <-- "Planner" : new action plan +deactivate "Planner" "Decision Engine" -> "Database" : save new action plan in database "Decision Engine" -> "Database" : update audit.state = SUCCEEDED "AMQP Bus" <[#blue]- "Decision Engine" : notify new audit state = SUCCEEDED deactivate "Decision Engine" +hnote over "Decision Engine" : Idle + @enduml diff --git a/doc/source/images/sequence_architecture_cdmc_sync.png b/doc/source/images/sequence_architecture_cdmc_sync.png new file mode 100644 index 000000000..16e00007b Binary files /dev/null and b/doc/source/images/sequence_architecture_cdmc_sync.png differ diff --git a/doc/source/images/sequence_trigger_audit_in_decision_engine.png b/doc/source/images/sequence_trigger_audit_in_decision_engine.png index 36848003f..33ddfd3ca 100644 Binary files a/doc/source/images/sequence_trigger_audit_in_decision_engine.png and b/doc/source/images/sequence_trigger_audit_in_decision_engine.png differ