00fea975e2
The "cpu_util" metric has been deprecated a few years ago. We'll obtain the same result by converting the cumulative cpu time to a percentage, leveraging the rate of change aggregation. Change-Id: I18fe0de6f74c785e674faceea0c48f44055818fe
306 lines
12 KiB
ReStructuredText
306 lines
12 KiB
ReStructuredText
..
|
|
Except where otherwise noted, this document is licensed under Creative
|
|
Commons Attribution 3.0 License. You can view the license at:
|
|
|
|
https://creativecommons.org/licenses/by/3.0/
|
|
|
|
.. _implement_strategy_plugin:
|
|
|
|
=================================
|
|
Build a new optimization strategy
|
|
=================================
|
|
|
|
Watcher Decision Engine has an external :ref:`strategy <strategy_definition>`
|
|
plugin interface which gives anyone the ability to integrate an external
|
|
strategy in order to make use of placement algorithms.
|
|
|
|
This section gives some guidelines on how to implement and integrate custom
|
|
strategies with Watcher. If you wish to create a third-party package for your
|
|
plugin, you can refer to our :ref:`documentation for third-party package
|
|
creation <plugin-base_setup>`.
|
|
|
|
|
|
Pre-requisites
|
|
==============
|
|
|
|
Before using any strategy, you should make sure you have your Telemetry service
|
|
configured so that it would provide you all the metrics you need to be able to
|
|
use your strategy.
|
|
|
|
|
|
Create a new strategy plugin
|
|
============================
|
|
|
|
In order to create a new strategy, you have to:
|
|
|
|
- Extend the :py:class:`~.UnclassifiedStrategy` class
|
|
- Implement its :py:meth:`~.BaseStrategy.get_name` class method to return the
|
|
**unique** ID of the new strategy you want to create. This unique ID should
|
|
be the same as the name of :ref:`the entry point we will declare later on
|
|
<strategy_plugin_add_entrypoint>`.
|
|
- Implement its :py:meth:`~.BaseStrategy.get_display_name` class method to
|
|
return the translated display name of the strategy you want to create.
|
|
Note: Do not use a variable to return the translated string so it can be
|
|
automatically collected by the translation tool.
|
|
- Implement its :py:meth:`~.BaseStrategy.get_translatable_display_name`
|
|
class method to return the translation key (actually the English display
|
|
name) of your new strategy. The value return should be the same as the
|
|
string translated in :py:meth:`~.BaseStrategy.get_display_name`.
|
|
- Implement its :py:meth:`~.BaseStrategy.execute` method to return the
|
|
solution you computed within your strategy.
|
|
|
|
Here is an example showing how you can write a plugin called ``NewStrategy``:
|
|
|
|
.. code-block:: python
|
|
|
|
# filepath: thirdparty/new.py
|
|
# import path: thirdparty.new
|
|
import abc
|
|
from watcher._i18n import _
|
|
from watcher.decision_engine.strategy.strategies import base
|
|
|
|
|
|
class NewStrategy(base.UnclassifiedStrategy):
|
|
|
|
def __init__(self, osc=None):
|
|
super(NewStrategy, self).__init__(osc)
|
|
|
|
def execute(self, original_model):
|
|
self.solution.add_action(action_type="nop",
|
|
input_parameters=parameters)
|
|
# Do some more stuff here ...
|
|
return self.solution
|
|
|
|
@classmethod
|
|
def get_name(cls):
|
|
return "new_strategy"
|
|
|
|
@classmethod
|
|
def get_display_name(cls):
|
|
return _("New strategy")
|
|
|
|
@classmethod
|
|
def get_translatable_display_name(cls):
|
|
return "New strategy"
|
|
|
|
|
|
As you can see in the above example, the :py:meth:`~.BaseStrategy.execute`
|
|
method returns a :py:class:`~.BaseSolution` instance as required. This solution
|
|
is what wraps the abstract set of actions the strategy recommends to you. This
|
|
solution is then processed by a :ref:`planner <watcher_planner_definition>` to
|
|
produce an action plan which contains the sequenced flow of actions to be
|
|
executed by the :ref:`Watcher Applier <watcher_applier_definition>`. This
|
|
solution also contains the various :ref:`efficacy indicators
|
|
<efficacy_indicator_definition>` alongside its computed :ref:`global efficacy
|
|
<efficacy_definition>`.
|
|
|
|
Please note that your strategy class will expect to find the same constructor
|
|
signature as BaseStrategy to instantiate you strategy. Therefore, you should
|
|
ensure that your ``__init__`` signature is identical to the
|
|
:py:class:`~.BaseStrategy` one.
|
|
|
|
|
|
Strategy efficacy
|
|
=================
|
|
|
|
As stated before, the ``NewStrategy`` class extends a class called
|
|
:py:class:`~.UnclassifiedStrategy`. This class actually implements a set of
|
|
abstract methods which are defined within the :py:class:`~.BaseStrategy` parent
|
|
class.
|
|
|
|
One thing this :py:class:`~.UnclassifiedStrategy` class defines is that our
|
|
``NewStrategy`` achieves the ``unclassified`` goal. This goal is a peculiar one
|
|
as it does not contain any indicator nor does it calculate a global efficacy.
|
|
This proves itself to be quite useful during the development of a new strategy
|
|
for which the goal has yet to be defined or in case a :ref:`new goal
|
|
<implement_goal_plugin>` has yet to be implemented.
|
|
|
|
|
|
Define Strategy Parameters
|
|
==========================
|
|
|
|
For each new added strategy, you can add parameters spec so that an operator
|
|
can input strategy parameters when creating an audit to control the
|
|
:py:meth:`~.BaseStrategy.execute` behavior of strategy. This is useful to
|
|
define some threshold for your strategy, and tune them at runtime.
|
|
|
|
To define parameters, just implements :py:meth:`~.BaseStrategy.get_schema` to
|
|
return parameters spec with `jsonschema
|
|
<http://json-schema.org/>`_ format.
|
|
It is strongly encouraged that provide default value for each parameter, or
|
|
else reference fails if operator specify no parameters.
|
|
|
|
Here is an example showing how you can define 2 parameters for
|
|
``DummyStrategy``:
|
|
|
|
.. code-block:: python
|
|
|
|
class DummyStrategy(base.DummyBaseStrategy):
|
|
|
|
@classmethod
|
|
def get_schema(cls):
|
|
return {
|
|
"properties": {
|
|
"para1": {
|
|
"description": "number parameter example",
|
|
"type": "number",
|
|
"default": 3.2,
|
|
"minimum": 1.0,
|
|
"maximum": 10.2,
|
|
},
|
|
"para2": {
|
|
"description": "string parameter example",
|
|
"type": "string",
|
|
"default": "hello",
|
|
},
|
|
},
|
|
}
|
|
|
|
|
|
You can reference parameters in :py:meth:`~.BaseStrategy.execute`:
|
|
|
|
.. code-block:: python
|
|
|
|
class DummyStrategy(base.DummyBaseStrategy):
|
|
|
|
def execute(self):
|
|
para1 = self.input_parameters.para1
|
|
para2 = self.input_parameters.para2
|
|
|
|
if para1 > 5:
|
|
...
|
|
|
|
|
|
Operator can specify parameters with following commands:
|
|
|
|
.. code:: bash
|
|
|
|
$ watcher audit create -a <your_audit_template> -p para1=6.0 -p para2=hi
|
|
|
|
Pls. check user-guide for details.
|
|
|
|
|
|
Abstract Plugin Class
|
|
=====================
|
|
|
|
Here below is the abstract :py:class:`~.BaseStrategy` class:
|
|
|
|
.. autoclass:: watcher.decision_engine.strategy.strategies.base.BaseStrategy
|
|
:members:
|
|
:special-members: __init__
|
|
:noindex:
|
|
|
|
.. _strategy_plugin_add_entrypoint:
|
|
|
|
Add a new entry point
|
|
=====================
|
|
|
|
In order for the Watcher Decision Engine to load your new strategy, the
|
|
strategy must be registered as a named entry point under the
|
|
``watcher_strategies`` entry point of your ``setup.py`` file. If you are using
|
|
pbr_, this entry point should be placed in your ``setup.cfg`` file.
|
|
|
|
The name you give to your entry point has to be unique and should be the same
|
|
as the value returned by the :py:meth:`~.BaseStrategy.get_name` class method of
|
|
your strategy.
|
|
|
|
Here below is how you would proceed to register ``NewStrategy`` using pbr_:
|
|
|
|
.. code-block:: ini
|
|
|
|
[entry_points]
|
|
watcher_strategies =
|
|
new_strategy = thirdparty.new:NewStrategy
|
|
|
|
|
|
To get a better understanding on how to implement a more advanced strategy,
|
|
have a look at the :py:class:`~.BasicConsolidation` class.
|
|
|
|
.. _pbr: https://docs.openstack.org/pbr/latest
|
|
|
|
Using strategy plugins
|
|
======================
|
|
|
|
The Watcher Decision Engine service will automatically discover any installed
|
|
plugins when it is restarted. If a Python package containing a custom plugin is
|
|
installed within the same environment as Watcher, Watcher will automatically
|
|
make that plugin available for use.
|
|
|
|
At this point, Watcher will scan and register inside the :ref:`Watcher Database
|
|
<watcher_database_definition>` all the strategies (alongside the goals they
|
|
should satisfy) you implemented upon restarting the :ref:`Watcher Decision
|
|
Engine <watcher_decision_engine_definition>`.
|
|
|
|
You should take care when installing strategy plugins. By their very nature,
|
|
there are no guarantees that utilizing them as is will be supported, as
|
|
they may require a set of metrics which is not yet available within the
|
|
Telemetry service. In such a case, please do make sure that you first
|
|
check/configure the latter so your new strategy can be fully functional.
|
|
|
|
Querying metrics
|
|
----------------
|
|
|
|
A large set of metrics, generated by OpenStack modules, can be used in your
|
|
strategy implementation. To collect these metrics, Watcher provides a
|
|
`DataSourceManager`_ for two data sources which are `Ceilometer`_
|
|
(with `Gnocchi`_ as API) and `Monasca`_. If you wish to query metrics from a
|
|
different data source, you can implement your own and use it via
|
|
DataSourceManager from within your new strategy. Indeed, strategies in Watcher
|
|
have the cluster data models decoupled from the data sources which means that
|
|
you may keep the former while changing the latter. The recommended way for you
|
|
to support a new data source is to implement a new helper that would
|
|
encapsulate within separate methods the queries you need to perform. To then
|
|
use it, you would just have to add it to appropriate watcher_strategies.*
|
|
section in config file.
|
|
|
|
If you want to use Ceilometer but with your own metrics database backend,
|
|
please refer to the `Ceilometer developer guide`_. The list of the available
|
|
Ceilometer backends is located here_. The `Ceilosca`_ project is a good example
|
|
of how to create your own pluggable backend. Moreover, if your strategy
|
|
requires new metrics not covered by Ceilometer, you can add them through a
|
|
`Ceilometer plugin`_.
|
|
|
|
|
|
.. _`DataSourceManager`: https://github.com/openstack/watcher/blob/master/watcher/datasource/manager.py
|
|
.. _`Ceilometer developer guide`: https://docs.openstack.org/ceilometer/latest/contributor/architecture.html#storing-accessing-the-data
|
|
.. _`Ceilometer`: https://docs.openstack.org/ceilometer/latest
|
|
.. _`Monasca`: https://github.com/openstack/monasca-api/blob/master/docs/monasca-api-spec.md
|
|
.. _`here`: https://docs.openstack.org/ceilometer/latest/contributor/install/dbreco.html#choosing-a-database-backend
|
|
.. _`Ceilometer plugin`: https://docs.openstack.org/ceilometer/latest/contributor/plugins.html
|
|
.. _`Ceilosca`: https://github.com/openstack/monasca-ceilometer/blob/master/ceilosca/ceilometer/storage/impl_monasca.py
|
|
.. _`Gnocchi`: https://gnocchi.xyz/
|
|
|
|
Read usage metrics using the Watcher Datasource Helper
|
|
------------------------------------------------------
|
|
|
|
The following code snippet shows how datasource_backend is defined:
|
|
|
|
.. code-block:: py
|
|
|
|
from watcher.datasource import manager as ds_manager
|
|
|
|
@property
|
|
def datasource_backend(self):
|
|
if not self._datasource_backend:
|
|
|
|
# Load the global preferred datasources order but override it
|
|
# if the strategy has a specific datasources config
|
|
datasources = CONF.watcher_datasources
|
|
if self.config.datasources:
|
|
datasources = self.config
|
|
|
|
self._datasource_backend = ds_manager.DataSourceManager(
|
|
config=datasources,
|
|
osc=self.osc
|
|
).get_backend(self.DATASOURCE_METRICS)
|
|
return self._datasource_backend
|
|
|
|
Using that you can now query the values for that specific metric:
|
|
|
|
.. code-block:: py
|
|
|
|
avg_meter = self.datasource_backend.statistic_aggregation(
|
|
instance.uuid, 'instance_cpu_usage', self.periods['instance'],
|
|
self.granularity,
|
|
aggregation=self.aggregation_method['instance'])
|