openstack/watcher
Code Issues Proposed changes

Merge "Deprecate prometheus datasource in favor of aetos"

Browse Source
This commit is contained in:
Zuul
2026-02-27 13:05:27 +00:00
committed by Gerrit Code Review
parent 5740bb98a5 9132e604b6
commit 7df0931593
9 changed files with 373 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
doc/source/datasources/aetos.rst
View File

@@ -2,6 +2,13 @@
Aetos datasource
================
.. note::
**RECOMMENDED**: Aetos is the recommended datasource for accessing
Prometheus metrics in Watcher. It provides all the functionality of the
Prometheus datasource with added multi-tenancy, Keystone authentication,
and role-based access control (RBAC). The direct Prometheus datasource
is deprecated and will be removed in a future release.
Synopsis
--------
The Aetos datasource allows Watcher to use an Aetos reverse proxy server as the
@@ -99,9 +106,9 @@ datasources for a specific strategy if preferred eg. under the
.. note::
Having both Prometheus and Aetos datasources configured at the same time
is not supported and will result in a configuration error. Allowing this
can be investigated in the future if a need or a proper use case is
identified.
is not supported and will result in a configuration error. Furthermore,
the prometheus datasource is deprecated, so aetos should be the preferred
choice.
The watcher.conf configuration file is also used to set the parameter values
required by the Watcher Aetos data source. The configuration can be

12
doc/source/datasources/index.rst
View File

@@ -1,8 +1,16 @@
Datasources
===========
.. note::
The Prometheus datasource is deprecated as of the 2026.1 release and will
be removed in a future release. Use the Aetos datasource instead, which
provides the same functionality with added multi-tenancy and Keystone
authentication support. See the :doc:`migrate-prometheus-to-aetos` guide.
.. toctree::
:glob:
:maxdepth: 1
./*
aetos
grafana
migrate-prometheus-to-aetos
prometheus

198
doc/source/datasources/migrate-prometheus-to-aetos.rst Normal file
View File

@@ -0,0 +1,198 @@
==================================
Migrating from Prometheus to Aetos
==================================
Overview
========
This guide provides step-by-step instructions for migrating an existing
Watcher deployment from using the Prometheus datasource to the Aetos
datasource.
Why Migrate?
============
The Aetos datasource provides:
* **Multi-tenancy**: Keystone-based authentication and role validation
* **Security**: RBAC for metric access, no direct Prometheus exposure
* **Compatibility**: All Prometheus APIs available
* **Integration**: Native OpenStack service catalog integration
Prerequisites
=============
Before migrating, ensure:
1. Aetos service is deployed and operational
2. Aetos endpoint registered in Keystone with service type 'metric-storage'
3. Watcher service account has appropriate roles (admin or service role)
Migration Steps
===============
Step 1: Verify Aetos Availability
---------------------------------
.. code-block:: bash
openstack catalog show metric-storage
Verify that the Aetos endpoint is registered and accessible.
Step 2: Backup Current Configuration
------------------------------------
.. code-block:: bash
sudo cp /etc/watcher/watcher.conf /etc/watcher/watcher.conf.backup.$(date +%Y%m%d)
This step is not strictly required for the migration, but provides a working
configuration to revert to in case something goes wrong.
Step 3: Remove Prometheus Configuration
---------------------------------------
Edit ``/etc/watcher/watcher.conf`` and remove or comment out the
``[prometheus_client]`` section:
.. code-block:: ini
# [prometheus_client]
# host = prometheus.example.com
# port = 9090
# ... (remove all prometheus_client options)
Step 4: Update Datasource Configuration
---------------------------------------
Change the datasources option in ``[watcher_datasources]``:
.. code-block:: ini
[watcher_datasources]
datasources = aetos
.. note::
If you have other datasources configured (e.g., grafana), you can keep them
alongside aetos:
.. code-block:: ini
[watcher_datasources]
datasources = aetos,grafana
Step 5: Configure Aetos Client
------------------------------
Add the ``[aetos_client]`` section with appropriate values:
.. code-block:: ini
[aetos_client]
# Keystone endpoint interface (public, internal, or admin)
interface = public
# Region name for Keystone catalog lookup
region_name = RegionOne
# Prometheus label for FQDN
fqdn_label = fqdn
# Prometheus label for instance UUID
instance_uuid_label = resource
See the aetos datasource documentation page for more details on how to
configure it :doc:`aetos`
Step 6: Restart Watcher Services
--------------------------------
.. code-block:: bash
sudo systemctl restart watcher-decision-engine
.. note::
The service name may vary depending on your deployment method. In DevStack,
it might be ``devstack@watcher-decision-engine``. Check your system's service
naming convention.
Verification
============
After migration, verify the Aetos datasource is working:
1. Check Watcher logs for successful datasource initialization:
.. code-block:: bash
sudo journalctl -u watcher-decision-engine -f
Look for messages indicating Aetos datasource loaded successfully.
2. Trigger a test audit to verify metric collection:
.. code-block:: bash
openstack optimize audit create -g <goal-name> -s <strategy-name>
3. Monitor audit execution and verify metrics are retrieved from Aetos.
Troubleshooting
===============
Aetos Endpoint Not Found
------------------------
**Error**: "Aetos service not registered in Keystone"
**Solution**:
- Verify Aetos is deployed: ``openstack catalog show metric-storage``
- Register Aetos endpoint if missing
- Check service type is exactly 'metric-storage'
Authentication Failures
-----------------------
**Error**: "Unauthorized" or "403 Forbidden"
**Solution**:
- Verify Watcher service account has admin or service role
- Check Keystone token is valid
Metric Labels Mismatch
----------------------
**Error**: Metrics not found or empty results
**Solution**:
- Verify ``fqdn_label`` matches your Prometheus exporter configuration
- Verify ``instance_uuid_label`` matches your Prometheus labels
- Check Prometheus metrics are properly labeled
Rollback Procedure
==================
If migration fails, rollback to Prometheus:
.. code-block:: bash
# Restore backup
sudo cp /etc/watcher/watcher.conf.backup.YYYYMMDD /etc/watcher/watcher.conf
# Restart services
sudo systemctl restart watcher-decision-engine
Additional Resources
====================
* Aetos documentation: https://docs.openstack.org/aetos/latest/
* Watcher datasources guide: :doc:`index`
* :doc:`prometheus`
* :doc:`aetos`

8
doc/source/datasources/prometheus.rst
View File

@@ -2,6 +2,14 @@
Prometheus datasource
=====================
.. note::
**DEPRECATED**: The Prometheus datasource is deprecated as of the 2026.1
release and will be removed in a future release. Please use the Aetos
datasource instead, which provides the same functionality with added
multi-tenancy and Keystone authentication support.
See :doc:`migrate-prometheus-to-aetos` for migration instructions.
Synopsis
--------
The Prometheus datasource allows Watcher to use a Prometheus server as the

38
releasenotes/notes/deprecate-prometheus-datasource-f8a3c2e1d9b4f5a7.yaml Normal file
View File

@@ -0,0 +1,38 @@
---
deprecations:
- |
The Prometheus datasource (``prometheus``) is now deprecated in favor of
the Aetos datasource (``aetos``). The Aetos datasource provides the same
Prometheus-based metrics collection functionality with the following
additional benefits:
* Multi-tenancy support through Keystone authentication
* Role-Based Access Control (RBAC) for metric access
* Better integration with OpenStack security patterns
* All Prometheus query functionality (PromQL) is preserved
The Prometheus datasource will be removed in a later release.
Deployers using the Prometheus datasource should migrate to Aetos.
See the migration guide at:
https://docs.openstack.org/watcher/latest/admin/datasources/migrate-prometheus-to-aetos.html
upgrade:
- |
If you are currently using the Prometheus datasource
(``datasources = prometheus`` in ``[watcher_datasources]``), you should
plan to migrate to the Aetos datasource.
The migration requires:
* Deploying the Aetos reverse-proxy service
* Updating Watcher configuration to use ``datasources = aetos``
* Configuring Aetos endpoint in Keystone catalog
* Updating the ``[aetos_client]`` configuration section
For detailed migration steps, see the migration guide.
The default value for the 'watcher_datasources' option has changed from
"['gnocchi', 'monasca', 'grafana', 'prometheus']" to
"['gnocchi', 'monasca', 'grafana', 'aetos']". See
https://docs.openstack.org/watcher/latest/configuration/watcher.html#watcher-datasources for more details

9
watcher/conf/datasources.py
View File

@@ -17,8 +17,8 @@
from oslo_config import cfg
from watcher.decision_engine.datasources import aetos
from watcher.decision_engine.datasources import manager
from watcher.decision_engine.datasources import prometheus
datasources = cfg.OptGroup(name='watcher_datasources',
title='Configuration Options for watcher'
@@ -27,10 +27,11 @@ datasources = cfg.OptGroup(name='watcher_datasources',
possible_datasources = list(manager.DataSourceManager.metric_map.keys())
# NOTE(jwysogla): Having the Aetos and Prometheus datasources specified at the
# same time raises a DataSourceConfigConflict exception. So remove the Aetos
# datasource from the list to have a valid default configuration.
# same time raises a DataSourceConfigConflict exception. The Prometheus
# datasource is deprecated in favor of Aetos, so we remove Prometheus from
# the default configuration.
default_datasources = list(possible_datasources)
default_datasources.remove(aetos.AetosHelper.NAME)
default_datasources.remove(prometheus.PrometheusHelper.NAME)
DATASOURCES_OPTS = [
cfg.ListOpt("datasources",

125
watcher/conf/prometheus_client.py
View File

@@ -15,43 +15,98 @@
from oslo_config import cfg
prometheus_client = cfg.OptGroup(name='prometheus_client',
title='Configuration Options for Prometheus',
help="See https://docs.openstack.org/watcher/"
"latest/datasources/prometheus.html for "
"details on how these options are used.")
prometheus_client = cfg.OptGroup(
name='prometheus_client',
title='Configuration Options for Prometheus (DEPRECATED)',
help="DEPRECATED: The Prometheus datasource is deprecated in favor "
"of the Aetos datasource. See https://docs.openstack.org/"
"watcher/latest/datasources/migrate-prometheus-to-aetos.html "
"for migration instructions.")
PROMETHEUS_CLIENT_OPTS = [
cfg.HostAddressOpt('host',
help="The hostname or IP address for the prometheus "
"server."),
cfg.PortOpt('port',
default=9090,
help="The port number used by the prometheus server."),
cfg.StrOpt('fqdn_label',
default="fqdn",
help="The label that Prometheus uses to store the fqdn of "
"exporters. Defaults to 'fqdn'."),
cfg.StrOpt('instance_uuid_label',
default="resource",
help="The label that Prometheus uses to store the uuid of "
"OpenStack instances. Defaults to 'resource'."),
cfg.StrOpt('username',
help="The basic_auth username to use to authenticate with the "
"Prometheus server."),
cfg.StrOpt('password',
secret=True,
help="The basic_auth password to use to authenticate with the "
"Prometheus server."),
cfg.StrOpt('cafile',
help="Path to the CA certificate for establishing a TLS "
"connection with the Prometheus server."),
cfg.StrOpt('certfile',
help="Path to the client certificate for establishing a TLS "
"connection with the Prometheus server."),
cfg.StrOpt('keyfile',
help="Path to the client key for establishing a TLS "
"connection with the Prometheus server."),
cfg.HostAddressOpt(
'host',
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="The hostname or IP address for the prometheus server. "
"DEPRECATED: Use Aetos datasource instead."),
cfg.PortOpt(
'port',
default=9090,
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="The port number used by the prometheus server. "
"DEPRECATED: Use Aetos datasource instead."),
cfg.StrOpt(
'fqdn_label',
default="fqdn",
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="The label that Prometheus uses to store the fqdn of "
"exporters. Defaults to 'fqdn'. "
"DEPRECATED: Use Aetos datasource instead."),
cfg.StrOpt(
'instance_uuid_label',
default="resource",
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="The label that Prometheus uses to store the uuid of "
"OpenStack instances. Defaults to 'resource'. "
"DEPRECATED: Use Aetos datasource instead."),
cfg.StrOpt(
'username',
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="The basic_auth username to use to authenticate with the "
"Prometheus server. DEPRECATED: Use Aetos datasource "
"instead."),
cfg.StrOpt(
'password',
secret=True,
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="The basic_auth password to use to authenticate with the "
"Prometheus server. DEPRECATED: Use Aetos datasource "
"instead."),
cfg.StrOpt(
'cafile',
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="Path to the CA certificate for establishing a TLS "
"connection with the Prometheus server. "
"DEPRECATED: Use Aetos datasource instead."),
cfg.StrOpt(
'certfile',
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="Path to the client certificate for establishing a TLS "
"connection with the Prometheus server. "
"DEPRECATED: Use Aetos datasource instead."),
cfg.StrOpt(
'keyfile',
deprecated_for_removal=True,
deprecated_reason='Prometheus datasource is deprecated in favor '
'of Aetos datasource',
deprecated_since='2026.1',
help="Path to the client key for establishing a TLS "
"connection with the Prometheus server. "
"DEPRECATED: Use Aetos datasource instead."),
]

3
watcher/decision_engine/datasources/manager.py
View File

@@ -62,6 +62,9 @@ class DataSourceManager:
LOG.warning('Invalid Datasource: %s. Allowed: %s ', *msgargs)
self.datasources = self.config.datasources
if self.datasources and 'prometheus' in self.datasources:
LOG.warning('The prometheus datasource is deprecated and will '
'be removed in a future release.')
self._validate_datasource_config()

11
watcher/decision_engine/datasources/prometheus.py
View File

@@ -13,9 +13,11 @@
# under the License.
#
from debtcollector import removals
from observabilityclient import prometheus_client
from oslo_config import cfg
from oslo_log import log
import warnings
from watcher._i18n import _
from watcher.common import exception
@@ -24,12 +26,21 @@ from watcher.decision_engine.datasources import prometheus_base
CONF = cfg.CONF
LOG = log.getLogger(__name__)
warnings.simplefilter("once")
@removals.removed_class("PrometheusHelper", version="2026.1",
removal_version="2027.1")
class PrometheusHelper(prometheus_base.PrometheusBase):
"""PrometheusHelper class for retrieving metrics from Prometheus server
This class implements the PrometheusBase to allow Watcher to query
Prometheus as a data source for metrics.
.. deprecated:: 2026.1
The Prometheus datasource is deprecated in favor of the Aetos
datasource. Use Aetos for the same functionality with added
multi-tenancy and Keystone authentication support.
"""
NAME = 'prometheus'