Add volume metrics dashboard
Implements a comprehensive admin dashboard for monitoring block storage Features: - Live metrics dashboard with HTMX auto-refresh - Prometheus datasource integration via Aetos pattern - Storage pool monitoring with capacity visualization - Capacity forecast using predict_linear() to identify pools at risk - Chart.js doughnut chart for volume/snapshot distribution - Responsive UI with Bootstrap grid layout - Configuration management via centralized config module - Support for both fake (development) and Prometheus datasources The capacity prediction feature analyzes 7 days of historical data to forecast when storage pools will run out of space within 30 days, helping administrators proactively manage storage resources. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Change-Id: I3dc2676d0d247df242a16df5f32c219ee0d29f53 Signed-off-by: Victoria Martinez de la Cruz <victoria@redhat.com>
This commit is contained in:
co-authored by
Claude Sonnet 4.5
parent
8be1498ce8
commit
7590259f25
@@ -67,3 +67,6 @@ releasenotes/build
|
||||
# DS_Store on Mac computers
|
||||
*.DS_Store
|
||||
CLAUDE.md
|
||||
|
||||
# Vendored JS (downloaded via get_vendor.sh)
|
||||
src/grian_ui/static/vendor/*.js
|
||||
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
include AUTHORS
|
||||
include ChangeLog
|
||||
exclude .gitignore
|
||||
exclude .gitreview
|
||||
|
||||
global-exclude *.pyc
|
||||
|
||||
recursive-include src/grian_ui *.py
|
||||
recursive-include src/grian_ui/templates *.html
|
||||
recursive-include src/grian_ui/static *.css *.js *.png *.jpg *.gif *.ico
|
||||
recursive-include src/grian_ui/enabled *.py
|
||||
@@ -0,0 +1,185 @@
|
||||
========================
|
||||
Volume Metrics Dashboard
|
||||
========================
|
||||
|
||||
The Volume Metrics dashboard provides cloud administrators with a real-time
|
||||
overview of Cinder block storage resources across the deployment. It is
|
||||
accessible under **Admin > Telemetry > Volume Metrics** and requires the
|
||||
``openstack.roles.admin`` permission.
|
||||
|
||||
All metrics originate from `Ceilometer <https://docs.openstack.org/ceilometer/latest/>`_
|
||||
volume pollster data, stored in Prometheus via the
|
||||
`sg-core <https://github.com/infrawatch/sg-core>`_ telemetry pipeline, and
|
||||
queried through the `Aetos <https://opendev.org/openstack/aetos>`_
|
||||
observability proxy (``service_type: metric-storage``).
|
||||
|
||||
Ceilometer meters follow a naming convention when exposed as Prometheus
|
||||
metrics: dots are replaced with underscores and a ``ceilometer_`` prefix is
|
||||
added (e.g. ``volume.size`` becomes ``ceilometer_volume_size``).
|
||||
|
||||
|
||||
Panels
|
||||
======
|
||||
|
||||
Summary Cards
|
||||
-------------
|
||||
|
||||
Four metric cards displayed at the top of the dashboard, auto-refreshing
|
||||
every **10 seconds**.
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 25 30 45
|
||||
|
||||
* - Card
|
||||
- Ceilometer Meter
|
||||
- PromQL
|
||||
* - Total Storage (GiB)
|
||||
- ``volume.provider.pool.capacity.total``
|
||||
- ``sum(ceilometer_volume_provider_pool_capacity_total)``
|
||||
* - Allocated Storage (GiB)
|
||||
- ``volume.provider.pool.capacity.allocated``
|
||||
- ``sum(ceilometer_volume_provider_pool_capacity_allocated)``
|
||||
* - Total Volumes
|
||||
- ``volume.size``
|
||||
- ``count(ceilometer_volume_size)``
|
||||
* - Total Snapshots
|
||||
- ``volume.snapshot.size``
|
||||
- ``count(ceilometer_volume_snapshot_size)``
|
||||
|
||||
**Total Storage** and **Allocated Storage** aggregate the capacity reported
|
||||
by the Cinder volume backend(s) (e.g. LVM, Ceph) across all storage pools.
|
||||
**Total Volumes** and **Total Snapshots** count the number of distinct
|
||||
Ceilometer samples, which corresponds to the number of active volumes and
|
||||
snapshots in the deployment.
|
||||
|
||||
|
||||
Volume and Snapshot Distribution (Doughnut Chart)
|
||||
-------------------------------------------------
|
||||
|
||||
A Chart.js doughnut chart showing the proportion of volumes to snapshots.
|
||||
Rendered on initial page load from the same ``count()`` queries used by the
|
||||
summary cards.
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 25 30 45
|
||||
|
||||
* - Segment
|
||||
- Ceilometer Meter
|
||||
- PromQL
|
||||
* - Volumes
|
||||
- ``volume.size``
|
||||
- ``count(ceilometer_volume_size)``
|
||||
* - Snapshots
|
||||
- ``volume.snapshot.size``
|
||||
- ``count(ceilometer_volume_snapshot_size)``
|
||||
|
||||
This chart gives a quick visual ratio of how storage objects are distributed
|
||||
between volumes and snapshots.
|
||||
|
||||
|
||||
Storage Pools
|
||||
-------------
|
||||
|
||||
A per-pool breakdown with capacity bars and a detailed table. Rendered on
|
||||
initial page load.
|
||||
|
||||
The dashboard first discovers all pools by querying the raw (un-aggregated)
|
||||
capacity total metric, then fetches per-pool details by filtering on the
|
||||
``resource`` label, which contains the Cinder pool name.
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 25 30 45
|
||||
|
||||
* - Metric
|
||||
- Ceilometer Meter
|
||||
- PromQL
|
||||
* - Total capacity
|
||||
- ``volume.provider.pool.capacity.total``
|
||||
- ``ceilometer_volume_provider_pool_capacity_total``
|
||||
* - Allocated capacity
|
||||
- ``volume.provider.pool.capacity.allocated``
|
||||
- ``ceilometer_volume_provider_pool_capacity_allocated{resource="<pool>"}``
|
||||
* - Free capacity
|
||||
- ``volume.provider.pool.capacity.free``
|
||||
- ``ceilometer_volume_provider_pool_capacity_free{resource="<pool>"}``
|
||||
* - Provisioned capacity
|
||||
- ``volume.provider.pool.capacity.provisioned``
|
||||
- ``ceilometer_volume_provider_pool_capacity_provisioned{resource="<pool>"}``
|
||||
|
||||
Each pool is displayed with:
|
||||
|
||||
- A **capacity bar** showing the percentage of allocated vs. total capacity.
|
||||
- A **detail row** with free and provisioned values.
|
||||
- A **collapsible table** with all numeric values and a color-coded usage
|
||||
percentage label (green < 75%, yellow 75-89%, red >= 90%).
|
||||
|
||||
All values are in GiB, matching the units reported by Cinder's volume
|
||||
backend statistics.
|
||||
|
||||
|
||||
Capacity Forecast
|
||||
-----------------
|
||||
|
||||
A prediction table that forecasts when each storage pool will run out of
|
||||
free space, auto-refreshing every **60 seconds**.
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 25 30 45
|
||||
|
||||
* - Metric
|
||||
- Ceilometer Meter
|
||||
- PromQL
|
||||
* - Current free capacity
|
||||
- ``volume.provider.pool.capacity.free``
|
||||
- ``ceilometer_volume_provider_pool_capacity_free{resource="<pool>"}``
|
||||
* - Predicted free in 30 days
|
||||
- ``volume.provider.pool.capacity.free``
|
||||
- ``predict_linear(ceilometer_volume_provider_pool_capacity_free{resource="<pool>"}[7d], 30 * 86400)``
|
||||
|
||||
The ``predict_linear()`` function uses linear regression over the last
|
||||
**7 days** of free capacity samples to project the value **30 days** into
|
||||
the future. If the predicted value is negative, the dashboard computes the
|
||||
estimated number of days until the pool reaches zero free capacity using
|
||||
linear interpolation::
|
||||
|
||||
decline_per_day = (current_free - predicted_value) / 30
|
||||
days_until_full = current_free / decline_per_day
|
||||
|
||||
Status labels:
|
||||
|
||||
- **Full in N days** (red) — predicted free capacity drops below zero within
|
||||
30 days.
|
||||
- **Declining** (yellow) — free capacity is decreasing but will not reach
|
||||
zero within 30 days.
|
||||
- **Healthy** (green) — free capacity is stable or increasing.
|
||||
- **Insufficient data** (grey) — not enough historical data for
|
||||
``predict_linear()`` to produce a result (requires at least 2 data points
|
||||
over the 7-day window).
|
||||
|
||||
.. note::
|
||||
|
||||
Predictions require at least 7 days of continuous Ceilometer data
|
||||
collection. On a fresh deployment, this panel will show "Insufficient
|
||||
data" until enough history accumulates.
|
||||
|
||||
|
||||
Data Pipeline
|
||||
=============
|
||||
|
||||
The metrics flow through the following pipeline::
|
||||
|
||||
Cinder → Ceilometer pollster → sg-core (STF) → Prometheus → Aetos → Horizon
|
||||
|
||||
1. **Ceilometer** polls Cinder every 600 seconds (default) for volume and
|
||||
pool statistics.
|
||||
2. **sg-core** receives Ceilometer samples over AMQP/TCP and writes them as
|
||||
Prometheus metrics.
|
||||
3. **Prometheus** stores the time-series data and serves PromQL queries.
|
||||
4. **Aetos** acts as a Keystone-authenticated proxy to Prometheus, registered
|
||||
in the service catalog as ``metric-storage``.
|
||||
5. **Horizon (grian-ui)** queries Aetos using ``python-observabilityclient``
|
||||
and renders the results.
|
||||
@@ -14,3 +14,5 @@ documentation for details.
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents:
|
||||
|
||||
admin-volume-metrics
|
||||
|
||||
+4
-2
@@ -9,21 +9,22 @@ readme = "README.rst"
|
||||
authors = [
|
||||
{name = "OpenStack", email = "openstack-discuss@lists.openstack.org"},
|
||||
]
|
||||
requires-python = ">=3.10"
|
||||
requires-python = ">=3.11"
|
||||
license = "Apache-2.0"
|
||||
classifiers = [
|
||||
"Environment :: OpenStack",
|
||||
"Intended Audience :: Information Technology",
|
||||
"Intended Audience :: System Administrators",
|
||||
"Operating System :: POSIX :: Linux",
|
||||
"Development Status :: 2 - Pre-Alpha",
|
||||
"Programming Language :: Python",
|
||||
"Programming Language :: Python :: Implementation :: CPython",
|
||||
"Programming Language :: Python :: 3 :: Only",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Programming Language :: Python :: 3.14",
|
||||
]
|
||||
dynamic = ["version"]
|
||||
keywords = ["openstack", "horizon", "telemetry"]
|
||||
@@ -64,6 +65,7 @@ ignore = [
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"tests/grian_ui_tests/*" = ["S"]
|
||||
"src/grian_ui/datasource/fake.py" = ["S311"]
|
||||
|
||||
[tool.ruff.lint.isort]
|
||||
known-first-party = ["grian_ui"]
|
||||
|
||||
@@ -7,3 +7,4 @@ oslo.utils>=4.7.0 # Apache-2.0
|
||||
# date but we do not test them so no guarantee of having them all correct. If
|
||||
# you find any incorrect lower bounds, let us know or propose a fix.
|
||||
pbr>=6.0.0 # Apache-2.0
|
||||
python-observabilityclient>=1.0.0 # Apache-2.0
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
"""
|
||||
Centralized configuration management for Grian UI.
|
||||
Validates and provides typed access to Django settings.
|
||||
"""
|
||||
|
||||
from django.conf import settings
|
||||
|
||||
|
||||
# Valid datasource types
|
||||
DATASOURCE_FAKE = "fake"
|
||||
DATASOURCE_PROMETHEUS = "prometheus"
|
||||
VALID_DATASOURCE_TYPES = (DATASOURCE_FAKE, DATASOURCE_PROMETHEUS)
|
||||
|
||||
|
||||
class ConfigurationError(Exception):
|
||||
"""Raised when configuration validation fails."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def _get_grian_plugin_config():
|
||||
"""Get and validate GRIAN_PLUGIN configuration."""
|
||||
config = getattr(settings, "GRIAN_PLUGIN", None)
|
||||
|
||||
if config is None:
|
||||
# Return default configuration
|
||||
return {"datasource": DATASOURCE_FAKE}
|
||||
|
||||
if not isinstance(config, dict):
|
||||
raise ConfigurationError(
|
||||
f"GRIAN_PLUGIN must be a dict, got {type(config).__name__}"
|
||||
)
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def get_datasource_type():
|
||||
"""
|
||||
Get the configured datasource type.
|
||||
|
||||
Returns:
|
||||
str: The datasource type (e.g., 'fake', 'prometheus')
|
||||
|
||||
Raises:
|
||||
ConfigurationError: If datasource type is invalid
|
||||
"""
|
||||
config = _get_grian_plugin_config()
|
||||
datasource = config.get("datasource", DATASOURCE_FAKE)
|
||||
|
||||
if not isinstance(datasource, str):
|
||||
raise ConfigurationError(
|
||||
f"GRIAN_PLUGIN['datasource'] must be a string, got {type(datasource).__name__}"
|
||||
)
|
||||
|
||||
if datasource not in VALID_DATASOURCE_TYPES:
|
||||
raise ConfigurationError(
|
||||
f"Invalid datasource type '{datasource}'. "
|
||||
f"Valid types: {', '.join(VALID_DATASOURCE_TYPES)}"
|
||||
)
|
||||
|
||||
return datasource
|
||||
|
||||
|
||||
def get_openstack_keystone_url():
|
||||
"""Get OpenStack Keystone URL from settings."""
|
||||
url = getattr(settings, "OPENSTACK_KEYSTONE_URL", None)
|
||||
if not url:
|
||||
raise ConfigurationError(
|
||||
"OPENSTACK_KEYSTONE_URL is not configured in settings"
|
||||
)
|
||||
return url
|
||||
|
||||
|
||||
def get_openstack_endpoint_type():
|
||||
"""Get OpenStack endpoint type (interface) from settings."""
|
||||
return getattr(settings, "OPENSTACK_ENDPOINT_TYPE", "publicURL")
|
||||
@@ -0,0 +1,31 @@
|
||||
================================
|
||||
Testing the Volume Metrics panel
|
||||
================================
|
||||
|
||||
.. note::
|
||||
These are development instructions. This content will be moved to the
|
||||
official documentation.
|
||||
|
||||
This document provides instructions on how to test the Volume Metrics panel.
|
||||
|
||||
Prerequisites
|
||||
=============
|
||||
|
||||
1. **Enable the panels**
|
||||
|
||||
Copy the files under ``src/grian_ui/local/enabled`` to the
|
||||
``local/enabled`` directory of your Horizon installation.
|
||||
|
||||
2. **Configure settings**
|
||||
|
||||
Copy the settings file from ``src/grian_ui/local/local_settings.d``
|
||||
to the ``local/local_settings.d`` directory of your Horizon
|
||||
installation.
|
||||
|
||||
3. **Install python-observabilityclient**
|
||||
|
||||
Install the latest ``python-observabilityclient`` in your environment.
|
||||
Follow the instructions at
|
||||
https://github.com/openstack/python-observabilityclient.
|
||||
|
||||
After completing these steps, restart your web server.
|
||||
@@ -0,0 +1,3 @@
|
||||
"""
|
||||
Volume metrics panel for Grian UI.
|
||||
"""
|
||||
@@ -0,0 +1,10 @@
|
||||
import horizon
|
||||
|
||||
from django.utils.translation import gettext_lazy as _
|
||||
|
||||
|
||||
class VolumeMetrics(horizon.Panel):
|
||||
name = _("Volume Metrics")
|
||||
slug = "volume_metrics"
|
||||
permissions = ("openstack.roles.admin",)
|
||||
urls = "grian_ui.content.volume_metrics.urls"
|
||||
@@ -0,0 +1,23 @@
|
||||
from django.urls import re_path
|
||||
|
||||
from grian_ui.content.volume_metrics import views
|
||||
|
||||
|
||||
urlpatterns = [
|
||||
re_path(r"^$", views.VolumesMetricsIndexView.as_view(), name="index"),
|
||||
re_path(
|
||||
r"^metrics-update/$",
|
||||
views.MetricsUpdateView.as_view(),
|
||||
name="metrics_update",
|
||||
),
|
||||
re_path(
|
||||
r"^chart-update/$",
|
||||
views.ChartUpdateView.as_view(),
|
||||
name="chart_update",
|
||||
),
|
||||
re_path(
|
||||
r"^capacity-prediction/$",
|
||||
views.CapacityPredictionView.as_view(),
|
||||
name="capacity_prediction",
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,93 @@
|
||||
from django.utils.translation import gettext_lazy as _
|
||||
from django.views import generic
|
||||
|
||||
from grian_ui.datasource.api import get_datasource
|
||||
|
||||
|
||||
class VolumesMetricsIndexView(generic.TemplateView):
|
||||
template_name = "admin/volume_metrics/index.html"
|
||||
page_title = _("Volume Metrics")
|
||||
|
||||
def get_context_data(self, **kwargs):
|
||||
context = super().get_context_data(**kwargs)
|
||||
context["page_title"] = self.page_title
|
||||
client = get_datasource(self.request)
|
||||
|
||||
# Get telemetry data from datasource
|
||||
try:
|
||||
context["overview"] = client.get_telemetry_overview()
|
||||
context["volumes"] = client.get_volumes_count()
|
||||
context["snapshots"] = client.get_snapshots_count()
|
||||
context["storage_pools"] = client.get_storage_pools()
|
||||
context["predictions"] = client.get_capacity_predictions()
|
||||
except Exception as e:
|
||||
context["error_message"] = f"Error loading telemetry data: {e}"
|
||||
context["data_loaded"] = False
|
||||
# Provide defaults on error
|
||||
context["overview"] = {}
|
||||
context["volumes"] = 0
|
||||
context["snapshots"] = 0
|
||||
context["storage_pools"] = []
|
||||
context["predictions"] = []
|
||||
|
||||
return context
|
||||
|
||||
|
||||
class MetricsUpdateView(generic.TemplateView):
|
||||
"""HTMX endpoint for updating metrics cards."""
|
||||
|
||||
template_name = "admin/volume_metrics/_metrics_cards.html"
|
||||
|
||||
def get_context_data(self, **kwargs):
|
||||
context = super().get_context_data(**kwargs)
|
||||
client = get_datasource(self.request)
|
||||
|
||||
try:
|
||||
context["overview"] = client.get_telemetry_overview()
|
||||
context["volumes"] = client.get_volumes_count()
|
||||
context["snapshots"] = client.get_snapshots_count()
|
||||
except Exception:
|
||||
# Provide defaults on error
|
||||
context["overview"] = {}
|
||||
context["volumes"] = 0
|
||||
context["snapshots"] = 0
|
||||
|
||||
return context
|
||||
|
||||
|
||||
class ChartUpdateView(generic.TemplateView):
|
||||
"""HTMX endpoint for updating the storage chart."""
|
||||
|
||||
template_name = "admin/volume_metrics/_storage_chart.html"
|
||||
|
||||
def get_context_data(self, **kwargs):
|
||||
context = super().get_context_data(**kwargs)
|
||||
client = get_datasource(self.request)
|
||||
|
||||
try:
|
||||
context["volumes"] = client.get_volumes_count()
|
||||
context["snapshots"] = client.get_snapshots_count()
|
||||
except Exception:
|
||||
# Provide defaults on error
|
||||
context["volumes"] = 0
|
||||
context["snapshots"] = 0
|
||||
|
||||
return context
|
||||
|
||||
|
||||
class CapacityPredictionView(generic.TemplateView):
|
||||
"""HTMX endpoint for capacity prediction visualization."""
|
||||
|
||||
template_name = "admin/volume_metrics/_capacity_prediction.html"
|
||||
|
||||
def get_context_data(self, **kwargs):
|
||||
context = super().get_context_data(**kwargs)
|
||||
client = get_datasource(self.request)
|
||||
|
||||
try:
|
||||
context["predictions"] = client.get_capacity_predictions()
|
||||
except Exception:
|
||||
# Provide defaults on error
|
||||
context["predictions"] = []
|
||||
|
||||
return context
|
||||
@@ -1,2 +1,63 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
"""
|
||||
Datasource API for telemetry data.
|
||||
Provides an abstraction layer for different telemetry backends.
|
||||
"""
|
||||
|
||||
import importlib
|
||||
|
||||
from abc import ABC
|
||||
from abc import abstractmethod
|
||||
|
||||
from grian_ui import config
|
||||
|
||||
|
||||
def get_datasource(request):
|
||||
"""
|
||||
Get the configured datasource backend.
|
||||
|
||||
Returns the datasource implementation based on GRIAN_PLUGIN['datasource']
|
||||
setting. Defaults to 'fake' if not configured.
|
||||
"""
|
||||
datasource_type = config.get_datasource_type()
|
||||
module_path = f"grian_ui.datasource.{datasource_type}"
|
||||
|
||||
try:
|
||||
datasource_module = importlib.import_module(module_path)
|
||||
return datasource_module.get_client(request)
|
||||
except ImportError:
|
||||
# Fallback to fake datasource
|
||||
import grian_ui.datasource.fake as fake_datasource
|
||||
|
||||
return fake_datasource.get_client(request)
|
||||
|
||||
|
||||
class DatasourceClient(ABC):
|
||||
"""Abstract base class for datasource clients."""
|
||||
|
||||
@abstractmethod
|
||||
def get_telemetry_overview(self) -> dict:
|
||||
"""Get overview telemetry data."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_volumes_count(self) -> int:
|
||||
"""Get volumes count telemetry data."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_snapshots_count(self) -> int:
|
||||
"""Get snapshots count telemetry data."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_storage_pools(self) -> list:
|
||||
"""Get storage pools information."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_capacity_predictions(self) -> list:
|
||||
"""Get capacity predictions for storage pools."""
|
||||
pass
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
"""
|
||||
Fake datasource implementation for development and testing.
|
||||
Provides realistic sample telemetry data.
|
||||
"""
|
||||
|
||||
import random # nosec
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from grian_ui.datasource.api import DatasourceClient
|
||||
|
||||
|
||||
class FakeTelemetryClient(DatasourceClient):
|
||||
"""Fake telemetry client that generates sample data."""
|
||||
|
||||
def get_telemetry_overview(self) -> dict:
|
||||
"""
|
||||
Get telemetry overview data.
|
||||
|
||||
Returns sample data for development/testing purposes.
|
||||
In a real implementation, this would aggregate multiple queries.
|
||||
"""
|
||||
return {
|
||||
"total_metrics": 1247,
|
||||
"active_instances": 18,
|
||||
"alerts_count": 3,
|
||||
"cpu_usage_avg": round(random.uniform(15.0, 85.0), 1), # nosec
|
||||
"memory_usage_avg": round(random.uniform(25.0, 75.0), 1), # nosec
|
||||
"network_throughput": round(random.uniform(50.0, 200.0), 1), # nosec
|
||||
"disk_io_rate": round(random.uniform(10.0, 100.0), 1), # nosec
|
||||
"uptime_percentage": round(random.uniform(98.5, 99.9), 2), # nosec
|
||||
"total_storage": 10240, # GB
|
||||
"allocated_storage": round(random.uniform(6000, 9000)), # nosec GB
|
||||
"last_updated": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
|
||||
}
|
||||
|
||||
def get_volumes_count(self) -> int:
|
||||
"""
|
||||
Get volumes count telemetry data.
|
||||
|
||||
Returns hardcoded placeholder value for development/testing.
|
||||
"""
|
||||
return 120
|
||||
|
||||
def get_snapshots_count(self) -> int:
|
||||
"""
|
||||
Get snapshots count telemetry data.
|
||||
|
||||
Returns hardcoded placeholder value for development/testing.
|
||||
"""
|
||||
return 45
|
||||
|
||||
def get_storage_pools(self) -> list:
|
||||
"""
|
||||
Get storage pools information.
|
||||
|
||||
Returns sample pool data for development/testing.
|
||||
"""
|
||||
return [
|
||||
{
|
||||
"name": "storage-pool-01",
|
||||
"total": 5120.0,
|
||||
"allocated": 4096.0,
|
||||
"free": 1024.0,
|
||||
"provisioned": 5120.0,
|
||||
},
|
||||
{
|
||||
"name": "storage-pool-02",
|
||||
"total": 5120.0,
|
||||
"allocated": 4096.0,
|
||||
"free": 1024.0,
|
||||
"provisioned": 8192.0,
|
||||
},
|
||||
]
|
||||
|
||||
def get_capacity_predictions(self) -> list:
|
||||
"""
|
||||
Get capacity predictions for storage pools.
|
||||
|
||||
Returns sample prediction data for development/testing.
|
||||
"""
|
||||
return [
|
||||
{
|
||||
"name": "storage-pool-01",
|
||||
"current_free": 1024.0,
|
||||
"predicted_value": 512.0,
|
||||
"days_until_full": None,
|
||||
"will_run_out": False,
|
||||
},
|
||||
{
|
||||
"name": "storage-pool-02",
|
||||
"current_free": 1024.0,
|
||||
"predicted_value": -200.0,
|
||||
"days_until_full": 24.5,
|
||||
"will_run_out": True,
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def get_client(request=None) -> DatasourceClient:
|
||||
"""
|
||||
Get a fake telemetry client instance.
|
||||
|
||||
Args:
|
||||
request: HTTP request (unused for fake client, accepted for interface compatibility)
|
||||
|
||||
Returns:
|
||||
FakeTelemetryClient instance
|
||||
"""
|
||||
return FakeTelemetryClient()
|
||||
@@ -0,0 +1,296 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
"""
|
||||
Prometheus datasource implementation via Aetos.
|
||||
Provides access to Prometheus metrics through OpenStack observability service.
|
||||
"""
|
||||
|
||||
from keystoneauth1.identity import v3 # noqa: I001
|
||||
from keystoneauth1 import session # noqa: I001
|
||||
from observabilityclient.utils import metric_utils as obs_client_utils
|
||||
|
||||
from grian_ui import config
|
||||
from grian_ui.datasource.api import DatasourceClient
|
||||
|
||||
|
||||
STALENESS_WINDOW = "10m"
|
||||
|
||||
|
||||
class PrometheusTelemetryClient(DatasourceClient):
|
||||
"""
|
||||
Prometheus telemetry client to retrieves data from the OSP deployment
|
||||
"""
|
||||
|
||||
def __init__(self, request):
|
||||
"""
|
||||
Initialize Prometheus Telemetry client via Aetos.
|
||||
|
||||
Args:
|
||||
request: Django HTTP request containing user authentication
|
||||
|
||||
Note:
|
||||
Uses get_prom_client_from_keystone for proper Aetos/Prometheus
|
||||
client initialization with Keystone session.
|
||||
A new client instance should be created per page request,
|
||||
but can be reused for multiple metric queries within that request.
|
||||
"""
|
||||
auth_url = config.get_openstack_keystone_url()
|
||||
token = request.user.token.id
|
||||
project_id = request.user.project_id
|
||||
domain_id = request.user.domain_id
|
||||
interface = config.get_openstack_endpoint_type()
|
||||
|
||||
auth = v3.Token(
|
||||
auth_url=auth_url,
|
||||
token=token,
|
||||
project_id=project_id,
|
||||
domain_id=domain_id,
|
||||
)
|
||||
|
||||
sess = session.Session(auth=auth)
|
||||
|
||||
adapter_options = {
|
||||
"service_type": "metric-storage",
|
||||
"region_name": request.user.services_region,
|
||||
"interface": interface,
|
||||
}
|
||||
|
||||
# discovers the Aetos endpoint via service catalog
|
||||
self.client = obs_client_utils.get_prom_client_from_keystone(
|
||||
sess, adapter_options=adapter_options
|
||||
)
|
||||
|
||||
def get_telemetry_overview(self) -> dict:
|
||||
"""
|
||||
Get telemetry overview data.
|
||||
|
||||
Aggregates multiple Prometheus queries to build a dashboard overview.
|
||||
Returns computed metrics rather than raw metric list.
|
||||
"""
|
||||
overview = {}
|
||||
|
||||
try:
|
||||
# Query for vCPUs count
|
||||
cpu_result = self.client.query(query="sum(ceilometer_vcpus)")
|
||||
if cpu_result and len(cpu_result) > 0:
|
||||
overview["cpu_usage_avg"] = round(
|
||||
float(cpu_result[0].value), 1
|
||||
)
|
||||
else:
|
||||
overview["cpu_usage_avg"] = 0.0
|
||||
|
||||
# Query for memory usage in MB
|
||||
memory_result = self.client.query(
|
||||
query="sum(ceilometer_memory_usage)"
|
||||
)
|
||||
if memory_result and len(memory_result) > 0:
|
||||
# Convert MB to GB for display
|
||||
overview["memory_usage_avg"] = round(
|
||||
float(memory_result[0].value) / 1024, 1
|
||||
)
|
||||
else:
|
||||
overview["memory_usage_avg"] = 0.0
|
||||
|
||||
# Query for total storage capacity
|
||||
total_storage_result = self.client.query(
|
||||
query="sum(ceilometer_volume_provider_pool_capacity_total)"
|
||||
)
|
||||
if total_storage_result and len(total_storage_result) > 0:
|
||||
overview["total_storage"] = int(
|
||||
float(total_storage_result[0].value)
|
||||
)
|
||||
else:
|
||||
overview["total_storage"] = 0
|
||||
|
||||
# Query for allocated storage
|
||||
allocated_storage_result = self.client.query(
|
||||
query="sum(ceilometer_volume_provider_pool_capacity_allocated)"
|
||||
)
|
||||
if allocated_storage_result and len(allocated_storage_result) > 0:
|
||||
overview["allocated_storage"] = int(
|
||||
float(allocated_storage_result[0].value)
|
||||
)
|
||||
else:
|
||||
overview["allocated_storage"] = 0
|
||||
|
||||
except Exception as e:
|
||||
# Return partial data if some queries fail
|
||||
overview["error"] = str(e)
|
||||
|
||||
return overview
|
||||
|
||||
def get_volumes_count(self) -> int:
|
||||
"""
|
||||
Get telemetry volumes count.
|
||||
|
||||
Returns:
|
||||
int: Current count of volumes from Ceilometer metrics
|
||||
"""
|
||||
result = self.client.query(
|
||||
query=f"count(last_over_time(ceilometer_volume_size[{STALENESS_WINDOW}]))"
|
||||
)
|
||||
if result and len(result) > 0:
|
||||
return int(float(result[0].value))
|
||||
return 0
|
||||
|
||||
def get_snapshots_count(self) -> int:
|
||||
"""
|
||||
Get telemetry snapshots count.
|
||||
|
||||
Returns:
|
||||
int: Current count of snapshots from Ceilometer metrics
|
||||
"""
|
||||
result = self.client.query(
|
||||
query=f"count(last_over_time(ceilometer_volume_snapshot_size[{STALENESS_WINDOW}]))"
|
||||
)
|
||||
if result and len(result) > 0:
|
||||
return int(float(result[0].value))
|
||||
return 0
|
||||
|
||||
def get_storage_pools(self) -> list:
|
||||
"""
|
||||
Get storage pools information from Ceilometer metrics.
|
||||
|
||||
Returns:
|
||||
list: List of storage pool dictionaries with capacity metrics
|
||||
"""
|
||||
pools = []
|
||||
|
||||
try:
|
||||
# Get all unique pool resources
|
||||
# We'll use the total capacity metric to identify pools
|
||||
total_result = self.client.query(
|
||||
query="ceilometer_volume_provider_pool_capacity_total"
|
||||
)
|
||||
|
||||
if not total_result:
|
||||
return pools
|
||||
|
||||
# For each pool, gather all metrics
|
||||
for pool_metric in total_result:
|
||||
pool_name = pool_metric.labels.get("resource", "unknown")
|
||||
|
||||
pool_data = {
|
||||
"name": pool_name,
|
||||
"total": float(pool_metric.value),
|
||||
"allocated": 0.0,
|
||||
"free": 0.0,
|
||||
"provisioned": 0.0,
|
||||
}
|
||||
|
||||
# Get allocated capacity for this pool
|
||||
allocated_query = f'ceilometer_volume_provider_pool_capacity_allocated{{resource="{pool_name}"}}'
|
||||
allocated_result = self.client.query(query=allocated_query)
|
||||
if allocated_result and len(allocated_result) > 0:
|
||||
pool_data["allocated"] = float(allocated_result[0].value)
|
||||
|
||||
# Get free capacity for this pool
|
||||
free_query = f'ceilometer_volume_provider_pool_capacity_free{{resource="{pool_name}"}}'
|
||||
free_result = self.client.query(query=free_query)
|
||||
if free_result and len(free_result) > 0:
|
||||
pool_data["free"] = float(free_result[0].value)
|
||||
|
||||
# Get provisioned capacity for this pool
|
||||
provisioned_query = f'ceilometer_volume_provider_pool_capacity_provisioned{{resource="{pool_name}"}}'
|
||||
provisioned_result = self.client.query(query=provisioned_query)
|
||||
if provisioned_result and len(provisioned_result) > 0:
|
||||
pool_data["provisioned"] = float(
|
||||
provisioned_result[0].value
|
||||
)
|
||||
|
||||
pools.append(pool_data)
|
||||
|
||||
except Exception:
|
||||
# Return empty list on error
|
||||
return []
|
||||
|
||||
return pools
|
||||
|
||||
def get_capacity_predictions(self) -> list:
|
||||
"""
|
||||
Predict when storage pools will run out of space.
|
||||
|
||||
Uses predict_linear to forecast when free capacity will hit zero
|
||||
based on the last 7 days of data, looking 30 days into the future.
|
||||
|
||||
Returns:
|
||||
list: List of predictions per pool with:
|
||||
- name: pool name
|
||||
- days_until_full: predicted days until capacity hits zero (None if not predicted to fill)
|
||||
- predicted_value: the predicted free capacity in 30 days
|
||||
- current_free: current free capacity
|
||||
"""
|
||||
predictions = []
|
||||
|
||||
try:
|
||||
# Get all unique pool resources
|
||||
total_result = self.client.query(
|
||||
query="ceilometer_volume_provider_pool_capacity_total"
|
||||
)
|
||||
|
||||
if not total_result:
|
||||
return predictions
|
||||
|
||||
# For each pool, calculate prediction
|
||||
for pool_metric in total_result:
|
||||
pool_name = pool_metric.labels.get("resource", "unknown")
|
||||
|
||||
# Get current free capacity
|
||||
free_query = f'ceilometer_volume_provider_pool_capacity_free{{resource="{pool_name}"}}'
|
||||
free_result = self.client.query(query=free_query)
|
||||
current_free = 0.0
|
||||
if free_result and len(free_result) > 0:
|
||||
current_free = float(free_result[0].value)
|
||||
|
||||
# Predict free capacity in 30 days based on 7 days of history
|
||||
# predict_linear(metric[7d], 30 * 86400) predicts value 30 days from now
|
||||
predict_query = f'predict_linear(ceilometer_volume_provider_pool_capacity_free{{resource="{pool_name}"}}[7d], 30 * 86400)'
|
||||
|
||||
predict_result = self.client.query(query=predict_query)
|
||||
|
||||
prediction_data = {
|
||||
"name": pool_name,
|
||||
"current_free": current_free,
|
||||
"predicted_value": None,
|
||||
"days_until_full": None,
|
||||
"will_run_out": False,
|
||||
}
|
||||
|
||||
if predict_result and len(predict_result) > 0:
|
||||
predicted_value = float(predict_result[0].value)
|
||||
prediction_data["predicted_value"] = predicted_value
|
||||
|
||||
# If predicted to go below zero, calculate when
|
||||
if predicted_value < 0 and current_free > 0:
|
||||
# Linear interpolation to find when it crosses zero
|
||||
# current_free - (days_until_full / 30) * (current_free - predicted_value) = 0
|
||||
# Solve for days_until_full
|
||||
decline_per_day = (current_free - predicted_value) / 30
|
||||
if decline_per_day > 0:
|
||||
days_until_full = current_free / decline_per_day
|
||||
prediction_data["days_until_full"] = round(
|
||||
days_until_full, 1
|
||||
)
|
||||
prediction_data["will_run_out"] = True
|
||||
|
||||
predictions.append(prediction_data)
|
||||
|
||||
except Exception:
|
||||
# Return empty list on error
|
||||
return []
|
||||
|
||||
return predictions
|
||||
|
||||
|
||||
def get_client(request) -> DatasourceClient:
|
||||
"""
|
||||
Get a Prometheus telemetry client instance.
|
||||
|
||||
Args:
|
||||
request: Django HTTP request with user authentication
|
||||
|
||||
Returns:
|
||||
PrometheusTelemetryClient: New client instance for this request
|
||||
"""
|
||||
return PrometheusTelemetryClient(request)
|
||||
@@ -1,2 +0,0 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
@@ -97,6 +97,11 @@ STATIC_URL = "static/"
|
||||
|
||||
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
|
||||
|
||||
# Add grian_ui to INSTALLED_APPS if not already present
|
||||
INSTALLED_APPS = list(INSTALLED_APPS) # noqa
|
||||
if "grian_ui" not in INSTALLED_APPS:
|
||||
INSTALLED_APPS.append("grian_ui")
|
||||
|
||||
GRIAN_PLUGIN = {"datasource": os.getenv("GRIAN_DATA_SOURCE", "fake")}
|
||||
|
||||
# COMPRESS_OFFLINE = True
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
@@ -0,0 +1,4 @@
|
||||
PANEL_GROUP = "telemetry"
|
||||
PANEL_GROUP_NAME = "Telemetry"
|
||||
PANEL_GROUP_DASHBOARD = "admin"
|
||||
AUTO_DISCOVER_STATIC_FILES = True
|
||||
@@ -0,0 +1,14 @@
|
||||
# The name of the panel to be added to HORIZON_CONFIG. Required.
|
||||
PANEL = "volume_metrics"
|
||||
|
||||
# The name of the dashboard the PANEL associated with. Required.
|
||||
PANEL_DASHBOARD = "admin"
|
||||
|
||||
# The name of the panel group the PANEL is associated with.
|
||||
PANEL_GROUP = "telemetry"
|
||||
|
||||
# Python panel class of the PANEL to be added.
|
||||
ADD_PANEL = "grian_ui.content.volume_metrics.panel.VolumeMetrics"
|
||||
|
||||
# A list of applications to be prepended to INSTALLED_APPS
|
||||
ADD_INSTALLED_APPS = ["grian_ui"]
|
||||
@@ -1,4 +1,9 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
GRIAN_PLUGIN = {"datasource": "fake"}
|
||||
# Grian UI configuration
|
||||
# datasource options: 'fake', 'prometheus'
|
||||
# Note: 'prometheus' datasource uses the Aetos pattern with authenticated
|
||||
# Keystone session (following Watcher and CloudKitty). All Prometheus access
|
||||
# goes through Aetos - no direct host:port specification.
|
||||
GRIAN_PLUGIN = {"datasource": "prometheus"}
|
||||
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
#!/bin/bash
|
||||
CHART_JS_VERSION=4.4.1
|
||||
HTMX_VERSION=2.0.4
|
||||
rm -f *.js
|
||||
wget "https://unpkg.com/chart.js@${CHART_JS_VERSION}/dist/chart.umd.js" -O chart.umd.js
|
||||
wget "https://unpkg.com/htmx.org@${HTMX_VERSION}/dist/htmx.js" -O htmx.js
|
||||
@@ -0,0 +1,70 @@
|
||||
{% load i18n %}
|
||||
<div class="panel panel-default">
|
||||
<div class="panel-heading">
|
||||
<h3 class="panel-title">
|
||||
<i class="fa fa-line-chart"></i> {% trans "Capacity Forecast" %}
|
||||
<small class="text-muted">{% trans "(30-day prediction based on 7-day trend)" %}</small>
|
||||
</h3>
|
||||
</div>
|
||||
<div class="panel-body">
|
||||
{% if predictions %}
|
||||
<div class="table-responsive">
|
||||
<table class="table table-hover">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>{% trans "Pool Name" %}</th>
|
||||
<th class="text-right">{% trans "Current Free (GiB)" %}</th>
|
||||
<th class="text-right">{% trans "Predicted in 30 Days (GiB)" %}</th>
|
||||
<th class="text-center">{% trans "Status" %}</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for pred in predictions %}
|
||||
<tr class="{% if pred.will_run_out %}danger{% endif %}">
|
||||
<td><strong>{{ pred.name }}</strong></td>
|
||||
<td class="text-right">{{ pred.current_free|floatformat:2 }}</td>
|
||||
<td class="text-right">
|
||||
{% if pred.predicted_value is not None %}
|
||||
{{ pred.predicted_value|floatformat:2 }}
|
||||
{% else %}
|
||||
<span class="text-muted">{% trans "N/A" %}</span>
|
||||
{% endif %}
|
||||
</td>
|
||||
<td class="text-center">
|
||||
{% if pred.will_run_out %}
|
||||
<span class="label label-danger">
|
||||
<i class="fa fa-exclamation-triangle"></i>
|
||||
{% trans "Full in" %} {{ pred.days_until_full }} {% trans "days" %}
|
||||
</span>
|
||||
{% elif pred.predicted_value is not None and pred.predicted_value < pred.current_free %}
|
||||
<span class="label label-warning">
|
||||
<i class="fa fa-arrow-down"></i> {% trans "Declining" %}
|
||||
</span>
|
||||
{% elif pred.predicted_value is not None %}
|
||||
<span class="label label-success">
|
||||
<i class="fa fa-check"></i> {% trans "Healthy" %}
|
||||
</span>
|
||||
{% else %}
|
||||
<span class="label label-default">
|
||||
<i class="fa fa-question"></i> {% trans "Insufficient data" %}
|
||||
</span>
|
||||
{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<div class="alert alert-info" style="margin-top: 15px; margin-bottom: 0;">
|
||||
<i class="fa fa-info-circle"></i>
|
||||
<strong>{% trans "Note:" %}</strong>
|
||||
{% trans "Predictions are based on linear regression of the last 7 days of capacity data. Actual usage may vary." %}
|
||||
</div>
|
||||
{% else %}
|
||||
<p class="text-muted text-center" style="padding: 20px;">
|
||||
<i class="fa fa-info-circle"></i> {% trans "No prediction data available." %}
|
||||
</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
@@ -0,0 +1,78 @@
|
||||
{% load i18n %}
|
||||
<style>
|
||||
.metric-card {
|
||||
min-height: 140px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
.metric-card .panel-heading {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding: 20px 15px;
|
||||
}
|
||||
.metric-icon {
|
||||
font-size: 3.5em;
|
||||
opacity: 0.3;
|
||||
margin-right: 15px;
|
||||
}
|
||||
.metric-value {
|
||||
font-size: 2.5em;
|
||||
font-weight: 300;
|
||||
line-height: 1;
|
||||
margin-bottom: 5px;
|
||||
}
|
||||
.metric-label {
|
||||
font-size: 0.9em;
|
||||
opacity: 0.9;
|
||||
font-weight: 400;
|
||||
}
|
||||
</style>
|
||||
|
||||
<div class="col-lg-3 col-md-6 col-sm-6">
|
||||
<div class="panel panel-primary metric-card">
|
||||
<div class="panel-heading">
|
||||
<i class="fa fa-hdd-o metric-icon"></i>
|
||||
<div class="text-right" style="flex: 1;">
|
||||
<div class="metric-value">{{ overview.total_storage|default:"N/A" }}</div>
|
||||
<div class="metric-label">{% trans "Total Storage (GiB)" %}</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-lg-3 col-md-6 col-sm-6">
|
||||
<div class="panel panel-green metric-card">
|
||||
<div class="panel-heading">
|
||||
<i class="fa fa-check-circle metric-icon"></i>
|
||||
<div class="text-right" style="flex: 1;">
|
||||
<div class="metric-value">{{ overview.allocated_storage|default:"N/A" }}</div>
|
||||
<div class="metric-label">{% trans "Allocated Storage (GiB)" %}</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-lg-3 col-md-6 col-sm-6">
|
||||
<div class="panel panel-yellow metric-card">
|
||||
<div class="panel-heading">
|
||||
<i class="fa fa-database metric-icon"></i>
|
||||
<div class="text-right" style="flex: 1;">
|
||||
<div class="metric-value">{{ volumes }}</div>
|
||||
<div class="metric-label">{% trans "Total Volumes" %}</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-lg-3 col-md-6 col-sm-6">
|
||||
<div class="panel panel-red metric-card">
|
||||
<div class="panel-heading">
|
||||
<i class="fa fa-camera metric-icon"></i>
|
||||
<div class="text-right" style="flex: 1;">
|
||||
<div class="metric-value">{{ snapshots }}</div>
|
||||
<div class="metric-label">{% trans "Total Snapshots" %}</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -0,0 +1,107 @@
|
||||
{% load i18n %}
|
||||
<div class="panel panel-default">
|
||||
<div class="panel-heading">
|
||||
<h3 class="panel-title">
|
||||
<i class="fa fa-bar-chart"></i> {% trans "Volume and Snapshot Distribution" %}
|
||||
</h3>
|
||||
</div>
|
||||
<div class="panel-body" style="padding: 15px;">
|
||||
{% if volumes or snapshots %}
|
||||
<canvas id="storageMetricsChart" style="max-height: 300px;"></canvas>
|
||||
{% else %}
|
||||
<p class="text-muted text-center" style="padding: 20px;">
|
||||
<i class="fa fa-info-circle"></i> {% trans "No volume or snapshot data available." %}
|
||||
</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% if volumes or snapshots %}
|
||||
<script>
|
||||
(function() {
|
||||
var ctx = document.getElementById('storageMetricsChart');
|
||||
if (!ctx) return;
|
||||
|
||||
if (window.storageChart) {
|
||||
window.storageChart.destroy();
|
||||
}
|
||||
|
||||
window.storageChart = new Chart(ctx.getContext('2d'), {
|
||||
type: 'doughnut',
|
||||
data: {
|
||||
labels: ['{% trans "Volumes" %}', '{% trans "Snapshots" %}'],
|
||||
datasets: [{
|
||||
label: '{% trans "Count" %}',
|
||||
data: [{{ volumes|default:0 }}, {{ snapshots|default:0 }}],
|
||||
backgroundColor: [
|
||||
'rgba(91, 192, 222, 0.8)',
|
||||
'rgba(240, 173, 78, 0.8)'
|
||||
],
|
||||
borderColor: [
|
||||
'rgba(91, 192, 222, 1)',
|
||||
'rgba(240, 173, 78, 1)'
|
||||
],
|
||||
borderWidth: 2
|
||||
}]
|
||||
},
|
||||
options: {
|
||||
responsive: true,
|
||||
maintainAspectRatio: true,
|
||||
plugins: {
|
||||
legend: {
|
||||
display: true,
|
||||
position: 'bottom',
|
||||
labels: {
|
||||
padding: 15,
|
||||
font: {
|
||||
size: 13
|
||||
},
|
||||
generateLabels: function(chart) {
|
||||
const data = chart.data;
|
||||
if (data.labels.length && data.datasets.length) {
|
||||
return data.labels.map(function(label, i) {
|
||||
const value = data.datasets[0].data[i];
|
||||
return {
|
||||
text: label + ': ' + value,
|
||||
fillStyle: data.datasets[0].backgroundColor[i],
|
||||
strokeStyle: data.datasets[0].borderColor[i],
|
||||
lineWidth: 2,
|
||||
hidden: false,
|
||||
index: i
|
||||
};
|
||||
});
|
||||
}
|
||||
return [];
|
||||
}
|
||||
}
|
||||
},
|
||||
title: {
|
||||
display: true,
|
||||
text: '{% trans "Volume and Snapshot Distribution" %}',
|
||||
font: {
|
||||
size: 14,
|
||||
weight: 'normal'
|
||||
},
|
||||
padding: {
|
||||
top: 5,
|
||||
bottom: 15
|
||||
}
|
||||
},
|
||||
tooltip: {
|
||||
callbacks: {
|
||||
label: function(context) {
|
||||
let label = context.label || '';
|
||||
if (label) {
|
||||
label += ': ';
|
||||
}
|
||||
label += context.parsed;
|
||||
return label;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
{% endif %}
|
||||
@@ -0,0 +1,173 @@
|
||||
{% extends 'base.html' %}
|
||||
{% load i18n %}
|
||||
{% load static %}
|
||||
{% block title %}{% trans "Volume Metrics" %}{% endblock %}
|
||||
|
||||
{% block page_header %}
|
||||
{% include "horizon/common/_page_header.html" with title=_("Volume Metrics") %}
|
||||
{% endblock page_header %}
|
||||
|
||||
{% block main %}
|
||||
<!-- HTMX -->
|
||||
<script src="{% static 'vendor/htmx.js' %}"></script>
|
||||
<!-- Chart.js -->
|
||||
<script src="{% static 'vendor/chart.umd.js' %}"></script>
|
||||
|
||||
<style>
|
||||
.metrics-section {
|
||||
margin-bottom: 25px;
|
||||
}
|
||||
.pool-capacity-bar {
|
||||
height: 20px;
|
||||
background-color: #f5f5f5;
|
||||
border-radius: 4px;
|
||||
overflow: hidden;
|
||||
position: relative;
|
||||
}
|
||||
.pool-capacity-fill {
|
||||
height: 100%;
|
||||
background: linear-gradient(90deg, #5cb85c 0%, #5bc0de 100%);
|
||||
transition: width 0.3s ease;
|
||||
}
|
||||
.pool-capacity-text {
|
||||
position: absolute;
|
||||
width: 100%;
|
||||
text-align: center;
|
||||
line-height: 20px;
|
||||
font-size: 0.85em;
|
||||
font-weight: 600;
|
||||
color: #333;
|
||||
}
|
||||
</style>
|
||||
|
||||
<!-- Auto-refreshing metrics cards -->
|
||||
<div class="row metrics-section"
|
||||
id="metrics-cards"
|
||||
hx-get="{% url 'horizon:admin:volume_metrics:metrics_update' %}"
|
||||
hx-trigger="load, every 10s"
|
||||
hx-swap="innerHTML">
|
||||
{% include "admin/volume_metrics/_metrics_cards.html" %}
|
||||
</div>
|
||||
|
||||
<!-- Two column layout for chart and pools -->
|
||||
<div class="row metrics-section">
|
||||
<!-- Storage Chart Column -->
|
||||
<div class="col-md-6" id="storage-chart">
|
||||
{% include "admin/volume_metrics/_storage_chart.html" %}
|
||||
</div>
|
||||
|
||||
<!-- Storage Pools Column -->
|
||||
<div class="col-md-6">
|
||||
<div class="panel panel-default">
|
||||
<div class="panel-heading">
|
||||
<h3 class="panel-title">
|
||||
<i class="fa fa-server"></i> {% trans "Storage Pools" %}
|
||||
</h3>
|
||||
</div>
|
||||
<div class="panel-body">
|
||||
{% if storage_pools %}
|
||||
{% for pool in storage_pools %}
|
||||
<div style="margin-bottom: 20px;">
|
||||
<div style="margin-bottom: 5px;">
|
||||
<strong>{{ pool.name }}</strong>
|
||||
<span class="pull-right text-muted">
|
||||
{% trans "Allocated" %}: {{ pool.allocated|floatformat:2 }} / {{ pool.total|floatformat:2 }} GiB {% trans "Total" %}
|
||||
</span>
|
||||
</div>
|
||||
<div class="pool-capacity-bar">
|
||||
{% widthratio pool.allocated pool.total 100 as usage_percent %}
|
||||
<div class="pool-capacity-fill" style="width: {{ usage_percent }}%;"></div>
|
||||
<div class="pool-capacity-text">{{ usage_percent }}% {% trans "allocated" %}</div>
|
||||
</div>
|
||||
<div style="margin-top: 5px; font-size: 0.85em; color: #666;">
|
||||
<span><i class="fa fa-check-circle text-success"></i> {% trans "Free" %}: {{ pool.free|floatformat:2 }} GiB</span>
|
||||
<span style="margin-left: 15px;"><i class="fa fa-hdd-o text-info"></i> {% trans "Provisioned" %}: {{ pool.provisioned|floatformat:2 }} GiB</span>
|
||||
</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
<p class="text-muted text-center" style="padding: 20px;">
|
||||
<i class="fa fa-info-circle"></i> {% trans "No storage pools found." %}
|
||||
</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Capacity Forecast Panel -->
|
||||
<div class="row metrics-section">
|
||||
<div class="col-sm-12"
|
||||
id="capacity-prediction"
|
||||
hx-get="{% url 'horizon:admin:volume_metrics:capacity_prediction' %}"
|
||||
hx-trigger="load, every 60s"
|
||||
hx-swap="innerHTML">
|
||||
{% include "admin/volume_metrics/_capacity_prediction.html" %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Detailed Pool Information Table (expandable) -->
|
||||
<div class="row">
|
||||
<div class="col-sm-12">
|
||||
<div class="panel panel-default">
|
||||
<div class="panel-heading" style="cursor: pointer;" onclick="togglePoolDetails()">
|
||||
<h3 class="panel-title">
|
||||
<i class="fa fa-table"></i> {% trans "Detailed Pool Information" %}
|
||||
<span class="pull-right"><i class="fa fa-chevron-up" id="pool-toggle-icon"></i></span>
|
||||
</h3>
|
||||
</div>
|
||||
<div class="panel-body" id="pool-details-table" style="display: block;">
|
||||
{% if storage_pools %}
|
||||
<div class="table-responsive">
|
||||
<table class="table table-striped table-hover">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>{% trans "Pool Name" %}</th>
|
||||
<th class="text-right">{% trans "Total (GiB)" %}</th>
|
||||
<th class="text-right">{% trans "Allocated (GiB)" %}</th>
|
||||
<th class="text-right">{% trans "Free (GiB)" %}</th>
|
||||
<th class="text-right">{% trans "Provisioned (GiB)" %}</th>
|
||||
<th class="text-right">{% trans "Usage %" %}</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for pool in storage_pools %}
|
||||
<tr>
|
||||
<td><strong>{{ pool.name }}</strong></td>
|
||||
<td class="text-right">{{ pool.total|floatformat:2 }}</td>
|
||||
<td class="text-right">{{ pool.allocated|floatformat:2 }}</td>
|
||||
<td class="text-right">{{ pool.free|floatformat:2 }}</td>
|
||||
<td class="text-right">{{ pool.provisioned|floatformat:2 }}</td>
|
||||
<td class="text-right">
|
||||
{% widthratio pool.allocated pool.total 100 as usage_percent %}
|
||||
<span class="label {% if usage_percent >= 90 %}label-danger{% elif usage_percent >= 75 %}label-warning{% else %}label-success{% endif %}">
|
||||
{{ usage_percent }}%
|
||||
</span>
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{% else %}
|
||||
<p class="text-muted text-center">{% trans "No storage pools found." %}</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
function togglePoolDetails() {
|
||||
var details = document.getElementById('pool-details-table');
|
||||
var icon = document.getElementById('pool-toggle-icon');
|
||||
if (details.style.display === 'none') {
|
||||
details.style.display = 'block';
|
||||
icon.className = 'fa fa-chevron-up';
|
||||
} else {
|
||||
details.style.display = 'none';
|
||||
icon.className = 'fa fa-chevron-down';
|
||||
}
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
@@ -4,3 +4,7 @@
|
||||
# our unit tests should not need to depend on horizon.
|
||||
|
||||
DEBUG = True
|
||||
|
||||
GRIAN_PLUGIN = {
|
||||
"datasource": "fake",
|
||||
}
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
from unittest import mock
|
||||
|
||||
from grian_ui import config
|
||||
from grian_ui_tests.unit import base
|
||||
|
||||
|
||||
class GetGrianPluginConfigTests(base.TestCase):
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_returns_default_when_not_configured(self, mock_settings):
|
||||
del mock_settings.GRIAN_PLUGIN
|
||||
result = config._get_grian_plugin_config()
|
||||
self.assertEqual(result, {"datasource": "fake"})
|
||||
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_returns_configured_dict(self, mock_settings):
|
||||
mock_settings.GRIAN_PLUGIN = {"datasource": "prometheus"}
|
||||
result = config._get_grian_plugin_config()
|
||||
self.assertEqual(result, {"datasource": "prometheus"})
|
||||
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_raises_on_non_dict(self, mock_settings):
|
||||
mock_settings.GRIAN_PLUGIN = "not_a_dict"
|
||||
self.assertRaisesRegex(
|
||||
config.ConfigurationError,
|
||||
"must be a dict",
|
||||
config._get_grian_plugin_config,
|
||||
)
|
||||
|
||||
|
||||
class GetDatasourceTypeTests(base.TestCase):
|
||||
@mock.patch("grian_ui.config._get_grian_plugin_config")
|
||||
def test_returns_fake(self, mock_config):
|
||||
mock_config.return_value = {"datasource": "fake"}
|
||||
self.assertEqual(config.get_datasource_type(), "fake")
|
||||
|
||||
@mock.patch("grian_ui.config._get_grian_plugin_config")
|
||||
def test_returns_prometheus(self, mock_config):
|
||||
mock_config.return_value = {"datasource": "prometheus"}
|
||||
self.assertEqual(config.get_datasource_type(), "prometheus")
|
||||
|
||||
@mock.patch("grian_ui.config._get_grian_plugin_config")
|
||||
def test_defaults_to_fake_when_key_missing(self, mock_config):
|
||||
mock_config.return_value = {}
|
||||
self.assertEqual(config.get_datasource_type(), "fake")
|
||||
|
||||
@mock.patch("grian_ui.config._get_grian_plugin_config")
|
||||
def test_raises_on_invalid_type(self, mock_config):
|
||||
mock_config.return_value = {"datasource": "invalid"}
|
||||
self.assertRaisesRegex(
|
||||
config.ConfigurationError,
|
||||
"Invalid datasource type",
|
||||
config.get_datasource_type,
|
||||
)
|
||||
|
||||
@mock.patch("grian_ui.config._get_grian_plugin_config")
|
||||
def test_raises_on_non_string(self, mock_config):
|
||||
mock_config.return_value = {"datasource": 123}
|
||||
self.assertRaisesRegex(
|
||||
config.ConfigurationError,
|
||||
"must be a string",
|
||||
config.get_datasource_type,
|
||||
)
|
||||
|
||||
|
||||
class GetOpenstackKeystoneUrlTests(base.TestCase):
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_returns_url(self, mock_settings):
|
||||
mock_settings.OPENSTACK_KEYSTONE_URL = "http://keystone:5000/v3"
|
||||
self.assertEqual(
|
||||
config.get_openstack_keystone_url(), "http://keystone:5000/v3"
|
||||
)
|
||||
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_raises_when_not_set(self, mock_settings):
|
||||
del mock_settings.OPENSTACK_KEYSTONE_URL
|
||||
self.assertRaisesRegex(
|
||||
config.ConfigurationError,
|
||||
"OPENSTACK_KEYSTONE_URL is not configured",
|
||||
config.get_openstack_keystone_url,
|
||||
)
|
||||
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_raises_when_empty(self, mock_settings):
|
||||
mock_settings.OPENSTACK_KEYSTONE_URL = ""
|
||||
self.assertRaisesRegex(
|
||||
config.ConfigurationError,
|
||||
"OPENSTACK_KEYSTONE_URL is not configured",
|
||||
config.get_openstack_keystone_url,
|
||||
)
|
||||
|
||||
|
||||
class GetOpenstackEndpointTypeTests(base.TestCase):
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_returns_configured_value(self, mock_settings):
|
||||
mock_settings.OPENSTACK_ENDPOINT_TYPE = "internalURL"
|
||||
self.assertEqual(config.get_openstack_endpoint_type(), "internalURL")
|
||||
|
||||
@mock.patch("grian_ui.config.settings")
|
||||
def test_defaults_to_public(self, mock_settings):
|
||||
del mock_settings.OPENSTACK_ENDPOINT_TYPE
|
||||
self.assertEqual(config.get_openstack_endpoint_type(), "publicURL")
|
||||
@@ -0,0 +1,52 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
import importlib
|
||||
|
||||
from unittest import mock
|
||||
|
||||
from grian_ui.datasource import api
|
||||
from grian_ui_tests.unit import base
|
||||
|
||||
|
||||
class GetDatasourceTests(base.TestCase):
|
||||
@mock.patch("grian_ui.datasource.api.config")
|
||||
def test_loads_fake_datasource(self, mock_config):
|
||||
mock_config.get_datasource_type.return_value = "fake"
|
||||
request = mock.MagicMock()
|
||||
|
||||
result = api.get_datasource(request)
|
||||
|
||||
from grian_ui.datasource.fake import FakeTelemetryClient
|
||||
|
||||
self.assertIsInstance(result, FakeTelemetryClient)
|
||||
|
||||
@mock.patch("grian_ui.datasource.api.config")
|
||||
def test_loads_configured_module_path(self, mock_config):
|
||||
mock_config.get_datasource_type.return_value = "fake"
|
||||
request = mock.MagicMock()
|
||||
|
||||
with mock.patch(
|
||||
"grian_ui.datasource.api.importlib.import_module",
|
||||
wraps=importlib.import_module,
|
||||
) as mock_import:
|
||||
api.get_datasource(request)
|
||||
|
||||
mock_import.assert_any_call("grian_ui.datasource.fake")
|
||||
|
||||
@mock.patch("grian_ui.datasource.api.config")
|
||||
def test_falls_back_to_fake_on_import_error(self, mock_config):
|
||||
mock_config.get_datasource_type.return_value = "nonexistent"
|
||||
request = mock.MagicMock()
|
||||
|
||||
result = api.get_datasource(request)
|
||||
|
||||
from grian_ui.datasource.fake import FakeTelemetryClient
|
||||
|
||||
self.assertIsInstance(result, FakeTelemetryClient)
|
||||
|
||||
|
||||
class DatasourceClientInterfaceTests(base.TestCase):
|
||||
def test_cannot_instantiate_abstract_class(self):
|
||||
with self.assertRaises(TypeError):
|
||||
api.DatasourceClient()
|
||||
@@ -0,0 +1,101 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
from grian_ui.datasource import api
|
||||
from grian_ui.datasource import fake
|
||||
from grian_ui_tests.unit import base
|
||||
|
||||
|
||||
class GetClientTests(base.TestCase):
|
||||
def test_returns_fake_client(self):
|
||||
client = fake.get_client()
|
||||
self.assertIsInstance(client, fake.FakeTelemetryClient)
|
||||
|
||||
def test_accepts_request_argument(self):
|
||||
client = fake.get_client(request=None)
|
||||
self.assertIsInstance(client, fake.FakeTelemetryClient)
|
||||
|
||||
def test_implements_datasource_interface(self):
|
||||
client = fake.get_client()
|
||||
self.assertIsInstance(client, api.DatasourceClient)
|
||||
|
||||
|
||||
class FakeTelemetryClientTests(base.TestCase):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.client = fake.FakeTelemetryClient()
|
||||
|
||||
def test_get_telemetry_overview_returns_dict(self):
|
||||
result = self.client.get_telemetry_overview()
|
||||
self.assertIsInstance(result, dict)
|
||||
|
||||
def test_get_telemetry_overview_keys(self):
|
||||
result = self.client.get_telemetry_overview()
|
||||
expected_keys = {
|
||||
"total_metrics",
|
||||
"active_instances",
|
||||
"alerts_count",
|
||||
"cpu_usage_avg",
|
||||
"memory_usage_avg",
|
||||
"network_throughput",
|
||||
"disk_io_rate",
|
||||
"uptime_percentage",
|
||||
"total_storage",
|
||||
"allocated_storage",
|
||||
"last_updated",
|
||||
}
|
||||
self.assertEqual(set(result.keys()), expected_keys)
|
||||
|
||||
def test_get_telemetry_overview_static_values(self):
|
||||
result = self.client.get_telemetry_overview()
|
||||
self.assertEqual(result["total_metrics"], 1247)
|
||||
self.assertEqual(result["active_instances"], 18)
|
||||
self.assertEqual(result["alerts_count"], 3)
|
||||
self.assertEqual(result["total_storage"], 10240)
|
||||
|
||||
def test_get_volumes_count(self):
|
||||
self.assertEqual(self.client.get_volumes_count(), 120)
|
||||
|
||||
def test_get_snapshots_count(self):
|
||||
self.assertEqual(self.client.get_snapshots_count(), 45)
|
||||
|
||||
def test_get_storage_pools_returns_list(self):
|
||||
result = self.client.get_storage_pools()
|
||||
self.assertIsInstance(result, list)
|
||||
self.assertEqual(len(result), 2)
|
||||
|
||||
def test_get_storage_pools_structure(self):
|
||||
pools = self.client.get_storage_pools()
|
||||
for pool in pools:
|
||||
self.assertIn("name", pool)
|
||||
self.assertIn("total", pool)
|
||||
self.assertIn("allocated", pool)
|
||||
self.assertIn("free", pool)
|
||||
self.assertIn("provisioned", pool)
|
||||
|
||||
def test_get_storage_pools_values(self):
|
||||
pools = self.client.get_storage_pools()
|
||||
self.assertEqual(pools[0]["name"], "storage-pool-01")
|
||||
self.assertEqual(pools[1]["name"], "storage-pool-02")
|
||||
self.assertEqual(pools[0]["total"], 5120.0)
|
||||
|
||||
def test_get_capacity_predictions_returns_list(self):
|
||||
result = self.client.get_capacity_predictions()
|
||||
self.assertIsInstance(result, list)
|
||||
self.assertEqual(len(result), 2)
|
||||
|
||||
def test_get_capacity_predictions_structure(self):
|
||||
predictions = self.client.get_capacity_predictions()
|
||||
for pred in predictions:
|
||||
self.assertIn("name", pred)
|
||||
self.assertIn("current_free", pred)
|
||||
self.assertIn("predicted_value", pred)
|
||||
self.assertIn("days_until_full", pred)
|
||||
self.assertIn("will_run_out", pred)
|
||||
|
||||
def test_get_capacity_predictions_values(self):
|
||||
predictions = self.client.get_capacity_predictions()
|
||||
self.assertFalse(predictions[0]["will_run_out"])
|
||||
self.assertIsNone(predictions[0]["days_until_full"])
|
||||
self.assertTrue(predictions[1]["will_run_out"])
|
||||
self.assertEqual(predictions[1]["days_until_full"], 24.5)
|
||||
@@ -0,0 +1,313 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
from unittest import mock
|
||||
|
||||
from grian_ui.datasource import api
|
||||
from grian_ui_tests.unit import base
|
||||
|
||||
|
||||
def _make_metric(value, labels=None):
|
||||
"""Create a mock Prometheus metric result."""
|
||||
metric = mock.MagicMock()
|
||||
metric.value = value
|
||||
metric.labels = labels or {}
|
||||
return metric
|
||||
|
||||
|
||||
def _make_client(query_map=None):
|
||||
"""Create a PrometheusTelemetryClient with a mocked prom client."""
|
||||
with (
|
||||
mock.patch("grian_ui.datasource.prometheus.config") as mock_config,
|
||||
mock.patch("grian_ui.datasource.prometheus.v3"),
|
||||
mock.patch("grian_ui.datasource.prometheus.session"),
|
||||
mock.patch(
|
||||
"grian_ui.datasource.prometheus.obs_client_utils"
|
||||
) as mock_obs,
|
||||
):
|
||||
mock_config.get_openstack_keystone_url.return_value = (
|
||||
"http://keystone:5000/v3"
|
||||
)
|
||||
mock_config.get_openstack_endpoint_type.return_value = "publicURL"
|
||||
|
||||
request = mock.MagicMock()
|
||||
request.user.token.id = "test-token"
|
||||
request.user.project_id = "test-project"
|
||||
request.user.domain_id = "test-domain"
|
||||
request.user.services_region = "RegionOne"
|
||||
|
||||
from grian_ui.datasource import prometheus
|
||||
|
||||
client = prometheus.get_client(request)
|
||||
|
||||
prom_client = mock_obs.get_prom_client_from_keystone.return_value
|
||||
if query_map:
|
||||
prom_client.query.side_effect = lambda query: query_map.get(
|
||||
query, []
|
||||
)
|
||||
|
||||
return client, prom_client
|
||||
|
||||
|
||||
class PrometheusTelemetryClientInitTests(base.TestCase):
|
||||
@mock.patch("grian_ui.datasource.prometheus.obs_client_utils")
|
||||
@mock.patch("grian_ui.datasource.prometheus.session")
|
||||
@mock.patch("grian_ui.datasource.prometheus.v3")
|
||||
@mock.patch("grian_ui.datasource.prometheus.config")
|
||||
def test_init_creates_client(
|
||||
self, mock_config, mock_v3, mock_session, mock_obs
|
||||
):
|
||||
mock_config.get_openstack_keystone_url.return_value = (
|
||||
"http://keystone:5000/v3"
|
||||
)
|
||||
mock_config.get_openstack_endpoint_type.return_value = "publicURL"
|
||||
|
||||
request = mock.MagicMock()
|
||||
request.user.token.id = "test-token"
|
||||
request.user.project_id = "test-project"
|
||||
request.user.domain_id = "test-domain"
|
||||
request.user.services_region = "RegionOne"
|
||||
|
||||
from grian_ui.datasource import prometheus
|
||||
|
||||
client = prometheus.PrometheusTelemetryClient(request)
|
||||
|
||||
mock_v3.Token.assert_called_once_with( # nosec B106
|
||||
auth_url="http://keystone:5000/v3",
|
||||
token="test-token",
|
||||
project_id="test-project",
|
||||
domain_id="test-domain",
|
||||
)
|
||||
mock_obs.get_prom_client_from_keystone.assert_called_once()
|
||||
self.assertIsInstance(client, api.DatasourceClient)
|
||||
|
||||
|
||||
class GetTelemetryOverviewTests(base.TestCase):
|
||||
def test_returns_metrics(self):
|
||||
query_map = {
|
||||
"sum(ceilometer_vcpus)": [_make_metric(64)],
|
||||
"sum(ceilometer_memory_usage)": [_make_metric(32768)],
|
||||
"sum(ceilometer_volume_provider_pool_capacity_total)": [
|
||||
_make_metric(10240)
|
||||
],
|
||||
"sum(ceilometer_volume_provider_pool_capacity_allocated)": [
|
||||
_make_metric(7000)
|
||||
],
|
||||
}
|
||||
client, _ = _make_client(query_map)
|
||||
|
||||
result = client.get_telemetry_overview()
|
||||
|
||||
self.assertEqual(result["cpu_usage_avg"], 64.0)
|
||||
self.assertEqual(result["memory_usage_avg"], 32.0)
|
||||
self.assertEqual(result["total_storage"], 10240)
|
||||
self.assertEqual(result["allocated_storage"], 7000)
|
||||
|
||||
def test_empty_results_return_zeros(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = []
|
||||
|
||||
result = client.get_telemetry_overview()
|
||||
|
||||
self.assertEqual(result["cpu_usage_avg"], 0.0)
|
||||
self.assertEqual(result["memory_usage_avg"], 0.0)
|
||||
self.assertEqual(result["total_storage"], 0)
|
||||
self.assertEqual(result["allocated_storage"], 0)
|
||||
|
||||
def test_exception_returns_partial_data_with_error(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.side_effect = Exception("connection refused")
|
||||
|
||||
result = client.get_telemetry_overview()
|
||||
|
||||
self.assertIn("error", result)
|
||||
self.assertIn("connection refused", result["error"])
|
||||
|
||||
|
||||
class GetVolumesCountTests(base.TestCase):
|
||||
def test_returns_count(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = [_make_metric(42)]
|
||||
|
||||
result = client.get_volumes_count()
|
||||
|
||||
self.assertEqual(result, 42)
|
||||
prom.query.assert_called_once_with(
|
||||
query="count(last_over_time(ceilometer_volume_size[10m]))"
|
||||
)
|
||||
|
||||
def test_empty_result_returns_zero(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = []
|
||||
|
||||
self.assertEqual(client.get_volumes_count(), 0)
|
||||
|
||||
def test_none_result_returns_zero(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = None
|
||||
|
||||
self.assertEqual(client.get_volumes_count(), 0)
|
||||
|
||||
|
||||
class GetSnapshotsCountTests(base.TestCase):
|
||||
def test_returns_count(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = [_make_metric(15)]
|
||||
|
||||
result = client.get_snapshots_count()
|
||||
|
||||
self.assertEqual(result, 15)
|
||||
prom.query.assert_called_once_with(
|
||||
query="count(last_over_time(ceilometer_volume_snapshot_size[10m]))"
|
||||
)
|
||||
|
||||
def test_empty_result_returns_zero(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = []
|
||||
|
||||
self.assertEqual(client.get_snapshots_count(), 0)
|
||||
|
||||
|
||||
class GetStoragePoolsTests(base.TestCase):
|
||||
def test_returns_pools(self):
|
||||
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
|
||||
query_map = {
|
||||
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
|
||||
'ceilometer_volume_provider_pool_capacity_allocated{resource="pool-01"}': [
|
||||
_make_metric(4096.0)
|
||||
],
|
||||
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
|
||||
_make_metric(1024.0)
|
||||
],
|
||||
'ceilometer_volume_provider_pool_capacity_provisioned{resource="pool-01"}': [
|
||||
_make_metric(6000.0)
|
||||
],
|
||||
}
|
||||
client, _ = _make_client(query_map)
|
||||
|
||||
result = client.get_storage_pools()
|
||||
|
||||
self.assertEqual(len(result), 1)
|
||||
self.assertEqual(result[0]["name"], "pool-01")
|
||||
self.assertEqual(result[0]["total"], 5120.0)
|
||||
self.assertEqual(result[0]["allocated"], 4096.0)
|
||||
self.assertEqual(result[0]["free"], 1024.0)
|
||||
self.assertEqual(result[0]["provisioned"], 6000.0)
|
||||
|
||||
def test_empty_result_returns_empty_list(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = None
|
||||
|
||||
self.assertEqual(client.get_storage_pools(), [])
|
||||
|
||||
def test_exception_returns_empty_list(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.side_effect = Exception("timeout")
|
||||
|
||||
self.assertEqual(client.get_storage_pools(), [])
|
||||
|
||||
def test_missing_sub_metrics_default_to_zero(self):
|
||||
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
|
||||
query_map = {
|
||||
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
|
||||
}
|
||||
client, _ = _make_client(query_map)
|
||||
|
||||
result = client.get_storage_pools()
|
||||
|
||||
self.assertEqual(result[0]["allocated"], 0.0)
|
||||
self.assertEqual(result[0]["free"], 0.0)
|
||||
self.assertEqual(result[0]["provisioned"], 0.0)
|
||||
|
||||
|
||||
class GetCapacityPredictionsTests(base.TestCase):
|
||||
def test_pool_predicted_to_fill(self):
|
||||
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
|
||||
predict_query = (
|
||||
'predict_linear(ceilometer_volume_provider_pool_capacity_free'
|
||||
'{resource="pool-01"}[7d], 30 * 86400)'
|
||||
)
|
||||
query_map = {
|
||||
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
|
||||
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
|
||||
_make_metric(1000.0)
|
||||
],
|
||||
predict_query: [_make_metric(-500.0)],
|
||||
}
|
||||
client, _ = _make_client(query_map)
|
||||
|
||||
result = client.get_capacity_predictions()
|
||||
|
||||
self.assertEqual(len(result), 1)
|
||||
self.assertTrue(result[0]["will_run_out"])
|
||||
self.assertIsNotNone(result[0]["days_until_full"])
|
||||
self.assertEqual(result[0]["predicted_value"], -500.0)
|
||||
self.assertEqual(result[0]["current_free"], 1000.0)
|
||||
|
||||
def test_pool_not_predicted_to_fill(self):
|
||||
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
|
||||
predict_query = (
|
||||
'predict_linear(ceilometer_volume_provider_pool_capacity_free'
|
||||
'{resource="pool-01"}[7d], 30 * 86400)'
|
||||
)
|
||||
query_map = {
|
||||
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
|
||||
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
|
||||
_make_metric(2000.0)
|
||||
],
|
||||
predict_query: [_make_metric(1500.0)],
|
||||
}
|
||||
client, _ = _make_client(query_map)
|
||||
|
||||
result = client.get_capacity_predictions()
|
||||
|
||||
self.assertFalse(result[0]["will_run_out"])
|
||||
self.assertIsNone(result[0]["days_until_full"])
|
||||
|
||||
def test_no_prediction_data(self):
|
||||
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
|
||||
query_map = {
|
||||
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
|
||||
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
|
||||
_make_metric(2000.0)
|
||||
],
|
||||
}
|
||||
client, _ = _make_client(query_map)
|
||||
|
||||
result = client.get_capacity_predictions()
|
||||
|
||||
self.assertFalse(result[0]["will_run_out"])
|
||||
self.assertIsNone(result[0]["predicted_value"])
|
||||
|
||||
def test_empty_total_returns_empty_list(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.return_value = None
|
||||
|
||||
self.assertEqual(client.get_capacity_predictions(), [])
|
||||
|
||||
def test_exception_returns_empty_list(self):
|
||||
client, prom = _make_client()
|
||||
prom.query.side_effect = Exception("timeout")
|
||||
|
||||
self.assertEqual(client.get_capacity_predictions(), [])
|
||||
|
||||
def test_days_until_full_calculation(self):
|
||||
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
|
||||
predict_query = (
|
||||
'predict_linear(ceilometer_volume_provider_pool_capacity_free'
|
||||
'{resource="pool-01"}[7d], 30 * 86400)'
|
||||
)
|
||||
query_map = {
|
||||
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
|
||||
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
|
||||
_make_metric(1500.0)
|
||||
],
|
||||
predict_query: [_make_metric(-1500.0)],
|
||||
}
|
||||
client, _ = _make_client(query_map)
|
||||
|
||||
result = client.get_capacity_predictions()
|
||||
|
||||
# decline_per_day = (1500 - (-1500)) / 30 = 100
|
||||
# days_until_full = 1500 / 100 = 15.0
|
||||
self.assertEqual(result[0]["days_until_full"], 15.0)
|
||||
@@ -0,0 +1,176 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
from unittest import mock
|
||||
|
||||
from grian_ui.content.volume_metrics import views
|
||||
from grian_ui_tests.unit import base
|
||||
|
||||
|
||||
def _make_mock_client():
|
||||
client = mock.MagicMock()
|
||||
client.get_telemetry_overview.return_value = {"cpu_usage_avg": 50.0}
|
||||
client.get_volumes_count.return_value = 10
|
||||
client.get_snapshots_count.return_value = 5
|
||||
client.get_storage_pools.return_value = [{"name": "pool-01"}]
|
||||
client.get_capacity_predictions.return_value = [
|
||||
{"name": "pool-01", "will_run_out": False}
|
||||
]
|
||||
return client
|
||||
|
||||
|
||||
class VolumesMetricsIndexViewTests(base.TestCase):
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data(self, mock_get_ds):
|
||||
client = _make_mock_client()
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.VolumesMetricsIndexView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertEqual(context["overview"], {"cpu_usage_avg": 50.0})
|
||||
self.assertEqual(context["volumes"], 10)
|
||||
self.assertEqual(context["snapshots"], 5)
|
||||
self.assertEqual(context["storage_pools"], [{"name": "pool-01"}])
|
||||
self.assertEqual(len(context["predictions"]), 1)
|
||||
self.assertIn("page_title", context)
|
||||
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data_on_error(self, mock_get_ds):
|
||||
client = mock.MagicMock()
|
||||
client.get_telemetry_overview.side_effect = Exception("fail")
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.VolumesMetricsIndexView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertIn("error_message", context)
|
||||
self.assertFalse(context["data_loaded"])
|
||||
self.assertEqual(context["overview"], {})
|
||||
self.assertEqual(context["volumes"], 0)
|
||||
self.assertEqual(context["snapshots"], 0)
|
||||
self.assertEqual(context["storage_pools"], [])
|
||||
self.assertEqual(context["predictions"], [])
|
||||
|
||||
def test_template_name(self):
|
||||
self.assertEqual(
|
||||
views.VolumesMetricsIndexView.template_name,
|
||||
"admin/volume_metrics/index.html",
|
||||
)
|
||||
|
||||
|
||||
class MetricsUpdateViewTests(base.TestCase):
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data(self, mock_get_ds):
|
||||
client = _make_mock_client()
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.MetricsUpdateView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertEqual(context["overview"], {"cpu_usage_avg": 50.0})
|
||||
self.assertEqual(context["volumes"], 10)
|
||||
self.assertEqual(context["snapshots"], 5)
|
||||
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data_on_error(self, mock_get_ds):
|
||||
client = mock.MagicMock()
|
||||
client.get_telemetry_overview.side_effect = Exception("fail")
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.MetricsUpdateView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertEqual(context["overview"], {})
|
||||
self.assertEqual(context["volumes"], 0)
|
||||
self.assertEqual(context["snapshots"], 0)
|
||||
|
||||
def test_template_name(self):
|
||||
self.assertEqual(
|
||||
views.MetricsUpdateView.template_name,
|
||||
"admin/volume_metrics/_metrics_cards.html",
|
||||
)
|
||||
|
||||
|
||||
class ChartUpdateViewTests(base.TestCase):
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data(self, mock_get_ds):
|
||||
client = _make_mock_client()
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.ChartUpdateView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertEqual(context["volumes"], 10)
|
||||
self.assertEqual(context["snapshots"], 5)
|
||||
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data_on_error(self, mock_get_ds):
|
||||
client = mock.MagicMock()
|
||||
client.get_volumes_count.side_effect = Exception("fail")
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.ChartUpdateView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertEqual(context["volumes"], 0)
|
||||
self.assertEqual(context["snapshots"], 0)
|
||||
|
||||
def test_template_name(self):
|
||||
self.assertEqual(
|
||||
views.ChartUpdateView.template_name,
|
||||
"admin/volume_metrics/_storage_chart.html",
|
||||
)
|
||||
|
||||
|
||||
class CapacityPredictionViewTests(base.TestCase):
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data(self, mock_get_ds):
|
||||
client = _make_mock_client()
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.CapacityPredictionView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertEqual(len(context["predictions"]), 1)
|
||||
|
||||
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
|
||||
def test_get_context_data_on_error(self, mock_get_ds):
|
||||
client = mock.MagicMock()
|
||||
client.get_capacity_predictions.side_effect = Exception("fail")
|
||||
mock_get_ds.return_value = client
|
||||
|
||||
view = views.CapacityPredictionView()
|
||||
view.request = mock.MagicMock()
|
||||
view.kwargs = {}
|
||||
|
||||
context = view.get_context_data()
|
||||
|
||||
self.assertEqual(context["predictions"], [])
|
||||
|
||||
def test_template_name(self):
|
||||
self.assertEqual(
|
||||
views.CapacityPredictionView.template_name,
|
||||
"admin/volume_metrics/_capacity_prediction.html",
|
||||
)
|
||||
Reference in New Issue
Block a user