Add volume metrics dashboard

Implements a comprehensive admin dashboard for monitoring block storage

Features:
- Live metrics dashboard with HTMX auto-refresh
- Prometheus datasource integration via Aetos pattern
- Storage pool monitoring with capacity visualization
- Capacity forecast using predict_linear() to identify pools at risk
- Chart.js doughnut chart for volume/snapshot distribution
- Responsive UI with Bootstrap grid layout
- Configuration management via centralized config module
- Support for both fake (development) and Prometheus datasources

The capacity prediction feature analyzes 7 days of historical data to
forecast when storage pools will run out of space within 30 days,
helping administrators proactively manage storage resources.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Change-Id: I3dc2676d0d247df242a16df5f32c219ee0d29f53
Signed-off-by: Victoria Martinez de la Cruz <victoria@redhat.com>
This commit is contained in:
Victoria Martinez de la Cruz
2026-06-29 17:32:44 +02:00
co-authored by Claude Sonnet 4.5
parent 8be1498ce8
commit 7590259f25
34 changed files with 2128 additions and 7 deletions
+3
View File
@@ -67,3 +67,6 @@ releasenotes/build
# DS_Store on Mac computers
*.DS_Store
CLAUDE.md
# Vendored JS (downloaded via get_vendor.sh)
src/grian_ui/static/vendor/*.js
+11
View File
@@ -0,0 +1,11 @@
include AUTHORS
include ChangeLog
exclude .gitignore
exclude .gitreview
global-exclude *.pyc
recursive-include src/grian_ui *.py
recursive-include src/grian_ui/templates *.html
recursive-include src/grian_ui/static *.css *.js *.png *.jpg *.gif *.ico
recursive-include src/grian_ui/enabled *.py
+185
View File
@@ -0,0 +1,185 @@
========================
Volume Metrics Dashboard
========================
The Volume Metrics dashboard provides cloud administrators with a real-time
overview of Cinder block storage resources across the deployment. It is
accessible under **Admin > Telemetry > Volume Metrics** and requires the
``openstack.roles.admin`` permission.
All metrics originate from `Ceilometer <https://docs.openstack.org/ceilometer/latest/>`_
volume pollster data, stored in Prometheus via the
`sg-core <https://github.com/infrawatch/sg-core>`_ telemetry pipeline, and
queried through the `Aetos <https://opendev.org/openstack/aetos>`_
observability proxy (``service_type: metric-storage``).
Ceilometer meters follow a naming convention when exposed as Prometheus
metrics: dots are replaced with underscores and a ``ceilometer_`` prefix is
added (e.g. ``volume.size`` becomes ``ceilometer_volume_size``).
Panels
======
Summary Cards
-------------
Four metric cards displayed at the top of the dashboard, auto-refreshing
every **10 seconds**.
.. list-table::
:header-rows: 1
:widths: 25 30 45
* - Card
- Ceilometer Meter
- PromQL
* - Total Storage (GiB)
- ``volume.provider.pool.capacity.total``
- ``sum(ceilometer_volume_provider_pool_capacity_total)``
* - Allocated Storage (GiB)
- ``volume.provider.pool.capacity.allocated``
- ``sum(ceilometer_volume_provider_pool_capacity_allocated)``
* - Total Volumes
- ``volume.size``
- ``count(ceilometer_volume_size)``
* - Total Snapshots
- ``volume.snapshot.size``
- ``count(ceilometer_volume_snapshot_size)``
**Total Storage** and **Allocated Storage** aggregate the capacity reported
by the Cinder volume backend(s) (e.g. LVM, Ceph) across all storage pools.
**Total Volumes** and **Total Snapshots** count the number of distinct
Ceilometer samples, which corresponds to the number of active volumes and
snapshots in the deployment.
Volume and Snapshot Distribution (Doughnut Chart)
-------------------------------------------------
A Chart.js doughnut chart showing the proportion of volumes to snapshots.
Rendered on initial page load from the same ``count()`` queries used by the
summary cards.
.. list-table::
:header-rows: 1
:widths: 25 30 45
* - Segment
- Ceilometer Meter
- PromQL
* - Volumes
- ``volume.size``
- ``count(ceilometer_volume_size)``
* - Snapshots
- ``volume.snapshot.size``
- ``count(ceilometer_volume_snapshot_size)``
This chart gives a quick visual ratio of how storage objects are distributed
between volumes and snapshots.
Storage Pools
-------------
A per-pool breakdown with capacity bars and a detailed table. Rendered on
initial page load.
The dashboard first discovers all pools by querying the raw (un-aggregated)
capacity total metric, then fetches per-pool details by filtering on the
``resource`` label, which contains the Cinder pool name.
.. list-table::
:header-rows: 1
:widths: 25 30 45
* - Metric
- Ceilometer Meter
- PromQL
* - Total capacity
- ``volume.provider.pool.capacity.total``
- ``ceilometer_volume_provider_pool_capacity_total``
* - Allocated capacity
- ``volume.provider.pool.capacity.allocated``
- ``ceilometer_volume_provider_pool_capacity_allocated{resource="<pool>"}``
* - Free capacity
- ``volume.provider.pool.capacity.free``
- ``ceilometer_volume_provider_pool_capacity_free{resource="<pool>"}``
* - Provisioned capacity
- ``volume.provider.pool.capacity.provisioned``
- ``ceilometer_volume_provider_pool_capacity_provisioned{resource="<pool>"}``
Each pool is displayed with:
- A **capacity bar** showing the percentage of allocated vs. total capacity.
- A **detail row** with free and provisioned values.
- A **collapsible table** with all numeric values and a color-coded usage
percentage label (green < 75%, yellow 75-89%, red >= 90%).
All values are in GiB, matching the units reported by Cinder's volume
backend statistics.
Capacity Forecast
-----------------
A prediction table that forecasts when each storage pool will run out of
free space, auto-refreshing every **60 seconds**.
.. list-table::
:header-rows: 1
:widths: 25 30 45
* - Metric
- Ceilometer Meter
- PromQL
* - Current free capacity
- ``volume.provider.pool.capacity.free``
- ``ceilometer_volume_provider_pool_capacity_free{resource="<pool>"}``
* - Predicted free in 30 days
- ``volume.provider.pool.capacity.free``
- ``predict_linear(ceilometer_volume_provider_pool_capacity_free{resource="<pool>"}[7d], 30 * 86400)``
The ``predict_linear()`` function uses linear regression over the last
**7 days** of free capacity samples to project the value **30 days** into
the future. If the predicted value is negative, the dashboard computes the
estimated number of days until the pool reaches zero free capacity using
linear interpolation::
decline_per_day = (current_free - predicted_value) / 30
days_until_full = current_free / decline_per_day
Status labels:
- **Full in N days** (red) — predicted free capacity drops below zero within
30 days.
- **Declining** (yellow) — free capacity is decreasing but will not reach
zero within 30 days.
- **Healthy** (green) — free capacity is stable or increasing.
- **Insufficient data** (grey) — not enough historical data for
``predict_linear()`` to produce a result (requires at least 2 data points
over the 7-day window).
.. note::
Predictions require at least 7 days of continuous Ceilometer data
collection. On a fresh deployment, this panel will show "Insufficient
data" until enough history accumulates.
Data Pipeline
=============
The metrics flow through the following pipeline::
Cinder → Ceilometer pollster → sg-core (STF) → Prometheus → Aetos → Horizon
1. **Ceilometer** polls Cinder every 600 seconds (default) for volume and
pool statistics.
2. **sg-core** receives Ceilometer samples over AMQP/TCP and writes them as
Prometheus metrics.
3. **Prometheus** stores the time-series data and serves PromQL queries.
4. **Aetos** acts as a Keystone-authenticated proxy to Prometheus, registered
in the service catalog as ``metric-storage``.
5. **Horizon (grian-ui)** queries Aetos using ``python-observabilityclient``
and renders the results.
+2
View File
@@ -14,3 +14,5 @@ documentation for details.
.. toctree::
:maxdepth: 2
:caption: Contents:
admin-volume-metrics
+4 -2
View File
@@ -9,21 +9,22 @@ readme = "README.rst"
authors = [
{name = "OpenStack", email = "openstack-discuss@lists.openstack.org"},
]
requires-python = ">=3.10"
requires-python = ">=3.11"
license = "Apache-2.0"
classifiers = [
"Environment :: OpenStack",
"Intended Audience :: Information Technology",
"Intended Audience :: System Administrators",
"Operating System :: POSIX :: Linux",
"Development Status :: 2 - Pre-Alpha",
"Programming Language :: Python",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]
dynamic = ["version"]
keywords = ["openstack", "horizon", "telemetry"]
@@ -64,6 +65,7 @@ ignore = [
[tool.ruff.lint.per-file-ignores]
"tests/grian_ui_tests/*" = ["S"]
"src/grian_ui/datasource/fake.py" = ["S311"]
[tool.ruff.lint.isort]
known-first-party = ["grian_ui"]
+1
View File
@@ -7,3 +7,4 @@ oslo.utils>=4.7.0 # Apache-2.0
# date but we do not test them so no guarantee of having them all correct. If
# you find any incorrect lower bounds, let us know or propose a fix.
pbr>=6.0.0 # Apache-2.0
python-observabilityclient>=1.0.0 # Apache-2.0
+79
View File
@@ -0,0 +1,79 @@
#
# SPDX-License-Identifier: Apache-2.0
"""
Centralized configuration management for Grian UI.
Validates and provides typed access to Django settings.
"""
from django.conf import settings
# Valid datasource types
DATASOURCE_FAKE = "fake"
DATASOURCE_PROMETHEUS = "prometheus"
VALID_DATASOURCE_TYPES = (DATASOURCE_FAKE, DATASOURCE_PROMETHEUS)
class ConfigurationError(Exception):
"""Raised when configuration validation fails."""
pass
def _get_grian_plugin_config():
"""Get and validate GRIAN_PLUGIN configuration."""
config = getattr(settings, "GRIAN_PLUGIN", None)
if config is None:
# Return default configuration
return {"datasource": DATASOURCE_FAKE}
if not isinstance(config, dict):
raise ConfigurationError(
f"GRIAN_PLUGIN must be a dict, got {type(config).__name__}"
)
return config
def get_datasource_type():
"""
Get the configured datasource type.
Returns:
str: The datasource type (e.g., 'fake', 'prometheus')
Raises:
ConfigurationError: If datasource type is invalid
"""
config = _get_grian_plugin_config()
datasource = config.get("datasource", DATASOURCE_FAKE)
if not isinstance(datasource, str):
raise ConfigurationError(
f"GRIAN_PLUGIN['datasource'] must be a string, got {type(datasource).__name__}"
)
if datasource not in VALID_DATASOURCE_TYPES:
raise ConfigurationError(
f"Invalid datasource type '{datasource}'. "
f"Valid types: {', '.join(VALID_DATASOURCE_TYPES)}"
)
return datasource
def get_openstack_keystone_url():
"""Get OpenStack Keystone URL from settings."""
url = getattr(settings, "OPENSTACK_KEYSTONE_URL", None)
if not url:
raise ConfigurationError(
"OPENSTACK_KEYSTONE_URL is not configured in settings"
)
return url
def get_openstack_endpoint_type():
"""Get OpenStack endpoint type (interface) from settings."""
return getattr(settings, "OPENSTACK_ENDPOINT_TYPE", "publicURL")
View File
@@ -0,0 +1,31 @@
================================
Testing the Volume Metrics panel
================================
.. note::
These are development instructions. This content will be moved to the
official documentation.
This document provides instructions on how to test the Volume Metrics panel.
Prerequisites
=============
1. **Enable the panels**
Copy the files under ``src/grian_ui/local/enabled`` to the
``local/enabled`` directory of your Horizon installation.
2. **Configure settings**
Copy the settings file from ``src/grian_ui/local/local_settings.d``
to the ``local/local_settings.d`` directory of your Horizon
installation.
3. **Install python-observabilityclient**
Install the latest ``python-observabilityclient`` in your environment.
Follow the instructions at
https://github.com/openstack/python-observabilityclient.
After completing these steps, restart your web server.
@@ -0,0 +1,3 @@
"""
Volume metrics panel for Grian UI.
"""
@@ -0,0 +1,10 @@
import horizon
from django.utils.translation import gettext_lazy as _
class VolumeMetrics(horizon.Panel):
name = _("Volume Metrics")
slug = "volume_metrics"
permissions = ("openstack.roles.admin",)
urls = "grian_ui.content.volume_metrics.urls"
@@ -0,0 +1,23 @@
from django.urls import re_path
from grian_ui.content.volume_metrics import views
urlpatterns = [
re_path(r"^$", views.VolumesMetricsIndexView.as_view(), name="index"),
re_path(
r"^metrics-update/$",
views.MetricsUpdateView.as_view(),
name="metrics_update",
),
re_path(
r"^chart-update/$",
views.ChartUpdateView.as_view(),
name="chart_update",
),
re_path(
r"^capacity-prediction/$",
views.CapacityPredictionView.as_view(),
name="capacity_prediction",
),
]
@@ -0,0 +1,93 @@
from django.utils.translation import gettext_lazy as _
from django.views import generic
from grian_ui.datasource.api import get_datasource
class VolumesMetricsIndexView(generic.TemplateView):
template_name = "admin/volume_metrics/index.html"
page_title = _("Volume Metrics")
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
context["page_title"] = self.page_title
client = get_datasource(self.request)
# Get telemetry data from datasource
try:
context["overview"] = client.get_telemetry_overview()
context["volumes"] = client.get_volumes_count()
context["snapshots"] = client.get_snapshots_count()
context["storage_pools"] = client.get_storage_pools()
context["predictions"] = client.get_capacity_predictions()
except Exception as e:
context["error_message"] = f"Error loading telemetry data: {e}"
context["data_loaded"] = False
# Provide defaults on error
context["overview"] = {}
context["volumes"] = 0
context["snapshots"] = 0
context["storage_pools"] = []
context["predictions"] = []
return context
class MetricsUpdateView(generic.TemplateView):
"""HTMX endpoint for updating metrics cards."""
template_name = "admin/volume_metrics/_metrics_cards.html"
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
client = get_datasource(self.request)
try:
context["overview"] = client.get_telemetry_overview()
context["volumes"] = client.get_volumes_count()
context["snapshots"] = client.get_snapshots_count()
except Exception:
# Provide defaults on error
context["overview"] = {}
context["volumes"] = 0
context["snapshots"] = 0
return context
class ChartUpdateView(generic.TemplateView):
"""HTMX endpoint for updating the storage chart."""
template_name = "admin/volume_metrics/_storage_chart.html"
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
client = get_datasource(self.request)
try:
context["volumes"] = client.get_volumes_count()
context["snapshots"] = client.get_snapshots_count()
except Exception:
# Provide defaults on error
context["volumes"] = 0
context["snapshots"] = 0
return context
class CapacityPredictionView(generic.TemplateView):
"""HTMX endpoint for capacity prediction visualization."""
template_name = "admin/volume_metrics/_capacity_prediction.html"
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
client = get_datasource(self.request)
try:
context["predictions"] = client.get_capacity_predictions()
except Exception:
# Provide defaults on error
context["predictions"] = []
return context
+61
View File
@@ -1,2 +1,63 @@
#
# SPDX-License-Identifier: Apache-2.0
"""
Datasource API for telemetry data.
Provides an abstraction layer for different telemetry backends.
"""
import importlib
from abc import ABC
from abc import abstractmethod
from grian_ui import config
def get_datasource(request):
"""
Get the configured datasource backend.
Returns the datasource implementation based on GRIAN_PLUGIN['datasource']
setting. Defaults to 'fake' if not configured.
"""
datasource_type = config.get_datasource_type()
module_path = f"grian_ui.datasource.{datasource_type}"
try:
datasource_module = importlib.import_module(module_path)
return datasource_module.get_client(request)
except ImportError:
# Fallback to fake datasource
import grian_ui.datasource.fake as fake_datasource
return fake_datasource.get_client(request)
class DatasourceClient(ABC):
"""Abstract base class for datasource clients."""
@abstractmethod
def get_telemetry_overview(self) -> dict:
"""Get overview telemetry data."""
pass
@abstractmethod
def get_volumes_count(self) -> int:
"""Get volumes count telemetry data."""
pass
@abstractmethod
def get_snapshots_count(self) -> int:
"""Get snapshots count telemetry data."""
pass
@abstractmethod
def get_storage_pools(self) -> list:
"""Get storage pools information."""
pass
@abstractmethod
def get_capacity_predictions(self) -> list:
"""Get capacity predictions for storage pools."""
pass
+113
View File
@@ -0,0 +1,113 @@
#
# SPDX-License-Identifier: Apache-2.0
"""
Fake datasource implementation for development and testing.
Provides realistic sample telemetry data.
"""
import random # nosec
from datetime import datetime
from grian_ui.datasource.api import DatasourceClient
class FakeTelemetryClient(DatasourceClient):
"""Fake telemetry client that generates sample data."""
def get_telemetry_overview(self) -> dict:
"""
Get telemetry overview data.
Returns sample data for development/testing purposes.
In a real implementation, this would aggregate multiple queries.
"""
return {
"total_metrics": 1247,
"active_instances": 18,
"alerts_count": 3,
"cpu_usage_avg": round(random.uniform(15.0, 85.0), 1), # nosec
"memory_usage_avg": round(random.uniform(25.0, 75.0), 1), # nosec
"network_throughput": round(random.uniform(50.0, 200.0), 1), # nosec
"disk_io_rate": round(random.uniform(10.0, 100.0), 1), # nosec
"uptime_percentage": round(random.uniform(98.5, 99.9), 2), # nosec
"total_storage": 10240, # GB
"allocated_storage": round(random.uniform(6000, 9000)), # nosec GB
"last_updated": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
}
def get_volumes_count(self) -> int:
"""
Get volumes count telemetry data.
Returns hardcoded placeholder value for development/testing.
"""
return 120
def get_snapshots_count(self) -> int:
"""
Get snapshots count telemetry data.
Returns hardcoded placeholder value for development/testing.
"""
return 45
def get_storage_pools(self) -> list:
"""
Get storage pools information.
Returns sample pool data for development/testing.
"""
return [
{
"name": "storage-pool-01",
"total": 5120.0,
"allocated": 4096.0,
"free": 1024.0,
"provisioned": 5120.0,
},
{
"name": "storage-pool-02",
"total": 5120.0,
"allocated": 4096.0,
"free": 1024.0,
"provisioned": 8192.0,
},
]
def get_capacity_predictions(self) -> list:
"""
Get capacity predictions for storage pools.
Returns sample prediction data for development/testing.
"""
return [
{
"name": "storage-pool-01",
"current_free": 1024.0,
"predicted_value": 512.0,
"days_until_full": None,
"will_run_out": False,
},
{
"name": "storage-pool-02",
"current_free": 1024.0,
"predicted_value": -200.0,
"days_until_full": 24.5,
"will_run_out": True,
},
]
def get_client(request=None) -> DatasourceClient:
"""
Get a fake telemetry client instance.
Args:
request: HTTP request (unused for fake client, accepted for interface compatibility)
Returns:
FakeTelemetryClient instance
"""
return FakeTelemetryClient()
+296
View File
@@ -0,0 +1,296 @@
#
# SPDX-License-Identifier: Apache-2.0
"""
Prometheus datasource implementation via Aetos.
Provides access to Prometheus metrics through OpenStack observability service.
"""
from keystoneauth1.identity import v3 # noqa: I001
from keystoneauth1 import session # noqa: I001
from observabilityclient.utils import metric_utils as obs_client_utils
from grian_ui import config
from grian_ui.datasource.api import DatasourceClient
STALENESS_WINDOW = "10m"
class PrometheusTelemetryClient(DatasourceClient):
"""
Prometheus telemetry client to retrieves data from the OSP deployment
"""
def __init__(self, request):
"""
Initialize Prometheus Telemetry client via Aetos.
Args:
request: Django HTTP request containing user authentication
Note:
Uses get_prom_client_from_keystone for proper Aetos/Prometheus
client initialization with Keystone session.
A new client instance should be created per page request,
but can be reused for multiple metric queries within that request.
"""
auth_url = config.get_openstack_keystone_url()
token = request.user.token.id
project_id = request.user.project_id
domain_id = request.user.domain_id
interface = config.get_openstack_endpoint_type()
auth = v3.Token(
auth_url=auth_url,
token=token,
project_id=project_id,
domain_id=domain_id,
)
sess = session.Session(auth=auth)
adapter_options = {
"service_type": "metric-storage",
"region_name": request.user.services_region,
"interface": interface,
}
# discovers the Aetos endpoint via service catalog
self.client = obs_client_utils.get_prom_client_from_keystone(
sess, adapter_options=adapter_options
)
def get_telemetry_overview(self) -> dict:
"""
Get telemetry overview data.
Aggregates multiple Prometheus queries to build a dashboard overview.
Returns computed metrics rather than raw metric list.
"""
overview = {}
try:
# Query for vCPUs count
cpu_result = self.client.query(query="sum(ceilometer_vcpus)")
if cpu_result and len(cpu_result) > 0:
overview["cpu_usage_avg"] = round(
float(cpu_result[0].value), 1
)
else:
overview["cpu_usage_avg"] = 0.0
# Query for memory usage in MB
memory_result = self.client.query(
query="sum(ceilometer_memory_usage)"
)
if memory_result and len(memory_result) > 0:
# Convert MB to GB for display
overview["memory_usage_avg"] = round(
float(memory_result[0].value) / 1024, 1
)
else:
overview["memory_usage_avg"] = 0.0
# Query for total storage capacity
total_storage_result = self.client.query(
query="sum(ceilometer_volume_provider_pool_capacity_total)"
)
if total_storage_result and len(total_storage_result) > 0:
overview["total_storage"] = int(
float(total_storage_result[0].value)
)
else:
overview["total_storage"] = 0
# Query for allocated storage
allocated_storage_result = self.client.query(
query="sum(ceilometer_volume_provider_pool_capacity_allocated)"
)
if allocated_storage_result and len(allocated_storage_result) > 0:
overview["allocated_storage"] = int(
float(allocated_storage_result[0].value)
)
else:
overview["allocated_storage"] = 0
except Exception as e:
# Return partial data if some queries fail
overview["error"] = str(e)
return overview
def get_volumes_count(self) -> int:
"""
Get telemetry volumes count.
Returns:
int: Current count of volumes from Ceilometer metrics
"""
result = self.client.query(
query=f"count(last_over_time(ceilometer_volume_size[{STALENESS_WINDOW}]))"
)
if result and len(result) > 0:
return int(float(result[0].value))
return 0
def get_snapshots_count(self) -> int:
"""
Get telemetry snapshots count.
Returns:
int: Current count of snapshots from Ceilometer metrics
"""
result = self.client.query(
query=f"count(last_over_time(ceilometer_volume_snapshot_size[{STALENESS_WINDOW}]))"
)
if result and len(result) > 0:
return int(float(result[0].value))
return 0
def get_storage_pools(self) -> list:
"""
Get storage pools information from Ceilometer metrics.
Returns:
list: List of storage pool dictionaries with capacity metrics
"""
pools = []
try:
# Get all unique pool resources
# We'll use the total capacity metric to identify pools
total_result = self.client.query(
query="ceilometer_volume_provider_pool_capacity_total"
)
if not total_result:
return pools
# For each pool, gather all metrics
for pool_metric in total_result:
pool_name = pool_metric.labels.get("resource", "unknown")
pool_data = {
"name": pool_name,
"total": float(pool_metric.value),
"allocated": 0.0,
"free": 0.0,
"provisioned": 0.0,
}
# Get allocated capacity for this pool
allocated_query = f'ceilometer_volume_provider_pool_capacity_allocated{{resource="{pool_name}"}}'
allocated_result = self.client.query(query=allocated_query)
if allocated_result and len(allocated_result) > 0:
pool_data["allocated"] = float(allocated_result[0].value)
# Get free capacity for this pool
free_query = f'ceilometer_volume_provider_pool_capacity_free{{resource="{pool_name}"}}'
free_result = self.client.query(query=free_query)
if free_result and len(free_result) > 0:
pool_data["free"] = float(free_result[0].value)
# Get provisioned capacity for this pool
provisioned_query = f'ceilometer_volume_provider_pool_capacity_provisioned{{resource="{pool_name}"}}'
provisioned_result = self.client.query(query=provisioned_query)
if provisioned_result and len(provisioned_result) > 0:
pool_data["provisioned"] = float(
provisioned_result[0].value
)
pools.append(pool_data)
except Exception:
# Return empty list on error
return []
return pools
def get_capacity_predictions(self) -> list:
"""
Predict when storage pools will run out of space.
Uses predict_linear to forecast when free capacity will hit zero
based on the last 7 days of data, looking 30 days into the future.
Returns:
list: List of predictions per pool with:
- name: pool name
- days_until_full: predicted days until capacity hits zero (None if not predicted to fill)
- predicted_value: the predicted free capacity in 30 days
- current_free: current free capacity
"""
predictions = []
try:
# Get all unique pool resources
total_result = self.client.query(
query="ceilometer_volume_provider_pool_capacity_total"
)
if not total_result:
return predictions
# For each pool, calculate prediction
for pool_metric in total_result:
pool_name = pool_metric.labels.get("resource", "unknown")
# Get current free capacity
free_query = f'ceilometer_volume_provider_pool_capacity_free{{resource="{pool_name}"}}'
free_result = self.client.query(query=free_query)
current_free = 0.0
if free_result and len(free_result) > 0:
current_free = float(free_result[0].value)
# Predict free capacity in 30 days based on 7 days of history
# predict_linear(metric[7d], 30 * 86400) predicts value 30 days from now
predict_query = f'predict_linear(ceilometer_volume_provider_pool_capacity_free{{resource="{pool_name}"}}[7d], 30 * 86400)'
predict_result = self.client.query(query=predict_query)
prediction_data = {
"name": pool_name,
"current_free": current_free,
"predicted_value": None,
"days_until_full": None,
"will_run_out": False,
}
if predict_result and len(predict_result) > 0:
predicted_value = float(predict_result[0].value)
prediction_data["predicted_value"] = predicted_value
# If predicted to go below zero, calculate when
if predicted_value < 0 and current_free > 0:
# Linear interpolation to find when it crosses zero
# current_free - (days_until_full / 30) * (current_free - predicted_value) = 0
# Solve for days_until_full
decline_per_day = (current_free - predicted_value) / 30
if decline_per_day > 0:
days_until_full = current_free / decline_per_day
prediction_data["days_until_full"] = round(
days_until_full, 1
)
prediction_data["will_run_out"] = True
predictions.append(prediction_data)
except Exception:
# Return empty list on error
return []
return predictions
def get_client(request) -> DatasourceClient:
"""
Get a Prometheus telemetry client instance.
Args:
request: Django HTTP request with user authentication
Returns:
PrometheusTelemetryClient: New client instance for this request
"""
return PrometheusTelemetryClient(request)
@@ -1,2 +0,0 @@
#
# SPDX-License-Identifier: Apache-2.0
+5
View File
@@ -97,6 +97,11 @@ STATIC_URL = "static/"
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
# Add grian_ui to INSTALLED_APPS if not already present
INSTALLED_APPS = list(INSTALLED_APPS) # noqa
if "grian_ui" not in INSTALLED_APPS:
INSTALLED_APPS.append("grian_ui")
GRIAN_PLUGIN = {"datasource": os.getenv("GRIAN_DATA_SOURCE", "fake")}
# COMPRESS_OFFLINE = True
-2
View File
@@ -1,2 +0,0 @@
#
# SPDX-License-Identifier: Apache-2.0
@@ -0,0 +1,4 @@
PANEL_GROUP = "telemetry"
PANEL_GROUP_NAME = "Telemetry"
PANEL_GROUP_DASHBOARD = "admin"
AUTO_DISCOVER_STATIC_FILES = True
@@ -0,0 +1,14 @@
# The name of the panel to be added to HORIZON_CONFIG. Required.
PANEL = "volume_metrics"
# The name of the dashboard the PANEL associated with. Required.
PANEL_DASHBOARD = "admin"
# The name of the panel group the PANEL is associated with.
PANEL_GROUP = "telemetry"
# Python panel class of the PANEL to be added.
ADD_PANEL = "grian_ui.content.volume_metrics.panel.VolumeMetrics"
# A list of applications to be prepended to INSTALLED_APPS
ADD_INSTALLED_APPS = ["grian_ui"]
@@ -1,4 +1,9 @@
#
# SPDX-License-Identifier: Apache-2.0
GRIAN_PLUGIN = {"datasource": "fake"}
# Grian UI configuration
# datasource options: 'fake', 'prometheus'
# Note: 'prometheus' datasource uses the Aetos pattern with authenticated
# Keystone session (following Watcher and CloudKitty). All Prometheus access
# goes through Aetos - no direct host:port specification.
GRIAN_PLUGIN = {"datasource": "prometheus"}
+6
View File
@@ -0,0 +1,6 @@
#!/bin/bash
CHART_JS_VERSION=4.4.1
HTMX_VERSION=2.0.4
rm -f *.js
wget "https://unpkg.com/chart.js@${CHART_JS_VERSION}/dist/chart.umd.js" -O chart.umd.js
wget "https://unpkg.com/htmx.org@${HTMX_VERSION}/dist/htmx.js" -O htmx.js
@@ -0,0 +1,70 @@
{% load i18n %}
<div class="panel panel-default">
<div class="panel-heading">
<h3 class="panel-title">
<i class="fa fa-line-chart"></i> {% trans "Capacity Forecast" %}
<small class="text-muted">{% trans "(30-day prediction based on 7-day trend)" %}</small>
</h3>
</div>
<div class="panel-body">
{% if predictions %}
<div class="table-responsive">
<table class="table table-hover">
<thead>
<tr>
<th>{% trans "Pool Name" %}</th>
<th class="text-right">{% trans "Current Free (GiB)" %}</th>
<th class="text-right">{% trans "Predicted in 30 Days (GiB)" %}</th>
<th class="text-center">{% trans "Status" %}</th>
</tr>
</thead>
<tbody>
{% for pred in predictions %}
<tr class="{% if pred.will_run_out %}danger{% endif %}">
<td><strong>{{ pred.name }}</strong></td>
<td class="text-right">{{ pred.current_free|floatformat:2 }}</td>
<td class="text-right">
{% if pred.predicted_value is not None %}
{{ pred.predicted_value|floatformat:2 }}
{% else %}
<span class="text-muted">{% trans "N/A" %}</span>
{% endif %}
</td>
<td class="text-center">
{% if pred.will_run_out %}
<span class="label label-danger">
<i class="fa fa-exclamation-triangle"></i>
{% trans "Full in" %} {{ pred.days_until_full }} {% trans "days" %}
</span>
{% elif pred.predicted_value is not None and pred.predicted_value < pred.current_free %}
<span class="label label-warning">
<i class="fa fa-arrow-down"></i> {% trans "Declining" %}
</span>
{% elif pred.predicted_value is not None %}
<span class="label label-success">
<i class="fa fa-check"></i> {% trans "Healthy" %}
</span>
{% else %}
<span class="label label-default">
<i class="fa fa-question"></i> {% trans "Insufficient data" %}
</span>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<div class="alert alert-info" style="margin-top: 15px; margin-bottom: 0;">
<i class="fa fa-info-circle"></i>
<strong>{% trans "Note:" %}</strong>
{% trans "Predictions are based on linear regression of the last 7 days of capacity data. Actual usage may vary." %}
</div>
{% else %}
<p class="text-muted text-center" style="padding: 20px;">
<i class="fa fa-info-circle"></i> {% trans "No prediction data available." %}
</p>
{% endif %}
</div>
</div>
@@ -0,0 +1,78 @@
{% load i18n %}
<style>
.metric-card {
min-height: 140px;
display: flex;
flex-direction: column;
}
.metric-card .panel-heading {
flex: 1;
display: flex;
align-items: center;
padding: 20px 15px;
}
.metric-icon {
font-size: 3.5em;
opacity: 0.3;
margin-right: 15px;
}
.metric-value {
font-size: 2.5em;
font-weight: 300;
line-height: 1;
margin-bottom: 5px;
}
.metric-label {
font-size: 0.9em;
opacity: 0.9;
font-weight: 400;
}
</style>
<div class="col-lg-3 col-md-6 col-sm-6">
<div class="panel panel-primary metric-card">
<div class="panel-heading">
<i class="fa fa-hdd-o metric-icon"></i>
<div class="text-right" style="flex: 1;">
<div class="metric-value">{{ overview.total_storage|default:"N/A" }}</div>
<div class="metric-label">{% trans "Total Storage (GiB)" %}</div>
</div>
</div>
</div>
</div>
<div class="col-lg-3 col-md-6 col-sm-6">
<div class="panel panel-green metric-card">
<div class="panel-heading">
<i class="fa fa-check-circle metric-icon"></i>
<div class="text-right" style="flex: 1;">
<div class="metric-value">{{ overview.allocated_storage|default:"N/A" }}</div>
<div class="metric-label">{% trans "Allocated Storage (GiB)" %}</div>
</div>
</div>
</div>
</div>
<div class="col-lg-3 col-md-6 col-sm-6">
<div class="panel panel-yellow metric-card">
<div class="panel-heading">
<i class="fa fa-database metric-icon"></i>
<div class="text-right" style="flex: 1;">
<div class="metric-value">{{ volumes }}</div>
<div class="metric-label">{% trans "Total Volumes" %}</div>
</div>
</div>
</div>
</div>
<div class="col-lg-3 col-md-6 col-sm-6">
<div class="panel panel-red metric-card">
<div class="panel-heading">
<i class="fa fa-camera metric-icon"></i>
<div class="text-right" style="flex: 1;">
<div class="metric-value">{{ snapshots }}</div>
<div class="metric-label">{% trans "Total Snapshots" %}</div>
</div>
</div>
</div>
</div>
@@ -0,0 +1,107 @@
{% load i18n %}
<div class="panel panel-default">
<div class="panel-heading">
<h3 class="panel-title">
<i class="fa fa-bar-chart"></i> {% trans "Volume and Snapshot Distribution" %}
</h3>
</div>
<div class="panel-body" style="padding: 15px;">
{% if volumes or snapshots %}
<canvas id="storageMetricsChart" style="max-height: 300px;"></canvas>
{% else %}
<p class="text-muted text-center" style="padding: 20px;">
<i class="fa fa-info-circle"></i> {% trans "No volume or snapshot data available." %}
</p>
{% endif %}
</div>
</div>
{% if volumes or snapshots %}
<script>
(function() {
var ctx = document.getElementById('storageMetricsChart');
if (!ctx) return;
if (window.storageChart) {
window.storageChart.destroy();
}
window.storageChart = new Chart(ctx.getContext('2d'), {
type: 'doughnut',
data: {
labels: ['{% trans "Volumes" %}', '{% trans "Snapshots" %}'],
datasets: [{
label: '{% trans "Count" %}',
data: [{{ volumes|default:0 }}, {{ snapshots|default:0 }}],
backgroundColor: [
'rgba(91, 192, 222, 0.8)',
'rgba(240, 173, 78, 0.8)'
],
borderColor: [
'rgba(91, 192, 222, 1)',
'rgba(240, 173, 78, 1)'
],
borderWidth: 2
}]
},
options: {
responsive: true,
maintainAspectRatio: true,
plugins: {
legend: {
display: true,
position: 'bottom',
labels: {
padding: 15,
font: {
size: 13
},
generateLabels: function(chart) {
const data = chart.data;
if (data.labels.length && data.datasets.length) {
return data.labels.map(function(label, i) {
const value = data.datasets[0].data[i];
return {
text: label + ': ' + value,
fillStyle: data.datasets[0].backgroundColor[i],
strokeStyle: data.datasets[0].borderColor[i],
lineWidth: 2,
hidden: false,
index: i
};
});
}
return [];
}
}
},
title: {
display: true,
text: '{% trans "Volume and Snapshot Distribution" %}',
font: {
size: 14,
weight: 'normal'
},
padding: {
top: 5,
bottom: 15
}
},
tooltip: {
callbacks: {
label: function(context) {
let label = context.label || '';
if (label) {
label += ': ';
}
label += context.parsed;
return label;
}
}
}
}
}
});
})();
</script>
{% endif %}
@@ -0,0 +1,173 @@
{% extends 'base.html' %}
{% load i18n %}
{% load static %}
{% block title %}{% trans "Volume Metrics" %}{% endblock %}
{% block page_header %}
{% include "horizon/common/_page_header.html" with title=_("Volume Metrics") %}
{% endblock page_header %}
{% block main %}
<!-- HTMX -->
<script src="{% static 'vendor/htmx.js' %}"></script>
<!-- Chart.js -->
<script src="{% static 'vendor/chart.umd.js' %}"></script>
<style>
.metrics-section {
margin-bottom: 25px;
}
.pool-capacity-bar {
height: 20px;
background-color: #f5f5f5;
border-radius: 4px;
overflow: hidden;
position: relative;
}
.pool-capacity-fill {
height: 100%;
background: linear-gradient(90deg, #5cb85c 0%, #5bc0de 100%);
transition: width 0.3s ease;
}
.pool-capacity-text {
position: absolute;
width: 100%;
text-align: center;
line-height: 20px;
font-size: 0.85em;
font-weight: 600;
color: #333;
}
</style>
<!-- Auto-refreshing metrics cards -->
<div class="row metrics-section"
id="metrics-cards"
hx-get="{% url 'horizon:admin:volume_metrics:metrics_update' %}"
hx-trigger="load, every 10s"
hx-swap="innerHTML">
{% include "admin/volume_metrics/_metrics_cards.html" %}
</div>
<!-- Two column layout for chart and pools -->
<div class="row metrics-section">
<!-- Storage Chart Column -->
<div class="col-md-6" id="storage-chart">
{% include "admin/volume_metrics/_storage_chart.html" %}
</div>
<!-- Storage Pools Column -->
<div class="col-md-6">
<div class="panel panel-default">
<div class="panel-heading">
<h3 class="panel-title">
<i class="fa fa-server"></i> {% trans "Storage Pools" %}
</h3>
</div>
<div class="panel-body">
{% if storage_pools %}
{% for pool in storage_pools %}
<div style="margin-bottom: 20px;">
<div style="margin-bottom: 5px;">
<strong>{{ pool.name }}</strong>
<span class="pull-right text-muted">
{% trans "Allocated" %}: {{ pool.allocated|floatformat:2 }} / {{ pool.total|floatformat:2 }} GiB {% trans "Total" %}
</span>
</div>
<div class="pool-capacity-bar">
{% widthratio pool.allocated pool.total 100 as usage_percent %}
<div class="pool-capacity-fill" style="width: {{ usage_percent }}%;"></div>
<div class="pool-capacity-text">{{ usage_percent }}% {% trans "allocated" %}</div>
</div>
<div style="margin-top: 5px; font-size: 0.85em; color: #666;">
<span><i class="fa fa-check-circle text-success"></i> {% trans "Free" %}: {{ pool.free|floatformat:2 }} GiB</span>
<span style="margin-left: 15px;"><i class="fa fa-hdd-o text-info"></i> {% trans "Provisioned" %}: {{ pool.provisioned|floatformat:2 }} GiB</span>
</div>
</div>
{% endfor %}
{% else %}
<p class="text-muted text-center" style="padding: 20px;">
<i class="fa fa-info-circle"></i> {% trans "No storage pools found." %}
</p>
{% endif %}
</div>
</div>
</div>
</div>
<!-- Capacity Forecast Panel -->
<div class="row metrics-section">
<div class="col-sm-12"
id="capacity-prediction"
hx-get="{% url 'horizon:admin:volume_metrics:capacity_prediction' %}"
hx-trigger="load, every 60s"
hx-swap="innerHTML">
{% include "admin/volume_metrics/_capacity_prediction.html" %}
</div>
</div>
<!-- Detailed Pool Information Table (expandable) -->
<div class="row">
<div class="col-sm-12">
<div class="panel panel-default">
<div class="panel-heading" style="cursor: pointer;" onclick="togglePoolDetails()">
<h3 class="panel-title">
<i class="fa fa-table"></i> {% trans "Detailed Pool Information" %}
<span class="pull-right"><i class="fa fa-chevron-up" id="pool-toggle-icon"></i></span>
</h3>
</div>
<div class="panel-body" id="pool-details-table" style="display: block;">
{% if storage_pools %}
<div class="table-responsive">
<table class="table table-striped table-hover">
<thead>
<tr>
<th>{% trans "Pool Name" %}</th>
<th class="text-right">{% trans "Total (GiB)" %}</th>
<th class="text-right">{% trans "Allocated (GiB)" %}</th>
<th class="text-right">{% trans "Free (GiB)" %}</th>
<th class="text-right">{% trans "Provisioned (GiB)" %}</th>
<th class="text-right">{% trans "Usage %" %}</th>
</tr>
</thead>
<tbody>
{% for pool in storage_pools %}
<tr>
<td><strong>{{ pool.name }}</strong></td>
<td class="text-right">{{ pool.total|floatformat:2 }}</td>
<td class="text-right">{{ pool.allocated|floatformat:2 }}</td>
<td class="text-right">{{ pool.free|floatformat:2 }}</td>
<td class="text-right">{{ pool.provisioned|floatformat:2 }}</td>
<td class="text-right">
{% widthratio pool.allocated pool.total 100 as usage_percent %}
<span class="label {% if usage_percent >= 90 %}label-danger{% elif usage_percent >= 75 %}label-warning{% else %}label-success{% endif %}">
{{ usage_percent }}%
</span>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted text-center">{% trans "No storage pools found." %}</p>
{% endif %}
</div>
</div>
</div>
</div>
<script>
function togglePoolDetails() {
var details = document.getElementById('pool-details-table');
var icon = document.getElementById('pool-toggle-icon');
if (details.style.display === 'none') {
details.style.display = 'block';
icon.className = 'fa fa-chevron-up';
} else {
details.style.display = 'none';
icon.className = 'fa fa-chevron-down';
}
}
</script>
{% endblock %}
+4
View File
@@ -4,3 +4,7 @@
# our unit tests should not need to depend on horizon.
DEBUG = True
GRIAN_PLUGIN = {
"datasource": "fake",
}
+104
View File
@@ -0,0 +1,104 @@
#
# SPDX-License-Identifier: Apache-2.0
from unittest import mock
from grian_ui import config
from grian_ui_tests.unit import base
class GetGrianPluginConfigTests(base.TestCase):
@mock.patch("grian_ui.config.settings")
def test_returns_default_when_not_configured(self, mock_settings):
del mock_settings.GRIAN_PLUGIN
result = config._get_grian_plugin_config()
self.assertEqual(result, {"datasource": "fake"})
@mock.patch("grian_ui.config.settings")
def test_returns_configured_dict(self, mock_settings):
mock_settings.GRIAN_PLUGIN = {"datasource": "prometheus"}
result = config._get_grian_plugin_config()
self.assertEqual(result, {"datasource": "prometheus"})
@mock.patch("grian_ui.config.settings")
def test_raises_on_non_dict(self, mock_settings):
mock_settings.GRIAN_PLUGIN = "not_a_dict"
self.assertRaisesRegex(
config.ConfigurationError,
"must be a dict",
config._get_grian_plugin_config,
)
class GetDatasourceTypeTests(base.TestCase):
@mock.patch("grian_ui.config._get_grian_plugin_config")
def test_returns_fake(self, mock_config):
mock_config.return_value = {"datasource": "fake"}
self.assertEqual(config.get_datasource_type(), "fake")
@mock.patch("grian_ui.config._get_grian_plugin_config")
def test_returns_prometheus(self, mock_config):
mock_config.return_value = {"datasource": "prometheus"}
self.assertEqual(config.get_datasource_type(), "prometheus")
@mock.patch("grian_ui.config._get_grian_plugin_config")
def test_defaults_to_fake_when_key_missing(self, mock_config):
mock_config.return_value = {}
self.assertEqual(config.get_datasource_type(), "fake")
@mock.patch("grian_ui.config._get_grian_plugin_config")
def test_raises_on_invalid_type(self, mock_config):
mock_config.return_value = {"datasource": "invalid"}
self.assertRaisesRegex(
config.ConfigurationError,
"Invalid datasource type",
config.get_datasource_type,
)
@mock.patch("grian_ui.config._get_grian_plugin_config")
def test_raises_on_non_string(self, mock_config):
mock_config.return_value = {"datasource": 123}
self.assertRaisesRegex(
config.ConfigurationError,
"must be a string",
config.get_datasource_type,
)
class GetOpenstackKeystoneUrlTests(base.TestCase):
@mock.patch("grian_ui.config.settings")
def test_returns_url(self, mock_settings):
mock_settings.OPENSTACK_KEYSTONE_URL = "http://keystone:5000/v3"
self.assertEqual(
config.get_openstack_keystone_url(), "http://keystone:5000/v3"
)
@mock.patch("grian_ui.config.settings")
def test_raises_when_not_set(self, mock_settings):
del mock_settings.OPENSTACK_KEYSTONE_URL
self.assertRaisesRegex(
config.ConfigurationError,
"OPENSTACK_KEYSTONE_URL is not configured",
config.get_openstack_keystone_url,
)
@mock.patch("grian_ui.config.settings")
def test_raises_when_empty(self, mock_settings):
mock_settings.OPENSTACK_KEYSTONE_URL = ""
self.assertRaisesRegex(
config.ConfigurationError,
"OPENSTACK_KEYSTONE_URL is not configured",
config.get_openstack_keystone_url,
)
class GetOpenstackEndpointTypeTests(base.TestCase):
@mock.patch("grian_ui.config.settings")
def test_returns_configured_value(self, mock_settings):
mock_settings.OPENSTACK_ENDPOINT_TYPE = "internalURL"
self.assertEqual(config.get_openstack_endpoint_type(), "internalURL")
@mock.patch("grian_ui.config.settings")
def test_defaults_to_public(self, mock_settings):
del mock_settings.OPENSTACK_ENDPOINT_TYPE
self.assertEqual(config.get_openstack_endpoint_type(), "publicURL")
@@ -0,0 +1,52 @@
#
# SPDX-License-Identifier: Apache-2.0
import importlib
from unittest import mock
from grian_ui.datasource import api
from grian_ui_tests.unit import base
class GetDatasourceTests(base.TestCase):
@mock.patch("grian_ui.datasource.api.config")
def test_loads_fake_datasource(self, mock_config):
mock_config.get_datasource_type.return_value = "fake"
request = mock.MagicMock()
result = api.get_datasource(request)
from grian_ui.datasource.fake import FakeTelemetryClient
self.assertIsInstance(result, FakeTelemetryClient)
@mock.patch("grian_ui.datasource.api.config")
def test_loads_configured_module_path(self, mock_config):
mock_config.get_datasource_type.return_value = "fake"
request = mock.MagicMock()
with mock.patch(
"grian_ui.datasource.api.importlib.import_module",
wraps=importlib.import_module,
) as mock_import:
api.get_datasource(request)
mock_import.assert_any_call("grian_ui.datasource.fake")
@mock.patch("grian_ui.datasource.api.config")
def test_falls_back_to_fake_on_import_error(self, mock_config):
mock_config.get_datasource_type.return_value = "nonexistent"
request = mock.MagicMock()
result = api.get_datasource(request)
from grian_ui.datasource.fake import FakeTelemetryClient
self.assertIsInstance(result, FakeTelemetryClient)
class DatasourceClientInterfaceTests(base.TestCase):
def test_cannot_instantiate_abstract_class(self):
with self.assertRaises(TypeError):
api.DatasourceClient()
@@ -0,0 +1,101 @@
#
# SPDX-License-Identifier: Apache-2.0
from grian_ui.datasource import api
from grian_ui.datasource import fake
from grian_ui_tests.unit import base
class GetClientTests(base.TestCase):
def test_returns_fake_client(self):
client = fake.get_client()
self.assertIsInstance(client, fake.FakeTelemetryClient)
def test_accepts_request_argument(self):
client = fake.get_client(request=None)
self.assertIsInstance(client, fake.FakeTelemetryClient)
def test_implements_datasource_interface(self):
client = fake.get_client()
self.assertIsInstance(client, api.DatasourceClient)
class FakeTelemetryClientTests(base.TestCase):
def setUp(self):
super().setUp()
self.client = fake.FakeTelemetryClient()
def test_get_telemetry_overview_returns_dict(self):
result = self.client.get_telemetry_overview()
self.assertIsInstance(result, dict)
def test_get_telemetry_overview_keys(self):
result = self.client.get_telemetry_overview()
expected_keys = {
"total_metrics",
"active_instances",
"alerts_count",
"cpu_usage_avg",
"memory_usage_avg",
"network_throughput",
"disk_io_rate",
"uptime_percentage",
"total_storage",
"allocated_storage",
"last_updated",
}
self.assertEqual(set(result.keys()), expected_keys)
def test_get_telemetry_overview_static_values(self):
result = self.client.get_telemetry_overview()
self.assertEqual(result["total_metrics"], 1247)
self.assertEqual(result["active_instances"], 18)
self.assertEqual(result["alerts_count"], 3)
self.assertEqual(result["total_storage"], 10240)
def test_get_volumes_count(self):
self.assertEqual(self.client.get_volumes_count(), 120)
def test_get_snapshots_count(self):
self.assertEqual(self.client.get_snapshots_count(), 45)
def test_get_storage_pools_returns_list(self):
result = self.client.get_storage_pools()
self.assertIsInstance(result, list)
self.assertEqual(len(result), 2)
def test_get_storage_pools_structure(self):
pools = self.client.get_storage_pools()
for pool in pools:
self.assertIn("name", pool)
self.assertIn("total", pool)
self.assertIn("allocated", pool)
self.assertIn("free", pool)
self.assertIn("provisioned", pool)
def test_get_storage_pools_values(self):
pools = self.client.get_storage_pools()
self.assertEqual(pools[0]["name"], "storage-pool-01")
self.assertEqual(pools[1]["name"], "storage-pool-02")
self.assertEqual(pools[0]["total"], 5120.0)
def test_get_capacity_predictions_returns_list(self):
result = self.client.get_capacity_predictions()
self.assertIsInstance(result, list)
self.assertEqual(len(result), 2)
def test_get_capacity_predictions_structure(self):
predictions = self.client.get_capacity_predictions()
for pred in predictions:
self.assertIn("name", pred)
self.assertIn("current_free", pred)
self.assertIn("predicted_value", pred)
self.assertIn("days_until_full", pred)
self.assertIn("will_run_out", pred)
def test_get_capacity_predictions_values(self):
predictions = self.client.get_capacity_predictions()
self.assertFalse(predictions[0]["will_run_out"])
self.assertIsNone(predictions[0]["days_until_full"])
self.assertTrue(predictions[1]["will_run_out"])
self.assertEqual(predictions[1]["days_until_full"], 24.5)
@@ -0,0 +1,313 @@
#
# SPDX-License-Identifier: Apache-2.0
from unittest import mock
from grian_ui.datasource import api
from grian_ui_tests.unit import base
def _make_metric(value, labels=None):
"""Create a mock Prometheus metric result."""
metric = mock.MagicMock()
metric.value = value
metric.labels = labels or {}
return metric
def _make_client(query_map=None):
"""Create a PrometheusTelemetryClient with a mocked prom client."""
with (
mock.patch("grian_ui.datasource.prometheus.config") as mock_config,
mock.patch("grian_ui.datasource.prometheus.v3"),
mock.patch("grian_ui.datasource.prometheus.session"),
mock.patch(
"grian_ui.datasource.prometheus.obs_client_utils"
) as mock_obs,
):
mock_config.get_openstack_keystone_url.return_value = (
"http://keystone:5000/v3"
)
mock_config.get_openstack_endpoint_type.return_value = "publicURL"
request = mock.MagicMock()
request.user.token.id = "test-token"
request.user.project_id = "test-project"
request.user.domain_id = "test-domain"
request.user.services_region = "RegionOne"
from grian_ui.datasource import prometheus
client = prometheus.get_client(request)
prom_client = mock_obs.get_prom_client_from_keystone.return_value
if query_map:
prom_client.query.side_effect = lambda query: query_map.get(
query, []
)
return client, prom_client
class PrometheusTelemetryClientInitTests(base.TestCase):
@mock.patch("grian_ui.datasource.prometheus.obs_client_utils")
@mock.patch("grian_ui.datasource.prometheus.session")
@mock.patch("grian_ui.datasource.prometheus.v3")
@mock.patch("grian_ui.datasource.prometheus.config")
def test_init_creates_client(
self, mock_config, mock_v3, mock_session, mock_obs
):
mock_config.get_openstack_keystone_url.return_value = (
"http://keystone:5000/v3"
)
mock_config.get_openstack_endpoint_type.return_value = "publicURL"
request = mock.MagicMock()
request.user.token.id = "test-token"
request.user.project_id = "test-project"
request.user.domain_id = "test-domain"
request.user.services_region = "RegionOne"
from grian_ui.datasource import prometheus
client = prometheus.PrometheusTelemetryClient(request)
mock_v3.Token.assert_called_once_with( # nosec B106
auth_url="http://keystone:5000/v3",
token="test-token",
project_id="test-project",
domain_id="test-domain",
)
mock_obs.get_prom_client_from_keystone.assert_called_once()
self.assertIsInstance(client, api.DatasourceClient)
class GetTelemetryOverviewTests(base.TestCase):
def test_returns_metrics(self):
query_map = {
"sum(ceilometer_vcpus)": [_make_metric(64)],
"sum(ceilometer_memory_usage)": [_make_metric(32768)],
"sum(ceilometer_volume_provider_pool_capacity_total)": [
_make_metric(10240)
],
"sum(ceilometer_volume_provider_pool_capacity_allocated)": [
_make_metric(7000)
],
}
client, _ = _make_client(query_map)
result = client.get_telemetry_overview()
self.assertEqual(result["cpu_usage_avg"], 64.0)
self.assertEqual(result["memory_usage_avg"], 32.0)
self.assertEqual(result["total_storage"], 10240)
self.assertEqual(result["allocated_storage"], 7000)
def test_empty_results_return_zeros(self):
client, prom = _make_client()
prom.query.return_value = []
result = client.get_telemetry_overview()
self.assertEqual(result["cpu_usage_avg"], 0.0)
self.assertEqual(result["memory_usage_avg"], 0.0)
self.assertEqual(result["total_storage"], 0)
self.assertEqual(result["allocated_storage"], 0)
def test_exception_returns_partial_data_with_error(self):
client, prom = _make_client()
prom.query.side_effect = Exception("connection refused")
result = client.get_telemetry_overview()
self.assertIn("error", result)
self.assertIn("connection refused", result["error"])
class GetVolumesCountTests(base.TestCase):
def test_returns_count(self):
client, prom = _make_client()
prom.query.return_value = [_make_metric(42)]
result = client.get_volumes_count()
self.assertEqual(result, 42)
prom.query.assert_called_once_with(
query="count(last_over_time(ceilometer_volume_size[10m]))"
)
def test_empty_result_returns_zero(self):
client, prom = _make_client()
prom.query.return_value = []
self.assertEqual(client.get_volumes_count(), 0)
def test_none_result_returns_zero(self):
client, prom = _make_client()
prom.query.return_value = None
self.assertEqual(client.get_volumes_count(), 0)
class GetSnapshotsCountTests(base.TestCase):
def test_returns_count(self):
client, prom = _make_client()
prom.query.return_value = [_make_metric(15)]
result = client.get_snapshots_count()
self.assertEqual(result, 15)
prom.query.assert_called_once_with(
query="count(last_over_time(ceilometer_volume_snapshot_size[10m]))"
)
def test_empty_result_returns_zero(self):
client, prom = _make_client()
prom.query.return_value = []
self.assertEqual(client.get_snapshots_count(), 0)
class GetStoragePoolsTests(base.TestCase):
def test_returns_pools(self):
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
query_map = {
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
'ceilometer_volume_provider_pool_capacity_allocated{resource="pool-01"}': [
_make_metric(4096.0)
],
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
_make_metric(1024.0)
],
'ceilometer_volume_provider_pool_capacity_provisioned{resource="pool-01"}': [
_make_metric(6000.0)
],
}
client, _ = _make_client(query_map)
result = client.get_storage_pools()
self.assertEqual(len(result), 1)
self.assertEqual(result[0]["name"], "pool-01")
self.assertEqual(result[0]["total"], 5120.0)
self.assertEqual(result[0]["allocated"], 4096.0)
self.assertEqual(result[0]["free"], 1024.0)
self.assertEqual(result[0]["provisioned"], 6000.0)
def test_empty_result_returns_empty_list(self):
client, prom = _make_client()
prom.query.return_value = None
self.assertEqual(client.get_storage_pools(), [])
def test_exception_returns_empty_list(self):
client, prom = _make_client()
prom.query.side_effect = Exception("timeout")
self.assertEqual(client.get_storage_pools(), [])
def test_missing_sub_metrics_default_to_zero(self):
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
query_map = {
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
}
client, _ = _make_client(query_map)
result = client.get_storage_pools()
self.assertEqual(result[0]["allocated"], 0.0)
self.assertEqual(result[0]["free"], 0.0)
self.assertEqual(result[0]["provisioned"], 0.0)
class GetCapacityPredictionsTests(base.TestCase):
def test_pool_predicted_to_fill(self):
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
predict_query = (
'predict_linear(ceilometer_volume_provider_pool_capacity_free'
'{resource="pool-01"}[7d], 30 * 86400)'
)
query_map = {
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
_make_metric(1000.0)
],
predict_query: [_make_metric(-500.0)],
}
client, _ = _make_client(query_map)
result = client.get_capacity_predictions()
self.assertEqual(len(result), 1)
self.assertTrue(result[0]["will_run_out"])
self.assertIsNotNone(result[0]["days_until_full"])
self.assertEqual(result[0]["predicted_value"], -500.0)
self.assertEqual(result[0]["current_free"], 1000.0)
def test_pool_not_predicted_to_fill(self):
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
predict_query = (
'predict_linear(ceilometer_volume_provider_pool_capacity_free'
'{resource="pool-01"}[7d], 30 * 86400)'
)
query_map = {
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
_make_metric(2000.0)
],
predict_query: [_make_metric(1500.0)],
}
client, _ = _make_client(query_map)
result = client.get_capacity_predictions()
self.assertFalse(result[0]["will_run_out"])
self.assertIsNone(result[0]["days_until_full"])
def test_no_prediction_data(self):
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
query_map = {
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
_make_metric(2000.0)
],
}
client, _ = _make_client(query_map)
result = client.get_capacity_predictions()
self.assertFalse(result[0]["will_run_out"])
self.assertIsNone(result[0]["predicted_value"])
def test_empty_total_returns_empty_list(self):
client, prom = _make_client()
prom.query.return_value = None
self.assertEqual(client.get_capacity_predictions(), [])
def test_exception_returns_empty_list(self):
client, prom = _make_client()
prom.query.side_effect = Exception("timeout")
self.assertEqual(client.get_capacity_predictions(), [])
def test_days_until_full_calculation(self):
pool_metric = _make_metric(5120.0, {"resource": "pool-01"})
predict_query = (
'predict_linear(ceilometer_volume_provider_pool_capacity_free'
'{resource="pool-01"}[7d], 30 * 86400)'
)
query_map = {
"ceilometer_volume_provider_pool_capacity_total": [pool_metric],
'ceilometer_volume_provider_pool_capacity_free{resource="pool-01"}': [
_make_metric(1500.0)
],
predict_query: [_make_metric(-1500.0)],
}
client, _ = _make_client(query_map)
result = client.get_capacity_predictions()
# decline_per_day = (1500 - (-1500)) / 30 = 100
# days_until_full = 1500 / 100 = 15.0
self.assertEqual(result[0]["days_until_full"], 15.0)
+176
View File
@@ -0,0 +1,176 @@
#
# SPDX-License-Identifier: Apache-2.0
from unittest import mock
from grian_ui.content.volume_metrics import views
from grian_ui_tests.unit import base
def _make_mock_client():
client = mock.MagicMock()
client.get_telemetry_overview.return_value = {"cpu_usage_avg": 50.0}
client.get_volumes_count.return_value = 10
client.get_snapshots_count.return_value = 5
client.get_storage_pools.return_value = [{"name": "pool-01"}]
client.get_capacity_predictions.return_value = [
{"name": "pool-01", "will_run_out": False}
]
return client
class VolumesMetricsIndexViewTests(base.TestCase):
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data(self, mock_get_ds):
client = _make_mock_client()
mock_get_ds.return_value = client
view = views.VolumesMetricsIndexView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertEqual(context["overview"], {"cpu_usage_avg": 50.0})
self.assertEqual(context["volumes"], 10)
self.assertEqual(context["snapshots"], 5)
self.assertEqual(context["storage_pools"], [{"name": "pool-01"}])
self.assertEqual(len(context["predictions"]), 1)
self.assertIn("page_title", context)
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data_on_error(self, mock_get_ds):
client = mock.MagicMock()
client.get_telemetry_overview.side_effect = Exception("fail")
mock_get_ds.return_value = client
view = views.VolumesMetricsIndexView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertIn("error_message", context)
self.assertFalse(context["data_loaded"])
self.assertEqual(context["overview"], {})
self.assertEqual(context["volumes"], 0)
self.assertEqual(context["snapshots"], 0)
self.assertEqual(context["storage_pools"], [])
self.assertEqual(context["predictions"], [])
def test_template_name(self):
self.assertEqual(
views.VolumesMetricsIndexView.template_name,
"admin/volume_metrics/index.html",
)
class MetricsUpdateViewTests(base.TestCase):
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data(self, mock_get_ds):
client = _make_mock_client()
mock_get_ds.return_value = client
view = views.MetricsUpdateView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertEqual(context["overview"], {"cpu_usage_avg": 50.0})
self.assertEqual(context["volumes"], 10)
self.assertEqual(context["snapshots"], 5)
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data_on_error(self, mock_get_ds):
client = mock.MagicMock()
client.get_telemetry_overview.side_effect = Exception("fail")
mock_get_ds.return_value = client
view = views.MetricsUpdateView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertEqual(context["overview"], {})
self.assertEqual(context["volumes"], 0)
self.assertEqual(context["snapshots"], 0)
def test_template_name(self):
self.assertEqual(
views.MetricsUpdateView.template_name,
"admin/volume_metrics/_metrics_cards.html",
)
class ChartUpdateViewTests(base.TestCase):
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data(self, mock_get_ds):
client = _make_mock_client()
mock_get_ds.return_value = client
view = views.ChartUpdateView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertEqual(context["volumes"], 10)
self.assertEqual(context["snapshots"], 5)
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data_on_error(self, mock_get_ds):
client = mock.MagicMock()
client.get_volumes_count.side_effect = Exception("fail")
mock_get_ds.return_value = client
view = views.ChartUpdateView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertEqual(context["volumes"], 0)
self.assertEqual(context["snapshots"], 0)
def test_template_name(self):
self.assertEqual(
views.ChartUpdateView.template_name,
"admin/volume_metrics/_storage_chart.html",
)
class CapacityPredictionViewTests(base.TestCase):
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data(self, mock_get_ds):
client = _make_mock_client()
mock_get_ds.return_value = client
view = views.CapacityPredictionView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertEqual(len(context["predictions"]), 1)
@mock.patch("grian_ui.content.volume_metrics.views.get_datasource")
def test_get_context_data_on_error(self, mock_get_ds):
client = mock.MagicMock()
client.get_capacity_predictions.side_effect = Exception("fail")
mock_get_ds.return_value = client
view = views.CapacityPredictionView()
view.request = mock.MagicMock()
view.kwargs = {}
context = view.get_context_data()
self.assertEqual(context["predictions"], [])
def test_template_name(self):
self.assertEqual(
views.CapacityPredictionView.template_name,
"admin/volume_metrics/_capacity_prediction.html",
)