Expand libvirt and ovs plugins to publish entity names
This is useful in dashboards where we display libvirt and ovs metrics for a physical host, it will elimnate the step of translating vm and tenant ids if needing to investigate project usage. The libvirt plugin metadata config array will now support 'vm_name' and 'tenant_name', and the ovs plugin will now support the metadata config value for 'tenant_name'. Change-Id: If76418b83119a12b8534fb7f7cb224339f357be8
This commit is contained in:
Brad Klein
Brad Klein
parent
0db9a7202e
commit
71a0d5dd26
@ -16,6 +16,10 @@ init_config:
|
||||
# If set, will submit raw counters from ovs-vsctl command output for the given
|
||||
# network interface
|
||||
use_absolute_metrics: true
|
||||
# List of router metadata keys to be sent as dimensions when cross posting
|
||||
# metrics to the infrastruture project.
|
||||
metadata:
|
||||
- tenant_name
|
||||
# Installations that don't allow usage of sudo should copy the `ovs-vsctl`
|
||||
# command to another location and use the `setcap` command to allow the
|
||||
# monasca-agent to run that command. The new location of the `ovs-vsctl`
|
||||
|
||||
@ -39,9 +39,19 @@ The Libvirt plugin uses a cache directory to persist data, which is `/dev/shm` b
|
||||
|
||||
If the owner of the VM is in a different tenant the Agent Cross-Tenant Metric Submission can be setup. See this [documentation](https://github.com/openstack/monasca-agent/blob/master/docs/MonascaMetrics.md#cross-tenant-metric-submission) for details.
|
||||
|
||||
`admin_user` is the username capable of making administrative nova calls.
|
||||
|
||||
`admin_password` password for the nova user.
|
||||
|
||||
`admin_tenant_name` is the project/tenant to POST metrics with the `vm.` prefix.
|
||||
|
||||
`identity_url` is the keystone endpoint for auth.
|
||||
|
||||
`region_name` is used to add the region dimension to metrics.
|
||||
|
||||
`nova_refresh` specifies the number of seconds between calls to the Nova API to refresh the instance cache. This is helpful for updating VM hostname and pruning deleted instances from the cache. By default, it is set to 14,400 seconds (four hours). Set to 0 to refresh every time the Collector runs, or to None to disable regular refreshes entirely (though the instance cache will still be refreshed if a new instance is detected).
|
||||
|
||||
`metadata` specifies the list of instance metadata keys to be included as dimensions with the cross-tenant metrics for the operations project. This is helpful to give more information about an instance. When using the agent setup scripts, by default `scale_group` metadata is enabled for supporting auto scaling in Heat.
|
||||
`metadata` specifies the list of instance metadata keys to be included as dimensions with the cross-tenant metrics for the operations project. This is helpful to give more information about an instance. When using the agent setup scripts, by default `scale_group` metadata is enabled for supporting auto scaling in Heat. VM name and tenant name (in addition to default IDs) can be provided as dimensions if `vm_name` and `tenant_name` are provided in the list of metadata keys.
|
||||
|
||||
`customer_metadata` specifies the list of instance metadata keys to be included as dimensions with customer metrics. This is helpful to give more information about an instance.
|
||||
|
||||
@ -59,7 +69,6 @@ If the owner of the VM is in a different tenant the Agent Cross-Tenant Metric Su
|
||||
|
||||
`vnic_collection_period` will cause vnic metrics to be output at a minimum `vnic_collection_period` second interval. This can be optionally set to have vnic metrics be outputted less often to reduce metric load on the system. If this is less than the agent collection period, it will be ignored. The default value is 0.
|
||||
|
||||
|
||||
`vm_cpu_check_enable` enables collecting of VM CPU metrics (Default True). Please see "Mapping Metrics to Configuration Parameters" section below for what metrics are controlled by this flag.
|
||||
|
||||
`vm_disks_check_enable` enables collecting of VM Disk metrics (Default True). Please see "Mapping Metrics to Configuration Parameters" section below for what metrics are controlled by this flag.
|
||||
@ -412,7 +421,7 @@ Please see table below for metrics in libvirt that are always enabled.
|
||||
All metrics include `resource_id` and `zone` (availability zone) dimensions. Because there is a separate set of metrics for the two target audiences (VM customers and Operations), other dimensions may differ.
|
||||
|
||||
| Dimension Name | Customer Value | Operations Value |
|
||||
| -------------- | ------------------------- | ----------------------- |
|
||||
| -------------- | ------------------------- | -------------------------------------------------------------- |
|
||||
| hostname | name of VM as provisioned | hypervisor's hostname |
|
||||
| zone | availability zone | availability zone |
|
||||
| resource_id | resource ID of VM | resource ID of VM |
|
||||
@ -421,6 +430,8 @@ All metrics include `resource_id` and `zone` (availability zone) dimensions. Be
|
||||
| device | name of net or disk dev | name of net or disk dev |
|
||||
| port_id | port ID of the VM port | port ID of the VM port |
|
||||
| tenant_id | (N/A) | owner of VM |
|
||||
| tenant_name | (N/A) | name of the project owner of the VM (if configured to publish) |
|
||||
| vm_name | (N/A) | name of the VM (if configured to publish) |
|
||||
|
||||
## Aggregate Metrics
|
||||
|
||||
|
||||
12
docs/Ovs.md
12
docs/Ovs.md
@ -32,7 +32,7 @@ configured using the configuration file example below.
|
||||
|
||||
`admin_password` password for the neutron user.
|
||||
|
||||
`admin_tenant_name` is the project/tenant to POST metrics with the `vm.` prefix.
|
||||
`admin_tenant_name` is the project/tenant to POST metrics with the `ovs.` prefix.
|
||||
|
||||
`admin_user` is the username capable of making administrative neutron calls.
|
||||
|
||||
@ -48,6 +48,8 @@ configured using the configuration file example below.
|
||||
|
||||
`check_router_ha` will check router HA status if set to true. This should be set to false if not configuring routers for HA, as setting this to true will cause the plugin to make additional neutron calls.
|
||||
|
||||
`metadata` specifies the list of router metadata keys to be included as dimensions with the cross-tenant metrics for the operations project. This is helpful to give more information about an router. Tenant name (in addition to default ID) can be provided as dimensions if `tenant_name` is provided in the list of metadata keys.
|
||||
|
||||
`ovs_cmd` is the location of the open vswitch command. Installations that allow sudo should set this to `sudo /usr/bin/ovs-vsctl` and add `mon-agent ALL=(ALL) NOPASSWD:/usr/bin/ovs-vsctl` to the `/etc/sudoers` file. Installations that don't allow usage of sudo should copy the `ovs-vsctl` command to another location and use the `setcap` command to allow the monasca-agent to run that command. The new location of the `ovs-vsctl` command should be what is set in the config file for `ovs_cmd`.
|
||||
|
||||
`instances` are not used and should be empty in `ovs.yaml` because like the ovs plugin it runs against all routers hosted on the node at once.
|
||||
@ -174,23 +176,25 @@ NOTE:
|
||||
## Router Metric Dimensions
|
||||
|
||||
| Dimension Name | Customer Value | Operations Value |
|
||||
| -------------- | -------------------------- | --------------------------- |
|
||||
| -------------- | -------------------------- | ------------------------------------------------------------------ |
|
||||
| hostname | (N/A) | hostname hosting the router |
|
||||
| resource_id | resource ID of router | resource ID of router |
|
||||
| service | "networking" | "networking" |
|
||||
| component | "ovs" | "ovs" |
|
||||
| router_name | name of the virtual router | name of the virtual router |
|
||||
| tenant_id | (N/A) | project owner of the router |
|
||||
| tenant_id | (N/A) | id of the project owner of the router |
|
||||
| tenant_name | (N/A) | name of the project owner of the router (if configured to publish) |
|
||||
| port_id | port ID of the router | port ID of the router |
|
||||
|
||||
## OVS Port Metric Dimensions
|
||||
| Dimension Name | Customer Value | Operations Value |
|
||||
| -------------- | -------------------------- | --------------------------- |
|
||||
| -------------- | -------------------------- | ------------------------------------------------------------------ |
|
||||
| hostname | (N/A) | hostname hosting the ports |
|
||||
| resource_id | resource ID of port | resource id of the port |
|
||||
| service | "networking" | "networking" |
|
||||
| component | "ovs" | "ovs" |
|
||||
| tenant_id | (N/A) | project owner of the port |
|
||||
| tenant_name | (N/A) | name of the project owner of the router (if configured to publish) |
|
||||
| port_id | port ID of VM | port ID of VM |
|
||||
|
||||
# License
|
||||
|
||||
@ -11,3 +11,39 @@ def add_basic_auth(request, username, password):
|
||||
auth_str = base64.encodestring('%s:%s' % (username, password)).strip()
|
||||
request.add_header('Authorization', 'Basic %s' % auth_str)
|
||||
return request
|
||||
|
||||
|
||||
def get_keystone_client(config):
|
||||
import keystoneclient.v2_0.client as kc
|
||||
kwargs = {
|
||||
'username': config.get('admin_user'),
|
||||
'project_name': config.get('admin_tenant_name'),
|
||||
'password': config.get('admin_password'),
|
||||
'auth_url': config.get('identity_uri'),
|
||||
'endpoint_type': 'internalURL',
|
||||
'region_name': config.get('region_name'),
|
||||
}
|
||||
|
||||
return kc.Client(**kwargs)
|
||||
|
||||
|
||||
def get_tenant_name(tenants, tenant_id):
|
||||
tenant_name = None
|
||||
for tenant in tenants:
|
||||
if tenant.id == tenant_id:
|
||||
tenant_name = tenant.name
|
||||
break
|
||||
return tenant_name
|
||||
|
||||
|
||||
def get_tenant_list(config, log):
|
||||
tenants = []
|
||||
try:
|
||||
log.debug("Retrieving Keystone tenant list")
|
||||
keystone = get_keystone_client(config)
|
||||
tenants = keystone.tenants.list()
|
||||
except Exception as e:
|
||||
msg = "Unable to get tenant list from keystone: {0}"
|
||||
log.error(msg.format(e))
|
||||
|
||||
return tenants
|
||||
|
||||
@ -18,6 +18,7 @@
|
||||
import json
|
||||
import libvirt
|
||||
import math
|
||||
import monasca_agent.collector.checks.utils as utils
|
||||
import os
|
||||
import stat
|
||||
import subprocess
|
||||
@ -145,6 +146,14 @@ class LibvirtCheck(AgentCheck):
|
||||
self.log.warn("Unable to ping VMs, no network namespaces found." +
|
||||
"Either no VMs are present, or routing is centralized.")
|
||||
|
||||
#
|
||||
# Only make the keystone call to get the tenant list
|
||||
# if we are configured to publish tenant names.
|
||||
#
|
||||
tenants = []
|
||||
if self.init_config.get('metadata') and 'tenant_name' in self.init_config.get('metadata'):
|
||||
tenants = utils.get_tenant_list(self.init_config, self.log)
|
||||
|
||||
for instance in instances:
|
||||
instance_ports = []
|
||||
inst_name = instance.__getattr__('OS-EXT-SRV-ATTR:instance_name')
|
||||
@ -170,6 +179,10 @@ class LibvirtCheck(AgentCheck):
|
||||
'disk': inst_flavor.disk,
|
||||
'instance_ports': instance_ports}
|
||||
|
||||
tenant_name = utils.get_tenant_name(tenants, instance.tenant_id)
|
||||
if tenant_name:
|
||||
id_cache[inst_name]['tenant_name'] = tenant_name
|
||||
|
||||
for config_var in ['metadata', 'customer_metadata']:
|
||||
if self.init_config.get(config_var):
|
||||
for metadata in self.init_config.get(config_var):
|
||||
@ -671,12 +684,7 @@ class LibvirtCheck(AgentCheck):
|
||||
# Add dimensions that would be helpful for operations
|
||||
dims_operations = dims_customer.copy()
|
||||
dims_operations['tenant_id'] = instance_cache.get(inst_name)['tenant_id']
|
||||
if self.init_config.get('metadata'):
|
||||
for metadata in self.init_config.get('metadata'):
|
||||
metadata_value = (instance_cache.get(inst_name).
|
||||
get(metadata))
|
||||
if metadata_value:
|
||||
dims_operations[metadata] = metadata_value
|
||||
dims_operations = self._update_dims_with_metadata(instance_cache, inst_name, dims_operations)
|
||||
if self.init_config.get('customer_metadata'):
|
||||
for metadata in self.init_config.get('customer_metadata'):
|
||||
metadata_value = (instance_cache.get(inst_name).
|
||||
@ -779,3 +787,18 @@ class LibvirtCheck(AgentCheck):
|
||||
#
|
||||
rate_value = -1
|
||||
return rate_value
|
||||
|
||||
def _update_dims_with_metadata(self, instance_cache, inst_name, dim_operations):
|
||||
"""Update operations dimensions with metadata."""
|
||||
dims = dim_operations
|
||||
if self.init_config.get('metadata'):
|
||||
for metadata in self.init_config.get('metadata'):
|
||||
if 'vm_name' == metadata:
|
||||
metadata_value = (instance_cache.get(inst_name).
|
||||
get('hostname'))
|
||||
else:
|
||||
metadata_value = (instance_cache.get(inst_name).
|
||||
get(metadata))
|
||||
if metadata_value:
|
||||
dims[metadata] = metadata_value
|
||||
return dims
|
||||
|
||||
@ -6,6 +6,7 @@ import datetime
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import monasca_agent.collector.checks.utils as utils
|
||||
import os
|
||||
import re
|
||||
import socket
|
||||
@ -154,6 +155,9 @@ class OvsCheck(AgentCheck):
|
||||
device_uuid = port_info['device_uuid']
|
||||
is_router_port = port_info['is_router_port']
|
||||
tenant_id = port_info['tenant_id']
|
||||
tenant_name = None
|
||||
if 'tenant_name' in port_info:
|
||||
tenant_name = port_info['tenant_name']
|
||||
if is_router_port and not self._is_active_router(device_uuid):
|
||||
continue
|
||||
if is_router_port:
|
||||
@ -193,6 +197,8 @@ class OvsCheck(AgentCheck):
|
||||
continue
|
||||
ops_dimensions = this_dimensions.copy()
|
||||
ops_dimensions.update({'tenant_id': tenant_id})
|
||||
if tenant_name:
|
||||
ops_dimensions.update({'tenant_name': tenant_name})
|
||||
if self.use_rate_metrics:
|
||||
self.gauge(metric_name_rate, value[interface_stats_key],
|
||||
dimensions=customer_dimensions,
|
||||
@ -393,6 +399,15 @@ class OvsCheck(AgentCheck):
|
||||
all_ports_data = all_ports_data['ports']
|
||||
all_routers_data = all_routers_data['routers']
|
||||
|
||||
#
|
||||
# Only make the keystone call to get the tenant list
|
||||
# if we are configured to publish tenant names.
|
||||
#
|
||||
if self.init_config.get('metadata') and 'tenant_name' in self.init_config.get('metadata'):
|
||||
tenants = utils.get_tenant_list(self.init_config, self.log)
|
||||
else:
|
||||
tenants = []
|
||||
|
||||
for port_data in all_ports_data:
|
||||
port_uuid = port_data['id']
|
||||
device_uuid = port_data['device_id']
|
||||
@ -410,6 +425,10 @@ class OvsCheck(AgentCheck):
|
||||
'is_router_port': is_router_port,
|
||||
'tenant_id': tenant_id}
|
||||
|
||||
tenant_name = utils.get_tenant_name(tenants, tenant_id)
|
||||
if tenant_name:
|
||||
port_cache[port_uuid]['tenant_name'] = tenant_name
|
||||
|
||||
port_cache['last_update'] = int(time.time())
|
||||
|
||||
# Write the updated cache
|
||||
|
||||
Loading…
Reference in New Issue
Block a user