openstack/keystone
Code Issues Proposed changes

Remove OS-STATS monitoring

Browse Source 
One time change to fully remove anything OS-STATS related.

implements bp removed-as-of-kilo

Change-Id: I31416d3e9bd4ea68f56f12269a3c68191403f197
This commit is contained in:
Steve Martinelli
2014-10-02 11:38:36 -04:00
parent c9d28a3ed2
commit af1f960411
12 changed files with 4 additions and 320 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
doc/source/configuration.rst
View File

@@ -98,7 +98,6 @@ following sections:
* ``[saml]`` - SAML configuration options
* ``[signing]`` - Cryptographic signatures for PKI based tokens
* ``[ssl]`` - SSL configuration
* ``[stats]`` - Stats system driver configuration
* ``[token]`` - Token driver & token provider configuration
* ``[trust]`` - Trust extension configuration
@@ -705,47 +704,6 @@ choosing the output levels and formats.
.. _Paste: http://pythonpaste.org/
.. _`Python logging module`: http://docs.python.org/library/logging.html
Monitoring
----------
Keystone provides some basic request/response monitoring statistics out of the
box.
Enable data collection by defining a ``stats_monitoring`` filter and including
it at the beginning of any desired WSGI pipelines:
.. code-block:: ini
[filter:stats_monitoring]
paste.filter_factory = keystone.contrib.stats:StatsMiddleware.factory
[pipeline:public_api]
pipeline = stats_monitoring [...] public_service
Enable the reporting of collected data by defining a ``stats_reporting`` filter
and including it near the end of your ``admin_api`` WSGI pipeline (After
``*_body`` middleware and before ``*_extension`` filters is recommended):
.. code-block:: ini
[filter:stats_reporting]
paste.filter_factory = keystone.contrib.stats:StatsExtension.factory
[pipeline:admin_api]
pipeline = [...] json_body stats_reporting ec2_extension [...] admin_service
Query the admin API for statistics using:
.. code-block:: bash
$ curl -H 'X-Auth-Token: ADMIN' http://localhost:35357/v2.0/OS-STATS/stats
Reset collected data using:
.. code-block:: bash
$ curl -H 'X-Auth-Token: ADMIN' -X DELETE http://localhost:35357/v2.0/OS-STATS/stats
SSL
---
@@ -835,7 +793,7 @@ and before the ``public_service`` app in the public_api WSGI pipeline in
paste.filter_factory = keystone.contrib.user_crud:CrudExtension.factory
[pipeline:public_api]
pipeline = stats_monitoring url_normalize token_auth admin_token_auth xml_body json_body debug ec2_extension user_crud_extension public_service
pipeline = url_normalize token_auth admin_token_auth xml_body json_body debug ec2_extension user_crud_extension public_service
Each user can then change their own password with a HTTP PATCH :

4
doc/source/external-auth.rst
View File

@@ -147,9 +147,9 @@ authentication.
For example, if the original pipeline looks like this::
[pipeline:public_api]
pipeline = stats_monitoring url_normalize token_auth admin_token_auth xml_body json_body debug ec2_extension user_crud_extension public_service
pipeline = url_normalize token_auth admin_token_auth xml_body json_body debug ec2_extension user_crud_extension public_service
Your modified pipeline might then look like this::
[pipeline:public_api]
pipeline = stats_monitoring url_normalize token_auth admin_token_auth xml_body json_body my_auth debug ec2_extension user_crud_extension public_service
pipeline = url_normalize token_auth admin_token_auth xml_body json_body my_auth debug ec2_extension user_crud_extension public_service

6
etc/keystone-paste.ini
View File

@@ -63,12 +63,6 @@ paste.filter_factory = keystone.middleware:NormalizingFilter.factory
[filter:sizelimit]
paste.filter_factory = keystone.middleware:RequestBodySizeLimiter.factory
[filter:stats_monitoring]
paste.filter_factory = keystone.contrib.stats:StatsMiddleware.factory
[filter:stats_reporting]
paste.filter_factory = keystone.contrib.stats:StatsExtension.factory
[filter:access_log]
paste.filter_factory = keystone.contrib.access:AccessLogMiddleware.factory

10
etc/keystone.conf.sample
View File

@@ -1501,16 +1501,6 @@
#cert_subject=/C=US/ST=Unset/L=Unset/O=Unset/CN=localhost
[stats]
#
# Options defined in keystone
#
# Stats backend driver. (string value)
#driver=keystone.contrib.stats.backends.kvs.Stats
[token]
#

6
keystone/common/config.py
View File

@@ -493,12 +493,6 @@ FILE_OPTIONS = {
'.sql.EndpointPolicy',
help='Endpoint policy backend driver'),
],
'stats': [
cfg.StrOpt('driver',
default=('keystone.contrib.stats.backends'
'.kvs.Stats'),
help='Stats backend driver.'),
],
'ldap': [
cfg.StrOpt('url', default='ldap://localhost',
help='URL for connecting to the LDAP server.'),

15
keystone/contrib/stats/__init__.py
View File

@@ -1,15 +0,0 @@
# Copyright 2012 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from keystone.contrib.stats.core import * # noqa

0
keystone/contrib/stats/backends/__init__.py
View File

32
keystone/contrib/stats/backends/kvs.py
View File

@@ -1,32 +0,0 @@
# Copyright 2012 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from keystone.common import kvs
from keystone.contrib import stats
class Stats(kvs.Base, stats.Driver):
def get_stats(self, api):
return self.db.get('stats-%s' % api, {})
def set_stats(self, api, stats_ref):
self.db.set('stats-%s' % api, stats_ref)
def increment_stat(self, api, category, value):
"""Increment a statistic counter, or create it if it doesn't exist."""
stats = self.get_stats(api)
stats.setdefault(category, dict())
counter = stats[category].setdefault(value, 0)
stats[category][value] = counter + 1
self.set_stats(api, stats)

160
keystone/contrib/stats/core.py
View File

@@ -1,160 +0,0 @@
# Copyright 2012 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from keystone.common import extension
from keystone.common import manager
from keystone.common import wsgi
from keystone import config
from keystone import exception
from keystone.openstack.common import log
from keystone.openstack.common import versionutils
CONF = config.CONF
LOG = log.getLogger(__name__)
extension_data = {
'name': 'OpenStack Keystone Stats API',
'namespace': 'http://docs.openstack.org/identity/api/ext/'
'OS-STATS/v1.0',
'alias': 'OS-STATS',
'updated': '2013-07-07T12:00:0-00:00',
'description': 'OpenStack Keystone Stats API.',
'links': [
{
'rel': 'describedby',
# TODO(ayoung): needs a description
'type': 'text/html',
'href': 'https://github.com/openstack/identity-api',
}
]}
extension.register_admin_extension(extension_data['alias'], extension_data)
class Manager(manager.Manager):
"""Default pivot point for the Stats backend.
See :mod:`keystone.common.manager.Manager` for more details on how this
dynamically calls the backend.
"""
def __init__(self):
super(Manager, self).__init__(CONF.stats.driver)
class Driver(object):
"""Interface description for a Stats driver."""
def get_stats(self, api):
"""Retrieve all previously-captured statistics for an interface."""
raise exception.NotImplemented()
def set_stats(self, api, stats_ref):
"""Update statistics for an interface."""
raise exception.NotImplemented()
def increment_stat(self, api, category, value):
"""Increment the counter for an individual statistic."""
raise exception.NotImplemented()
class StatsExtension(wsgi.ExtensionRouter):
"""Reports on previously-collected request/response statistics."""
def add_routes(self, mapper):
stats_controller = StatsController()
mapper.connect(
'/OS-STATS/stats',
controller=stats_controller,
action='get_stats',
conditions=dict(method=['GET']))
mapper.connect(
'/OS-STATS/stats',
controller=stats_controller,
action='reset_stats',
conditions=dict(method=['DELETE']))
class StatsController(wsgi.Application):
def __init__(self):
self.stats_api = Manager()
super(StatsController, self).__init__()
def get_stats(self, context):
self.assert_admin(context)
return {
'OS-STATS:stats': [
{
'type': 'identity',
'api': 'admin',
'extra': self.stats_api.get_stats('admin'),
},
{
'type': 'identity',
'api': 'public',
'extra': self.stats_api.get_stats('public'),
},
]
}
def reset_stats(self, context):
self.assert_admin(context)
self.stats_api.set_stats('public', dict())
self.stats_api.set_stats('admin', dict())
class StatsMiddleware(wsgi.Middleware):
"""Monitors various request/response attribute statistics."""
request_attributes = ['application_url',
'method',
'path',
'path_qs',
'remote_addr']
response_attributes = ['status_int']
@versionutils.deprecated(
what='keystone.contrib.stats.core.StatsMiddleware',
as_of=versionutils.deprecated.ICEHOUSE,
in_favor_of='external tooling',
remove_in=+2)
def __init__(self, *args, **kwargs):
self.stats_api = Manager()
return super(StatsMiddleware, self).__init__(*args, **kwargs)
def _resolve_api(self, host):
if host.endswith(':%s' % (CONF.admin_port)):
return 'admin'
elif host.endswith(':%s' % (CONF.public_port)):
return 'public'
else:
return host
def capture_stats(self, host, obj, attributes):
"""Collect each attribute from the given object."""
for attribute in attributes:
self.stats_api.increment_stat(
self._resolve_api(host), attribute, getattr(obj, attribute))
def process_request(self, request):
"""Monitor incoming request attributes."""
self.capture_stats(request.host, request, self.request_attributes)
def process_response(self, request, response):
"""Monitor outgoing response attributes."""
self.capture_stats(request.host, response, self.response_attributes)
return response

42
keystone/tests/test_contrib_stats_core.py
View File

@@ -1,42 +0,0 @@
# Copyright 2012 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from keystone import config
from keystone.contrib import stats
from keystone import tests
CONF = config.CONF
class StatsContribCore(tests.TestCase):
def setUp(self):
super(StatsContribCore, self).setUp()
self.stats_middleware = stats.StatsMiddleware(None)
def test_admin_request(self):
host_admin = "127.0.0.1:%s" % CONF.admin_port
self.assertEqual("admin",
self.stats_middleware._resolve_api(host_admin))
def test_public_request(self):
host_public = "127.0.0.1:%s" % CONF.public_port
self.assertEqual("public",
self.stats_middleware._resolve_api(host_public))
def test_other_request(self):
host_public = "127.0.0.1:%s" % CONF.public_port
host_other = host_public + "1"
self.assertEqual(host_other,
self.stats_middleware._resolve_api(host_other))

3
keystone/tests/test_singular_plural.py
View File

@@ -16,7 +16,6 @@ import ast
from keystone.contrib.admin_crud import core as admin_crud_core
from keystone.contrib.s3 import core as s3_core
from keystone.contrib.stats import core as stats_core
from keystone.contrib.user_crud import core as user_crud_core
from keystone.identity import core as identity_core
from keystone import service
@@ -25,7 +24,7 @@ from keystone import service
class TestSingularPlural(object):
def test_keyword_arg_condition_or_methods(self):
"""Raise if we see a keyword arg called 'condition' or 'methods'."""
modules = [admin_crud_core, s3_core, stats_core,
modules = [admin_crud_core, s3_core,
user_crud_core, identity_core, service]
for module in modules:
filename = module.__file__

2
tox.ini
View File

@@ -26,7 +26,6 @@ commands =
keystone/tests/test_backend.py \
keystone/tests/test_backend_rules.py \
keystone/tests/test_cache_backend_mongo.py \
keystone/tests/test_contrib_stats_core.py \
keystone/tests/test_driver_hints.py \
keystone/tests/test_hacking_checks.py \
keystone/tests/test_injection.py \
@@ -54,7 +53,6 @@ commands =
keystone/tests/test_backend.py \
keystone/tests/test_backend_rules.py \
keystone/tests/test_cache_backend_mongo.py \
keystone/tests/test_contrib_stats_core.py \
keystone/tests/test_driver_hints.py \
keystone/tests/test_hacking_checks.py \
keystone/tests/test_injection.py \