Keystone Caching Layer for Manager Calls
Implements core elements of Keystone Caching layer. dogpile.cache is used as the caching library to provide flexibility to the cache backend (out of the box, Redis, Memcached, File, and in-Memory). The keystone.common.cache.on_arguments decorator is used cache the return value of method and functions that are meant to be cached. The default behavior is to not cache anything (using a no-op cacher). developing.rst has been updated to give an outline on how to approach adding the caching layer onto a manager object. Subsequent patches will build upon this code to enable caching across the various keystone subsystems. DocImpact partial-blueprint: caching-layer-for-driver-calls Change-Id: I664ddccd8d1d393cf50d24054f946512093fb790
This commit is contained in:
parent
5dc50bbf0f
commit
369be47fdb
@ -76,6 +76,7 @@ following sections:
|
||||
* ``[identity]`` - identity system driver configuration
|
||||
* ``[catalog]`` - service catalog driver configuration
|
||||
* ``[token]`` - token driver & token provider configuration
|
||||
* ``[cache]`` - caching layer configuration
|
||||
* ``[policy]`` - policy system driver configuration for RBAC
|
||||
* ``[signing]`` - cryptographic signatures for PKI based tokens
|
||||
* ``[ssl]`` - SSL configuration
|
||||
@ -198,6 +199,75 @@ Conversely, if ``provider`` is ``keystone.token.providers.uuid.Provider``,
|
||||
For a customized provider, ``token_format`` must not set to ``PKI`` or
|
||||
``UUID``.
|
||||
|
||||
|
||||
Caching Layer
|
||||
-------------
|
||||
|
||||
Keystone supports a caching layer that is above the configurable subsystems (e.g ``token``,
|
||||
``identity``, etc). Keystone uses the `dogpile.cache`_ library which allows for flexible
|
||||
cache backends. The majority of the caching configuration options are set in the ``[cache]``
|
||||
section. However, each section that has the capability to be cached usually has a ``caching``
|
||||
boolean value that will toggle caching for that specific section. The current default
|
||||
behavior is that subsystem caching is enabled, but the global toggle is set to disabled.
|
||||
|
||||
``[cache]`` configuration section:
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
* ``enabled`` - enables/disables caching across all of keystone
|
||||
* ``debug_cache_backend`` - enables more in-depth logging from the cache backend (get, set, delete, etc)
|
||||
* ``backend`` - the caching backend module to use e.g. ``dogpile.cache.memcache``
|
||||
|
||||
.. NOTE::
|
||||
A given ``backend`` must be registered with ``dogpile.cache`` before it
|
||||
can be used. The default backend is the ``Keystone`` no-op backend
|
||||
(``keystone.common.cache.noop``). If caching is desired a different backend will
|
||||
need to be specified. Current functional backends are:
|
||||
|
||||
* ``dogpile.cache.memcached`` - Memcached backend using the standard `python-memcached`_ library
|
||||
* ``dogpile.cache.pylibmc`` - Memcached backend using the `pylibmc`_ library
|
||||
* ``dogpile.cache.bmemcached`` - Memcached using `python-binary-memcached`_ library.
|
||||
* ``dogpile.cache.redis`` - `Redis`_ backend
|
||||
* ``dogpile.cache.dbm`` - local DBM file backend
|
||||
* ``dogpile.cache.memory`` - in-memory cache
|
||||
|
||||
.. WARNING::
|
||||
``dogpile.cache.memory`` is not suitable for use outside of testing or
|
||||
small workloads as it does not cleanup it's internal cache on cache
|
||||
expiration and does not share cache between processes. This means
|
||||
that caching and cache invalidation will not be consistent or reliable
|
||||
when using ``Keystone`` under HTTPD or similar configurations.
|
||||
|
||||
* ``expiration_time`` - int, the default length of time to cache a specific value. A value of ``0``
|
||||
indicates to not cache anything. It is recommended that the ``enabled`` option be used to disable
|
||||
cache instead of setting this to ``0``.
|
||||
* ``backend_argument`` - an argument passed to the backend when instantiated
|
||||
``backend_argument`` should be specified once per argument to be passed to the
|
||||
back end and in the format of ``<argument name>:<argument value>``.
|
||||
e.g.: ``backend_argument = host:localhost``
|
||||
* ``proxies`` - comma delimited list of `ProxyBackends`_ e.g. ``my.example.Proxy, my.example.Proxy2``
|
||||
* ``use_key_mangler`` - Use a key-mangling function (sha1) to ensure fixed length cache-keys.
|
||||
This is toggle-able for debugging purposes, it is highly recommended to always
|
||||
leave this set to True. If the cache backend provides a key-mangler, this
|
||||
option has no effect.
|
||||
|
||||
For more information about the different backends (and configuration options):
|
||||
* `dogpile.cache.backends.memory`_
|
||||
* `dogpile.cache.backends.memcached`_
|
||||
* `dogpile.cache.backends.redis`_
|
||||
* `dogpile.cache.backends.file`_
|
||||
|
||||
.. _`dogpile.cache`: http://dogpilecache.readthedocs.org/en/latest/
|
||||
.. _`python-memcached`: http://www.tummy.com/software/python-memcached/
|
||||
.. _`pylibmc`: http://sendapatch.se/projects/pylibmc/index.html
|
||||
.. _`python-binary-memcached`: https://github.com/jaysonsantos/python-binary-memcached
|
||||
.. _`Redis`: http://redis.io/
|
||||
.. _`dogpile.cache.backends.memory`: http://dogpilecache.readthedocs.org/en/latest/api.html#memory-backend
|
||||
.. _`dogpile.cache.backends.memcached`: http://dogpilecache.readthedocs.org/en/latest/api.html#memcached-backends
|
||||
.. _`dogpile.cache.backends.redis`: http://dogpilecache.readthedocs.org/en/latest/api.html#redis-backends
|
||||
.. _`dogpile.cache.backends.file`: http://dogpilecache.readthedocs.org/en/latest/api.html#file-backends
|
||||
.. _`ProxyBackends`: http://dogpilecache.readthedocs.org/en/latest/api.html#proxy-backends
|
||||
|
||||
|
||||
Certificates for PKI
|
||||
--------------------
|
||||
|
||||
|
@ -259,6 +259,86 @@ Now you can get a translated error response::
|
||||
}
|
||||
|
||||
|
||||
Caching Layer
|
||||
-------------
|
||||
|
||||
The caching layer is designed to be applied to any ``manager`` object within Keystone
|
||||
via the use of the ``on_arguments`` decorator provided in the ``keystone.common.cache``
|
||||
module. This decorator leverages `dogpile.cache`_ caching system to provide a flexible
|
||||
caching backend.
|
||||
|
||||
It is recommended that each of the managers have an independent toggle within the config
|
||||
file to enable caching. The easiest method to utilize the toggle within the
|
||||
configuration file is to define a ``caching`` boolean option within that manager's
|
||||
configuration section (e.g. ``identity``). Once that option is defined you can
|
||||
pass function to the ``on_arguments`` decorator with the named argument ``should_cache_fn``.
|
||||
In the ``keystone.common.cache`` module, there is a function called ``should_cache_fn``,
|
||||
which will provide a reference, to a function, that will consult the global cache
|
||||
``enabled`` option as well as the specific manager's caching enable toggle.
|
||||
|
||||
.. NOTE::
|
||||
If a section-specific boolean option is not defined in the config section specified when
|
||||
calling ``should_cache_fn``, the returned function reference will default to enabling
|
||||
caching for that ``manager``.
|
||||
|
||||
Example use of cache and ``should_cache_fn`` (in this example, ``token`` is the manager)::
|
||||
|
||||
from keystone.common import cache
|
||||
SHOULD_CACHE = cache.should_cache_fn('token')
|
||||
|
||||
@cache.on_arguments(should_cache_fn=SHOULD_CACHE)
|
||||
def cacheable_function(arg1, arg2, arg3):
|
||||
...
|
||||
return some_value
|
||||
|
||||
With the above example, each call to the ``cacheable_function`` would check to see if
|
||||
the arguments passed to it matched a currently valid cached item. If the return value
|
||||
was cached, the caching layer would return the cached value; if the return value was
|
||||
not cached, the caching layer would call the function, pass the value to the ``SHOULD_CACHE``
|
||||
function reference, which would then determine if caching was globally enabled and enabled
|
||||
for the ``token`` manager. If either caching toggle is disabled, the value is returned but
|
||||
not cached.
|
||||
|
||||
It is recommended that each of the managers have an independent configurable time-to-live (TTL).
|
||||
If a configurable TTL has been defined for the manager configuration section, it is possible to
|
||||
pass it to the ``cache.on_arguments`` decorator with the named-argument ``expiration_time``. For
|
||||
consistency, it is recommended that this option be called ``cache_time`` and default to ``None``.
|
||||
If the ``expiration_time`` argument passed to the decorator is set to ``None``, the expiration
|
||||
time will be set to the global default (``expiration_time`` option in the ``[cache]``
|
||||
configuration section.
|
||||
|
||||
Example of using a section specific ``cache_time`` (in this example, ``identity`` is the manager)::
|
||||
|
||||
from keystone.common import cache
|
||||
SHOULD_CACHE = cache.should_cache_fn('identity')
|
||||
|
||||
@cache.on_arguments(should_cache_fn=SHOULD_CACHE,
|
||||
expiration_time=CONF.identity.cache_time)
|
||||
def cachable_function(arg1, arg2, arg3):
|
||||
...
|
||||
return some_value
|
||||
|
||||
For cache invalidation, the ``on_arguments`` decorator will add an ``invalidate`` method
|
||||
(attribute) to your decorated function. To invalidate the cache, you pass the same arguments
|
||||
to the ``invalidate`` method as you would the normal function.
|
||||
|
||||
Example (using the above cacheable_function)::
|
||||
|
||||
def invalidate_cache(arg1, arg2, arg3):
|
||||
cacheable_function.invalidate(arg1, arg2, arg3)
|
||||
|
||||
.. WARNING::
|
||||
The ``on_arguments`` decorator does not accept keyword-arguments/named arguments. An
|
||||
exception will be raised if keyword arguments are passed to a caching-decorated function.
|
||||
|
||||
.. NOTE::
|
||||
In all cases methods work the same as functions except if you are attempting to invalidate
|
||||
the cache on a decorated bound-method, you need to pass ``self`` to the ``invalidate``
|
||||
method as the first argument before the arguments.
|
||||
|
||||
.. _`dogpile.cache`: http://dogpilecache.readthedocs.org/
|
||||
|
||||
|
||||
Building the Documentation
|
||||
==========================
|
||||
|
||||
|
@ -160,6 +160,47 @@
|
||||
# mode e.g. kerberos or x509 to require binding to that authentication.
|
||||
# enforce_token_bind = permissive
|
||||
|
||||
[cache]
|
||||
# Global cache functionality toggle.
|
||||
# enabled = False
|
||||
|
||||
# Prefix for building the configuration dictionary for the cache region. This
|
||||
# should not need to be changed unless there is another dogpile.cache region
|
||||
# with the same configuration name
|
||||
# config_prefix = cache.keystone
|
||||
|
||||
# Default TTL, in seconds, for any cached item in the dogpile.cache region.
|
||||
# This applies to any cached method that doesn't have an explicit cache
|
||||
# expiration time defined for it.
|
||||
# expiration_time = 600
|
||||
|
||||
# Dogpile.cache backend module. It is recommended that Memcache
|
||||
# (dogpile.cache.memcache) or Redis (dogpile.cache.redis) be used in production
|
||||
# deployments. Small workloads (single process) like devstack can use the
|
||||
# dogpile.cache.memory backend.
|
||||
# backend = keystone.common.cache.noop
|
||||
|
||||
# Arguments supplied to the backend module. Specify this option once per
|
||||
# argument to be passed to the dogpile.cache backend.
|
||||
# Example format: <argname>:<value>
|
||||
# backend_argument =
|
||||
|
||||
# Proxy Classes to import that will affect the way the dogpile.cache backend
|
||||
# functions. See the dogpile.cache documentation on changing-backend-behavior.
|
||||
# Comma delimited list e.g. my.dogpile.proxy.Class, my.dogpile.proxyClass2
|
||||
# proxies =
|
||||
|
||||
# Use a key-mangling function (sha1) to ensure fixed length cache-keys. This
|
||||
# is toggle-able for debugging purposes, it is highly recommended to always
|
||||
# leave this set to True.
|
||||
# use_key_mangler = True
|
||||
|
||||
# Extra debugging from the cache backend (cache keys, get/set/delete/etc calls)
|
||||
# This is only really useful if you need to see the specific cache-backend
|
||||
# get/set/delete calls with the keys/values. Typically this should be left
|
||||
# set to False.
|
||||
# debug_cache_backend = False
|
||||
|
||||
[policy]
|
||||
# driver = keystone.policy.backends.sql.Policy
|
||||
|
||||
|
17
keystone/common/cache/__init__.py
vendored
Normal file
17
keystone/common/cache/__init__.py
vendored
Normal file
@ -0,0 +1,17 @@
|
||||
# vim: tabstop=4 shiftwidth=4 softtabstop=4
|
||||
|
||||
# Copyright 2013 Metacloud
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from keystone.common.cache.core import * # flake8: noqa
|
15
keystone/common/cache/backends/__init__.py
vendored
Normal file
15
keystone/common/cache/backends/__init__.py
vendored
Normal file
@ -0,0 +1,15 @@
|
||||
# vim: tabstop=4 shiftwidth=4 softtabstop=4
|
||||
|
||||
# Copyright 2013 Metacloud
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
51
keystone/common/cache/backends/noop.py
vendored
Normal file
51
keystone/common/cache/backends/noop.py
vendored
Normal file
@ -0,0 +1,51 @@
|
||||
# vim: tabstop=4 shiftwidth=4 softtabstop=4
|
||||
|
||||
# Copyright 2013 Metacloud
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from dogpile.cache import api
|
||||
|
||||
|
||||
NO_VALUE = api.NO_VALUE
|
||||
|
||||
|
||||
class NoopCacheBackend(api.CacheBackend):
|
||||
"""A no op backend as a default caching backend.
|
||||
|
||||
The no op backend is provided as the default caching backend for keystone
|
||||
to ensure that ``dogpile.cache.memory`` is not used in any real-world
|
||||
circumstances unintentionally. ``dogpile.cache.memory`` does not have a
|
||||
mechanism to cleanup it's internal dict and therefore could cause run-away
|
||||
memory utilization.
|
||||
"""
|
||||
def __init__(self, *args):
|
||||
return
|
||||
|
||||
def get(self, key):
|
||||
return NO_VALUE
|
||||
|
||||
def get_multi(self, keys):
|
||||
return [NO_VALUE for x in keys]
|
||||
|
||||
def set(self, key, value):
|
||||
return
|
||||
|
||||
def set_multi(self, mapping):
|
||||
return
|
||||
|
||||
def delete(self, key):
|
||||
return
|
||||
|
||||
def delete_multi(self, keys):
|
||||
return
|
183
keystone/common/cache/core.py
vendored
Normal file
183
keystone/common/cache/core.py
vendored
Normal file
@ -0,0 +1,183 @@
|
||||
# vim: tabstop=4 shiftwidth=4 softtabstop=4
|
||||
|
||||
# Copyright 2013 Metacloud
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""Keystone Caching Layer Implementation."""
|
||||
|
||||
import dogpile.cache
|
||||
|
||||
from dogpile.cache import proxy
|
||||
from dogpile.cache import util
|
||||
|
||||
from keystone.common import config
|
||||
from keystone import exception
|
||||
from keystone.openstack.common import importutils
|
||||
from keystone.openstack.common import log
|
||||
|
||||
|
||||
CONF = config.CONF
|
||||
LOG = log.getLogger(__name__)
|
||||
REGION = dogpile.cache.make_region()
|
||||
|
||||
on_arguments = REGION.cache_on_arguments
|
||||
|
||||
dogpile.cache.register_backend(
|
||||
'keystone.common.cache.noop',
|
||||
'keystone.common.cache.backends.noop',
|
||||
'NoopCacheBackend')
|
||||
|
||||
|
||||
class DebugProxy(proxy.ProxyBackend):
|
||||
"""Extra Logging ProxyBackend."""
|
||||
def get(self, key):
|
||||
value = self.proxied.get(key)
|
||||
msg = _('CACHE_GET: Key: "%(key)s)" "%(value)s"')
|
||||
LOG.debug(msg % {'key': key, 'value': value})
|
||||
return value
|
||||
|
||||
def get_multi(self, keys):
|
||||
values = self.proxied.get_multi(keys)
|
||||
msg = _('CACHE_GET_MULTI: "%(key)s" "%(value)s"')
|
||||
LOG.debug(msg % {'keys': keys, 'values': values})
|
||||
return values
|
||||
|
||||
def set(self, key, value):
|
||||
msg = _('CACHE_SET: Key: %(key)s Value: %(value)s')
|
||||
LOG.debug(msg % {'key': key, 'value': value})
|
||||
return self.proxied.set(key, value)
|
||||
|
||||
def set_multi(self, keys):
|
||||
LOG.debug(_('CACHE_SET_MULTI: %s') % keys)
|
||||
self.proxied.set_multi(keys)
|
||||
|
||||
def delete(self, key):
|
||||
self.proxied.delete(key)
|
||||
LOG.debug(_('CACHE_DELETE: %s') % key)
|
||||
|
||||
def delete_multi(self, keys):
|
||||
LOG.debug(_('CACHE_DELETE_MULTI: %s') % keys)
|
||||
self.proxied.delete_multi(keys)
|
||||
|
||||
|
||||
def build_cache_config(conf):
|
||||
"""Build the cache region dictionary configuration.
|
||||
|
||||
:param conf: configuration object for keystone
|
||||
:returns: dict
|
||||
"""
|
||||
prefix = conf.cache.config_prefix
|
||||
conf_dict = {}
|
||||
conf_dict['%s.backend' % prefix] = conf.cache.backend
|
||||
conf_dict['%s.expiration_time' % prefix] = conf.cache.expiration_time
|
||||
for argument in conf.cache.backend_argument:
|
||||
try:
|
||||
(argname, argvalue) = argument.split(':', 1)
|
||||
except ValueError:
|
||||
msg = _('Unable to build cache config-key. Expected format '
|
||||
'"<argname>:<value>". Skipping unknown format: %s')
|
||||
LOG.error(msg, argument)
|
||||
continue
|
||||
|
||||
arg_key = '.'.join([prefix, 'arguments', argname])
|
||||
conf_dict[arg_key] = argvalue
|
||||
|
||||
LOG.debug(_('Keystone Cache Config: %s'), conf_dict)
|
||||
|
||||
return conf_dict
|
||||
|
||||
|
||||
def configure_cache_region(conf, region=None):
|
||||
"""Configure a cache region.
|
||||
|
||||
:param conf: global keystone configuration object
|
||||
:param region: optional CacheRegion object, if not provided a new region
|
||||
will be instantiated
|
||||
:raises: exception.ValidationError
|
||||
:returns: dogpile.cache.CacheRegion
|
||||
"""
|
||||
if region is not None and not isinstance(region,
|
||||
dogpile.cache.CacheRegion):
|
||||
raise exception.ValidationError(
|
||||
_('region not type dogpile.cache.CacheRegion'))
|
||||
|
||||
if region is None:
|
||||
region = dogpile.cache.make_region()
|
||||
|
||||
if 'backend' not in region.__dict__:
|
||||
# NOTE(morganfainberg): this is how you tell if a region is configured.
|
||||
# There is a request logged with dogpile.cache upstream to make this
|
||||
# easier / less ugly.
|
||||
|
||||
config_dict = build_cache_config(conf)
|
||||
region.configure_from_config(config_dict,
|
||||
'%s.' % conf.cache.config_prefix)
|
||||
|
||||
if conf.cache.debug_cache_backend:
|
||||
region.wrap(DebugProxy)
|
||||
|
||||
# NOTE(morganfainberg): if the backend requests the use of a
|
||||
# key_mangler, we should respect that key_mangler function. If a
|
||||
# key_mangler is not defined by the backend, use the sha1_mangle_key
|
||||
# mangler provided by dogpile.cache. This ensures we always use a fixed
|
||||
# size cache-key. This is toggle-able for debug purposes; if disabled
|
||||
# this could cause issues with certain backends (such as memcached) and
|
||||
# its limited key-size.
|
||||
if region.key_mangler is None:
|
||||
if CONF.cache.use_key_mangler:
|
||||
region.key_mangler = util.sha1_mangle_key
|
||||
|
||||
for class_path in conf.cache.proxies:
|
||||
# NOTE(morganfainberg): if we have any proxy wrappers, we should
|
||||
# ensure they are added to the cache region's backend. Since
|
||||
# configure_from_config doesn't handle the wrap argument, we need
|
||||
# to manually add the Proxies. For information on how the
|
||||
# ProxyBackends work, see the dogpile.cache documents on
|
||||
# "changing-backend-behavior"
|
||||
cls = importutils.import_class(class_path)
|
||||
LOG.debug(_('Adding cache-proxy \'%s\' to backend.') % class_path)
|
||||
region.wrap(cls)
|
||||
|
||||
return region
|
||||
|
||||
|
||||
def should_cache_fn(section):
|
||||
"""Build a function that returns a config section's caching status.
|
||||
|
||||
For any given driver in keystone that has caching capabilities, a boolean
|
||||
config option for that driver's section (e.g. ``token``) should exist and
|
||||
default to ``True``. This function will use that value to tell the caching
|
||||
decorator if caching for that driver is enabled. To properly use this
|
||||
with the decorator, pass this function the configuration section and assign
|
||||
the result to a variable. Pass the new variable to the caching decorator
|
||||
as the named argument ``should_cache_fn``. e.g.:
|
||||
|
||||
from keystone.common import cache
|
||||
|
||||
SHOULD_CACHE = cache.should_cache_fn('token')
|
||||
|
||||
@cache.on_arguments(should_cache_fn=SHOULD_CACHE)
|
||||
def function(arg1, arg2):
|
||||
...
|
||||
|
||||
:param section: name of the configuration section to examine
|
||||
:type section: string
|
||||
:returns: function reference
|
||||
"""
|
||||
def should_cache(value):
|
||||
if not CONF.cache.enabled:
|
||||
return False
|
||||
conf_group = getattr(CONF, section)
|
||||
return getattr(conf_group, 'caching', True)
|
||||
return should_cache
|
@ -71,6 +71,25 @@ FILE_OPTIONS = {
|
||||
cfg.StrOpt('provider', default=None),
|
||||
cfg.StrOpt('driver',
|
||||
default='keystone.token.backends.sql.Token')],
|
||||
'cache': [
|
||||
cfg.StrOpt('config_prefix', default='cache.keystone'),
|
||||
cfg.IntOpt('expiration_time', default=600),
|
||||
# NOTE(morganfainberg): the dogpile.cache.memory acceptable in devstack
|
||||
# and other such single-process/thread deployments. Running
|
||||
# dogpile.cache.memory in any other configuration has the same pitfalls
|
||||
# as the KVS token backend. It is recommended that either Redis or
|
||||
# Memcached are used as the dogpile backend for real workloads. To
|
||||
# prevent issues with the memory cache ending up in "production"
|
||||
# unintentionally, we register a no-op as the keystone default caching
|
||||
# backend.
|
||||
cfg.StrOpt('backend', default='keystone.common.cache.noop'),
|
||||
cfg.BoolOpt('use_key_mangler', default=True),
|
||||
cfg.MultiStrOpt('backend_argument', default=[]),
|
||||
cfg.ListOpt('proxies', default=[]),
|
||||
# Global toggle for all caching using the should_cache_fn mechanism.
|
||||
cfg.BoolOpt('enabled', default=False),
|
||||
# caching backend specific debugging.
|
||||
cfg.BoolOpt('debug_cache_backend', default=False)],
|
||||
'ssl': [
|
||||
cfg.BoolOpt('enable', default=False),
|
||||
cfg.StrOpt('certfile',
|
||||
|
@ -20,6 +20,7 @@ import routes
|
||||
from keystone import assignment
|
||||
from keystone import auth
|
||||
from keystone import catalog
|
||||
from keystone.common import cache
|
||||
from keystone.common import dependency
|
||||
from keystone.common import wsgi
|
||||
from keystone import config
|
||||
@ -39,6 +40,9 @@ CONF = config.CONF
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Ensure the cache is configured and built before we instantiate the managers
|
||||
cache.configure_cache_region(CONF, cache.REGION)
|
||||
|
||||
# Ensure that the identity driver is created before the assignment manager.
|
||||
# The default assignment driver is determined by the identity driver, so the
|
||||
# identity driver must be available to the assignment manager.
|
||||
|
@ -35,6 +35,7 @@ environment.use_eventlet()
|
||||
|
||||
from keystone import assignment
|
||||
from keystone import catalog
|
||||
from keystone.common import cache
|
||||
from keystone.common import dependency
|
||||
from keystone.common import kvs
|
||||
from keystone.common import sql
|
||||
@ -276,6 +277,16 @@ class TestCase(NoModule, unittest.TestCase):
|
||||
|
||||
setattr(self, manager_name, manager.Manager())
|
||||
|
||||
# NOTE(morganfainberg): ensure the cache region is setup. It is safe
|
||||
# to call configure_cache_region on the same region multiple times.
|
||||
# The region wont be configured more than one time.
|
||||
cache.configure_cache_region(CONF, cache.REGION)
|
||||
|
||||
# Invalidate all cache between tests. This should probably be extended
|
||||
# to purge the cache_region's backend as well to avoid odd memory bloat
|
||||
# during a given test sequence since we use dogpile.cache.memory.
|
||||
cache.REGION.invalidate()
|
||||
|
||||
dependency.resolve_future_dependencies()
|
||||
|
||||
def load_fixtures(self, fixtures):
|
||||
|
@ -112,6 +112,7 @@ class MemcacheClient(object):
|
||||
class MemcacheToken(test.TestCase, test_backend.TokenTests):
|
||||
def setUp(self):
|
||||
super(MemcacheToken, self).setUp()
|
||||
self.load_backends()
|
||||
fake_client = MemcacheClient()
|
||||
self.token_man = token.Manager()
|
||||
self.token_man.driver = token_memcache.Token(client=fake_client)
|
||||
|
96
keystone/tests/test_cache.py
Normal file
96
keystone/tests/test_cache.py
Normal file
@ -0,0 +1,96 @@
|
||||
# vim: tabstop=4 shiftwidth=4 softtabstop=4
|
||||
|
||||
# Copyright 2013 Metacloud
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from dogpile.cache import api
|
||||
from dogpile.cache import proxy
|
||||
|
||||
from keystone.common import cache
|
||||
from keystone.common import config
|
||||
from keystone.tests import core as test
|
||||
|
||||
|
||||
CONF = config.CONF
|
||||
NO_VALUE = api.NO_VALUE
|
||||
SHOULD_CACHE = cache.should_cache_fn('cache')
|
||||
|
||||
|
||||
class TestProxy(proxy.ProxyBackend):
|
||||
def get(self, key):
|
||||
value = self.proxied.get(key)
|
||||
if value != NO_VALUE:
|
||||
value[0].cached = True
|
||||
return value
|
||||
|
||||
|
||||
class TestProxyValue(object):
|
||||
def __init__(self, value):
|
||||
self.value = value
|
||||
self.cached = False
|
||||
|
||||
|
||||
class CacheRegionTest(test.TestCase):
|
||||
def setUp(self):
|
||||
super(CacheRegionTest, self).setUp()
|
||||
self.region = cache.configure_cache_region(CONF)
|
||||
self.region.wrap(TestProxy)
|
||||
|
||||
def test_region_built_with_proxy_direct_cache_test(self):
|
||||
"""Verify cache regions are properly built with proxies."""
|
||||
test_value = TestProxyValue('Direct Cache Test')
|
||||
self.region.set('cache_test', test_value)
|
||||
cached_value = self.region.get('cache_test')
|
||||
self.assertTrue(cached_value.cached)
|
||||
|
||||
def test_cache_region_no_error_multiple_config(self):
|
||||
"""Verify configuring the CacheRegion again doesn't error."""
|
||||
cache.configure_cache_region(CONF, self.region)
|
||||
|
||||
def test_should_cache_fn(self):
|
||||
"""Verify should_cache_fn generates a sane function."""
|
||||
test_value = TestProxyValue('Decorator Test')
|
||||
|
||||
@self.region.cache_on_arguments(should_cache_fn=SHOULD_CACHE)
|
||||
def cacheable_function(value):
|
||||
return value
|
||||
|
||||
setattr(CONF.cache, 'caching', False)
|
||||
cacheable_function(test_value)
|
||||
cached_value = cacheable_function(test_value)
|
||||
self.assertFalse(cached_value.cached)
|
||||
|
||||
setattr(CONF.cache, 'caching', True)
|
||||
cacheable_function(test_value)
|
||||
cached_value = cacheable_function(test_value)
|
||||
self.assertTrue(cached_value.cached)
|
||||
|
||||
def test_cache_dictionary_config_builder(self):
|
||||
"""Validate we build a sane dogpile.cache dictionary config."""
|
||||
CONF.cache.config_prefix = 'test_prefix'
|
||||
CONF.cache.backend = 'some_test_backend'
|
||||
CONF.cache.expiration_time = 86400
|
||||
CONF.cache.backend_argument = ['arg1:test', 'arg2:test:test',
|
||||
'arg3.invalid']
|
||||
|
||||
config_dict = cache.build_cache_config(CONF)
|
||||
self.assertEquals(
|
||||
config_dict['test_prefix.backend'], CONF.cache.backend)
|
||||
self.assertEquals(
|
||||
config_dict['test_prefix.expiration_time'],
|
||||
CONF.cache.expiration_time)
|
||||
self.assertEquals(config_dict['test_prefix.arguments.arg1'], 'test')
|
||||
self.assertEquals(config_dict['test_prefix.arguments.arg2'],
|
||||
'test:test')
|
||||
self.assertFalse('test_prefix.arguments.arg3' in config_dict)
|
@ -14,6 +14,10 @@ driver = keystone.trust.backends.kvs.Trust
|
||||
[token]
|
||||
driver = keystone.token.backends.kvs.Token
|
||||
|
||||
[cache]
|
||||
backend = dogpile.cache.memory
|
||||
enabled = True
|
||||
|
||||
[oauth1]
|
||||
driver = keystone.contrib.oauth1.backends.kvs.OAuth1
|
||||
|
||||
|
@ -18,3 +18,4 @@ python-keystoneclient>=0.3.0
|
||||
oslo.config>=1.1.0
|
||||
Babel>=0.9.6
|
||||
oauth2
|
||||
dogpile.cache>=0.5.0
|
||||
|
Loading…
Reference in New Issue
Block a user