From 28512ca56996372d92831ad4ca6dbbc24a2137db Mon Sep 17 00:00:00 2001 From: Vishakha Agarwal Date: Mon, 27 Apr 2020 22:44:21 +0530 Subject: [PATCH] Update caching-layer.rst This patch updates about the implementation of oslo.cache for caching in keystone. Change-Id: I8b5b4d3c8d8b3ae99305cc69657571f5983e1004 --- doc/source/admin/caching-layer.inc | 4 + doc/source/contributor/caching-layer.rst | 150 ++++++----------------- 2 files changed, 42 insertions(+), 112 deletions(-) diff --git a/doc/source/admin/caching-layer.inc b/doc/source/admin/caching-layer.inc index 400b6e869a..4d53df466d 100644 --- a/doc/source/admin/caching-layer.inc +++ b/doc/source/admin/caching-layer.inc @@ -1,5 +1,7 @@ .. -*- rst -*- +.. _caching_layer: + Caching layer ============= @@ -162,6 +164,8 @@ options), see: - `dogpile.cache.dbm `__ +.. _cache_invalidation: + Cache invalidation ------------------ diff --git a/doc/source/contributor/caching-layer.rst b/doc/source/contributor/caching-layer.rst index fe810f3005..22632577e6 100644 --- a/doc/source/contributor/caching-layer.rst +++ b/doc/source/contributor/caching-layer.rst @@ -19,136 +19,62 @@ Caching Layer ============= The caching layer is designed to be applied to any ``manager`` object within Keystone -via the use of the ``on_arguments`` decorator provided in the ``keystone.common.cache`` -module. This decorator leverages `dogpile.cache`_ caching system to provide a flexible -caching backend. +via the use of ``keystone.common.cache`` module. This leverages `oslo.cache`_ caching +system to provide a flexible caching backend. -It is recommended that each of the managers have an independent toggle within the config -file to enable caching. The easiest method to utilize the toggle within the -configuration file is to define a ``caching`` boolean option within that manager's -configuration section (e.g. ``identity``). Once that option is defined you can -pass function to the ``on_arguments`` decorator with the named argument ``should_cache_fn``. -In the ``keystone.common.cache`` module, there is a function called ``should_cache_fn``, -which will provide a reference, to a function, that will consult the global cache -``enabled`` option as well as the specific manager's caching enable toggle. +.. _oslo.cache: https://opendev.org/openstack/oslo.cache - .. NOTE:: - If a section-specific boolean option is not defined in the config section specified when - calling ``should_cache_fn``, the returned function reference will default to enabling - caching for that ``manager``. +The caching can be setup for all or some subsystems. It is recommended that each of the +managers have an independent toggle within the config file to enable caching. The easiest +method to utilize the toggle within the configuration file is to define a ``caching`` +boolean option within that manager's configuration section (e.g. ``identity``). Enable the +global cache ``enabled`` option as well as the specific manager's caching enable toggle in +order to cache that subsystem. -Example use of cache and ``should_cache_fn`` (in this example, ``token`` is the manager): +The `oslo.cache`_ is simple and easy to adopt by any system. See the `usage guide`_ of +it. There are various cache :ref:`backends ` supported by it. Example use of +`oslo.cache`_ in keystone (in this example, ``token`` is the manager): .. code-block:: python from keystone.common import cache - SHOULD_CACHE = cache.should_cache_fn('token') - @cache.on_arguments(should_cache_fn=SHOULD_CACHE) - def cacheable_function(arg1, arg2, arg3): + TOKENS_REGION = cache.create_region(name='tokens') + MEMOIZE_TOKENS = cache.get_memoization_decorator( + group='token', + region=TOKENS_REGION) + + @MEMOIZE_TOKENS + def _validate_token(self, token_id): ... - return some_value + return token + +.. _usage guide: https://docs.openstack.org/oslo.cache/latest/user/usage.html With the above example, each call to the ``cacheable_function`` would check to see if the arguments passed to it matched a currently valid cached item. If the return value was cached, the caching layer would return the cached value; if the return value was -not cached, the caching layer would call the function, pass the value to the ``SHOULD_CACHE`` -function reference, which would then determine if caching was globally enabled and enabled -for the ``token`` manager. If either caching toggle is disabled, the value is returned but -not cached. +not cached, the caching layer would call the function, pass the value to the +``MEMOIZE_TOKEN`` decorator, which would then determine if caching was globally enabled +and enabled for the ``token`` manager. If either caching toggle is disabled, the value +is returned but not cached. -It is recommended that each of the managers have an independent configurable time-to-live (TTL). -If a configurable TTL has been defined for the manager configuration section, it is possible to -pass it to the ``cache.on_arguments`` decorator with the named-argument ``expiration_time``. For -consistency, it is recommended that this option be called ``cache_time`` and default to ``None``. -If the ``expiration_time`` argument passed to the decorator is set to ``None``, the expiration -time will be set to the global default (``expiration_time`` option in the ``[cache]`` -configuration section. +It is recommended that each of the managers have an independent configurable time-to-live +(TTL). The option ``cache_time`` is to be set for every manager under its section in +keystone.conf file. If the ``cache_time`` is set to ``None``, the expiration time will be +set to the global default ``expiration_time`` option in the ``[cache]`` configuration section. +These options are passed to and handled by oslo.cache. -Example of using a section specific ``cache_time`` (in this example, ``identity`` is the manager): +:ref:`Cache invalidation ` can be done if specific cache entries are changed. +Example of invalidating a cache (in this example, ``token`` is the manager): .. code-block:: python - from keystone.common import cache - SHOULD_CACHE = cache.should_cache_fn('identity') - - @cache.on_arguments(should_cache_fn=SHOULD_CACHE, - expiration_time=CONF.identity.cache_time) - def cachable_function(arg1, arg2, arg3): + def invalidate_individual_token_cache(self, token_id): ... - return some_value + self._validate_token.invalidate(self, token_id) -For cache invalidation, the ``on_arguments`` decorator will add an ``invalidate`` method -(attribute) to your decorated function. To invalidate the cache, you pass the same arguments -to the ``invalidate`` method as you would the normal function. +For cache invalidation, there is an ``invalidate`` method (attribute) on the decorated function. +To invalidate the cache, pass the same arguments to the ``invalidate`` method as you would the +normal function. This means you need to pass ``self`` as the first argument. -Example (using the above cacheable_function): - -.. code-block:: python - - def invalidate_cache(arg1, arg2, arg3): - cacheable_function.invalidate(arg1, arg2, arg3) - -.. WARNING:: - The ``on_arguments`` decorator does not accept keyword-arguments/named arguments. An - exception will be raised if keyword arguments are passed to a caching-decorated function. - -.. NOTE:: - In all cases methods work the same as functions except if you are attempting to invalidate - the cache on a decorated bound-method, you need to pass ``self`` to the ``invalidate`` - method as the first argument before the arguments. - -.. _`dogpile.cache`: http://dogpilecache.readthedocs.org/ - - -dogpile.cache based MongoDB (NoSQL) backend -------------------------------------------- - -The ``dogpile.cache`` based MongoDB backend implementation allows for various MongoDB -configurations, e.g., standalone, a replica set, sharded replicas, with or without SSL, -use of TTL type collections, etc. - -Example of typical configuration for MongoDB backend: - -.. code-block:: python - - from dogpile.cache import region - - arguments = { - 'db_hosts': 'localhost:27017', - 'db_name': 'ks_cache', - 'cache_collection': 'cache', - 'username': 'test_user', - 'password': 'test_password', - - # optional arguments - 'son_manipulator': 'my_son_manipulator_impl' - } - - region.make_region().configure('keystone.cache.mongo', - arguments=arguments) - -The optional `son_manipulator` is used to manipulate custom data type while its saved in -or retrieved from MongoDB. If the dogpile cached values contain built-in data types and no -custom classes, then the provided implementation class is sufficient. For further details, refer -http://api.mongodb.org/python/current/examples/custom_type.html#automatic-encoding-and-decoding - -Similar to other backends, this backend can be added via Keystone configuration in -``keystone.conf``:: - - [cache] - # Global cache functionality toggle. - enabled = True - - # Referring to specific cache backend - backend = keystone.cache.mongo - - # Backend specific configuration arguments - backend_argument = db_hosts:localhost:27017 - backend_argument = db_name:ks_cache - backend_argument = cache_collection:cache - backend_argument = username:test_user - backend_argument = password:test_password - -This backend is registered in ``keystone.common.cache.core`` module. So, its usage -is similar to other dogpile caching backends as it implements the same dogpile APIs.