Initial support for versioned driver classes

- Adds a helper to deprecate old driver classes
- Implements the versioned driver for keystone.catalog
- Documents developing drivers

partially implements bp stable-driver-interfaces

Change-Id: I58f6781a4e1256ffeb0cf226140b8be245c32aac
This commit is contained in:
David Stanek 2015-08-28 21:28:22 +00:00
parent b33cbef784
commit c8e2364240
8 changed files with 210 additions and 5 deletions

View File

@ -0,0 +1,130 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
===========================
Developing Keystone Drivers
===========================
A driver, also known as a backend, is an important architectural
component of Keystone. It is an abstraction around the data access
needed by a particular subsystem. This pluggable implementation is not
only how Keystone implements its own data access, but how you can
implement your own!
Each major subsystem (that has data access needs) implements the data access
by using drivers. Some examples of Keystone's drivers:
- :class:`keystone.identity.backends.ldap.Identity`
- :class:`keystone.token.providers.fernet.Provider`
- :class:`keystone.contrib.federation.backends.sql.Federation`
In/Out of Tree
--------------
It's best to start developing your custom driver outside of the Keystone
development process. This means developing it in your own public or private git
repository and not worrying about getting it upstream (for now).
This is better for you because it gives you more freedom and you are not bound
to the strict OpenStack development rules or schedule. You can iterate faster
and take whatever shortcuts you need to get your product out of the door.
This is also good for Keystone because it will limit the amount of drivers
that must be maintained by the team. If the team had to maintain a
driver for each NoSQL DB that deployers want to use in production there
would be less time to make Keystone itself better. Not to mention that
the team would have to start gaining expertise in potentially dozens of
new technologies.
As you'll see below there is no penalty for open sourcing your driver,
on GitHub for example, or even keeping your implementation private. We
use `Setuptools entry points`_ to load your driver from anywhere in the
Python path.
.. _Setuptools entry points: no good resource?
How To Make a Driver
--------------------
The TLDR; steps (and too long didn't write yet):
1. Determine which subsystem you would like write a driver for
2. Subclass the most current version of the driver interface
3. Implement each of the abstract methods for that driver
a. We are currently not documenting the exact input/outputs of the
driver methods. The best approach right now is to use an existing
driver as an example of what data your driver will receive and
what data your driver will be required to return.
b. There is a plan in place to document these APIs in more detail.
4. Register your new driver as an entry point
5. Configure your new driver in ``keystone.conf``
6. Sit back and enjoy!
Driver Versioning
-----------------
In the past the driver class was named ``Driver`` and changes would
sometimes be devastating to developers that depend on our driver
contracts. To help alleviate some of the issues we are now creating
version driver classes, e.g. ``Driver11``.
We'll be supporting the current driver version for at least one version back.
This gives developers a full cycle to update their drivers. Some cases, such
as critical security flaws, may require a change to be introduced that breaks
compatibility. These special cases will be communicated as widely as possible
via the typical OpenStack communication channels.
As new driver interface versions are added old ones will be moved to a
"deprecated" state and will output deprecation messages when used. When a
driver version moves from "deprecated" to "unsupported" it will be
removed from the keystone source tree.
Removing Methods
~~~~~~~~~~~~~~~~
Newer driver interfaces may remove methods that are currently required.
Methods are removed when the are no longer required or invoked by Keystone.
There is no reason why methods removed from the Keystone interface need to be
removed from custom drivers.
Adding Methods
--------------
The most common API changes will be adding method to support new
features. We'll do our best to add methods in a way that is backward
compatible. The new version of the driver will define the new method as
an ``abc.abstractmethod`` that must be implemented by driver
implementations. When possible we'll also go back to our supported drivers and
add the method, with a default implementation.
For example, given a ``thing.DriverV11`` that added a new method
``list_things_by_name()``, we will go back to ``thing.Driver10`` and
implement that method. This is good because in many cases your driver
will just work, but there are a couple of unfortunate side effects.
First if you have already used that method name you will have to rename
your method and cut a new version. Second is that the default
implementation may cause a performance penalty due to its naive
implementation.
Updating Methods
~~~~~~~~~~~~~~~~
We will try not to update existing methods in ways that will break old
driver implementations. That means that:
* we will respect existing parameters and not just delete them. If they are
to be removed we will respect their behavior and deprecate then in older
versions.
* we will add new parameters as optional with backward compatible defaults.

View File

@ -74,6 +74,7 @@ Developers Documentation
:maxdepth: 1
developing
developing_drivers
architecture
middlewarearchitecture
http-api

View File

@ -18,7 +18,7 @@ from keystone.common import driver_hints
from keystone.common import kvs
class Catalog(kvs.Base, catalog.Driver):
class Catalog(kvs.Base, catalog.CatalogDriverV8):
# Public interface
def get_catalog(self, user_id, tenant_id):
return self.db.get('catalog-%s-%s' % (tenant_id, user_id))

View File

@ -86,7 +86,7 @@ class Endpoint(sql.ModelBase, sql.DictBase):
extra = sql.Column(sql.JsonBlob())
class Catalog(catalog.Driver):
class Catalog(catalog.CatalogDriverV8):
# Regions
def list_regions(self, hints):
session = sql.get_session()

View File

@ -278,8 +278,8 @@ class Manager(manager.Manager):
@six.add_metaclass(abc.ABCMeta)
class Driver(object):
"""Interface description for an Catalog driver."""
class CatalogDriverV8(object):
"""Interface description for the Catalog driver."""
def _get_list_limit(self):
return CONF.catalog.list_limit or CONF.list_limit
@ -543,3 +543,6 @@ class Driver(object):
v3_catalog.append(service_v3)
return v3_catalog
Driver = manager.create_legacy_driver(CatalogDriverV8)

View File

@ -104,3 +104,35 @@ class Manager(object):
f = getattr(self.driver, name)
setattr(self, name, f)
return f
def create_legacy_driver(driver_class):
"""Helper function to deprecate the original driver classes.
The keystone.{subsystem}.Driver classes are deprecated in favor of the
new versioned classes. This function creates a new class based on a
versioned class and adds a deprecation message when it is used.
This will allow existing custom drivers to work when the Driver class is
renamed to include a version.
Example usage:
Driver = create_legacy_driver(CatalogDriverV8)
"""
module_name = driver_class.__module__
class_name = driver_class.__name__
class Driver(driver_class):
@versionutils.deprecated(
as_of=versionutils.deprecated.LIBERTY,
what='%s.Driver' % module_name,
in_favor_of='%s.%s' % (module_name, class_name),
remove_in=+2)
def __init__(self, *args, **kwargs):
super(Driver, self).__init__(*args, **kwargs)
return Driver

View File

@ -0,0 +1,39 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import mock
from keystone import catalog
from keystone.common import manager
from keystone.tests import unit
class TestCreateLegacyDriver(unit.BaseTestCase):
@mock.patch('oslo_log.versionutils.report_deprecated_feature')
def test_class_is_properly_deprecated(self, mock_reporter):
Driver = manager.create_legacy_driver(catalog.CatalogDriverV8)
# NOTE(dstanek): I want to subvert the requirement for this
# class to implement all of the abstractmethods.
Driver.__abstractmethods__ = set()
impl = Driver()
details = {
'as_of': 'Liberty',
'what': 'keystone.catalog.core.Driver',
'in_favor_of': 'keystone.catalog.core.CatalogDriverV8',
'remove_in': 'N',
}
mock_reporter.assert_called_with(mock.ANY, mock.ANY, details)
self.assertIsInstance(impl, catalog.CatalogDriverV8)

View File

@ -4729,7 +4729,7 @@ class CatalogTests(object):
region_two['id'],
{'parent_region_id': region_four['id']})
@mock.patch.object(core.Driver,
@mock.patch.object(core.CatalogDriverV8,
"_ensure_no_circle_in_hierarchical_regions")
def test_circular_regions_can_be_deleted(self, mock_ensure_on_circle):
# turn off the enforcement so that cycles can be created for the test