boilerplate extension descriptor for api-def

Today we have a number of extensions that must provide
boilerplate/redundant code in their extension class to return
basic API definition attributes such as name, description, alias,
etc.. For example [1].

This patch proposes we can provide most of that boilerplate code
right in a base extension descriptor class; just have extension
sub-classes provide their API definition. With this patch, code such
as [1] gets reduced to a class definition with a single class-level attribute
to specify the API definition for the extension.

For sample usage, have a look at [2] that used as a dummy patch
to test PS4/PS7 of this change with neutron master.

[1] https://review.openstack.org/#/c/421562/2/neutron/extensions/providernet.py@36
[2] https://review.openstack.org/#/c/433929/1/neutron/extensions/providernet.py

Change-Id: I25135b39a1d26c11006bc2a7b6080cdd839f0085

doc/source/devref/api_extensions.rst

@@ -0,0 +1,77 @@
+..
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+
+      Convention for heading levels in Neutron devref:
+      =======  Heading 0 (reserved for the title in a document)
+      -------  Heading 1
+      ~~~~~~~  Heading 2
+      +++++++  Heading 3
+      '''''''  Heading 4
+      (Avoid deeper levels because they do not render well.)
+
+
+API Extensions
+==============
+
+API extensions provide a standardized way of introducing new API functionality.
+While the ``neutron-lib`` project itself does not serve an API, the ``neutron``
+project does and leverages the API extension framework from ``neutron-lib``.
+
+API extensions consist of the following high-level constructs:
+
+- API definitions that specify the extension's static metadata. This metadata
+  includes basic details about the extension such as its name, description,
+  alias, etc. as well as its extended resources/sub-resources and
+  required/optional extensions. These definitions live in the
+  ``neutron_lib.api.definitions`` package.
+- API reference documenting the APIs/resources added/modified by the extension.
+  This documentation is in ``rst`` format and is used to generate the
+  `openstack API reference <https://developer.openstack.org/api-ref/networking/
+  v2/>`_. The API reference lives under the ``api-ref/source/v2``
+  directory of the ``neutron-lib`` project repository.
+- An extension descriptor class that must be defined in an extension directory
+  for ``neutron`` or other sub-project that supports extensions. This concrete
+  class provides the extension's metadata to the API server. These extension
+  classes reside outside of ``neutron-lib``, but leverage the base classes
+  from ``neutron_lib.api.extensions``. For more details see the section below
+  on using neutron-lib's extension classes.
+- The API extension plugin implementation itself. This is the code that
+  implements the extension's behavior and should carry out the operations
+  defined by the extension. This code resides under its respective project
+  repository, not in ``neutron-lib``. For more details see the `neutron api
+  extension dev-ref <https://github.com/openstack/neutron/blob/master/doc/
+  source/devref/api_extensions.rst>`_.
+
+
+Using neutron-lib's base extension classes
+------------------------------------------
+
+The ``neutron_lib.api.extensions`` module provides a set of base extension
+descriptor classes consumers can use to define their extension descriptor(s).
+For those extensions that have an API definition in
+``neutron_lib.api.definitions``, the ``APIExtensionDescriptor`` class can
+be used. For example::
+
+    from neutron_lib.api.definitions import provider_net
+    from neutron_lib.api import extensions
+
+
+    class Providernet(extensions.APIExtensionDescriptor):
+        api_definition = provider_net
+        # nothing else needed if default behavior is acceptable
+
+
+For extensions that do not yet have a definition in
+``neutron_lib.api.definitions``, they can continue to use the
+``ExtensionDescriptor`` as has been done historically.

doc/source/devref/index.rst

@@ -32,6 +32,7 @@ Neutron Lib Internals
 .. toctree::
    :maxdepth: 3
 
+   api_extensions
    api_converters
    api_validators
    callbacks

neutron_lib/api/extensions.py

@@ -16,6 +16,12 @@
 import abc
 import six
 
+from neutron_lib._i18n import _
+from neutron_lib import constants
+
+
+_UNSET = constants.Sentinel()
+
 
 def is_extension_supported(plugin, alias):
     """Validate that the extension is supported.
@@ -92,7 +98,6 @@ class ExtensionDescriptor(object):
         map[<resource_name>][<attribute_name>][<attribute_property>]
         specifying the extended resource attribute properties required
         by that API version.
-
         Extension can add resources and their attr definitions too.
         The returned map can be integrated into RESOURCE_ATTRIBUTE_MAP.
         """
@@ -128,7 +133,6 @@ class ExtensionDescriptor(object):
         An extension can use this method and supplying its own resource
         attribute map in extension_attrs_map argument to extend all its
         attributes that needs to be extended.
-
         If an extension does not implement update_attributes_map, the method
         does nothing and just return.
         """
@@ -147,8 +151,83 @@ class ExtensionDescriptor(object):
         The controllers associated with each instance of
         extensions.ResourceExtension should be a subclass of
         neutron.pecan_wsgi.controllers.utils.NeutronPecanController.
-
         If a resource is defined in both get_resources and get_pecan_resources,
         the resource defined in get_pecan_resources will take precedence.
         """
         return []
+
+
+class APIExtensionDescriptor(ExtensionDescriptor):
+    """Base class that defines the contract for extensions.
+
+    Concrete implementations of API extensions should first provide
+    an API definition in neutron_lib.api.definitions. The API
+    definition module (object reference) can then be specified as a
+    class level attribute on the concrete extension.
+
+    For example::
+
+        from neutron_lib.api.definitions import provider_net
+        from neutron_lib.api import extensions
+
+
+        class Providernet(extensions.APIExtensionDescriptor):
+            api_definition = provider_net
+            # nothing else needed if default behavior is acceptable
+
+
+    If extension implementations need to override the default behavior of
+    this class they can override the respective method directly.
+    """
+    api_definition = _UNSET
+
+    @classmethod
+    def _assert_api_definition(cls, attr=None):
+        if cls.api_definition == _UNSET:
+            raise NotImplementedError(
+                _("Extension module API definition not set."))
+        if attr and getattr(cls.api_definition, attr, _UNSET) == _UNSET:
+            raise NotImplementedError(_("Extension module API definition "
+                                        "does not define '%s'") % attr)
+
+    @classmethod
+    def get_name(cls):
+        """The name of the API definition."""
+        cls._assert_api_definition('NAME')
+        return cls.api_definition.NAME
+
+    @classmethod
+    def get_alias(cls):
+        """The alias for the API definition."""
+        cls._assert_api_definition('ALIAS')
+        return cls.api_definition.ALIAS
+
+    @classmethod
+    def get_description(cls):
+        """Friendly description for the API definition."""
+        cls._assert_api_definition('DESCRIPTION')
+        return cls.api_definition.DESCRIPTION
+
+    @classmethod
+    def get_updated(cls):
+        """The timestamp when the API definition was last updated."""
+        cls._assert_api_definition('UPDATED_TIMESTAMP')
+        return cls.api_definition.UPDATED_TIMESTAMP
+
+    def get_extended_resources(self, version):
+        """Retrieve the resource attribute map for the API definition."""
+        if version == "2.0":
+            self._assert_api_definition('RESOURCE_ATTRIBUTE_MAP')
+            return self.api_definition.RESOURCE_ATTRIBUTE_MAP
+        else:
+            return {}
+
+    def get_required_extensions(self):
+        """Returns the API definition's required extensions."""
+        self._assert_api_definition('REQUIRED_EXTENSIONS')
+        return self.api_definition.REQUIRED_EXTENSIONS
+
+    def get_optional_extensions(self):
+        """Returns the API definition's optional extensions."""
+        self._assert_api_definition('OPTIONAL_EXTENSIONS')
+        return self.api_definition.OPTIONAL_EXTENSIONS

neutron_lib/tests/unit/api/test_extensions.py

@@ -108,3 +108,95 @@ class TestExtensionIsSupported(base.BaseTestCase):
     def test_extension_does_not_exist(self):
         self.assertFalse(extensions.is_extension_supported(self._plugin,
                                                            "gordon"))
+
+
+class TestAPIExtensionDescriptor(base.BaseTestCase):
+
+    # API definition attributes; acts as an API definition module
+    NAME = 'Test API'
+    ALIAS = 'test-api'
+    DESCRIPTION = 'A test API definition'
+    UPDATED_TIMESTAMP = '2017-02-01T10:00:00-00:00'
+    RESOURCE_ATTRIBUTE_MAP = {'ports': {}}
+    REQUIRED_EXTENSIONS = ['l3']
+    OPTIONAL_EXTENSIONS = ['fw']
+
+    def setUp(self):
+        super(TestAPIExtensionDescriptor, self).setUp()
+        self.extn = _APIDefinition()
+        self.empty_extn = _EmptyAPIDefinition()
+
+    def test__assert_api_definition_no_defn(self):
+        self.assertRaises(NotImplementedError,
+                          _NoAPIDefinition._assert_api_definition)
+
+    def test__assert_api_definition_no_attr(self):
+        self.assertRaises(
+            NotImplementedError, self.extn._assert_api_definition, attr='NOPE')
+
+    def test_get_name(self):
+        self.assertEqual(self.NAME, self.extn.get_name())
+
+    def test_get_name_unset(self):
+        self.assertRaises(NotImplementedError, _EmptyAPIDefinition.get_name)
+
+    def test_get_alias(self):
+        self.assertEqual(self.ALIAS, self.extn.get_alias())
+
+    def test_get_alias_unset(self):
+        self.assertRaises(NotImplementedError, _EmptyAPIDefinition.get_alias)
+
+    def test_get_description(self):
+        self.assertEqual(self.DESCRIPTION, self.extn.get_description())
+
+    def test_get_description_unset(self):
+        self.assertRaises(NotImplementedError,
+                          _EmptyAPIDefinition.get_description)
+
+    def test_get_updated(self):
+        self.assertEqual(self.UPDATED_TIMESTAMP, self.extn.get_updated())
+
+    def test_get_updated_unset(self):
+        self.assertRaises(NotImplementedError, _EmptyAPIDefinition.get_updated)
+
+    def test_get_extended_resources_v2(self):
+        self.assertEqual(self.RESOURCE_ATTRIBUTE_MAP,
+                         self.extn.get_extended_resources('2.0'))
+
+    def test_get_extended_resources_v2_unset(self):
+        self.assertRaises(NotImplementedError,
+                          self.empty_extn.get_extended_resources, '2.0')
+
+    def test_get_extended_resources_v1(self):
+        self.assertEqual({}, self.extn.get_extended_resources('1.0'))
+
+    def test_get_extended_resources_v1_unset(self):
+        self.assertEqual({}, self.empty_extn.get_extended_resources('1.0'))
+
+    def test_get_required_extensions(self):
+        self.assertEqual(self.REQUIRED_EXTENSIONS,
+                         self.extn.get_required_extensions())
+
+    def test_get_required_extensions_unset(self):
+        self.assertRaises(NotImplementedError,
+                          self.empty_extn.get_required_extensions)
+
+    def test_get_optional_extensions(self):
+        self.assertEqual(self.OPTIONAL_EXTENSIONS,
+                         self.extn.get_optional_extensions())
+
+    def test_get_optional_extensions_unset(self):
+        self.assertRaises(NotImplementedError,
+                          self.empty_extn.get_optional_extensions)
+
+
+class _APIDefinition(extensions.APIExtensionDescriptor):
+    api_definition = TestAPIExtensionDescriptor
+
+
+class _NoAPIDefinition(extensions.APIExtensionDescriptor):
+    pass
+
+
+class _EmptyAPIDefinition(extensions.APIExtensionDescriptor):
+    api_definition = {}

releasenotes/notes/boilerplate-ext-descriptor-a5cec8b9b900cbfd.yaml

@@ -0,0 +1,6 @@
+---
+features:
+  - The ``APIExtensionDescriptor`` was added to ``neutron_lib.api.extensions``
+    and can be used with extensions that have an API definition in neutron-lib
+    to minimize the boilplate code needed in the extension definition class.
+    For more details, see the dev-ref.