Rehome neutron.api.v2.attributes

This patch rehomes a bulk of neutron.api.v2.attributes into lib
with the following notable aspects:
- A new AttributeInfo class is available and wraps a resource's
attribute dict to provide operations with the attributes. The
class can be used as a operational wrapper on a dict of attrs, or
it an be used as a transport of attrs based on the consumer's need.
- A global singleton attribute map RESOURCES is available and
is initially populated with core resources.
- Some unit tests are beefed a little and make them more modular.
- Some huge comments into the devref.
- The fixture for API defs is updated to support backing up the
global RESOURCES dict.

Change-Id: Ic455e1af2796e58025381160dc5c3b83217413fa

doc/source/devref/api_attributes.rst

@@ -0,0 +1,90 @@
+..
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+
+      Convention for heading levels in Neutron devref:
+      =======  Heading 0 (reserved for the title in a document)
+      -------  Heading 1
+      ~~~~~~~  Heading 2
+      +++++++  Heading 3
+      '''''''  Heading 4
+      (Avoid deeper levels because they do not render well.)
+
+
+API Attributes
+==============
+
+Neutron's resource attributes are defined in dictionaries
+in ``api/definitions``.
+
+The map containing all installed resources (for core and active extensions)
+is in ``api/attributes.py``.
+
+
+Attribute map structure
+-----------------------
+
+Example attribute definitions for ``dns_name``:
+
+.. code-block:: python
+
+        'dns_name': {
+            'allow_post': True,
+            'allow_put': True,
+            'default': '',
+            'convert_to': convert_to_lowercase,
+            'validate': {'type:dns_name': FQDN_MAX_LEN},
+            'is_visible': True
+        },
+
+
+The ``validate`` item specifies rules for `validating <api_validators.html>`_
+the attribute.
+
+The ``convert_to`` item specifies rules for `converting <api_converters.html>`_
+the attribute.
+
+Example attribute definitions for ``gateway_ip``:
+
+.. code-block:: python
+
+        'gateway_ip': {
+            'allow_post': True,
+            'allow_put': True,
+            'default': constants.ATTR_NOT_SPECIFIED,
+            'validate': {'type:ip_address_or_none': None},
+            'is_visible': True
+        }
+
+Note: a default of ``ATTR_NOT_SPECIFIED`` indicates that an attribute is not
+required, but will be generated by the plugin if it is not specified.
+Particularly, a value of ``ATTR_NOT_SPECIFIED`` is different from an
+attribute that has been specified with a value of ``None``.  For example,
+if ``gateway_ip`` is omitted in a request to create a subnet, the plugin
+will receive ``ATTR_NOT_SPECIFIED`` and the default gateway IP will be
+generated.  However, if ``gateway_ip`` is specified as ``None``, this means
+that the subnet does not have a gateway IP.
+
+The following are the defined keys for attribute maps:
+
+======================  ======
+``default``             default value of the attribute (if missing, the attribute becomes mandatory)
+``allow_post``          the attribute can be used on ``POST`` requests
+``allow_put``           the attribute can be used on ``PUT`` requests
+``validate``            specifies rules for validating data in the attribute
+``convert_to``          transformation to apply to the value before it is returned
+``convert_list_to``     if the value is a list, apply this transformation to the value before it is returned
+``is_visible``          the attribute is returned in ``GET`` responses
+``required_by_policy``  the attribute is required by the policy engine and should therefore be filled by the API layer even if not present in request body
+``enforce_policy``      the attribute is actively part of the policy enforcing mechanism, ie: there might be rules which refer to this attribute
+======================  ======

doc/source/devref/api_validators.rst

@@ -64,7 +64,6 @@ dictionary key for the validator. For example:
 
 ::
 
-  RESOURCE_ATTRIBUTE_MAP = {
       NETWORKS: {
           'id': {'allow_post': False, 'allow_put': False,
                  'validate': {'type:uuid': None},
@@ -72,7 +71,7 @@ dictionary key for the validator. For example:
                  'primary_key': True},
           'name': {'allow_post': True, 'allow_put': True,
                    'validate': {'type:string': NAME_MAX_LEN},
-                   'default': '', 'is_visible': True},
+                   'default': '', 'is_visible': True}}
 
 Here, the networks resource has an 'id' attribute with a UUID validator,
 as seen by the 'validate' key containing a dictionary with a key of
@@ -88,7 +87,7 @@ could have a validator defined as follows:
    'ip_version': {'allow_post': True, 'allow_put': False,
                   'convert_to': conversions.convert_to_int,
                   'validate': {'type:values': [4, 6]},
-                  'is_visible': True},
+                  'is_visible': True}}
 
 Here, the validate_values() method will take the list of values as the
 allowable values that can be specified for this attribute.
@@ -99,4 +98,3 @@ Test The Validator
 Do the right thing, and make sure you've created a unit test for any
 validator that you add to verify that it works as expected, even for
 simple validators.
-

doc/source/devref/index.rst

@@ -32,6 +32,7 @@ Neutron Lib Internals
 .. toctree::
    :maxdepth: 3
 
+   api_attributes
    api_extensions
    api_converters
    api_validators

neutron_lib/api/attributes.py

@@ -13,6 +13,22 @@
 from webob import exc
 
 from neutron_lib._i18n import _
+from neutron_lib.api.definitions import network
+from neutron_lib.api.definitions import port
+from neutron_lib.api.definitions import subnet
+from neutron_lib.api.definitions import subnetpool
+from neutron_lib.api import validators
+from neutron_lib import constants
+from neutron_lib import exceptions
+
+
+def _validate_privileges(context, res_dict):
+    if ('project_id' in res_dict and
+            res_dict['project_id'] != context.project_id and
+            not context.is_admin):
+        msg = _("Specifying 'project_id' or 'tenant_id' other than the "
+                "authenticated project in request requires admin privileges")
+        raise exc.HTTPBadRequest(msg)
 
 
 def populate_project_info(attributes):
@@ -23,12 +39,11 @@ def populate_project_info(attributes):
 
     If neither are present then attributes is not updated.
 
-    :param attributes: a dictionary of resource/API attributes
+    :param attributes: A dictionary of resource/API attributes
-    :type attributes: dict
+    or API request/response dict.
-
+    :returns: attributes (updated with project_id if applicable).
-    :return: the updated attributes dictionary
+    :raises: HTTPBadRequest if the attributes project_id and tenant_id
-    :rtype: dict
+    don't match.
-
     """
     if 'tenant_id' in attributes and 'project_id' not in attributes:
         attributes['project_id'] = attributes['tenant_id']
@@ -41,3 +56,159 @@ def populate_project_info(attributes):
         raise exc.HTTPBadRequest(msg)
 
     return attributes
+
+
+class AttributeInfo(object):
+    """Provides operations on a resource's attribute map.
+
+    AttributeInfo wraps an API resource's attribute dict and provides methods
+    for filling defaults, validating, converting, etc. based on the
+    underlying attributes.
+    """
+
+    def __init__(self, resource_attrs):
+        """Create a new instance that wraps the given resource attributes.
+
+        :param resource_attrs: The resource's attributes that can be any
+        of the following types: an instance of AttributeInfo, an API definition
+        that contains a RESOURCE_ATTRIBUTE_MAP attribute or a dict of
+        attributes for the resource.
+        """
+        if isinstance(resource_attrs, AttributeInfo):
+            resource_attrs = resource_attrs.attributes
+        elif getattr(resource_attrs,
+                     'RESOURCE_ATTRIBUTE_MAP', None) is not None:
+            # handle neutron_lib API definitions
+            resource_attrs = resource_attrs.RESOURCE_ATTRIBUTE_MAP
+
+        self.attributes = resource_attrs
+
+    def fill_post_defaults(
+            self, res_dict,
+            exc_cls=lambda m: exceptions.InvalidInput(error_message=m),
+            check_allow_post=True):
+        """Fill in default values for attributes in a POST request.
+
+        When a POST request is made, the attributes with default values do not
+        need to be specified by the user. This function fills in the values of
+        any unspecified attributes if they have a default value.
+
+        If an attribute is not specified and it does not have a default value,
+        an exception is raised.
+
+        If an attribute is specified and it is not allowed in POST requests, an
+        exception is raised. The caller can override this behavior by setting
+        check_allow_post=False (used by some internal admin operations).
+
+        :param res_dict: The resource attributes from the request.
+        :param exc_cls: Exception to be raised on error that must take
+        a single error message as it's only constructor arg.
+        :param check_allow_post: Raises an exception if a non-POST-able
+        attribute is specified.
+        :raises: exc_cls If check_allow_post is True and this instance of
+        ResourceAttributes doesn't support POST.
+        """
+        for attr, attr_vals in self.attributes.items():
+            if attr_vals['allow_post']:
+                if 'default' not in attr_vals and attr not in res_dict:
+                    msg = _("Failed to parse request. Required "
+                            "attribute '%s' not specified") % attr
+                    raise exc_cls(msg)
+                res_dict[attr] = res_dict.get(attr,
+                                              attr_vals.get('default'))
+            elif check_allow_post:
+                if attr in res_dict:
+                    msg = _("Attribute '%s' not allowed in POST") % attr
+                    raise exc_cls(msg)
+
+    def convert_values(
+            self, res_dict,
+            exc_cls=lambda m: exceptions.InvalidInput(error_message=m)):
+        """Convert and validate attribute values for a request.
+
+        :param res_dict: The resource attributes from the request.
+        :param exc_cls: Exception to be raised on error that must take
+        a single error message as it's only constructor arg.
+        :raises: exc_cls If any errors occur converting/validating the
+        res_dict.
+        """
+        for attr, attr_vals in self.attributes.items():
+            if (attr not in res_dict or
+                    res_dict[attr] is constants.ATTR_NOT_SPECIFIED):
+                continue
+            # Convert values if necessary
+            if 'convert_to' in attr_vals:
+                res_dict[attr] = attr_vals['convert_to'](res_dict[attr])
+            # Check that configured values are correct
+            if 'validate' not in attr_vals:
+                continue
+            for rule in attr_vals['validate']:
+                validator = validators.get_validator(rule)
+                res = validator(res_dict[attr], attr_vals['validate'][rule])
+
+                if res:
+                    msg_dict = dict(attr=attr, reason=res)
+                    msg = _("Invalid input for %(attr)s. "
+                            "Reason: %(reason)s.") % msg_dict
+                    raise exc_cls(msg)
+
+    def populate_project_id(self, context, res_dict, is_create):
+        """Populate the owner information in a request body.
+
+        Ensure both project_id and tenant_id attributes are present.
+        Validate that the requestor has the required privileges.
+        For a create request, copy owner info from context to request body
+        if needed and verify that owner is specified if required.
+
+        :param context: The request context.
+        :param res_dict: The resource attributes from the request.
+        :param attr_info: The attribute map for the resource.
+        :param is_create: Is this a create request?
+        :raises: HTTPBadRequest If neither the project_id nor tenant_id
+        are specified in the res_dict.
+
+        """
+        populate_project_info(res_dict)
+        _validate_privileges(context, res_dict)
+
+        if is_create and 'project_id' not in res_dict:
+            if context.project_id:
+                res_dict['project_id'] = context.project_id
+
+                # For backward compatibility
+                res_dict['tenant_id'] = context.project_id
+
+            elif 'tenant_id' in self.attributes:
+                msg = _("Running without keystone AuthN requires "
+                        "that tenant_id is specified")
+                raise exc.HTTPBadRequest(msg)
+
+    def verify_attributes(self, attrs_to_verify):
+        """Reject unknown attributes.
+
+        Consumers should ensure the project info is populated in the
+        attrs_to_verify before calling this method.
+
+        :param attrs_to_verify: The attributes to verify against this
+        resource attributes.
+        :raises: HTTPBadRequest: If attrs_to_verify contains any unrecognized
+        for this resource attributes instance.
+        """
+        extra_keys = set(attrs_to_verify.keys()) - set(self.attributes.keys())
+        if extra_keys:
+            msg = _("Unrecognized attribute(s) '%s'") % ', '.join(extra_keys)
+            raise exc.HTTPBadRequest(msg)
+
+
+def _core_resource_attributes():
+    resources = {}
+    for core_def in [network.RESOURCE_ATTRIBUTE_MAP,
+                     port.RESOURCE_ATTRIBUTE_MAP,
+                     subnet.RESOURCE_ATTRIBUTE_MAP,
+                     subnetpool.RESOURCE_ATTRIBUTE_MAP]:
+        resources.update(core_def)
+    return resources
+
+
+# populate core resources into singleton global
+RESOURCES = _core_resource_attributes()

neutron_lib/api/definitions/base.py

@@ -123,19 +123,6 @@ KNOWN_EXTENSIONS = (
     'bgpvpn',  # https://git.openstack.org/cgit/openstack/networking-bgpvpn
 )
 
-# The following is a short reference for understanding attribute info:
-# allow_post: the attribute can be used on POST requests.
-# allow_put: the attribute can be used on PUT requests.
-# convert_to: transformation to apply to the value before it is returned
-# default: default value of the attribute (if missing, the attribute
-# becomes mandatory.
-# enforce_policy: the attribute is actively part of the policy enforcing
-# mechanism, ie: there might be rules which refer to this attribute.
-# is_visible: the attribute is returned in GET responses.
-# required_by_policy: the attribute is required by the policy engine and
-# should therefore be filled by the API layer even if not present in
-# request body.
-# validate: specifies rules for validating data in the attribute.
 KNOWN_KEYWORDS = (
     'allow_post',
     'allow_put',

neutron_lib/fixture.py

@@ -13,6 +13,7 @@
 import copy
 import fixtures
 
+from neutron_lib.api import attributes
 from neutron_lib.api import definitions
 from neutron_lib.callbacks import manager
 from neutron_lib.callbacks import registry
@@ -98,11 +99,29 @@ class APIDefinitionFixture(fixtures.Fixture):
     This fixture saves and restores 1 or more neutron-lib API definitions
     attribute maps. It should be used anywhere multiple tests can be run
     that might update an extension attribute map.
+
+    In addition the fixture backs up and restores the global attribute
+    RESOURCES base on the boolean value of its backup_global_resources
+    attribute.
     """
 
     def __init__(self, *api_definitions):
-        self.definitions = api_definitions
+        """Create a new instance.
+
+        Consumers can also control if the fixture should handle the global
+        attribute RESOURCE map using the backup_global_resources of the
+        fixture instance. If True the fixture will also handle
+        neutron_lib.api.attributes.RESOURCES.
+
+        :param api_definitions: Zero or more API definitions the fixture
+        should handle. If no api_definitions are passed, the default is
+        to handle all neutron_lib API definitions as well as the global
+        RESOURCES attribute map.
+        """
+        self.definitions = api_definitions or definitions._ALL_API_DEFINITIONS
         self._orig_attr_maps = {}
+        self._orig_resources = {}
+        self.backup_global_resources = True
 
     def _setUp(self):
         for api_def in self.definitions:
@@ -110,12 +129,17 @@ class APIDefinitionFixture(fixtures.Fixture):
                 api_def, api_def.RESOURCE_ATTRIBUTE_MAP)
             api_def.RESOURCE_ATTRIBUTE_MAP = copy.deepcopy(
                 api_def.RESOURCE_ATTRIBUTE_MAP)
+        if self.backup_global_resources:
+            for resource, attrs in attributes.RESOURCES.items():
+                self._orig_resources[resource] = copy.deepcopy(attrs)
         self.addCleanup(self._restore)
 
     def _restore(self):
         for alias, def_and_map in self._orig_attr_maps.items():
             api_def, attr_map = def_and_map[0], def_and_map[1]
             api_def.RESOURCE_ATTRIBUTE_MAP = attr_map
+        if self.backup_global_resources:
+            attributes.RESOURCES = self._orig_resources
 
     @classmethod
     def all_api_definitions_fixture(cls):

neutron_lib/tests/unit/api/test_attributes.py

@@ -10,15 +10,24 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
+from oslo_utils import uuidutils
+import testtools
 from webob import exc
 
 from neutron_lib.api import attributes
-from oslo_utils import uuidutils
+from neutron_lib.api import converters
+from neutron_lib.api.definitions import network
+from neutron_lib.api.definitions import port
+from neutron_lib.api.definitions import subnet
+from neutron_lib.api.definitions import subnetpool
+from neutron_lib import constants
+from neutron_lib import context
+from neutron_lib import exceptions
 
 from neutron_lib.tests import _base as base
 
 
-class TestApiUtils(base.BaseTestCase):
+class TestPopulateProjectInfo(base.BaseTestCase):
 
     def test_populate_project_info_add_project(self):
         attrs_in = {'tenant_id': uuidutils.generate_uuid()}
@@ -47,3 +56,216 @@ class TestApiUtils(base.BaseTestCase):
         }
         self.assertRaises(exc.HTTPBadRequest,
                           attributes.populate_project_info, attrs)
+
+
+class TestAttributeInfo(base.BaseTestCase):
+
+    class _MyException(Exception):
+        pass
+
+    _EXC_CLS = _MyException
+    _RESOURCE_NAME = 'thing'
+    _RESOURCE_ATTRS = {'name': {}, 'type': {}}
+    _RESOURCE_MAP = {_RESOURCE_NAME: _RESOURCE_ATTRS}
+    _ATTRS_INSTANCE = attributes.AttributeInfo(_RESOURCE_MAP)
+
+    def test_create_from_attribute_info_instance(self):
+        cloned_attrs = attributes.AttributeInfo(
+            TestAttributeInfo._ATTRS_INSTANCE)
+
+        self.assertEqual(TestAttributeInfo._ATTRS_INSTANCE.attributes,
+                         cloned_attrs.attributes)
+
+    def test_create_from_api_def(self):
+        self.assertEqual(
+            port.RESOURCE_ATTRIBUTE_MAP,
+            attributes.AttributeInfo(port.RESOURCE_ATTRIBUTE_MAP).attributes)
+
+    def _test_fill_default_value(self, attr_inst, expected, res_dict,
+                                 check_allow_post=True):
+        attr_inst.fill_post_defaults(
+            res_dict, check_allow_post=check_allow_post)
+        self.assertEqual(expected, res_dict)
+
+    def test_fill_default_value_ok(self):
+        attr_info = {
+            'key': {
+                'allow_post': True,
+                'default': constants.ATTR_NOT_SPECIFIED,
+            },
+        }
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self._test_fill_default_value(attr_inst, {'key': 'X'}, {'key': 'X'})
+        self._test_fill_default_value(
+            attr_inst, {'key': constants.ATTR_NOT_SPECIFIED}, {})
+
+    def test_override_no_allow_post(self):
+        attr_info = {
+            'key': {
+                'allow_post': False,
+                'default': constants.ATTR_NOT_SPECIFIED,
+            },
+        }
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self._test_fill_default_value(attr_inst, {'key': 'X'}, {'key': 'X'},
+                                      check_allow_post=False)
+
+    def test_fill_no_default_value_allow_post(self):
+        attr_info = {
+            'key': {
+                'allow_post': True,
+            },
+        }
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self._test_fill_default_value(attr_inst, {'key': 'X'}, {'key': 'X'})
+        self.assertRaises(exceptions.InvalidInput,
+                          self._test_fill_default_value,
+                          attr_inst, {'key': 'X'}, {})
+        self.assertRaises(self._EXC_CLS, attr_inst.fill_post_defaults,
+                          {}, self._EXC_CLS)
+
+    def test_fill_no_default_value_no_allow_post(self):
+        attr_info = {
+            'key': {
+                'allow_post': False,
+            },
+        }
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self.assertRaises(exceptions.InvalidInput,
+                          self._test_fill_default_value,
+                          attr_inst, {'key': 'X'}, {'key': 'X'})
+        self._test_fill_default_value(attr_inst, {}, {})
+        self.assertRaises(self._EXC_CLS, attr_inst.fill_post_defaults,
+                          {'key': 'X'}, self._EXC_CLS)
+
+    def _test_convert_value(self, attr_inst, expected, res_dict):
+        attr_inst.convert_values(res_dict)
+        self.assertEqual(expected, res_dict)
+
+    def test_convert_value(self):
+        attr_info = {
+            'key': {
+            },
+        }
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self._test_convert_value(attr_inst,
+                                 {'key': constants.ATTR_NOT_SPECIFIED},
+                                 {'key': constants.ATTR_NOT_SPECIFIED})
+        self._test_convert_value(attr_inst, {'key': 'X'}, {'key': 'X'})
+        self._test_convert_value(attr_inst,
+                                 {'other_key': 'X'}, {'other_key': 'X'})
+
+        attr_info = {
+            'key': {
+                'convert_to': converters.convert_to_int,
+            },
+        }
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self._test_convert_value(attr_inst,
+                                 {'key': constants.ATTR_NOT_SPECIFIED},
+                                 {'key': constants.ATTR_NOT_SPECIFIED})
+        self._test_convert_value(attr_inst, {'key': 1}, {'key': '1'})
+        self._test_convert_value(attr_inst, {'key': 1}, {'key': 1})
+        self.assertRaises(exceptions.InvalidInput, self._test_convert_value,
+                          attr_inst, {'key': 1}, {'key': 'a'})
+
+        attr_info = {
+            'key': {
+                'validate': {'type:uuid': None},
+            },
+        }
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self._test_convert_value(attr_inst,
+                                 {'key': constants.ATTR_NOT_SPECIFIED},
+                                 {'key': constants.ATTR_NOT_SPECIFIED})
+        uuid_str = '01234567-1234-1234-1234-1234567890ab'
+        self._test_convert_value(attr_inst,
+                                 {'key': uuid_str}, {'key': uuid_str})
+        self.assertRaises(exceptions.InvalidInput, self._test_convert_value,
+                          attr_inst, {'key': 1}, {'key': 1})
+        self.assertRaises(self._EXC_CLS, attr_inst.convert_values,
+                          {'key': 1}, self._EXC_CLS)
+
+    def test_populate_project_id_admin_req(self):
+        tenant_id_1 = uuidutils.generate_uuid()
+        tenant_id_2 = uuidutils.generate_uuid()
+        # non-admin users can't create a res on behalf of another project
+        ctx = context.Context(user_id=None, tenant_id=tenant_id_1)
+        res_dict = {'tenant_id': tenant_id_2}
+        attr_inst = attributes.AttributeInfo({})
+        self.assertRaises(exc.HTTPBadRequest,
+                          attr_inst.populate_project_id,
+                          ctx, res_dict, None)
+        # but admin users can
+        ctx.is_admin = True
+        attr_inst.populate_project_id(ctx, res_dict, is_create=False)
+
+    def test_populate_project_id_from_context(self):
+        tenant_id = uuidutils.generate_uuid()
+        ctx = context.Context(user_id=None, tenant_id=tenant_id)
+        # for each create request, the tenant_id should be added to the
+        # req body
+        res_dict = {}
+        attr_inst = attributes.AttributeInfo({})
+        attr_inst.populate_project_id(ctx, res_dict, is_create=True)
+        self.assertEqual(
+            {'tenant_id': ctx.tenant_id, 'project_id': ctx.tenant_id},
+            res_dict)
+
+    def test_populate_project_id_mandatory_not_specified(self):
+        tenant_id = uuidutils.generate_uuid()
+        ctx = context.Context(user_id=None, tenant_id=tenant_id)
+        # if the tenant_id is mandatory for the resource and not specified
+        # in the request nor in the context, an exception should be raised
+        res_dict = {}
+        attr_info = {'tenant_id': {'allow_post': True}}
+        ctx.tenant_id = None
+        attr_inst = attributes.AttributeInfo(attr_info)
+        self.assertRaises(exc.HTTPBadRequest,
+                          attr_inst.populate_project_id,
+                          ctx, res_dict, True)
+
+    def test_populate_project_id_not_mandatory(self):
+        ctx = context.Context(user_id=None)
+        # if the tenant_id is not mandatory for the resource it should be
+        # OK if it is not in the request.
+        res_dict = {'name': 'test_port'}
+        attr_inst = attributes.AttributeInfo({})
+        ctx.tenant_id = None
+        attr_inst.populate_project_id(ctx, res_dict, True)
+
+    def test_verify_attributes_null(self):
+        attributes.AttributeInfo({}).verify_attributes({})
+
+    def test_verify_attributes_ok_with_project_id(self):
+        attributes.AttributeInfo(
+            {'tenant_id': 'foo', 'project_id': 'foo'}).verify_attributes(
+            {'tenant_id': 'foo'})
+
+    def test_verify_attributes_ok_subset(self):
+        attributes.AttributeInfo(
+            {'attr1': 'foo', 'attr2': 'bar'}).verify_attributes(
+            {'attr1': 'foo'})
+
+    def test_verify_attributes_unrecognized(self):
+        with testtools.ExpectedException(exc.HTTPBadRequest) as bad_req:
+            attributes.AttributeInfo(
+                {'attr1': 'foo'}).verify_attributes(
+                {'attr1': 'foo', 'attr2': 'bar'})
+            self.assertEqual(bad_req.message,
+                             "Unrecognized attribute(s) 'attr2'")
+
+
+class TestCoreResources(base.BaseTestCase):
+
+    CORE_DEFS = [network, port, subnet, subnetpool]
+
+    def test_core_resource_names(self):
+        self.assertEqual(
+            sorted([r.COLLECTION_NAME for r in TestCoreResources.CORE_DEFS]),
+            sorted(attributes.RESOURCES.keys()))
+
+    def test_core_resource_attrs(self):
+        for r in TestCoreResources.CORE_DEFS:
+            self.assertIs(r.RESOURCE_ATTRIBUTE_MAP[r.COLLECTION_NAME],
+                          attributes.RESOURCES[r.COLLECTION_NAME])

neutron_lib/tests/unit/test_fixture.py

@@ -16,10 +16,12 @@ from oslo_config import cfg
 from oslo_db import options
 from oslotest import base
 
+from neutron_lib.api import attributes
 from neutron_lib.callbacks import registry
 from neutron_lib.db import model_base
 from neutron_lib import fixture
 from neutron_lib.plugins import directory
+from neutron_lib.tests.unit.api import test_attributes
 
 
 class PluginDirectoryFixtureTestCase(base.BaseTestCase):
@@ -63,27 +65,30 @@ class SqlFixtureTestCase(base.BaseTestCase):
 
 class APIDefinitionFixtureTestCase(base.BaseTestCase):
 
-    _ATTR_MAP_1 = {'routers': {'name': 'a'}}
+    def _test_all_api_definitions_fixture(self, global_cleanup=True):
-    _ATTR_MAP_2 = {'ports': {'description': 'a'}}
+        apis = fixture.APIDefinitionFixture.all_api_definitions_fixture()
+        apis.backup_global_resources = global_cleanup
+        apis.setUp()
 
-    def setUp(self):
+        asserteq = self.assertNotEqual if global_cleanup else self.assertEqual
-        super(APIDefinitionFixtureTestCase, self).setUp()
+        asserteq({}, apis._orig_resources)
-        self.routers_def = mock.Mock()
-        self.routers_def.RESOURCE_ATTRIBUTE_MAP = (
-            APIDefinitionFixtureTestCase._ATTR_MAP_1)
-        self.ports_def = mock.Mock()
-        self.ports_def.RESOURCE_ATTRIBUTE_MAP = (
-            APIDefinitionFixtureTestCase._ATTR_MAP_2)
-        self.useFixture(fixture.APIDefinitionFixture(
-            self.routers_def, self.ports_def))
 
-    def test_fixture(self):
+        for r in test_attributes.TestCoreResources.CORE_DEFS:
-        # assert same contents, but different instances
+            attributes.RESOURCES[r.COLLECTION_NAME]['_test_'] = {}
-        self.assertEqual(APIDefinitionFixtureTestCase._ATTR_MAP_1,
+            r.RESOURCE_ATTRIBUTE_MAP['_test_'] = {}
-                         self.routers_def.RESOURCE_ATTRIBUTE_MAP)
+
-        self.assertEqual(APIDefinitionFixtureTestCase._ATTR_MAP_2,
+        apis.cleanUp()
-                         self.ports_def.RESOURCE_ATTRIBUTE_MAP)
+        for r in test_attributes.TestCoreResources.CORE_DEFS:
-        self.assertIsNot(APIDefinitionFixtureTestCase._ATTR_MAP_1,
+            self.assertNotIn('_test_', r.RESOURCE_ATTRIBUTE_MAP)
-                         self.routers_def.RESOURCE_ATTRIBUTE_MAP)
+            global_assert = (self.assertNotIn
-        self.assertIsNot(APIDefinitionFixtureTestCase._ATTR_MAP_2,
+                             if global_cleanup else self.assertIn)
-                         self.ports_def.RESOURCE_ATTRIBUTE_MAP)
+            global_assert('_test_', attributes.RESOURCES[r.COLLECTION_NAME])
+            # cleanup
+            if not global_cleanup:
+                del attributes.RESOURCES[r.COLLECTION_NAME]['_test_']
+
+    def test_all_api_definitions_fixture_no_global_backup(self):
+        self._test_all_api_definitions_fixture(global_cleanup=False)
+
+    def test_all_api_definitions_fixture_with_global_backup(self):
+        self._test_all_api_definitions_fixture(global_cleanup=True)

releasenotes/notes/core-attributes-43e6969f1b187e5c.yaml

@@ -0,0 +1,14 @@
+---
+features:
+  - A bulk of the ``neutron.api.v2.attributes`` functionality is now available
+    in ``neutron_lib.api.attributes``. A new ``AttributeInfo`` class is
+    available in that acts as a wrapper for an API resource's attribute dict
+    and allows consumers to perform operations with the underlying attribute
+    dict. The ``populate_project_info`` function is now available. The global
+    attribute map ``RESOURCES`` is now available and will take the place of
+    neutron's global ``RESOURCE_ATTRIBUTE_MAP``.
+  - The ``neutron_lib.fixture.APIDefinitionFixture`` has been updated to handle
+    backing-up and restoring the global ``RESOURCES`` dict. By default the
+    constructor now also uses all API definitions if none are passed to it's
+    constructor. This is the default behavior almost all consumers need and is
+    thus a convenience change.