openstack
/
neutron
Code Issues Proposed changes

Add tag mechanism for network resources 

Introduce a generic mechanism to allow the user to set tags
on Neutron resources. This patch adds the function for "network"
resource with tags.

APIImpact
DocImpact: allow users to set tags on network resources

Partial-Implements: blueprint add-tags-to-core-resources
Related-Bug: #1489291
Change-Id: I4d9e80d2c46d07fc22de8015eac4bd3dacf4c03a
This commit is contained in:
Hirofumi Ichihara 2016-03-01 11:05:56 +09:00
parent 0ae3c172ae
commit ec1457dd75
13 changed files with 682 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/source/devref/index.rst
View File

@ -77,6 +77,7 @@ Neutron Internals
address_scopes
openvswitch_firewall
network_ip_availability
tag
Testing
-------

119
doc/source/devref/tag.rst Normal file
View File

@ -0,0 +1,119 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
Convention for heading levels in Neutron devref:
======= Heading 0 (reserved for the title in a document)
------- Heading 1
~~~~~~~ Heading 2
+++++++ Heading 3
''''''' Heading 4
(Avoid deeper levels because they do not render well.)
Add Tags to Neutron Resources
=============================
Tag service plugin allows users to set tags on their resources. Tagging
resources can be used by external systems or any other clients of the Neutron
REST API (and NOT backend drivers).
The following use cases refer to adding tags to networks, but the same
can be applicable to any other Neutron resource:
1) Ability to map different networks in different OpenStack locations
to one logically same network (for Multi site OpenStack)
2) Ability to map Id's from different management/orchestration systems to
OpenStack networks in mixed environments, for example for project Kuryr,
map docker network id to neutron network id
3) Leverage tags by deployment tools
4) allow operators to tag information about provider networks
(e.g. high-bandwith, low-latency, etc)
5) new features like get-me-a-network or a similar port scheduler
could choose a network for a port based on tags
Which Resources
---------------
Tag system uses standardattr mechanism so it's targeting to resources have the
mechanism. In Mitaka, they are networks, ports, routers, floating IPs, security
group, security group rules and subnet pools but now tag system supports
networks only.
Model
-----
Tag is not standalone resource. Tag is always related to existing
resources. The following shows tag model::
+------------------+ +------------------+
| Network | | Tag |
+------------------+ +------------------+
| standard_attr_id +------> | standard_attr_id |
| | | tag |
| | | |
+------------------+ +------------------+
Tag has two columns only and tag column is just string. These tags are
defined per resource. Tag is unique in a resource but it can be
overlapped throughout.
API
---
The following shows basic API for tag. Tag is regarded as a subresource of
resource so API always includes id of resource related to tag.
Add a single tag on a network ::
PUT /v2.0/networks/{network_id}/tags/{tag}
Returns `201 Created`. If the tag already exists, no error is raised, it
just returns the `201 Created` because the `OpenStack Development Mailing List
<http://lists.openstack.org/pipermail/openstack-dev/2016-February/087638.html>`_
discussion told us that PUT should be no issue updating an existing tag.
Replace set of tags on a network ::
PUT /v2.0/networks/{network_id}/tags
with request payload ::
{
'tags': ['foo', 'bar', 'baz']
}
Response ::
{
'tags': ['foo', 'bar', 'baz']
}
Check if a tag exists or not on a network ::
GET /v2.0/networks/{network_id}/tags/{tag}
Remove a single tag on a network ::
DELETE /v2.0/networks/{network_id}/tags/{tag}
Remove all tags on a network ::
DELETE /v2.0/networks/{network_id}/tags
PUT and DELETE for collections are the motivation of `extending the API
framework <https://review.openstack.org/#/c/284519/>`_.

2
neutron/db/migration/alembic_migrations/versions/EXPAND_HEAD
View File

@ -1 +1 @@
31ed664953e6
2f9e956e7532

39
neutron/db/migration/alembic_migrations/versions/mitaka/expand/2f9e956e7532_tag_support.py Normal file
View File

@ -0,0 +1,39 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
"""tag support
Revision ID: 2f9e956e7532
Revises: 31ed664953e6
Create Date: 2016-01-21 08:11:49.604182
"""
# revision identifiers, used by Alembic.
revision = '2f9e956e7532'
down_revision = '31ed664953e6'
from alembic import op
import sqlalchemy as sa
def upgrade():
op.create_table(
'tags',
sa.Column('standard_attr_id', sa.BigInteger(),
sa.ForeignKey('standardattributes.id', ondelete='CASCADE'),
nullable=False, primary_key=True),
sa.Column('tag', sa.String(length=60), nullable=False,
primary_key=True)
)

1
neutron/db/migration/models/head.py
View File

@ -49,6 +49,7 @@ from neutron.db.quota import models # noqa
from neutron.db import rbac_db_models # noqa
from neutron.db import securitygroups_db # noqa
from neutron.db import servicetype_db # noqa
from neutron.db import tag_db # noqa
from neutron.ipam.drivers.neutrondb_ipam import db_models # noqa
from neutron.plugins.ml2.drivers import type_flat # noqa
from neutron.plugins.ml2.drivers import type_geneve # noqa

29
neutron/db/tag_db.py Normal file
View File

@ -0,0 +1,29 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
import sqlalchemy as sa
from sqlalchemy import orm
from neutron.db import model_base
class Tag(model_base.BASEV2):
standard_attr_id = sa.Column(
sa.BigInteger().with_variant(sa.Integer(), 'sqlite'),
sa.ForeignKey(model_base.StandardAttribute.id, ondelete="CASCADE"),
nullable=False, primary_key=True)
tag = sa.Column(sa.String(60), nullable=False, primary_key=True)
standard_attr = orm.relationship(
'StandardAttribute',
backref=orm.backref('tags', lazy='joined', viewonly=True))

207
neutron/extensions/tag.py Normal file
View File

@ -0,0 +1,207 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import abc
import six
from oslo_log import log as logging
import webob.exc
from neutron._i18n import _
from neutron.api import extensions
from neutron.api.v2 import attributes
from neutron.api.v2 import base
from neutron.api.v2 import resource as api_resource
from neutron.common import exceptions
from neutron import manager
from neutron.services import service_base
LOG = logging.getLogger(__name__)
TAG = 'tag'
TAGS = TAG + 's'
MAX_TAG_LEN = 60
TAG_PLUGIN_TYPE = 'TAG'
TAG_SUPPORTED_RESOURCES = {
attributes.NETWORKS: attributes.NETWORK,
# other resources can be added
}
TAG_ATTRIBUTE_MAP = {
TAGS: {'allow_post': False, 'allow_put': False, 'is_visible': True}
}
class TagResourceNotFound(exceptions.NotFound):
message = _("Resource %(resource)s %(resource_id)s could not be found.")
class TagNotFound(exceptions.NotFound):
message = _("Tag %(tag)s could not be found.")
def get_parent_resource_and_id(kwargs):
for key in kwargs:
for resource in TAG_SUPPORTED_RESOURCES:
if key == TAG_SUPPORTED_RESOURCES[resource] + '_id':
return resource, kwargs[key]
return None, None
def validate_tag(tag):
msg = attributes._validate_string(tag, MAX_TAG_LEN)
if msg:
raise exceptions.InvalidInput(error_message=msg)
def validate_tags(body):
if 'tags' not in body:
raise exceptions.InvalidInput(error_message="Invalid tags body.")
msg = attributes.validate_list_of_unique_strings(body['tags'], MAX_TAG_LEN)
if msg:
raise exceptions.InvalidInput(error_message=msg)
class TagController(object):
def __init__(self):
self.plugin = (manager.NeutronManager.get_service_plugins()
[TAG_PLUGIN_TYPE])
def index(self, request, **kwargs):
# GET /v2.0/networks/{network_id}/tags
parent, parent_id = get_parent_resource_and_id(kwargs)
return self.plugin.get_tags(request.context, parent, parent_id)
def show(self, request, id, **kwargs):
# GET /v2.0/networks/{network_id}/tags/{tag}
# id == tag
validate_tag(id)
parent, parent_id = get_parent_resource_and_id(kwargs)
return self.plugin.get_tag(request.context, parent, parent_id, id)
def create(self, request, **kwargs):
# not supported
# POST /v2.0/networks/{network_id}/tags
raise webob.exc.HTTPNotFound("not supported")
def update(self, request, id, **kwargs):
# PUT /v2.0/networks/{network_id}/tags/{tag}
# id == tag
validate_tag(id)
parent, parent_id = get_parent_resource_and_id(kwargs)
return self.plugin.update_tag(request.context, parent, parent_id, id)
def update_all(self, request, body, **kwargs):
# PUT /v2.0/networks/{network_id}/tags
# body: {"tags": ["aaa", "bbb"]}
validate_tags(body)
parent, parent_id = get_parent_resource_and_id(kwargs)
return self.plugin.update_tags(request.context, parent, parent_id,
body)
def delete(self, request, id, **kwargs):
# DELETE /v2.0/networks/{network_id}/tags/{tag}
# id == tag
validate_tag(id)
parent, parent_id = get_parent_resource_and_id(kwargs)
return self.plugin.delete_tag(request.context, parent, parent_id, id)
def delete_all(self, request, **kwargs):
# DELETE /v2.0/networks/{network_id}/tags
parent, parent_id = get_parent_resource_and_id(kwargs)
return self.plugin.delete_tags(request.context, parent, parent_id)
class Tag(extensions.ExtensionDescriptor):
"""Extension class supporting tags."""
@classmethod
def get_name(cls):
return "Tag support"
@classmethod
def get_alias(cls):
return "tag"
@classmethod
def get_description(cls):
return "Enables to set tag on resources."
@classmethod
def get_updated(cls):
return "2016-01-01T00:00:00-00:00"
@classmethod
def get_resources(cls):
"""Returns Ext Resources."""
exts = []
action_status = {'index': 200, 'show': 204, 'update': 201,
'update_all': 200, 'delete': 204, 'delete_all': 204}
controller = api_resource.Resource(TagController(),
base.FAULT_MAP,
action_status=action_status)
collection_methods = {"delete_all": "DELETE",
"update_all": "PUT"}
exts = []
for collection_name, member_name in TAG_SUPPORTED_RESOURCES.items():
parent = {'member_name': member_name,
'collection_name': collection_name}
exts.append(extensions.ResourceExtension(
TAGS, controller, parent,
collection_methods=collection_methods))
return exts
def get_extended_resources(self, version):
if version != "2.0":
return {}
EXTENDED_ATTRIBUTES_2_0 = {}
for collection_name in TAG_SUPPORTED_RESOURCES:
EXTENDED_ATTRIBUTES_2_0[collection_name] = TAG_ATTRIBUTE_MAP
return EXTENDED_ATTRIBUTES_2_0
@six.add_metaclass(abc.ABCMeta)
class TagPluginBase(service_base.ServicePluginBase):
"""REST API to operate the Tag."""
def get_plugin_description(self):
return "Tag support"
def get_plugin_type(self):
return TAG_PLUGIN_TYPE
@abc.abstractmethod
def get_tags(self, context, resource, resource_id):
pass
@abc.abstractmethod
def get_tag(self, context, resource, resource_id, tag):
pass
@abc.abstractmethod
def update_tags(self, context, resource, resource_id, body):
pass
@abc.abstractmethod
def update_tag(self, context, resource, resource_id, tag):
pass
@abc.abstractmethod
def delete_tags(self, context, resource, resource_id):
pass
@abc.abstractmethod
def delete_tag(self, context, resource, resource_id, tag):
pass

1
neutron/plugins/common/constants.py
View File

@ -41,6 +41,7 @@ EXT_TO_SERVICE_MAPPING = {
# Maps default service plugins entry points to their extension aliases
DEFAULT_SERVICE_PLUGINS = {
'auto_allocate': 'auto-allocated-topology',
'tag': 'tag',
}
# Service operation status constants

0
neutron/services/tag/__init__.py Normal file
View File

123
neutron/services/tag/tag_plugin.py Normal file
View File

@ -0,0 +1,123 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
from oslo_db import api as oslo_db_api
from oslo_db import exception as db_exc
from oslo_log import helpers as log_helpers
from oslo_log import log as logging
from sqlalchemy.orm import exc
from neutron.api.v2 import attributes
from neutron.db import api as db_api
from neutron.db import common_db_mixin
from neutron.db import models_v2
from neutron.db import tag_db as tag_model
from neutron.extensions import tag as tag_ext
LOG = logging.getLogger(__name__)
resource_model_map = {
attributes.NETWORKS: models_v2.Network,
# other resources can be added
}
def _extend_tags_dict(plugin, response_data, db_data):
tags = [tag_db.tag for tag_db in db_data.standard_attr.tags]
response_data['tags'] = tags
class TagPlugin(common_db_mixin.CommonDbMixin, tag_ext.TagPluginBase):
"""Implementation of the Neutron Tag Service Plugin."""
supported_extension_aliases = ['tag']
def _get_resource(self, context, resource, resource_id):
model = resource_model_map[resource]
try:
return self._get_by_id(context, model, resource_id)
except exc.NoResultFound:
raise tag_ext.TagResourceNotFound(resource=resource,
resource_id=resource_id)
@log_helpers.log_method_call
def get_tags(self, context, resource, resource_id):
res = self._get_resource(context, resource, resource_id)
tags = [tag_db.tag for tag_db in res.standard_attr.tags]
return dict(tags=tags)
@log_helpers.log_method_call
def get_tag(self, context, resource, resource_id, tag):
res = self._get_resource(context, resource, resource_id)
if not any(tag == tag_db.tag for tag_db in res.standard_attr.tags):
raise tag_ext.TagNotFound(tag=tag)
@log_helpers.log_method_call
@oslo_db_api.wrap_db_retry(
max_retries=db_api.MAX_RETRIES,
exception_checker=lambda e: isinstance(e, db_exc.DBDuplicateEntry))
def update_tags(self, context, resource, resource_id, body):
res = self._get_resource(context, resource, resource_id)
new_tags = set(body['tags'])
old_tags = {tag_db.tag for tag_db in res.standard_attr.tags}
tags_added = new_tags - old_tags
tags_removed = old_tags - new_tags
with context.session.begin(subtransactions=True):
for tag_db in res.standard_attr.tags:
if tag_db.tag in tags_removed:
context.session.delete(tag_db)
for tag in tags_added:
tag_db = tag_model.Tag(standard_attr_id=res.standard_attr_id,
tag=tag)
context.session.add(tag_db)
return body
@log_helpers.log_method_call
def update_tag(self, context, resource, resource_id, tag):
res = self._get_resource(context, resource, resource_id)
if any(tag == tag_db.tag for tag_db in res.standard_attr.tags):
return
try:
with context.session.begin(subtransactions=True):
tag_db = tag_model.Tag(standard_attr_id=res.standard_attr_id,
tag=tag)
context.session.add(tag_db)
except db_exc.DBDuplicateEntry:
pass
@log_helpers.log_method_call
def delete_tags(self, context, resource, resource_id):
res = self._get_resource(context, resource, resource_id)
with context.session.begin(subtransactions=True):
query = context.session.query(tag_model.Tag)
query = query.filter_by(standard_attr_id=res.standard_attr_id)
query.delete()
@log_helpers.log_method_call
def delete_tag(self, context, resource, resource_id, tag):
res = self._get_resource(context, resource, resource_id)
with context.session.begin(subtransactions=True):
query = context.session.query(tag_model.Tag)
query = query.filter_by(tag=tag,
standard_attr_id=res.standard_attr_id)
if not query.delete():
raise tag_ext.TagNotFound(tag=tag)
# support only _apply_dict_extend_functions supported resources
# at the moment.
for resource in resource_model_map:
common_db_mixin.CommonDbMixin.register_dict_extend_funcs(
resource, [_extend_tags_dict])

155
neutron/tests/unit/extensions/test_tag.py Normal file
View File

@ -0,0 +1,155 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from neutron.api import extensions
from neutron.common import config
import neutron.extensions
from neutron.services.tag import tag_plugin
from neutron.tests.unit.db import test_db_base_plugin_v2
extensions_path = ':'.join(neutron.extensions.__path__)
class TestTagApiBase(test_db_base_plugin_v2.NeutronDbPluginV2TestCase):
def setUp(self):
service_plugins = {'TAG': "neutron.services.tag.tag_plugin.TagPlugin"}
super(TestTagApiBase, self).setUp(service_plugins=service_plugins)
plugin = tag_plugin.TagPlugin()
ext_mgr = extensions.PluginAwareExtensionManager(
extensions_path, {'TAG': plugin}
)
app = config.load_paste_app('extensions_test_app')
self.ext_api = extensions.ExtensionMiddleware(app, ext_mgr=ext_mgr)
def _get_resource_tags(self, resource_id):
res = self._show(self.resource, resource_id)
return res[self.member]['tags']
def _put_tag(self, resource_id, tag):
req = self._req('PUT', self.resource, id=resource_id,
subresource='tags', sub_id=tag)
return req.get_response(self.ext_api)
def _put_tags(self, resource_id, tags):
body = {'tags': tags}
req = self._req('PUT', self.resource, data=body, id=resource_id,
subresource='tags')
return req.get_response(self.ext_api)
def _get_tag(self, resource_id, tag):
req = self._req('GET', self.resource, id=resource_id,
subresource='tags', sub_id=tag)
return req.get_response(self.ext_api)
def _delete_tag(self, resource_id, tag):
req = self._req('DELETE', self.resource, id=resource_id,
subresource='tags', sub_id=tag)
return req.get_response(self.ext_api)
def _delete_tags(self, resource_id):
req = self._req('DELETE', self.resource, id=resource_id,
subresource='tags')
return req.get_response(self.ext_api)
def _assertEqualTags(self, expected, actual):
self.assertEqual(set(expected), set(actual))
class TestNetworkTagApi(TestTagApiBase):
resource = 'networks'
member = 'network'
def test_put_tag(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tag(net_id, 'red')
self.assertEqual(201, res.status_int)
tags = self._get_resource_tags(net_id)
self._assertEqualTags(['red'], tags)
res = self._put_tag(net_id, 'blue')
self.assertEqual(201, res.status_int)
tags = self._get_resource_tags(net_id)
self._assertEqualTags(['red', 'blue'], tags)
def test_put_tag_exists(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tag(net_id, 'blue')
self.assertEqual(201, res.status_int)
res = self._put_tag(net_id, 'blue')
self.assertEqual(201, res.status_int)
def test_put_tags(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tags(net_id, ['red', 'green'])
self.assertEqual(200, res.status_int)
tags = self._get_resource_tags(net_id)
self._assertEqualTags(['red', 'green'], tags)
def test_put_tags_replace(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tags(net_id, ['red', 'green'])
self.assertEqual(200, res.status_int)
tags = self._get_resource_tags(net_id)
self._assertEqualTags(['red', 'green'], tags)
res = self._put_tags(net_id, ['blue', 'red'])
self.assertEqual(200, res.status_int)
tags = self._get_resource_tags(net_id)
self._assertEqualTags(['blue', 'red'], tags)
def test_get_tag(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tag(net_id, 'red')
self.assertEqual(201, res.status_int)
res = self._get_tag(net_id, 'red')
self.assertEqual(204, res.status_int)
def test_get_tag_notfound(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tag(net_id, 'red')
self.assertEqual(201, res.status_int)
res = self._get_tag(net_id, 'green')
self.assertEqual(404, res.status_int)
def test_delete_tag(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tags(net_id, ['red', 'green'])
self.assertEqual(200, res.status_int)
res = self._delete_tag(net_id, 'red')
self.assertEqual(204, res.status_int)
tags = self._get_resource_tags(net_id)
self._assertEqualTags(['green'], tags)
def test_delete_tag_notfound(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tags(net_id, ['red', 'green'])
self.assertEqual(200, res.status_int)
res = self._delete_tag(net_id, 'blue')
self.assertEqual(404, res.status_int)
def test_delete_tags(self):
with self.network() as net:
net_id = net['network']['id']
res = self._put_tags(net_id, ['red', 'green'])
self.assertEqual(200, res.status_int)
res = self._delete_tags(net_id)
self.assertEqual(204, res.status_int)
tags = self._get_resource_tags(net_id)
self._assertEqualTags([], tags)

5
releasenotes/notes/add-tags-to-core-resources-b05330a129900609.yaml Normal file
View File

@ -0,0 +1,5 @@
---
prelude: >
Add tag mechanism for network resources
features:
- Users can set tags on their network resources.

1
setup.cfg
View File

@ -78,6 +78,7 @@ neutron.service_plugins =
neutron.services.vpn.plugin.VPNDriverPlugin = neutron_vpnaas.services.vpn.plugin:VPNDriverPlugin
qos = neutron.services.qos.qos_plugin:QoSPlugin
bgp = neutron.services.bgp.bgp_plugin:BgpPlugin
tag = neutron.services.tag.tag_plugin:TagPlugin
flavors = neutron.services.flavors.flavors_plugin:FlavorsPlugin
auto_allocate = neutron.services.auto_allocate.plugin:Plugin
network_ip_availability = neutron.services.network_ip_availability.plugin:NetworkIPAvailabilityPlugin