Commit Graph

34 Commits (99f4495c940011293e3cabbb590770dc1e7b6900)

Author SHA1 Message Date
Thomas Morin 605364ad78 Support that an extension extends a sub-resource
The neutron.api.extensions code assumes that the resource to update are
base resources and updates its dictionary. When the resource
is a sub-resource, ie. a {'parent': ... , 'parameters': {..} } dictionary,
the result of the update is that the content of 'parameters' is

The correct thing to do here, in the case where the extended resource is
a sub-resource, is to update the content of parameters with the new/changed

This change also removes a workaround that was made in
the qos-bw-limit-direction extension, and which after the change in API
extension code, is not needed anymore.

Needed-By: I263e1ee6cf4e1a91be91a4a78f4a160f64d33cc6
Change-Id: I4cb61481205c3689c41e62670cec113adb2a0362
Closes-Bug: 1722842
6 years ago
Boden R 277996b68d cleanup unit test usage of api extension maps
Today our unit test code uses various ways to "patch" the global
RESOURCE_ATTRIBUTE_MAP as well as extension specific maps in some cases.
This patch consolidates such patching whereby tests should use neutron's
AttributeMapMemento in their setup() chain (only once) if they update
the global map and they should individually handle backup/restore of per
extension map updates. This change will simplify the code and make it
easier to phase-in API definition usage with neutron-lib where we have
some as API definitions and others not. Longer term the
AttributeMapMemento will be replace with neutron-lib's fixture as we
move all extension maps to API definitions in neutron-lib.

Change-Id: I2586f0b11b107d7f57214a0d65bcf7c38a5f0ebb
6 years ago
Boden R 95f1e03446 use plugin constants from neutron-lib
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.


Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
6 years ago
Boden R 0a1405794f use service type constants from neutron_lib plugins
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.

Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
6 years ago
Viktor Varga c4d65601ed Replaced assertTrue(False) with fail()
In some unit and functional tests self.assertTrue(False) was used
instead of, which might be against readability.

Using assertTrue(False) gives the following message on fail:

  File "C:\Python361\lib\unittest\", line 678, in assertTrue
    raise self.failureException(msg)
    AssertionError: False is not true

After replacing assertTrue(False) with fail():

  File "C:\Python361\lib\unittest\", line 666, in fail
    raise self.failureException(msg)
    AssertionError: None

Although the results are the same (both tests failed), the
message 'False is not true' is unnecessary, and can be omitted
from the log by using fail().

Change-Id: I81e21040fd6a2f9713889912fafd2b19bd056b5a
6 years ago
Kevin Benton 1a055fe01e Stop extension warnings in UTs
Gets rid of these log lines dumped to output when executing a test with

Did not find expected name "Test_portsecurity" in
Did not find expected name "Test_providernet" in
Did not find expected name "Test_quotasv2" in

Change-Id: I6a7326e1ef4097455303b963526fe1d638a2c59c
6 years ago
fpxie d2976d46d0 Replace six.iteritems with dict.items(Part-1)
according to,
now we should avoid using six.iteritems and replace
it with dict.items.

Change-Id: I8753e80b34c0f86cf70aebc3bcbd3392ee933f62
Partial-Bug: #1680761
6 years ago
Boden R a6a344c7b5 Consume ServicePluginBase from neutron-lib
Neutron lib now contains ServicePluginBase [1]. In accordance
with the discussion in [2], this patch:
- Removes ServicePluginBase from neutron and replaces all uses
with neutron-lib.
- Removes PluginInterface; the plugin interface for extensions
is ServicePluginBase; that's what's used by everyone today.
- Updates a few UTs, based on the above.

NB: A subsequent patch will address the WorkerSupportServiceMixin
so its untouched herein.


[1] I2b1131ac53e9bfeb42a92f9ef134be6ff4cfe5a3

Change-Id: I1e8f2ee6a96df9ba07dae37f1a80e61ad20693cc
6 years ago
Gary Kotton ad16d63b26 neutron-lib: use CORE from neutron lib constants
Make use of the constant defined in the neutron-lib project.


Change-Id: I46d48f731b557383d00c0abd5fd582a1c0fb78c1
Partially-implements: blueprint neutron-lib
7 years ago
Armando Migliaccio 17563a802e Adopt neutron-lib plugin directory
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.

Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.

This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.

Usage pattern [1,2] can be translated to [3,4] respectively.

[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()

The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.

Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.

Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.


Partially-implements: blueprint neutron-lib

Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
7 years ago
Armando Migliaccio 720a8b3ff0 Broken extensions should not show up in the extension list
This patch ensures that invalid extensions are not exposed to
the user and thus allow clients that rely on them to fail
fast rather than unexpectedly.

Closes-bug: #1547678

Change-Id: Iaace8a9e41562cb16b8c4b475eea01956eb6aab7
7 years ago
Ihar Hrachyshka 5e0878f476 Added API extensions to detect sorting/pagination features
Those features are available only when allow_sorting and
allow_pagination options are enabled (the current default is False).

They don't depend on plugin support, because when plugins don't
implement them natively, emulated mode is applied by API router itself.
So to make it plugin agnostic, we introduce a way to register custom
per-extension checks to override support detection for cases like that

Now that we have a way to detect support for those features via API,
there is little reason to keep tempest configuration options to enable
those features. Instead, just inspect [network-feature-enabled]
api_extensions option in tempest.conf.

used in a single place only (in allow_sorting/allow_pagination
definitions), removed them and replaced with a literal.

Added first in-tree API tests for /extensions entry point.

DocImpact Update API documentation to cover new extensions.
APIImpact Document the new extensions.
Related-Bug: #1566514
Change-Id: I0aaaa037a8ad52060a68dd75c0a1accc6add238e
7 years ago
Brandon Logan c40403eee0 Make service plugins' get_plugin_type classmethods
Any service plugin that implements the get_plugin_type method should
make it a classmethod.  It should not need to be instantiated to
retrieve the simple constant it returns in every case.

Change-Id: Ia3a1237a5e07169ebc9378b1cd4188085e20d71c
Closes-Bug: #1590117
7 years ago
Victor Stinner 6355173e4c Reuse to_utf8() and jsonutils.dump_as_bytes()
Reactor code:

* Reuse oslo_utils.encodeutils.to_utf8() instead of existing
  isinstance(text, six.text_type) test
* Replace jsonutils.dumps(obj).encode('utf-8') with
* Other minor bytes/Unicode changes

Change-Id: I03b8eff0fd70ab65ac66d6f3221e8ced0a56db17
7 years ago
Kevin Benton 5dacbba701 Add a description field to all standard resources
In order to give users and operators more flexibility in
annotating the purpose of various Neutron resources, this patch
adds a description field limited to 255 chars to all of the
Neutron resources that reference the standard attribute table.
The resource that reference the table are the following:
security_group_rules, security_groups, ports, subnets,
networks, routers, floatingips, subnetpools

This patch adds a description table related to standard attributes
and migrates over the existing security group description to the new
table as well.

Co-Authored-By: James Dempsey <>

DocImpact: Adds a description field to all resources outline in
           commit message.
Closes-Bug: #1483480
Change-Id: I6e1ef53d7aae7d04a5485810cc1db0a8eb125953
7 years ago
Hirofumi Ichihara 0ae3c172ae Make API framework more flexible for various extensions
This patch adds a function to Neutron extension mechanism
so that we can add API methods to "/resources" path rather than
"/resources/action" path.

This patch also allow extentions to control correspondences
between action and status in API request.

Change-Id: I862086f7528583e65d7bee794f011ddff6ae8901
Partial-Implements: blueprint add-tags-to-core-resources
Related-Bug: #1489291
7 years ago
armando-migliaccio aadf2f30f8 Add the ability to load a set of service plugins on startup
Service plugins are a great way of adding functionality in a
cohesive way. Some plugins (e.g. network ip availability or
auto_allocate) extend the capabilities of the Neutron server
by being completely orthogonal to the core plugin, and yet may
be considered an integral part of functionality available in
any Neutron deployment. For this reason, it makes sense to
include them seamlessly in the service plugin loading process.

This patch, in particular, introduces the 'auto_allocate' service
plugin for default loading, as we'd want this feature to be enabled
for Nova to use irrespective of the chosen underlying core plugin.

The feature requires subnetpools, external_net and router, while
the first is part of the core, the others can be plugin specific
and they must be explicitly advertised. That said, they all are
features that any deployment can hardly live without.

DocImpact: The "get-me-a-network" feature simplifies the process
for launching an instance with basic network connectivity (via an
externally connected private tenant network).

Once leveraged by Nova, a tenant/admin is no longer required to
provision networking resources ahead of boot process in order to
successfully launch an instance.

Implements: blueprint get-me-a-network

Change-Id: Ia35e8a946bf0ac0bb085cde46b675d17b0bb2f51
7 years ago
armando-migliaccio b14c06b5ed Fail if required extensions are missing
If the required extensions are missing, we currently log an error
that is going to be practically ignored. That said, the unfulfilled
requirement will most definitely going to lead to other failures,
so we might as well fail fast.

This patch also cleans up some <barf>dns-integration nonsense</barf>
within the ML2 framework: the extension must not be declared statically
as it's being loaded by the extension manager, and this fixes the lousy
unit tests we have to live with. As for the db base plugin, some cleanup
is still overdue, but it will have to be taken care of in a follow-up

Closes-bug: #1538623

Change-Id: Id50eeb52c5d209170042b48821a29af3421c2f5c
7 years ago
Jenkins e1195647d2 Merge "Removes the use of mutables as default args" 8 years ago
Brandon Palm 2503dfb239 Fixed a bunch of typos throughout Neutron
Went through all of the docstrings in Neutron and did
some cleanup.  I'm sure there are bunch more that I have missed.

Change-Id: Ib29d2de1c580880c89ed4fd069e1515d0977a3e7
8 years ago
Gary Kotton 2b7fb6ff08 Removes the use of mutables as default args
Passing mutable objects as default args is a known Python pitfall.
We'd better avoid this. This commit changes mutable default args with
None, then use 'arg = arg or {}', 'arg = arg or []'. For unit code which
doesn't use the args , just set with None. This commit also adds hacking

This code was taken from commit 0bea84ac20fe498bd08f7212a0017196c8cb0812
in Nova.

Change-Id: I36d07cade687690dc02a8f6cc3d70f5d00caf112
Co-Authored-By: ChangBo Guo(gcb) <>
8 years ago
Elena Ezhova 216d2d0b75 Consume sslutils and wsgi modules from oslo.service
sslutils and basic WSGI functionality have been moved to
oslo.service and now Neutron can reuse them.

Marked ssl options that were renamed in oslo.service as

Added a note about possible implications for out-of-tree plugins
to neutron_api.rst

Bumped oslo.service version to 0.9.0.

Related-Bug: #1482633

Depends-On: I0424a6c261fae447dbc25b3abf00258c860a88f5
Change-Id: Ibfdf07e665fcfcd093a0e31274e1a6116706aec2
8 years ago
armando-migliaccio 61121c5f2a Decentralize the managemement of service providers
After the service split, some of the configuration, parsing and
validation was kept in the neutron core; ultimately this needs to
get closer to the services where it belongs.

This patch starts from ProviderConfiguration and ServiceTypeManager
classes, and aims at removing the hard-coded elements, like the list
of known advanced services, so that in the long run we can make
Neutron easier to plug with external services.

Partial-bug: #1473110

Depends-on: I44edcceba37ac58efcc0a53c9d1f835d9530344a
Depends-on: I8924234aadf786801ffc100d7daa27acc145a195
Change-Id: Ia4cad678e6c722ca05821dbdbf05d61523246a86
8 years ago
Kevin Benton 4595899f7f Neutron RBAC API and network support
This adds the new API endpoint to create, update, and delete
role-based access control entries. These entries enable tenants
to grant access to other tenants to perform an action on an object
they do not own.

This was previously done using a single 'shared' flag; however, this
was too coarse because an object would either be private to a tenant
or it would be shared with every tenant.

In addition to introducing the API, this patch also adds support to
for the new entries in Neutron networks. This means tenants can now
share their networks with specific tenants as long as they know the
tenant ID.

This feature is backwards-compatible with the previous 'shared'
attribute in the API. So if a deployer doesn't want this new feature
enabled, all of the RBAC operations can be blocked in policy.json and
networks can still be globally shared in the legacy manner.

Even though this feature is referred to as role-based access control,
this first version only supports sharing networks with specific
tenant IDs because Neutron currently doesn't have integration with
Keystone to handle changes in a tenant's roles/groups/etc.


Change-Id: Ib90e2a931df068f417faf26e9c3780dc3c468867
Partially-Implements: blueprint rbac-networks
8 years ago
Cyril Roelandt 0a4812d6e8 Python 3: fix neutron.tests.unit.api.test_extensions
Change-Id: I4db43d93ae22c4a480aa1d103c8e7cf1427d49db
8 years ago
Cedric Brandily 07d3d69663 Python 3: encode unicode response bodies
WebOb disallows in py3K to set webob.Response.body to a unicode object,
we should encode unicode bodies in such case.

Change-Id: Ie0dc57fbe3ed9b19dac2e958de14387bc4c1a260
Blueprint: neutron-python3
8 years ago
Kevin Benton c5cb76315f Alter unit test to match bug and cleanup ext logic
The unit test for bug #1443342 was only testing that a side effect
leading to the bug didn't occur (comparing object identities). This
patch updates the unit test to fully assert that the bug itself
doesn't occur without checking implementation details.

This also eliminates the branching that led to the original issue by
using setdefault to always return a dict to update.

Related-Bug: #1443342
Change-Id: I6c48681aa49c17bbadffa12bad1eea272c483437
8 years ago
Wei Wang 75859529cf Fix bug that resources in attr_map may point to same object
The assignment of attr_map may cause resources' attr_map point
to a same object. So once one resource's attr_map update, it maybe
affecte others.

Change-Id: Ica2c3082f9579b297f8c6323e04f8fd17c4da222
Closes-Bug: #1443342
8 years ago
Jenkins 043829d4c3 Merge "Use string exception casting everywhere" 8 years ago
Martin Roy fe6654b250 Use string exception casting everywhere
Instead of the deprecated "message" member access,
casting to a string invokes the __str__ method of the exception
that is wired to return the message

Added a test of the failure cases of IpRouteCommand::delete_gateway
because they were missing

Running unit and functional tests locally no longer shows the warning
reported in the bug.

Change-Id: Ia79f526aa973ece1145615d65349f860aa3fd465
Closes-Bug: #1466542
8 years ago
Sean M. Collins ad1c7a35de Remove get_namespace from API extensions
Based on the conversation on the ML.


Closes-Bug: #1464023
Depends-On: 6f900fc429bf24cb31e0d2f149aa732055fd5956
Change-Id: I3c406910991c33cf959c5345d76153eabe3ace2d
8 years ago
Ihar Hrachyshka d0be7bc57f Make pep8 job succeed when /etc/neutron/neutron.conf is not installed
Currently, if /etc/neutron/neutron.conf is not installed in the system,
neutron-db-manage fails in oslo.config code when trying to determine the
default configuration file to use.

Test job should not rely on any contents inside /etc/.

Instead, pass --config-file with test-only configuration explicitly into
the utility.

neutron.conf.test was renamed into neutron.conf since for some reason
oslo.config does not support a name that does not have .conf at its
filename end.

Change-Id: I719829fc83a7b20a49c338aaf1dbef916dcc768c
8 years ago
Cyril Roelandt 3751f9ab34 Python3: use six.iteritems() instead of dict.iteritems()
This also adds a check to neutron/hacking/ that should catch this
error in the future.

Blueprint: neutron-python3
Change-Id: Ie7b833ffa173772d39b85ee3ecaddace18e1274f
8 years ago
Maru Newby 1105782e39 Reorganize unit test tree
This change ensures that the structure of the unit test tree matches
that of the code tree to make it obvious where to find tests for a
given module.  A check is added to the pep8 job to protect against

The plugin test paths are relocated to neutron/tests/unit/plugins
but are otherwise ignored for now.

Change-Id: If307593259139171be21a71c58e3a34bf148cc7f
Partial-Bug: #1440834
8 years ago