The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
Add common parent owner check for the resources which introduced by
Then port forwarding resource will share the same tenant_id with
floatingip. That means only the fip owner can create/update/get/delete
the associated port forwarding resource.
Partially-Implements: blueprint port-forwarding
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with https://review.openstack.org/#/c/557040/
This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib
Today we shim the RESOURCE_ATTRIBUTE_MAP in neutron; it references the
equivelant in neutron-lib named RESOURCES.
This patch removes neutron's RESOURCE_ATTRIBUTE_MAP and cleans up
neutron.api.v2.attributes in prep to delete it.
To do so:
- CORE_RESOURCES and RESOURCE_FOREIGN_KEYS are moved to the single
module that references them respectively and the are made private (no
consumers use them).
- get_collection_info is removed and instead the 2 uses in neutron
just use the get() method of the RESOURCES map. There are no external
uses of get_collection_info.
- References using RESOURCE_ATTRIBUTE_MAP are switched over to
- The neutron.api.v2.attributes module is removed as it's empty now.
- A few api attribute UTs are removed; there's nothing to test as per
The SHARED constant was rehomed into neutron-lib with commit
This patch removes the SHARED constant from neutron.
This patch switches callbacks over to the payload object style events
 for BEFORE_RESPONSE and AFTER_REQUEST based notifications. To do
so an APIEventPayload object is used with the publish() method to
pass along the API related data. In addition a few UTs are updated to
work with the changes.
The core resource API definitions are in neutron-lib and are already
setup in a RESOURCES map, similar to neutron's global
RESOURCE_ATTRIBUTE_MAP. However, a number of consumers directly use
RESOURCE_ATTRIBUTE_MAP and moreover can perform some interesting
operations on it while mockin" for tests. For that reason this
patch proposes we phase in the use of neutron's RESOURCES map rather
than a rip and replace of RESOURCE_ATTRIBUTE_MAP.
This patch removes the API definitions for the core neutron resources
by updating neutron's global RESOURCE_ATTRIBUTE_MAP to reference lib's
RESOURCE map. In addition the AttributeMapMemento class is removed; it
no longer servers a purpose and neutron-lib's fixture should be used
in its place.
This patch should be safe to merge without worry of impacting consumers.
No one is using AttributeMapMemento  and the global map only changed
its reference value.
Idfd956fa213e1ba6c3b1c983f246c5b515951e6d rehomed the FAULT_MAP into
neutron-lib. This patch consumes it from lib and removes its definition
Idfd956fa213e1ba6c3b1c983f246c5b515951e6d rehomed the FAULT_MAP into
neutron-lib. Since consumers expect to access the latest faults in this
map which are updated by neutron at runtime, we can't just move
consumers over to lib's version (it won't be updated by neutron).
Therefore this patch shims neutron's FAULT_MAP to use lib's. Once
this shim lands consumers can move to lib's FAULT_MAP (neutron will
ref it via shim) and finally we can then consume in neutron and remove
in a subsequent patch.
The shim herein shouldn't impact consumers so should be free to land
without worry of their impact.
This will drop Pecan in place of the old APIRouter for all
unit tests. This will significantly increase the UT coverage
of pecan and will allow us to drop the old API routing logic
The rest of the inline test changes are due to slight plugin loading
Implements: blueprint wsgi-pecan-switch
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Use the new constraint interface on the context rather than
setting an ugly attribute.
neutron-lib now contains the API definitions for neutron's core
resources. This patch removes the constant core resource and collection
variables and uses them from lib. Subsequent patches will consume the
actual core resource attribute definitions.
Allows posting revision number matching in the If-Match header
so updates/deletes will only be satisfied if the current revision
number of the object matches.
DocImpact: The Neutron API now supports conditional updates to resources
that contain the standard 'revision_number' attribute by
setting the revision_number in an HTTP If-Match header.
Partially-Implements: blueprint push-notifications
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
neutron-lib now contains the functionality from neutron's attributes
module. This patch moves the local references over to neturon-lib's
implementation and removes the functionality from the attributes
Routes allows for auxiliary format suffix. Sadly it doesn't distinguish
between an actual format suffix (.json) and any other suffix that may be
part of the id. (like for first.second resource tag). To work this
behavior around, we will reattach the 'format' suffix if it is not of a
supported format (json only at the time of writing).
This of course leaves a corner case where there is a tag where .json is
a part of its id. This seems to be a reasonable balance to leave it
unfixed, because an alternative would probably be not backwards
The callback modules have been available in neutron-lib since commit 
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need .
When a user is not authorized to see a given resource, we need to
convert HTTP 403s into HTTP 404s to avoid giving away information
that the resource exists. However, the previous code was being
overaggressive and doing this conversion even in some cases where
the user is allowed to see the resource and really needs to know
that what they were trying to do is forbidden, not be told that the
resource doesn't exist. This fixes that logic to only do the 403
to 404 conversion when truly appropriate.
The legacy framework return a list of resources as the response to:
Current pecan 2.0 controller seems return the same as root controller
content. This patch will make it return the same response as legacy
This sets up the factory methods needed to have paste deploy create the
pecan app and return it. It also changes the legacy factory methods to
conditionally use the pecan factory methods if the web_framework config
option is set to 'pecan'. This way, all deployments of neutron will not
need to change their api-paste.ini files to get pecan toggled on. It
should just happen without notice once pecan becomes the default.
Also, by moving this to be loaded by paste deploy, there is a good chunk of
code that has been removed because it is no longer necessary.
Co-Authored-By: Brandon Logan <firstname.lastname@example.org>
Tags controller was not being picked up from the extension. This also
allows the usage of collection_methods that tags uses. Also, the tags
extension made use of action_status on a resource, which allowed
overriding the default status codes, so this patch handles action_status
Remove The following _MAX_LEN constants from
neutron/api/v2/attributes.py and use the corresponding DB field size
constants from neutron_lib.db.constants instead.
NAME_MAX_LEN --> NAME_FIELD_SIZE
TENANT_ID_MAX_LEN --> PROJECT_ID_FIELD_SIZE
DESCRIPTION_MAX_LEN --> DESCRIPTION_FIELD_SIZE
LONG_DESCRIPTION_MAX_LEN --> LONG_DESCRIPTION_FIELD_SIZE
DEVICE_ID_MAX_LEN --> DEVICE_ID_FIELD_SIZE
DEVICE_OWNER_MAX_LEN --> DEVICE_NAME_FIELD_SIZE
In alembic migration scripts, the raw numerical value is used.
For more information, see:
They were deprecated in Newton. This patch cleans them up.
Note: it does not mean that the features will not work anymore. On the
contrary, now API will consistently sort and paginate for plugins that
honour the relevant sorting/pagination parameters.
Note2: base resource controller still allows to pass
allow_pagination=False and allow_sorting=False parameters to disable the
features if a registered plugin does not support the features yet.
It's written to but never read from!
This patch is required by neutron-lib adoption because it is part of
rehoming neutron/api/v2/attributes.py. All subprojects that reference
PLURALS must be patched to remove the reference.
By changing pecan to store plugins by collection (plural) instead of
resource (singular) we don't need REVERSED_PLURALS and
These are not used outside of neutron core.
Consequently, this exposed a bug with some controllers not honoring an
extension's intention of disallowing pagination and sorting. This is
also fixed in this patch to get successful test runs.
Co-Authored-By: Brandon Logan <email@example.com>
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
Partially-implements: blueprint neutron-lib
In some places, indentation was wrong.
In this patchset, they are fixed.
Also, I did some readibility improvements which I've found
during indentation fixing.
Update the API to accept project_id in requests and return
project_id in responses.
For now, the API treats tenant_id and project_id equivalently.
It accepts either or both in requests.
It returns both in responses, depending on filters.
We include an extension to indicate that support for project_id
is enabled in the API.
Completes: blueprint keystone-v3
APIImpact: Describe how the Networking API supports Keystone V3.
Co-Authored-By: Henry Gessau <HenryG@gessau.net>
Co-Authored-By: Akihiro Motoki <firstname.lastname@example.org>
Retrying mutating operations at the API layer caused a
couple of problems. First, when components would call
the core plugin using the neutron manager, they would
have to handle the retriable failures on their own or
undo any work they had done so far and allow retriable
failures to be propagated up to the API. Second, retrying
at the API makes composite operations (e.g. auto allocate,
add_router_interface, etc) painful because they have to
consider if exceptions are retriable before raising
fatal exceptions on failures of core plugin calls.
This patch begins the process of moving them down to the
core operations with a new decorator called
'retry_if_session_inactive', which ensures that the
retry logic isn't triggered if there is an ongoing transaction
since retrying inside of a transaction is normally ineffective.
Follow-up patches apply them to various parts of the code-base.
Additionally, the args and kwargs of the method are automatically
deep copied in retries to ensure that any mangling the methods
do to their arguments don't impact their retriability.
Finally, since we are leaving the API decorators in place for now,
the retry logic will not be triggered by another decorator if an
exception has already been retried. This prevents an exponential
explosion of retries on nested retry decorators.
The ultimate goal will be to get rid of the API decorators entirely
so retries are up to each individual plugin.
The deprecation shim created by Doug turns out to be rather useful.
It emits a warning when an global (attribute of a module) is
referenced but that global has been moved to another module.
This update makes the following changes to the shim:
- Rename it to _MovedGlobals to better describe what it is a
- Use inspect to get the original reference and to check that
_MovedGlobals is called from the last line of a module.
- Save the old reference automatically in the instance to prevent it
from getting garbage collected.
- Beef up the _moved_global() method for moving/renaming individual
globals, allowing it to move and rename or rename in place.