The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
Add common parent owner check for the resources which introduced by
service plugin.
Then port forwarding resource will share the same tenant_id with
floatingip. That means only the fip owner can create/update/get/delete
the associated port forwarding resource.
Partially-Implements: blueprint port-forwarding
Partial-Bug: #1491317
Change-Id: I450c674e55ca15e1d9a6a6224138f3305427da68
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
be presented.
Depends-On: Ic3ab5b3ffdc378d570678b9c967cb42b0c7a8a9b
Depends-On: I4397df1c35463a8b532afdc9c5d28b37224a37b4
Depends-On: I3f2e6e861adaeef81a1a5819a57b28f5c6281d80
Depends-On: I1189bc9a50308df5c7e18c329f3a1262c90b9e12
Depends-On: I057cd917628c77dd20c0ff7747936c3fec7b4844
Depends-On: I0b24a304cc3466a2c05426cdbb6f9d99f1797edd
Change-Id: I21bf8a752813802822fd9966dda6ab3b6c4abfdc
Partial-Bug: #1749820
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with https://review.openstack.org/#/c/557040/
This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib
NeutronLibImpact
Change-Id: I1feb842d3e0e92c945efb01ece29856335a398fe
Handle an edge case that API users send a POST/PUT request with
invalid data in request body.
Closes-Bug: #1780327
Change-Id: I3877c18a18ac506dc8f4a9ded2a18b53b9f6cfae
Today we shim the RESOURCE_ATTRIBUTE_MAP in neutron; it references the
equivelant in neutron-lib named RESOURCES.
This patch removes neutron's RESOURCE_ATTRIBUTE_MAP and cleans up
neutron.api.v2.attributes in prep to delete it.
To do so:
- CORE_RESOURCES and RESOURCE_FOREIGN_KEYS are moved to the single
module that references them respectively and the are made private (no
consumers use them).
- get_collection_info is removed and instead the 2 uses in neutron
just use the get() method of the RESOURCES map. There are no external
uses of get_collection_info.
- References using RESOURCE_ATTRIBUTE_MAP are switched over to
neutron-lib's RESOURCES.
- The neutron.api.v2.attributes module is removed as it's empty now.
- A few api attribute UTs are removed; there's nothing to test as per
this patch.
NeutronLibImpact
Change-Id: Iaacee584d499c4d33d6d2dd9609c7ac0f2cfc386
The SHARED constant was rehomed into neutron-lib with commit
If123044496b54a8bfc115ab41b8506ec5d7ba5a3
This patch removes the SHARED constant from neutron.
NeutronLibImpact
Change-Id: I9304ca7bdec6698e549c294c124c60230b6b9eaf
This patch switches callbacks over to the payload object style events
[1] for BEFORE_RESPONSE and AFTER_REQUEST based notifications. To do
so an APIEventPayload object is used with the publish() method to
pass along the API related data. In addition a few UTs are updated to
work with the changes.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Ibd8559e0db9dcc995abf8937a0cb764b21a18531
The core resource API definitions are in neutron-lib and are already
setup in a RESOURCES map, similar to neutron's global
RESOURCE_ATTRIBUTE_MAP. However, a number of consumers directly use
RESOURCE_ATTRIBUTE_MAP and moreover can perform some interesting
operations on it while mockin" for tests. For that reason this
patch proposes we phase in the use of neutron's RESOURCES map rather
than a rip and replace of RESOURCE_ATTRIBUTE_MAP.
This patch removes the API definitions for the core neutron resources
by updating neutron's global RESOURCE_ATTRIBUTE_MAP to reference lib's
RESOURCE map. In addition the AttributeMapMemento class is removed; it
no longer servers a purpose and neutron-lib's fixture should be used
in its place.
This patch should be safe to merge without worry of impacting consumers.
No one is using AttributeMapMemento [1] and the global map only changed
its reference value.
[1] http://codesearch.openstack.org/?q=AttributeMapMemento
Change-Id: Ib04fddcbd4465074452b71b16befa4e33b27259e
Idfd956fa213e1ba6c3b1c983f246c5b515951e6d rehomed the FAULT_MAP into
neutron-lib. This patch consumes it from lib and removes its definition
from neutron.
NeutronLibImpact
Change-Id: Ib051555ea8496ed7eec8ed30733c964012e31c32
Idfd956fa213e1ba6c3b1c983f246c5b515951e6d rehomed the FAULT_MAP into
neutron-lib. Since consumers expect to access the latest faults in this
map which are updated by neutron at runtime, we can't just move
consumers over to lib's version (it won't be updated by neutron).
Therefore this patch shims neutron's FAULT_MAP to use lib's. Once
this shim lands consumers can move to lib's FAULT_MAP (neutron will
ref it via shim) and finally we can then consume in neutron and remove
in a subsequent patch.
The shim herein shouldn't impact consumers so should be free to land
without worry of their impact.
Change-Id: I8e79f29206ec244d870e19f99c18ccaaa50b4761
This code is no longer used now that we have switched to
Pecan and have no option to switch to the legacy API
framework.
Change-Id: I0701dfe13024132cf5b9696560b0d027712e7739
This will drop Pecan in place of the old APIRouter for all
unit tests. This will significantly increase the UT coverage
of pecan and will allow us to drop the old API routing logic
completely.
The rest of the inline test changes are due to slight plugin loading
differences.
Implements: blueprint wsgi-pecan-switch
Change-Id: I76dc23fb7b96d82b0da50285bd0aac76142e81e5
This option is no longer needed as we have switched to pecan
and will be deleting the legacy API layer.
Implements: blueprint wsgi-pecan-switch
Change-Id: Ib153d75bb10375e048a8417139873bdf9dca8769
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
Use the new constraint interface on the context rather than
setting an ugly attribute.
Depends-On: I6bc2539a1ddbf7990164abeb8bb951ddcb45c993
Related-Bug: #1493714
Change-Id: I9142ca96a40092b2a4c94920c4ded9bbc3a0b35b
neutron-lib now contains the API definitions for neutron's core
resources. This patch removes the constant core resource and collection
variables and uses them from lib. Subsequent patches will consume the
actual core resource attribute definitions.
NeutronLibImpact
Change-Id: Ia9afdf620cd538b2aa420593277d6403a45c996b
Allows posting revision number matching in the If-Match header
so updates/deletes will only be satisfied if the current revision
number of the object matches.
DocImpact: The Neutron API now supports conditional updates to resources
that contain the standard 'revision_number' attribute by
setting the revision_number in an HTTP If-Match header.
APIImpact
Partial-Bug: #1493714
Partially-Implements: blueprint push-notifications
Change-Id: I7d97d6044378eb59cb2c7bdc788dc6c174783299
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
neutron-lib now contains the functionality from neutron's attributes
module. This patch moves the local references over to neturon-lib's
implementation and removes the functionality from the attributes
module.
NeutronLibImpact
Change-Id: I773702ebd4b66d28059cb3ced64b2b477ceff8af
Routes allows for auxiliary format suffix. Sadly it doesn't distinguish
between an actual format suffix (.json) and any other suffix that may be
part of the id. (like for first.second resource tag). To work this
behavior around, we will reattach the 'format' suffix if it is not of a
supported format (json only at the time of writing).
This of course leaves a corner case where there is a tag where .json is
a part of its id. This seems to be a reasonable balance to leave it
unfixed, because an alternative would probably be not backwards
compatible.
Closes-Bug: #1694897
Change-Id: I271107150166f0ee680faaa2e3ca6044cf4e8d4f
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
according to https://wiki.openstack.org/wiki/Python3, now we should avoid
using six.iteritems and replace it with dict.items.
Change-Id: I58a399baa2275f280acc0e6d649f81838648ce5c
Closes-Bug: #1680761
When a user is not authorized to see a given resource, we need to
convert HTTP 403s into HTTP 404s to avoid giving away information
that the resource exists. However, the previous code was being
overaggressive and doing this conversion even in some cases where
the user is allowed to see the resource and really needs to know
that what they were trying to do is forbidden, not be told that the
resource doesn't exist. This fixes that logic to only do the 403
to 404 conversion when truly appropriate.
Change-Id: I7a5b0a9e89c8a71490dd74497794a52489f46cd2
Closes-Bug: 1682621
The legacy framework return a list of resources as the response to:
GET /v2.0/
Current pecan 2.0 controller seems return the same as root controller
content. This patch will make it return the same response as legacy
framework
Change-Id: Icae8c1b417d4d5a65a98eeafe80a26886aa254a3
This sets up the factory methods needed to have paste deploy create the
pecan app and return it. It also changes the legacy factory methods to
conditionally use the pecan factory methods if the web_framework config
option is set to 'pecan'. This way, all deployments of neutron will not
need to change their api-paste.ini files to get pecan toggled on. It
should just happen without notice once pecan becomes the default.
Also, by moving this to be loaded by paste deploy, there is a good chunk of
code that has been removed because it is no longer necessary.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I8b1bbea8d90fdc62715cd8b6738ad955df53d7cd
Tags controller was not being picked up from the extension. This also
allows the usage of collection_methods that tags uses. Also, the tags
extension made use of action_status on a resource, which allowed
overriding the default status codes, so this patch handles action_status
as well.
Change-Id: Ic7f5cfddd30f26737469516235053656f1ba6a24
Remove The following _MAX_LEN constants from
neutron/api/v2/attributes.py and use the corresponding DB field size
constants from neutron_lib.db.constants instead.
NAME_MAX_LEN --> NAME_FIELD_SIZE
TENANT_ID_MAX_LEN --> PROJECT_ID_FIELD_SIZE
DESCRIPTION_MAX_LEN --> DESCRIPTION_FIELD_SIZE
LONG_DESCRIPTION_MAX_LEN --> LONG_DESCRIPTION_FIELD_SIZE
DEVICE_ID_MAX_LEN --> DEVICE_ID_FIELD_SIZE
DEVICE_OWNER_MAX_LEN --> DEVICE_NAME_FIELD_SIZE
In alembic migration scripts, the raw numerical value is used.
For more information, see:
http://lists.openstack.org/pipermail/openstack-dev/2016-October/105789.html
NeutronLibImpact
Change-Id: I734890372584fe27e5d6ec38c0cad2de882ff11c
They were deprecated in Newton. This patch cleans them up.
Note: it does not mean that the features will not work anymore. On the
contrary, now API will consistently sort and paginate for plugins that
honour the relevant sorting/pagination parameters.
Note2: base resource controller still allows to pass
allow_pagination=False and allow_sorting=False parameters to disable the
features if a registered plugin does not support the features yet.
Change-Id: I5fd30b20f645846d9366740372c4815c4b33e2eb
Related-Bug: #1566514
It's written to but never read from!
Related-Bug: #1642426
This patch is required by neutron-lib adoption because it is part of
rehoming neutron/api/v2/attributes.py. All subprojects that reference
PLURALS must be patched to remove the reference.
NeutronLibImpact
Change-Id: Ifdf29b8fc4c824c0ee840c4f51593a5aa8a22127
By changing pecan to store plugins by collection (plural) instead of
resource (singular) we don't need REVERSED_PLURALS and
get_resource_info().
These are not used outside of neutron core.
Consequently, this exposed a bug with some controllers not honoring an
extension's intention of disallowing pagination and sorting. This is
also fixed in this patch to get successful test runs.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I9d529d0a2ad369e2be0a8df3c6f06a6532e8b13d
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
Subprojects have had enough time to switch to neutron-lib for
the deprecated items from these modules.
Change-Id: Ib2f4acf7e0153667389c57a52f2db33a23758b24
Converters and Validators were moved to neutron_lib in change
I6b9079e9e703c6fd75adbed3846e7257685433e8
Change-Id: I37c1e2ee1634a7fb72c401caee9122dadd669a64
In some places, indentation was wrong.
In this patchset, they are fixed.
Also, I did some readibility improvements which I've found
during indentation fixing.
Change-Id: I298b9b0c557c07848c69ed23b0faa3a3147d055d
Update the API to accept project_id in requests and return
project_id in responses.
For now, the API treats tenant_id and project_id equivalently.
It accepts either or both in requests.
It returns both in responses, depending on filters.
We include an extension to indicate that support for project_id
is enabled in the API.
Completes: blueprint keystone-v3
APIImpact: Describe how the Networking API supports Keystone V3.
Co-Authored-By: Henry Gessau <HenryG@gessau.net>
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: I8775aa8a477191ef21e7c3c6da31d098befefc3c
Retrying mutating operations at the API layer caused a
couple of problems. First, when components would call
the core plugin using the neutron manager, they would
have to handle the retriable failures on their own or
undo any work they had done so far and allow retriable
failures to be propagated up to the API. Second, retrying
at the API makes composite operations (e.g. auto allocate,
add_router_interface, etc) painful because they have to
consider if exceptions are retriable before raising
fatal exceptions on failures of core plugin calls.
This patch begins the process of moving them down to the
core operations with a new decorator called
'retry_if_session_inactive', which ensures that the
retry logic isn't triggered if there is an ongoing transaction
since retrying inside of a transaction is normally ineffective.
Follow-up patches apply them to various parts of the code-base.
Additionally, the args and kwargs of the method are automatically
deep copied in retries to ensure that any mangling the methods
do to their arguments don't impact their retriability.
Finally, since we are leaving the API decorators in place for now,
the retry logic will not be triggered by another decorator if an
exception has already been retried. This prevents an exponential
explosion of retries on nested retry decorators.
The ultimate goal will be to get rid of the API decorators entirely
so retries are up to each individual plugin.
Partial-Bug: #1596075
Partial-Bug: #1612798
Change-Id: I7b8a4a105aabfa1b5f5dd7a638099007b0933e66
The deprecation shim created by Doug turns out to be rather useful.
It emits a warning when an global (attribute of a module) is
referenced but that global has been moved to another module.
This update makes the following changes to the shim:
- Rename it to _MovedGlobals to better describe what it is a
debtcollector for.
- Use inspect to get the original reference and to check that
_MovedGlobals is called from the last line of a module.
- Save the old reference automatically in the instance to prevent it
from getting garbage collected.
- Beef up the _moved_global() method for moving/renaming individual
globals, allowing it to move and rename or rename in place.
Change-Id: I868aa4a3129dd05467a103364088efbb86bc5d0f
Boden R