Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
As a complement to the create_port_bulk optimizations that took place in
the Stein cycle, create the ability to have an ML2 plugin optionally
expose an 'update_port_bulk' operation as well. This will permit ML2
plugins to optimize this code path for bulk operations as well.
Common neutron resource(e.g, Port) consists of:
1. Resource Attributes, e.g: Port.mac_address, etc.
2. Standard Attributes, e.g: created_at, and are shared among all
The `sort` opt only supports limited attributes. We need to filter
attributes that are defined with `is_sort_key=True` and it's preferred
to explicitly warn CLI & API users of illegal sort keys rather than
just accept without check, pass forward and then hit a internal error
which's quite confusing.
There are some extreme conditions which will result the unbound
router gateway port. Then all the centralized floating IPs will
not be reachable since the gateway port was set to 4095 tag.
This patch adds the HA status to the router related port
processing code path. If it is HA router, the gateway port
will go to the right HA router processing code branch.
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
The neutron.common.constants.EXT_PARENT_RESOURCE_MAPPING constant lives
in neutron-lib. This patch consumes it by removing it from neutron
and switching all uses over to libs constant.
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Co-Authored-By: Akihiro Motoki <firstname.lastname@example.org>
If l3-agent was restarted by a regular action, such as config change,
package upgrade, manually service restart etc. We should not set the
HA port down during such scenarios. Unless the physical host was
rebooted, aka the VRRP processes were all terminated.
This patch adds a new RPC call during l3 agent init, it will try to
retrieve the HA router count first. And then compare the VRRP process
(keepalived) count and 'neutron-keepalived-state-change' count
with the hosting router count. If the count matches, then that
set HA port to 'DOWN' state action will not be triggered anymore.
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
Currently, the dhcp Provisioning of ports is the crucial bottleneck
of that concurrently boot multiple VM.
The root cause is that these ports will be processed one by one by dhcp
agent when they belong to the same network, And the 'Provisioning complete'
port is still blocked other port's processing in other dhcp agents. The
patch aim to optimize the dispatch strategy of the port cast to agent to
improve the Provisioning process.
In server side, I classify messages to multi levels. Especially, I classify
the port_update_end or port_create_end message to two levels, the high-level
message only cast to one agent, the low-level message cast to all agent. In
agent side I put these messages to `resource_processing_queue`, with the queue,
We can delete `_net_lock` and process these messages in order of priority.
Additonally, I modified the `resource_processing_queue` for my demand. I update
`_queue` from LIST to PriorityQueue in `ExclusiveResourceProcessor`, by this
way, we can sort all message which cached in `ExclusiveResourceProcessor` by
Current DHCP port management in Neutron makes the server to clear the
device_id while the agent is responsible for setting it.
This may cause a potential race condition, for example during network
rescheduling. The server aims to clear the device_id on a DHCP port and
assign the network to another agent while the old agent might just be
taking possession of the port. If the DHCP agent takes possession of the
port (i.e., update port...set the device_id) before the server clears
it, then there is no issue. However, if this happens after the clear
operation by server then the DHCP port would be updated/marked to be
owned by the old agent.
When the new agent takes over the network scheduled to it, it won't be
able to find a port to reuse so that an extra port might need to be
created. This leads to two issues:
1) an extra port is created and never deleted;
2) the extra port creation may fail if there are no available IP
This patch proposes a validation check to prevent an agent from updating
a DHCP port unless the network is bound to that agent.
Co-authored-by: Allain Legacy <Allain.email@example.com>
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
Add common parent owner check for the resources which introduced by
Then port forwarding resource will share the same tenant_id with
floatingip. That means only the fip owner can create/update/get/delete
the associated port forwarding resource.
Partially-Implements: blueprint port-forwarding
This patch implements the plugin.
This patch introduces an new service plugin for port forwarding resources,
named 'pf_plugin', and supports create/update/delete port forwarding
operation towards a free Floating IP.
This patch including some works below:
* Introduces portforwarding extension and the base class of plugin
* Introduces portforwarding plugin, support CRUD port forwarding
* Add the policy of portforwarding
The race issue fix in:
Fip extend port forwarding field addition in:
Partially-Implements: blueprint port-forwarding
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with https://review.openstack.org/#/c/557040/
This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib
The remainder of the neutron.plugins.common.utils were rehomed into
neutron-lib with . This patch consumes them by using the functions
from neutron-lib, and removing the neutron.plugins.common.utils module
all together as it's fully rehomed now.
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8. This ensures future python3 compatibility
and a bit better code styling.
According to , when a network contains more that one IPv4
subnet, they are returned in the 'classless-static-routes'
DHCP option, regardless of whether DHCP is enabled for them
However, the get_active_networks_info() method used for
synchronizing networks after the dhcp agent restarts filters
subnets with "enable_dhcp=True", which differs from the
get_network_info() method. This will block VM access to
other VMs in the dhcp disabled subnets, even though they are
in the same network. This is visible by looking at the "opts"
file before and after a restart.
Change the dhcp agent to ask for all subnets in its
get_active_networks_info() RPC call by adding an
enable_dhcp_filter argument to toggle the behavior, with the
default being True to not break backwards compatibility.
Based on https://review.openstack.org/#/c/352530/ by Quan Tian.
Sorting and filtering will rely on the attributes information. It is
necessary to populate project info before using it to sort/filter.
The port delete events are not synchronized with network rpc events. This
creates a condition which makes it possible for a port delete event to be
processed just before a previously started network query completes.
The problematic order of operations is as follows:
1) a network is scheduled to an agent; a network rpc is sent to the
2) the agent queries the network data from the server
3) while that query is in progress a port on that network is deleted; a
port rpc is sent to the agent
4) that port delete rpc is received before the network query rpc
5) the port delete results in no action because the port was not present
on the agent
6) the network query finishes and adds the port to the cache (even
though the port has already been deleted)
7) some time passes and a new port is configured with the same IP
address as the port that was deleted in (3)
8) the dhcp host file is corrupted with 2 entries for the same IP
9) dhcp queries for the newest port is rejected because of the duplicate
entry in the dhcp host file.
The solution is to add the network_id to the port_delete_end rpc event
so that the _net_lock(network_id) synchronization point can be acquired
so that it is processed serially with other network related events.
To ensure backwards compatibility with newer agents running against older
servers the determination of which network_id value to use in the lock is
handled using a utility that will fallback to the previous mode of operation
whenever the network_id attribute is not present in the *_delete_end RPC
events. That utility can be removed in the future when it is guaranteed
that the network_id attribute will be present in RPC messages from the
Signed-off-by: Allain Legacy <firstname.lastname@example.org>
This will enable users to filter list of results with attributes
with empty value. For example, the request below will list
all unbound ports (unbound ports have blank device_id).
Post-binding information about router ports is missing in results of RPC
calls made by l3 agents. sync_routers code ensures that bindings are
present, however, it does not refresh router objects before returning
them - for RPC clients ports remain unbound before the next sync and
there is no necessary address scope information present to create routes
from fip namespaces to qrouter namespaces.
'Versions.factory()' is deprecated in version 'Queens'
and was marked to be removed 'Rocky'.
This patch removes it and switches to use
In addition, neutron.api.versions is removed
because this deprecated module isn't used anywhere.
The neutron.common.topics module was rehomed into neutron-lib with
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
The is_extension_supported function now lives in neutron-lib. This patch
removes the function from neutron and uses lib's version instead.