Commit Graph

231 Commits (4fe17646769ffe7fac9223b2d6db260ee04167ff)

Author SHA1 Message Date
Brian Haley 291eabb0b6 Fix some pylint indentation warnings
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/api.


Change-Id: I1258b04f64a18036407e1d9de9ddca7472af0d11
7 months ago
Sahid Orentino Ferdjaoui 7ceb935da8 dhcp/rpc: retrieve network details with segments
When segment plugin is enabled, we should return segments details as
they are part of network.

Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <>
Change-Id: I1dab155bc812f8764d22e78ebb7d80aaaad65515
9 months ago
Slawek Kaplonski cd8bf18150 [L3HA] Don't update HA router's ports if router isn't active on agents
In case when HA router isn't active on any L3 agent,
_ensure_host_set_on_port method shouldn't try to update port's host to
the host from which there was an rpc message sent, as this can be host
on which router is in the "standby" mode.
This method should only update port's host to the router's "active_host"
if there is such active_host found already.


Closes-Bug: #1973162
Closes-Bug: #1942190
Change-Id: Ib3945d294601b35f9b268c25841cd284b52c4ca3
1 year ago
Slawek Kaplonski 8fd88fd223 Pass host parameter to the get_network_info method
Host parameter is needed there to filter subnets per segment when
segments plugin is enabled.
When dhcp agent requests informations about networks, and segments
plugin is enabled, subnets which belongs to the network are filtered out
based on the host passed as argument to the get_network_info() method.
But we never passed host to that method, even when we should e.g.
during the full sync of the DHCP agent, when it requests details about
each network.
This patch fixes that issue by passing host parameter to that method.

Closes-Bug: #1958955
Change-Id: Ib5eef501493f6735a47ea085196242a5807c4565
1 year ago
Slawek Kaplonski 1c1814aa6b Fix reference before assignment error in the dhcp_rpc module
In patch [1] method get_network_info was refactored and that causes
NameError in the DHCP agent when there is "network object passed in
kwargs and there are subnets with segments in network. See related bug
for details.


Closes-Bug: #1958955
Change-Id: Iad8d85c79f8b11a24b1bb1ca44c776e909b610c3
1 year ago
Rodolfo Alonso Hernandez 5710d3407b Improve DHCP RPC handler
Remove unnecessary DB retrieval operations from
"get_active_networks_info" method.

Partial-Bug: #1950662
Change-Id: I4ea7b86e3f544d5dddcdac562208bb8afd1fc36a
1 year ago
Rodolfo Alonso Hernandez c686a2b555 Improve DHCP RPC handler
Remove unnecessary DB retrieval operations from "get_network_info"

Partial-Bug: #1950662
Change-Id: If4b33c8437dba411fed913e7e1c7f06d899c08f7
2 years ago
Nurmatov Mamatisa 92c636d8b2 Use payload callback for RPC resource_cache
This patch switches the code over to the payload style of callbacks [1].


Change-Id: I2e65470e6f37ebccde01bdf3f9ed31b29567398f
2 years ago
Nurmatov Mamatisa 1483b63ffa Add enable_dhcp filter in get_network_info rpc
Add enable_dhcp, to make a filter to avoid unnecessary
net_info data transfer through rpc.

Change-Id: Ibcef366f5b1f4b7da4f47f1f538a17111da0faa1
Closes-Bug: #1552614
2 years ago
LIU Yulong 8e3a83c213 Config option to disable the DHCP functions
This patch adds a new config option ``enable_traditional_dhcp``,
if set False, neutron-server will disable:
* DHCP provisioning block
* DHCP scheduler API extension
* Network scheduling mechanism
* DHCP RPC/notification

Partially-Implements: bp/distributed-dhcp-for-ml2-ovs
Related-Bug: #1900934

Change-Id: Icfbfc9691c5cf837406ff4291b3e3ed4970b26ee
2 years ago
Hang Yang 9f09b1fb19 Support address group in OVS firewall agent
Support security group rules with remote_address_group_id in openvswitch
firewall. This change reuses most of the firewall functions handling remote
security groups to also process remote address groups. The conjunctive flows
for a rule with remote_adress_group_id are similar to others with
remote_group_id but have different conj_ids.

Change-Id: I8c69e62ba56b0d3204e9c12df3133126071b92f7
Implements: blueprint address-groups-in-sg-rules
2 years ago
Zuul f84d2f74f5 Merge "Get sec group ids after address group update" 2 years ago
Miguel Lavalle 92359b6fb9 Get sec group ids after address group update
This change adds code to retrieve for the agent the security group ids
affected by an update or deletion of an address group.

Also adds event notificatoins to add and remove addresses from address

Co-authored-by: Hang Yang <>
Change-Id: I34766b96cb775356664f5e0d48a08a22ac6898e2
2 years ago
Zuul 0a852dab99 Merge "Handle router HA port concurrently deleting" 3 years ago
LIU Yulong 91eb3d8346 Handle router HA port concurrently deleting
Router HA port may be deleted concurrently while the plugin
is trying to update. This patch catches the known exceptions.
Should not `plugin.update_port_statuses` use because:
1. plugin.update_port_statuses will hide all exception
   no matter the port exists.
2. The code just needs to catch the port not found error,
   but let all other exception raised if port still exists.

Closes-Bug: #1906375
Change-Id: Id5d9c99be3bd6854568d2b1baa86c25c0cfd4756
3 years ago
LIU Yulong 510089bc5f Upgrade RPC version of SecurityGroup*Rpc
A partial upgrading of neutron cluster, neutron-server
has a newer version while neutron-agents not, does not
run well after a RPC data structure upgrading. This
patch upgrades the security group related RPC version
between neutron-server and agents. A partial upgrading
neutron cluster will explicitly raise error. The RPC
version should be aligned.

Closes-bug: #1903531
Related-bug: #1867119
Change-Id: I6cb2ba05fa3337be46eb01f2d9f869efa41e4db6
3 years ago
Miguel Lavalle 25a694c098 Agent side push notifications for address groups
Adds agent side code to enable the OVS agent to receive address groups
from the push notifications cache.

Change-Id: I1f27eccb2a69c553631fdc12d34e9025925844c5
Partial-Bug: #1592028
3 years ago
Bernard Cafarelli cebdd77af8
Bump pylint version to support python 3.8
As spotted in Focal testing patch [0], pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 [1]

Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel


Change-Id: Ie646b7093aa8634fd950c136a0eba9adcf56591c
3 years ago
LIU Yulong 00298fe6e8 [Security] fix allowed-address-pair issue
When add allowed-address-pair to one port, it will
unexpectedly open all others' protocol under same security
group. IPv6 has the same problem.

The root cause is the openflow rules calculation of the
security group, it will unexpectedly allow all IP(4&6)
traffic to get through.

For openvswitch openflow firewall, this patch adds a source
mac address match for the allowed-address-pair which has
prefix lenght 0, that means all ethernet packets from this
mac will be accepted. It exactly will meet the request of
accepting any IP address from the configured VM.

Test result shows that the remote security group and
allowed address pair works:
1. Port has allowed-address-pair clould send any
   IP (src) packet out.
2. Port has x.x.x.x/y allowed-address-pair could be accepted
   for those VMs under same security group.
3. Ports under same network can reach each other (remote
   security group).
4. Protocol port number could be accessed only when there
   has related rule.

Closes-bug: #1867119
Change-Id: I2e3aa7c400d7bb17cc117b65faaa160b41013dde
3 years ago
Zuul 230c25d209 Merge "Remove leftovers of get_external_network_id for router" 3 years ago
Aditya Reddy Nagaram cbc473e066 Support for stateless security groups
Blueprint: stateless-security-groups

Change-Id: Iae39a89b762786e4f05aa61aa0db634941806d41
3 years ago
Slawek Kaplonski 483cc047fa Remove leftovers of get_external_network_id for router
Some time ago in patch [1] deprecated option
'gateway_external_network_id' was removed.
After that commit [2] removed rpc function "get_external_network_id"
but there still left some leftovers from this.
This patch removes them completly.



Change-Id: Ie58ea7f021db051b68be80a1d98f5985ff19fe23
3 years ago
Miguel Lavalle 12760c94c8 Adding LOG statements to debug 1838449
Adding LOG statements to debug 1838449

Change-Id: I6f9cbe2e6b4ea0f122f5f8318dbbc31fce6b61f4
Related-Bug: #1838449
4 years ago
Rodolfo Alonso Hernandez b0a93df476 Update DHCP port information during setup
When setting up the DHCP agent of a network, the DHCP namespace external
port is configured. If this port already exists and the fixed IP
addresses are correctly configured (in the DHCP subnets range), the port
is used as is.

Sometimes, because of 1627480 or 1841636, the port information is not
correctly retrieved. This patch does not solve it but mitigates the
process of resynchronizing the network DHCP. If the stored DHCP port
does not have the correct information, the agent calls the RPC plugin to
retrieve from the server the DHCP port updated information, including
the fixed IP address and the subnets.

Change-Id: Iff40e7bba645ee12c2001d7ce735a36e0ddc81e9
Related-Bug: #1627480
Related-Bug: #1841636
4 years ago
LIU Yulong dd96f37759 Optimize DVR related port DB query
Save order by in port query when not require fixed_ips,
and save some useless query for dvr subnet mac.

Closes-Bug: #1834308
Change-Id: I6836840edcaa5a21fd2ba9f65ffd24f7e5038fa3
4 years ago
Adrian Chiris c62c67f413 Add RPC method to get networks for L3 and DHCP agents
- Added get_networks() RPC call for DHCP agent
- Added get_networks() RPC call for L3 agent

This change is required in order to support out of tree
MultiInterfaceDriver and IPoIBInterfaceDriver interface drivers
as they require information on the network a port is being plugged

These RPCs will be passed as kwargs when loading the relevant
interface driver.

get_networks() keyword args map to the keyword arguments of:

Change-Id: I11d82380aad8655a4fdc9656737b912b16e2859b
Partial-Bug: #1834176
4 years ago
Brian Haley b79842f289 Start enforcing E125 flake8 directive
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues.  Didn't think it was going to be
close to 100 files when I started.

Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
4 years ago
Zuul b7a37b3192 Merge "Optimize the code that fixes the race condition of DHCP agent." 4 years ago
Rodolfo Alonso Hernandez 5d35e7d360 Switch to new engine facade for L3RpcCallback
Partially-Implements blueprint: enginefacade-switch

Change-Id: Ida1bcb896c02f1ad04090a99296bc641bd21896d
4 years ago
Yang JianFeng 494b65d951 Optimize the code that fixes the race condition of DHCP agent.
The above patchs that resolve the race condition of DHCP agent will
result in neutron-server raise DhcpPortInUse ERROR log. And, the
second patch may result in old dhcp agent create a redundant port.

Closes-Bug: #1829332
Change-Id: If7a7ac2f88ce5b0e799c1104c936735a6cc860aa
4 years ago
LIU Yulong 3d99147e73 Ensure dvr ha router gateway port binding host
There are some extreme conditions which will result the unbound
router gateway port. Then all the centralized floating IPs will
not be reachable since the gateway port was set to 4095 tag.

This patch adds the HA status to the router related port
processing code path. If it is HA router, the gateway port
will go to the right HA router processing code branch.

Closes-Bug: #1827754
Change-Id: Ida1c9f3a38171ea82adc2f11cb17945d6e2434be
4 years ago
Boden R 9bbe9911c4 remove neutron.common.constants
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.


Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
4 years ago
Brian Haley eaf990b2bc Fix pep8 E128 warnings in non-test code
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.

Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <>
4 years ago
LIU Yulong 5b7d444b31 Not set the HA port down at regular l3-agent restart
If l3-agent was restarted by a regular action, such as config change,
package upgrade, manually service restart etc. We should not set the
HA port down during such scenarios. Unless the physical host was
rebooted, aka the VRRP processes were all terminated.

This patch adds a new RPC call during l3 agent init, it will try to
retrieve the HA router count first. And then compare the VRRP process
(keepalived) count and 'neutron-keepalived-state-change' count
with the hosting router count. If the count matches, then that
set HA port to 'DOWN' state action will not be triggered anymore.

Closes-Bug: #1798475
Change-Id: I5e2bb64df0aaab11a640a798963372c8d91a06a8
4 years ago
Boden R 024802aafd remove neutron.common.rpc
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.


Change-Id: I733f07a8c4a2af071b3467bd710290eee11a4f4c
4 years ago
Boden R 68fd13af40 remove neutron.common.exceptions
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.


Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
4 years ago
Kailun Qin b70ee4df88 Block port update from unbound DHCP agent
Current DHCP port management in Neutron makes the server to clear the
device_id while the agent is responsible for setting it.

This may cause a potential race condition, for example during network
rescheduling. The server aims to clear the device_id on a DHCP port and
assign the network to another agent while the old agent might just be
taking possession of the port. If the DHCP agent takes possession of the
port (i.e., update port...set the device_id) before the server clears
it, then there is no issue. However, if this happens after the clear
operation by server then the DHCP port would be updated/marked to be
owned by the old agent.

When the new agent takes over the network scheduled to it, it won't be
able to find a port to reuse so that an extra port might need to be
created. This leads to two issues:
1) an extra port is created and never deleted;
2) the extra port creation may fail if there are no available IP

This patch proposes a validation check to prevent an agent from updating
a DHCP port unless the network is bound to that agent.

Co-authored-by: Allain Legacy <>

Closes-Bug: #1795126
Story: 2003919
Change-Id: Ie619516c07fb3dc9d025f64c0e1e59d5d808cb6f
5 years ago
Swaminathan Vasudevan fd72643a61 Revert "DVR: Inter Tenant Traffic between networks not possible with shared net"
This reverts commit d019790fe4.

Closes-Bug: #1783654
Change-Id: I4fd2610e185fb60cae62693cd4032ab700209b5f
5 years ago
ZhaoBo 21ae99d5b3 [server side] Floating IP port forwarding plugin
This patch implements the plugin.
This patch introduces an new service plugin for port forwarding resources,
named 'pf_plugin', and supports create/update/delete port forwarding
operation towards a free Floating IP.

This patch including some works below:
* Introduces portforwarding extension and the base class of plugin
* Introduces portforwarding plugin, support CRUD port forwarding
* Add the policy of portforwarding

The race issue fix in:

Fip extend port forwarding field addition in:

Partially-Implements: blueprint port-forwarding
Change-Id: Ibc446f8234bff80d5b16c988f900d3940245ba89
Partial-Bug: #1491317
5 years ago
Boden R e4348eb1e1 use retry_db_errors from neutron-lib
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with

This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib


Change-Id: I1feb842d3e0e92c945efb01ece29856335a398fe
5 years ago
Boden R 839e575fa6 use plugin utils from neutron-lib
The remainder of the neutron.plugins.common.utils were rehomed into
neutron-lib with [1][2]. This patch consumes them by using the functions
from neutron-lib, and removing the neutron.plugins.common.utils module
all together as it's fully rehomed now.



Change-Id: Ic0f7b37861f078ce8c5ee92d97e977b8d2b468ad
5 years ago
Brian Haley 2e34279ec3 Fix lack of routes for neighbour IPv4 subnets
According to [1], when a network contains more that one IPv4
subnet, they are returned in the 'classless-static-routes'
DHCP option, regardless of whether DHCP is enabled for them
or not.

However, the get_active_networks_info() method used for
synchronizing networks after the dhcp agent restarts filters
subnets with "enable_dhcp=True", which differs from the
get_network_info() method.  This will block VM access to
other VMs in the dhcp disabled subnets, even though they are
in the same network.  This is visible by looking at the "opts"
file before and after a restart.

Change the dhcp agent to ask for all subnets in its
get_active_networks_info() RPC call by adding an
enable_dhcp_filter argument to toggle the behavior, with the
default being True to not break backwards compatibility.

Based on by Quan Tian.


Change-Id: I11ca1d1a603d02587f3b8d4a5a52a96b0587d61f
Closes-Bug: #1652654
5 years ago
Brian Haley 90cd939047 Fix W503 pep8 warnings
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.


Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
5 years ago
Dmitrii Shcherbakov ff5e8d7d6c Refresh router objects after port binding
Post-binding information about router ports is missing in results of RPC
calls made by l3 agents. sync_routers code ensures that bindings are
present, however, it does not refresh router objects before returning
them - for RPC clients ports remain unbound before the next sync and
there is no necessary address scope information present to create routes
from fip namespaces to qrouter namespaces.

Change-Id: Ia135f0ed7ca99887d5208fa78fe4df1ff6412c26
Closes-Bug: #1759971
5 years ago
Zuul 522da182b4 Merge "DVR: Inter Tenant Traffic between networks not possible with shared net" 5 years ago
Boden R ef93f7e7f0 use common agent topics from neutron-lib
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.


Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
5 years ago
Boden R 062ef79381 use is_extension_supported from neutron-lib
The is_extension_supported function now lives in neutron-lib. This patch
removes the function from neutron and uses lib's version instead.


Change-Id: Iccb72e00f85043b3dff0299df7eb1279655e313e
5 years ago
Swaminathan Vasudevan d019790fe4 DVR: Inter Tenant Traffic between networks not possible with shared net
Inter Tenant Traffic between two different networks that belong
to two different Tenants is not possible when connected through
a shared network that are internally connected through DVR

This issue can be seen in multinode environment where there
is network isolation.

The issue is, we have two different IP for the ports that are
connecting the two routers and DVR does not expose the router
interfaces outside a compute and is blocked by ovs tunnel bridge

This patch fixes the issue by not applying the DVR specific
rules in the tunnel-bridge to the shared network ports that
are connecting the routers.

Closes-Bug: #1751396
Change-Id: I0717f29209f1354605d2f4128949ddbaefd99629
5 years ago
Boden R 75ed3bcf34 remove neutron.callbacks package
Neutron lib contains the latest callbacks and thus this patch removes
the callbacks package from neutron entirely.


Change-Id: I14e45fd5d2d3c816bb39f8ace56f7be460bac0d6
5 years ago
Zuul bfaae98396 Merge "use l3 api def from neutron-lib" 6 years ago