Commit Graph

76 Commits (master)

Author SHA1 Message Date
Brian Haley 291eabb0b6 Fix some pylint indentation warnings
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/api.


Change-Id: I1258b04f64a18036407e1d9de9ddca7472af0d11
7 months ago
Sahid Orentino Ferdjaoui 7ceb935da8 dhcp/rpc: retrieve network details with segments
When segment plugin is enabled, we should return segments details as
they are part of network.

Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <>
Change-Id: I1dab155bc812f8764d22e78ebb7d80aaaad65515
9 months ago
Slawek Kaplonski 8fd88fd223 Pass host parameter to the get_network_info method
Host parameter is needed there to filter subnets per segment when
segments plugin is enabled.
When dhcp agent requests informations about networks, and segments
plugin is enabled, subnets which belongs to the network are filtered out
based on the host passed as argument to the get_network_info() method.
But we never passed host to that method, even when we should e.g.
during the full sync of the DHCP agent, when it requests details about
each network.
This patch fixes that issue by passing host parameter to that method.

Closes-Bug: #1958955
Change-Id: Ib5eef501493f6735a47ea085196242a5807c4565
1 year ago
Slawek Kaplonski 1c1814aa6b Fix reference before assignment error in the dhcp_rpc module
In patch [1] method get_network_info was refactored and that causes
NameError in the DHCP agent when there is "network object passed in
kwargs and there are subnets with segments in network. See related bug
for details.


Closes-Bug: #1958955
Change-Id: Iad8d85c79f8b11a24b1bb1ca44c776e909b610c3
1 year ago
Rodolfo Alonso Hernandez 5710d3407b Improve DHCP RPC handler
Remove unnecessary DB retrieval operations from
"get_active_networks_info" method.

Partial-Bug: #1950662
Change-Id: I4ea7b86e3f544d5dddcdac562208bb8afd1fc36a
1 year ago
Rodolfo Alonso Hernandez c686a2b555 Improve DHCP RPC handler
Remove unnecessary DB retrieval operations from "get_network_info"

Partial-Bug: #1950662
Change-Id: If4b33c8437dba411fed913e7e1c7f06d899c08f7
2 years ago
Nurmatov Mamatisa 1483b63ffa Add enable_dhcp filter in get_network_info rpc
Add enable_dhcp, to make a filter to avoid unnecessary
net_info data transfer through rpc.

Change-Id: Ibcef366f5b1f4b7da4f47f1f538a17111da0faa1
Closes-Bug: #1552614
2 years ago
LIU Yulong 8e3a83c213 Config option to disable the DHCP functions
This patch adds a new config option ``enable_traditional_dhcp``,
if set False, neutron-server will disable:
* DHCP provisioning block
* DHCP scheduler API extension
* Network scheduling mechanism
* DHCP RPC/notification

Partially-Implements: bp/distributed-dhcp-for-ml2-ovs
Related-Bug: #1900934

Change-Id: Icfbfc9691c5cf837406ff4291b3e3ed4970b26ee
2 years ago
Rodolfo Alonso Hernandez b0a93df476 Update DHCP port information during setup
When setting up the DHCP agent of a network, the DHCP namespace external
port is configured. If this port already exists and the fixed IP
addresses are correctly configured (in the DHCP subnets range), the port
is used as is.

Sometimes, because of 1627480 or 1841636, the port information is not
correctly retrieved. This patch does not solve it but mitigates the
process of resynchronizing the network DHCP. If the stored DHCP port
does not have the correct information, the agent calls the RPC plugin to
retrieve from the server the DHCP port updated information, including
the fixed IP address and the subnets.

Change-Id: Iff40e7bba645ee12c2001d7ce735a36e0ddc81e9
Related-Bug: #1627480
Related-Bug: #1841636
4 years ago
Adrian Chiris c62c67f413 Add RPC method to get networks for L3 and DHCP agents
- Added get_networks() RPC call for DHCP agent
- Added get_networks() RPC call for L3 agent

This change is required in order to support out of tree
MultiInterfaceDriver and IPoIBInterfaceDriver interface drivers
as they require information on the network a port is being plugged

These RPCs will be passed as kwargs when loading the relevant
interface driver.

get_networks() keyword args map to the keyword arguments of:

Change-Id: I11d82380aad8655a4fdc9656737b912b16e2859b
Partial-Bug: #1834176
4 years ago
Brian Haley b79842f289 Start enforcing E125 flake8 directive
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues.  Didn't think it was going to be
close to 100 files when I started.

Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
4 years ago
Yang JianFeng 494b65d951 Optimize the code that fixes the race condition of DHCP agent.
The above patchs that resolve the race condition of DHCP agent will
result in neutron-server raise DhcpPortInUse ERROR log. And, the
second patch may result in old dhcp agent create a redundant port.

Closes-Bug: #1829332
Change-Id: If7a7ac2f88ce5b0e799c1104c936735a6cc860aa
4 years ago
Boden R 9bbe9911c4 remove neutron.common.constants
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.


Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
4 years ago
Brian Haley eaf990b2bc Fix pep8 E128 warnings in non-test code
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.

Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <>
4 years ago
Boden R 68fd13af40 remove neutron.common.exceptions
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.


Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
4 years ago
Kailun Qin b70ee4df88 Block port update from unbound DHCP agent
Current DHCP port management in Neutron makes the server to clear the
device_id while the agent is responsible for setting it.

This may cause a potential race condition, for example during network
rescheduling. The server aims to clear the device_id on a DHCP port and
assign the network to another agent while the old agent might just be
taking possession of the port. If the DHCP agent takes possession of the
port (i.e., update port...set the device_id) before the server clears
it, then there is no issue. However, if this happens after the clear
operation by server then the DHCP port would be updated/marked to be
owned by the old agent.

When the new agent takes over the network scheduled to it, it won't be
able to find a port to reuse so that an extra port might need to be
created. This leads to two issues:
1) an extra port is created and never deleted;
2) the extra port creation may fail if there are no available IP

This patch proposes a validation check to prevent an agent from updating
a DHCP port unless the network is bound to that agent.

Co-authored-by: Allain Legacy <>

Closes-Bug: #1795126
Story: 2003919
Change-Id: Ie619516c07fb3dc9d025f64c0e1e59d5d808cb6f
5 years ago
Boden R e4348eb1e1 use retry_db_errors from neutron-lib
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with

This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib


Change-Id: I1feb842d3e0e92c945efb01ece29856335a398fe
5 years ago
Boden R 839e575fa6 use plugin utils from neutron-lib
The remainder of the neutron.plugins.common.utils were rehomed into
neutron-lib with [1][2]. This patch consumes them by using the functions
from neutron-lib, and removing the neutron.plugins.common.utils module
all together as it's fully rehomed now.



Change-Id: Ic0f7b37861f078ce8c5ee92d97e977b8d2b468ad
5 years ago
Brian Haley 2e34279ec3 Fix lack of routes for neighbour IPv4 subnets
According to [1], when a network contains more that one IPv4
subnet, they are returned in the 'classless-static-routes'
DHCP option, regardless of whether DHCP is enabled for them
or not.

However, the get_active_networks_info() method used for
synchronizing networks after the dhcp agent restarts filters
subnets with "enable_dhcp=True", which differs from the
get_network_info() method.  This will block VM access to
other VMs in the dhcp disabled subnets, even though they are
in the same network.  This is visible by looking at the "opts"
file before and after a restart.

Change the dhcp agent to ask for all subnets in its
get_active_networks_info() RPC call by adding an
enable_dhcp_filter argument to toggle the behavior, with the
default being True to not break backwards compatibility.

Based on by Quan Tian.


Change-Id: I11ca1d1a603d02587f3b8d4a5a52a96b0587d61f
Closes-Bug: #1652654
5 years ago
Brian Haley 90cd939047 Fix W503 pep8 warnings
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.


Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
5 years ago
Boden R 062ef79381 use is_extension_supported from neutron-lib
The is_extension_supported function now lives in neutron-lib. This patch
removes the function from neutron and uses lib's version instead.


Change-Id: Iccb72e00f85043b3dff0299df7eb1279655e313e
5 years ago
Zuul 048c880c30 Merge "consume common constants from lib" 6 years ago
Boden R 0822b0aef4 consume common constants from lib
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.


Change-Id: Ie4bcffccf626a6e1de84af01f3487feb825f8b65
6 years ago
SapanaJadhav d8807e2fa9 Wrong path of rpc_api.rst in class docstring
Changing rpc_api.rst file path from doc/source/devref/rpc_api.rst
to /doc/source/contributor/internals/rpc_api.rst. Because rpc_api.rst
file is located at this path

Closes-Bug #1722072

Change-Id: Ic243aab9e3428bfec69db61a94b4129cd768e233
6 years ago
Inessa Vasilevskaya 7322bd6efb Make code follow log translation guideline
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.

Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
6 years ago
Kevin Benton 4dc81f0a53 DHCP RPC: Separate local from non-local subnets
In order to allow the DHCP agent to service other subnets
on the network in other segments via DHCP relay, we need to
return all subnets regardless of segment association.

However, this behavior will break older DHCP agents that
then try to get IPs on the subnets belonging to other segments.
This patch adds a new subnet attribute, 'non_local_subnets'
that will be returned in the DHCP RPC calls, so that agents
that can deal with off-link subnets can handle them

Change-Id: I9cce7b8a19c1201435df0c6baac7be57c57847e6
Partial-Bug: #1692486
6 years ago
Boden R 0e2b667bf1 use neutron-lib callbacks
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.

As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by

This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.

There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].


[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432

Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
6 years ago
Boden R 7bd521e7ce use neutron_lib's portbindings api-def
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.

- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.

[1] 87e42f993c07ae320159d5123662ee9f3bd4d903

Change-Id: I669af9b4c712877772d91a03857ab108714001d4
6 years ago
Kevin Benton bb3653f337 Don't IPAddressGenerationFailure for DHCP ports
This is a runtime state users can easily create by creating tiny subnets
with no room for DHCP agent addresses. It's not something we need to
dump tracebacks on the server for because it's not a bug or unexpected

These traces can be seen all over the API test logs where small subnets
are created without the need to actuall put things on them.


Change-Id: I663b0df07608d8511a40c82e589875c50c7a6066
6 years ago
Armando Migliaccio 17563a802e Adopt neutron-lib plugin directory
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.

Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.

This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.

Usage pattern [1,2] can be translated to [3,4] respectively.

[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()

The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.

Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.

Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.


Partially-implements: blueprint neutron-lib

Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
7 years ago
Armando Migliaccio d6df96ddf2 Skip warnings during DHCP port actions if no error is raised
With multiple concurrent DHCP operations, the server log may be
filled with warning traces about the inability to complete the
DHCP action correctly. These should be emitted only when reraising
the underlying exception, namely when the underlying error is
not an expected nor a handled one.

Related-bug: 1621345
Related-bug: 1620864

Change-Id: I1f00609f8181e3765238550f45e6e114d2df7e6e
7 years ago
Kevin Benton 9d24490da8 Handle racey teardowns in DHCP agent
Capture port not found exceptions from port updates of DHCP ports
that no longer exist. The DHCP agent already checks the return
value for None in case any of the other things went missing
(e.g. Subnet, Network), so checking for ports disappearing makes
sense. The corresponding agent-side log message for this has also
been downgraded to debug since this is a normal occurrence.

This also cleans up log noise from calling reload_allocations on
networks that have already been torn down due to all of the subnets
being removed.

Closes-Bug: #1621650
Change-Id: I495401d225c664b8f1cf7b3d51747f3b47c24fc0
7 years ago
Jenkins 1d4a621718 Merge "Narrow down DBError to DBReferenceError in DHCP action handler" 7 years ago
Armando Migliaccio 9726a00c90 Narrow down DBError to DBReferenceError in DHCP action handler
Commit 85ed7017ff removed the DBError
handling to let the retry decorator do its magic, however the
full implications of this change were not evaluated. As a result,
DBReferenceError (which derives from DBError) is not processed
correctly and that caused a regression of the existing logic.

Rather than bloat the retry's responsibility even further, this
patch partially reverts commit 85ed7017f by narrowing down the
exception handling to DBReferenceError only.

Related-bug: #1618216

Change-Id: Icf4e5e4145dcdcdc710b8e42044467913ed01ec1
7 years ago
Jenkins aae2e8dcbc Merge "Don't catch DBError in DHCP action handler" 7 years ago
Kevin Benton 85ed7017ff Don't catch DBError in DHCP action handler
The DHCP port action handler has been catching DBErrors
since f1b9ac5a54, which is
well before we had the retry decorator to deal with these.
With the port action handler catching these, it means there
will not be retries on deadlocks or connection errors so
transient situations can result in a permanently broken
DHCP service for a network.

This removes the catch for DBError so the decorator can retry
the operation.

Closes-Bug: #1618216
Change-Id: I42031b481958bbfdb8f52902c294022717af7adf
7 years ago
Kevin Benton c05751a556 Catch InvalidInput in DHCP port actions
The exception handler checking for concurrently deleted subnets
was missing InvalidInput, which comes from IPAM when a port creation
request references a subnet that doesn't exist.

This patch just catches that exception.

Change-Id: I14f9e4bddde845e3fef2b0d8649d3954ba5c93bd
Closes-Bug: #1618187
7 years ago
Jenkins 39a4793a55 Merge "Filter out subnets on different segments for dhcp" 7 years ago
Brandon Logan 45552b5c2c Filter out subnets on different segments for dhcp
The dhcp agent will request network information from neutron-server.
neutron-server will give the dhcp agent the entire network tree back
to the agent including subnets.  If that agent is on a host that
is mapped to a segment, subnets that do not belong to that
segment are also returned and the dhcp agent will attempt to create
or update the dhcp port based on those subnets.  These requests
will fail because the ipam logic forbids it for multiple segments.

This patch strips out those subnets that do not belong to the host's
segment.  If the segments plugin is not enabled or the network is not
a routed network then the neutron-server will return back to the agent
what it normally would without routed networks.

Change-Id: I379b56f510853c60bb52c2d710117fecfb9aca2e
Partially-Implements: blueprint routed-networks
7 years ago
Henry Gessau 890c30fc52 Add retry decorator to dhcp_ready_on_ports
Enable DB retries for this provisioning blocks phase.

Closes-Bug: #1602833

Change-Id: Ie6444d6552b4c1e73c00d5624e3866417204c9e8
7 years ago
Aaron Rosen 2ddbbce7ba remove unused rpc method get_active_networks
This method was originally left for backwards compatibility during
an upgrade of neutron. It stopped being used in havana so I think it
should be safe to remove now.

Closes-bug: #1583835
Change-Id: I503d10ee59ba2fa046b5030e8281fd41a49ee74e
7 years ago
Jenkins b342ca6447 Merge "Restart dsnmasq on any network subnet change" 7 years ago
Kevin Benton e45add7b07 Restart dsnmasq on any network subnet change
When a new subnet is added to a network, the network cache
is updated with the list of subnets regardless of which ones
have DHCP enabled. This changes the index order of the subnet
list which means that the tags used for each subnet change.

This means we must restart the process because the opts file
will be using different tags than the process args. This patch
implements that change. It also sorts the subnets on the RPC
side so the agent indexes don't change if subnets aren't

The previous logic was only restarting the process when DHCP
enabled subnets changed, which meant that adding a DHCP disabled
subnet would break the association between the opts file tags and
the process arg tags, which led to the reported bug.

Closes-Bug: #1581918
Change-Id: If1452c0e8fe95eb94cd78c7a05b57aead75662b5
7 years ago
Kevin Benton b672c26cb4 Add provisioning blocks to status ACTIVE transition
Sometimes an object requires multiple disjoint actors to complete
a set of tasks before the status of the object should be transitioned
to ACTIVE. The main example of this is when a port is being created.
The L2 agent has to do its business to wire up the VIF, but at the same
time the DHCP agent has to setup the DHCP reservation. This led to
Nova booting the VM when the L2 agent was done even though the DHCP
agent may have been nowhere near ready.

This patch introduces a provisioning blocks mechansim that allows the
entities to be tracked that need to be involved to make a transition
to ACTIVE happen. See the devref in the dependent patch for a high-level
view of how this works.

The ML2 code is updated to use this new mechanism to prevent updating
the port status to ACTIVE without both the DHCP agent and L2 agent
reporting that the port is ready.

The DHCP RPC API required a version bump to allow the port ready

This also adds a devref doc for the provisioning_blocks
module with a high-level overview of how it works in addition
to a detailed description of how it is used specifically with
ML2, the L2 agents, and the DHCP agents.

Closes-Bug: #1453350
Change-Id: Id85ff6de1a14a550ab50baf4f79d3130af3680c8
7 years ago
Henry Gessau 4148a347b3 Use constants from neutron-lib
With this we enable the deprecation warnings by default.

Related-Blueprint: neutron-lib

Change-Id: I5b9e53751dd164010e5bbeb15f534ac0fe2a5105
7 years ago
Henry Gessau ae5bad49cc Use exceptions from neutron-lib
Related-Blueprint: neutron-lib

Change-Id: Ia014468bd621c4ee6aea95bf19328c61070174c4
7 years ago
Gary Kotton cefb1079f0 DHCP: remove update_lease_expiration
This code has not been around since commit d9832282cf

Change-Id: I5c0a75443ee9330505c2762d53c739c20ddaaea3
Closes-bug: #1571503
7 years ago
LiuNanke 83ef6b5677 Using LOG.warning replace LOG.warn
Python 3 deprecated the logger.warn method, see:
so we prefer to use warning to avoid DeprecationWarning.

Closes-Bugs: #1529913

Change-Id: Icc01ce5fbd10880440cf75a2e0833394783464a0
Co-Authored-By: Gary Kotton <>
7 years ago
Kevin Benton 0db4ba3f85 Downgrade network not found log in DHCP RPC
This is a normal occurence as networks are created and
deleted. It's not something that an operator should be
warned about.

Change-Id: I3bb498a29d93a88b059a669d510d21b4c65ea014
Closes-Bug: #1554695
7 years ago
hgangwx 283e5db691 Wrong usage of "a"
Wrong usage of "a" in the messages:
"'%s' is not a integer"
"Create a address scope"
"Return True if port has as a allowed address pair"
"But if a agent does not report its status"
"a ipv4 link-local address"
"Retrieve and return a extended information about a network"
"It could be a eui-64 address, a random IPv6 address"
"push a elastic-recheck query for it (see below)"
"is not a enforced requirement"
"a ovs_lib.VifPort object"

Should be:
"'%s' is not an integer"
"Create an address scope"
"Return True if port has as an allowed address pair"
"But if an agent does not report its status"
"an ipv4 link-local address"
"Retrieve and return extended information about a network"
"It could be an eui-64 address, a random IPv6 address"
"push an elastic-recheck query for it (see below)"
"is not an enforced requirement"
"an ovs_lib.VifPort object"

Totally 9 occurrences in Neutron base code.

Change-Id: I0f980fc8c98524db3d194ecb779f76abb37eb31c
8 years ago