Commit Graph

156 Commits (master)

Author SHA1 Message Date
Brian Haley 86badcfe2d Fix some pylint indentation warnings
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
some remaining ones in miscellaneous directories.

Also cleanup any remaining code that I missed in this
series, or has changed since I started.


Change-Id: I17b4779020a7bfb369c3e721ab6638cd4a6ab50c
6 months ago
Brian Haley b1714a2b9d Fix some pylint indentation warnings
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/plugins.


Change-Id: Id9138652f5f07ef12fa682e182fe210019e8f975
7 months ago
Takashi Kajinami 17106dc6f5 ml2: Use the base module to register common ml2 agent config
The neutron.plugins.ml2.drivers.agent.config module registers options
commonly used by the ml2 agents but in fact it is used only by linux
bridge agent and macvtap agent.

This change makes all ml2 agents use that base module consistently in
individual config modules.

Change-Id: Ib3ec8a8eaf347721bb06f092a0887e62f3a6bffd
12 months ago
Jakub Libosvar 4d3a274765 Don't register config options on imports
Importing some modules lead to registering config options that may
collide with config options from a project that calls the import. This
patch wraps the side effect that registers config options into a
function that needs to be called in case the caller wants to register
the options.

This solution is also not perfect as it guards the common options to be
registered only once even if the function is called multiple times. This
is to solve problems in unittests, ideally we should always call the
function just once even in our testing suites.

Resolves-Bug: #1968606
Change-Id: Ic1532eb8de887ff1b1085206df11f53e22f7f524
Signed-off-by: Jakub Libosvar <>
1 year ago
elajkat 837fda7566 Change set_device_rate calls according to new signature
[1] changed the signature of set_device_rate to be alligned with how
pyroute2 expects rate parameters, some method which calls
set_device_rate needs to be changed also to use the new dict format for
calling it.


Related-Bug: #1962844
Closes-Bug: #1968206
Change-Id: I7256747f45308b011edbb0f8c802439ea277423d
1 year ago
Rodolfo Alonso Hernandez cdff281f64 [SR-IOV] Fix QoS extension to set min/max values
"ip link" commands allow to define VF rates independently.
That means, first "rate" (max BW) can be set and in a second
command "min" (min BW) (check LP bug description).

However Pyroute2 command to set the VF rates requires to set both.
If one value is missing ("min_tx_rate", "max_tx_rate"), the library
sets this value to 0; in other words, the value is deleted.

The Pyroute2 structures are built depending on the parameter names.
In this case, {'vf': {'rate': ...}} will create a "vf_rate" [1]
nla structure, that requires "min_tx_rate" and "max_tx_rate".
This is part of the full structure passed to the "iproute" library

This is an example of code that only sets the "max_tx_rate" for
the 15th VF of "enp196s0f0":
  $ from neutron.plugins.ml2.drivers.mech_sriov.agent import pci_lib
    pci = pci_lib.PciDeviceIPWrapper("enp196s0f0")
    pci.set_vf_rate(15, {'max_tx_rate': 10})

The "msg" [3] (structure passed to "iproute" library) is this: The
"min_tx_rate" is set to the default value 0.

This patch reads first the existing rates ("min_tx_rate",
"max_tx_rate") and populates the command parameters accordingly.

[1]a9564dff8e/pyroute2.core/pr2modules/netlink/rtnl/ifinfmsg/ (L712-L717)
[2]c8d9d92544/ip/ipaddress.c (L454-L470)
[3]a9564dff8e/pyroute2.core/pr2modules/iproute/ (L1499)

Closes-Bug: #1962844
Change-Id: Ibbb6d938355440c42850812e368224b76b1fce19
1 year ago
Rodolfo Alonso Hernandez 0fe6c0b8ca Use the "connectivity" property of "MechanismDriver"
The base class "MechanismDriver" now has a property called
"connectivity". This patch overrides the default value in the
in-tree drivers.

The method "_check_drivers_connectivity" now uses this property
that is available in all drivers.


Closes-Bug: #1959125
bp boot-vm-with-unaddressed-port
Related-Bug: #1821058

Change-Id: I91734835b07d804365b46adfb26e984557107d80
1 year ago
Rodolfo Alonso Hernandez c8427c82da [OVN] Check new added segments in OVN mech driver
Check new segments added to OVN mechanism driver. This mechanism
driver does not inherit from ``SimpleAgentMechanismDriverBase``
because OVN has no agents (NOTE 1). However OVN mechanism driver
requires an OVN controller service in each deployed chassis. This
is how OVN driver knows the hosts using this backend.

The segments, attached to an external network (connected to a
physical network), can be mapped to a host if ther is a chassis (OVN
controller agent) in this host.

NOTE 1: OVN provides agent API compatibility, presenting the
controller and the metadata services as agents. But OVN mechanism
driver has no control or provides any information (via RPC) to
those services.

Closes-Bug: #1949967
Change-Id: I570b2251da8a4a25d155ba838346b24afafd727f
2 years ago
Rodolfo Alonso Hernandez f8ca735e5f Move mech driver VNIC validation to SimpleAgentMechanismDriverBase
The mechanism driver support VNIC types validation is done now in the
"SimpleAgentMechanismDriverBase" class __init__ method. If a subclass
needs to administratively prohibit any VNIC type supported by default,
"vnic_type_prohibit_list" must be passed to the base class __init__

Related-Bug: #1578989

Change-Id: Ic25a8a7c716b4980ad2542b44519f77c6fdad309
2 years ago
Zuul 4b0a225e80 Merge "SR-IOV agent can handle ports with same MAC addresses" 2 years ago
Zuul ca3ec4133b Merge "Add a single option to override the default hypervisor name" 2 years ago
Takashi Kajinami be43141a5f Provide the rpc_response_max_timeout parameter to sriov-agent
The rpc_response_max_timeout parameter is used in comminucation over
messaging queue, thus should be available for sriov-agent which
communicate with neutron-server over messaging queue.

Change-Id: Ie6ae31e40488fd8f3d43e83b25e536a7dd9d938c
Closes-Bug: #1930996
2 years ago
Rodolfo Alonso Hernandez 77ac42d2ee SR-IOV agent can handle ports with same MAC addresses
SR-IOV agent can handle ports with same MAC address (located in
different networks). The agent can retrieve, from the system, the
MAC address and the PCI slot; because the PCI slot is unique per
port in the same host, this parameter is used to match with the
Neutron port ID stored in the database (published via RPC).

RPC API bumped to version 1.9.

Closes-Bug: #1791159

Change-Id: Id8c3e0485bebc55c778ecaadaabca1c28ec56205
2 years ago
Takashi Kajinami ddf0fef28b Add a single option to override the default hypervisor name
Currently neutron uses socket.gethostname() to determine hypervisor
names, but this implementation is not fully compatible with libvirt
driver which uses canonical name for hypervisor name.
This incompatibility causes an issue with root resource provider
detection if a deployment uses FQDNs as canonicanl names.

This change introduces the resource_provider_default_hypervisor option,
so that users can override the hypervisor name by the single option(*1)
instead of setting two list options(*2). This is especially useful if
the deployment has multiple bridges or interfaces.



Related-Bug: #1926693
Change-Id: I692219200535df3af1265248e88c96947e4d8f9d
2 years ago
Rodolfo Alonso Hernandez 3a9e81bbb8 [SR-IOV] Do not fail if ip-link vf "min_tx_rate" is not supported
Some NICs do not support ip-link vf "min_tx_rate" parameter. This is
not an API issue (ip-link or Pyroute2); the parameter is already
supported. The error is returned by the specific NIC driver.

A sanity check is implemented, reading the SR-IOV configured devices
("physical_device_mappings") and the excluded VFs ("exclude_devices").

Change-Id: If70de0a7112777ac4011ad42af0ac98969765011
Closes-Bug: #1918464
2 years ago
Rodolfo Alonso Hernandez c33f0588cb [SR-IOV] Add support for ACCELERATOR_DIRECT VNIC type
Add support in SR-IOV mechanism driver for VNIC type

This VNIC type refers to any device that supports hardware
acceleration of any kind and is provided by Cyborg [1].

NOTE: "accelerator-direct-physical" is not supported yet.


Change-Id: I529e6a2a445a140dca7686976efeefcd13d333f0
Closes-Bug: #1909100
2 years ago
Brian Haley 055036ba2b Improve terminology in the Neutron tree
There is no real reason we should be using some of the
terms we do, they're outdated, and we're behind other
open-source projects in this respect. Let's switch to
using more inclusive terms in all possible places.

Change-Id: I99913107e803384b34cbd5ca588451b1cf64d594
3 years ago
Rodolfo Alonso Hernandez ddd5832323 Remove "six" library
Last step to remove "six" library usage in Neutron.

Change-Id: Idd42e0c51c8c3bd598c9cf91602596be238bccae
3 years ago
Bernard Cafarelli 5ce0595803
Set process name for agents
Now that we use setproctitle for neutron-server workers (and
neutron-keepalived-state-change), this has the side effect of changing
the process name for agents, impacting some monitoring systems. More
details in launchpad bug.

This patch fixes it by setting the name with setproctitle to:
agent name (original process name).

Also use the newly introduced name constants to replace existing
hardcoded uses.

Change-Id: I74c3a4d3e9f833752571a75f196560cd45529385
Closes-Bug: #1881297
3 years ago
Rodolfo Alonso Hernandez 954f2f28da Configure privsep in SR-IOV agent
With the introduction in [1] of commands executed under a privsep
context, privsep should be initialized first.


Change-Id: Ifdb8ad910ff6ffb31668ebb1e97d885ee924b470
Closes-Bug: #1883757
3 years ago
Rodolfo Alonso Hernandez c5d8fd6329 Use pyroute2 for SRIOV VF commands
Recent changes in some versions of iproute2 CLI output (v4.18),
have invalidated the regular expression used to parse the
"ip link" output.

To solve this problem and avoid future ones, pyroute2 is used to
retrieve the virtual functions information and set the VF attributes
(spoofcheck, min_tx_rate, max_tx_rate and link_state).

pyroute2 extended the "ip link" support to retrieve this information,
adding "ext_mask=1" in the get command. If no virtual functions are
present in this particular network interface, the added method,
"get_link_vfs",  will return an empty list.

The set commands can return a "InterfaceOperationNotSupported" in
case the operation is not supported. For min_tx_rate, if the driver
does not support to set a minimum bandwidth, an "InvalidArgument"
(from a pyroute2.NetlinkError(22)) exception will be raised.

Change-Id: I680da4f64bd114f1caecaaeedbf8a4b1915a0849
Closes-Bug: #1878042
3 years ago
Rodolfo Alonso Hernandez 37fafeb678 Add SR-IOV agent resource provider information
When the SR-IOV agent starts, print information about the Placement
resource providers: bandwidth, inventory default configuration and
hypervisor names.

Change-Id: Ib34f7a4f850301b2a8bf739c9d8d5e73bcc1bc48
Related-Bug: #1578989
3 years ago
Adrian Chiris 6f7b88d080 Use effective MAC address for macvtap assigned VFs
This commit modifies the way sriov agent retrieves mac
addresses for macvtap assigned VFs to use the effective
mac taken from the macvtap net device instead of the
administrative mac set on the PF.

This commit rids sriov agent from depending on hypervisor/nova
to set both administrative and effective mac for macvtap ports.

Related-Bug: #1841067

Change-Id: Id499bcc49d27f13f7f03481922a3383b4a255da1
3 years ago
Bence Romsics 258eebea71 Locate RP-tree parent by hypervisor name
Previously we assumed that we can look up the resource provider (created
by nova) to be used as the parent of the agent and physical NIC resource
provider tree by the name set in the config option This
assumption was wrong.

While nova-compute's and neutron-agent's
must match for port binding to work, the root resource provider created
by nova does not belong to the compute host (where nova-compute runs)
but it belongs to the compute nodes (i.e. hypervisors). Actually there
may be multiple compute nodes managed by a single nova-compute (think
of ironic). Plus the value of and the compute node's ID
may be different even when nova-compute manages a hypervisor on the
same host because of various deployment considerations. For example
when tripleo does not manage the undercloud (so a libvirt hypervisor
returns the plain hostname), but the same tripleo enforces it's host
naming conventions in nova's and neutron's settings.

This change enables neutron to use the hypervisor name to locate the
root of the resource provider tree.

We introduce a new configuration option for

(1) ovs-agent: resource_provider_hypervisors, for example:

bridge_mappings = physnet0:br-physnet0,...
resource_provider_bandwidths = br-physnet0:10000000:10000000,...
resource_provider_hypervisors = br-physnet0:hypervisor0,...

(2) sriov-agent: resource_provider_hypervisors, for example:

bridge_mappings = physnet1:ens5,...
resource_provider_bandwidths = ens5:10000000:10000000,...
resource_provider_hypervisors = ens5:hypervisor1,...

For both agents 'resource_provider_hypervisors' values default to
socket.gethostname() for each key in resource_provider_bandwidths.

We try to not block later developments in which one neutron
agent may manage devices on multiple hosts. That's why we allow
the each physdev to be associated with a different hypervisor.

But here we do not try to solve the problem that the natural physdev
identifiers may not be unique accross multiple hosts. We leave solving
this problem to whoever wants to implement an agent handling devices of
multiple hosts.

(3) We extend report_state message's configurations field alike:

'bridge_mappings': {'physnet0': 'br-physnet0'},
'resource_provider_bandwidths': {
    'br-physnet0': {'egress': 10000000, 'ingress': 10000000}},
'resource_provider_hypervisors': {'br-physnet0': 'hypervisor0'},

(4) In neutron-server we use
when selecting parent resource provider for agent and physdev
RP-tree. When not available in the message we fall back to using as before.

Since we only changed the free-format configurations field of the
report_state message rpc version is not bumped and we expect this
change to be backported to stein and train.

Change-Id: I9b08a3a9c20b702b745b41d4885fb5120fd665ce
Closes-Bug: #1853840
4 years ago
Adrian Chiris 76de8a715d SR-IOV: macvtap assigned vf check using sysfs
This reverts commits:

As the issue is relevant only for old kernels and almost 4 years
have passed since this fix was introduced, it is safe to undo the
workaround to use ip link show command for determining whether or not
a VF is assigned with macvtap(and followup patch that fixed excessive
use of ip link show commans).

reverting this commit will benefit the agent by reducing the amount of
ip link calls.

While it is possible to perform a revert-per-commit, this change should
really go in as one commit for clarity and avoid introducing a
performance degradation in case one is merged without the other.


 Conflicts details:
   In eswitch_manager: merge tool was unable to properly revert
                       is_assigned_vf() due to changes in later patches.
   In pci_lib: conflicts with later changes to is_macvtap_assigned()
               and link_show() that are no longer required and revert
               of IPCommandDeviceError definition.
   In unit tests: conflicts due to newly added tests and reverting the
                  IPCommandDeviceError exception definition.
                  Sorting out mocks as link_show() method was removed

Related-bug: #1523083

Change-Id: I04ea8eb63de07a6e1e51c2790c5920b086b8542c
4 years ago
Rodolfo Alonso Hernandez a70f07deb1 Add "connectivity" parameter in vif_details
Added "connectivity" parameter to mech driver vif_details. The default
value is "legacy". The in-tree drivers (Linux Bridge, Open vSwitch, SR-IOV
and MacVTap) have "l2" connectivity.

Change-Id: I45480986fc89e0b2f475ee0ceb13d8742fb8c530
Related-Bug: #1821058
4 years ago
Oleg Bondarev c148c6df46 SRIOV agent: wait VFs initialization on embedded switch create
Check for configured and actual number of VFs to prevent
device registaration with 0 VFs.

Closes-Bug: #1831622
Change-Id: Ie699d245f8ae2fc1d16b96432d2962788d9dba57
4 years ago
Boden R 9bbe9911c4 remove neutron.common.constants
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.


Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
4 years ago
Brian Haley eaf990b2bc Fix pep8 E128 warnings in non-test code
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.

Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <>
4 years ago
Zuul 5b01279ba5 Merge "Add support for binding activate and deactivate" 4 years ago
Brian Haley 7b973d151e Fix pylint R1716 (chained-comparison) refactor messages
Use "a < b < c" instead of "a < b and b < c".

Change-Id: I1f0c46a609c56ffe869486b25fe6e82faaf3d5fb
4 years ago
Adrian Chiris a05e1369b0 Add support for binding activate and deactivate
Add support for binding activate and deactivate callbacks
in SR-IOV mechanism driver agent.

this work item is required for supporting VM live-migration
with SR-IOV in nova with libvirt as hypervisor.

This commits implemets the following:
- Implement binding_activate() and binding_deactivate() methods in agent.
- Configure agent to listen on the relevant topics.
- RPC version bump on agend side.
- When processing an activated binding treat it as a newly added port.
- Deactivated binding will trigger a NO-OP as once a binding is
  deactivated it will be removed by Hypervisor.

Closes-Bug: #1815361

Change-Id: I6b7195e08ed8932cfc2785b66627de2854ead85d
4 years ago
Brian Haley 2147c70745 Change SR-IOV agent to log message after failure
If the SR-IOV agent fails to send its report_state
to the server, it logs an exception:

   Failed reporting state!: MessagingTimeout: Timed out...

If it then tries a second time and succeeds it just
goes on happily. It would be nice if it logged that
it had success on the subsequent attempt so someone
looking at the logs know it recovered.

Change-Id: I0782599a4f0ffe48c2dd4fa2abc9fe5111715c0c
4 years ago
Lajos Katona 97c9c4656d get_standard_device_mappings for mechdriver
get_mappings for machanism drivers return back different dicts for
different mechanism drivers. By adding get_standard_device_mappings
method SRIOV and OVS mechanism drivers can return a mapping dict in a
form like this one:
{'physnet_name': ['device_or_bridge_1', 'device_or_bridge_2']}

Change-Id: Ieddd9b3a4f3d7269aafc373f040c55b025f5c201
Related-Bug: #1578989
5 years ago
Hongbin Lu f0678b9b09 Add propagate_uplink_status to port
Introduce an attribute 'propagate_uplink_status' to port.
This attribute can be implemented for VF port to indicate if the VF
link state should follow the state of the PF.

Note: ML2 extension driver loaded on request via configuration:

  extension_drivers = uplink_status_propagation

Other related patches:
* neutron-lib:
* tempest test:
* OSC:
* neutronclient:

APIImpact Add 'propagate_uplink_status' attribute to 'port' resource

Change-Id: Ie8260c332e24c1880f9f82e6b6dacca8415be842
Close-Bug: #1722720
5 years ago
Zuul 6efb09fe22 Merge "ovs/sriov mech drivers: resource_provider_uuid5_namespace" 5 years ago
Zuul c4f081d024 Merge "supported_vnic_type configurable for sriov" 5 years ago
Lajos Katona d860109bf6 supported_vnic_type configurable for sriov
Now supported_vnic_types is hardcoded to the mechanism drivers, but that
can depend on several factors, like type of the NIC, admin decision,
With this patch we put the right to decide which vnic types are
supported for ovs agent into the hands of the admin, by allowing
blacklisting items from the mechanism driver specific list.

Change-Id: Iad9e2e966df53b4164d2a56a93215c69825b5241
Partial-Bug: #1578989
See-Also: (nova spec)
See-Also: (neutron spec)
5 years ago
Bence Romsics 69d6b86f56 ovs/sriov mech drivers: resource_provider_uuid5_namespace
Set a UUID v5 namespace for the Open vSwitch mechanism driver and
another for the SR-IOV mechanism driver. These will be used to
deterministically generate Placement Resource Provider UUIDs.

Change-Id: I365a0aeefffe68bd7d5ba5f6697b814bf2713a19
Partial-Bug: #1578989
See-Also: (nova spec)
See-Also: (neutron spec)
5 years ago
Zuul e5726c7bda Merge "sriov-agent: Report resource info in heartbeat" 5 years ago
Lajos Katona 4df02ee93c Enable ingress direction for min_bw rule
Enable QoS minimum-bandwidth rule for ovs and sriov backends for both
ingress and egress directions.
Add qos-bw-limit-direction as supported extension to qos-plugin, and
add create/delete/update_minimum_bandwidth and
delete_minimum_bandwidth_ingress empty methods to sriov/ovs/linuxbridge
extension drivers.

Change-Id: I6eb21ccf0400ea9adae90ff0bf97e08cdb09b8eb
Partial-Bug: #1578989
See-Also: (nova spec)
See-Also: (neutron spec)
5 years ago
Bence Romsics d7d433f729 sriov-agent: Report resource info in heartbeat
Example config for sriov-agent:

physical_device_mappings = physnet0:ens5,physnet0:ens6
resource_provider_bandwidths = ens5:100000:100000,ens6:100000:100000

Agent configurations now includes 'resource_provider_bandwidths' and

Change-Id: I1af798d404ed94cd30da3f92734bea911bbc3ac7
Co-Authored-By: Lajos Katona <>
Partial-Bug: #1578989
See-Also: (nova spec)
See-Also: (neutron spec)
5 years ago
Brian Haley eea5aaac4f Fix corner case in failure assigning MAC to SR-IOV NIC
Sometimes due to NIC driver incorrect behavior, VFs might be
missing in 'ip link show' output.  This may lead to a VM boot
failure as agent will just skip such missing devices.

Make the agent do a resync in case a newly added device
'disappears' during processing, which should cause a MAC to
get assigned.

Co-authored-by: Oleg Bondarev <>

Change-Id: I148b5a025fc388821fd1269d02908cc8ce1882fe
Closes-bug: #1784484
5 years ago
Brian Haley 7cfdf4aa81 Fix all pep8 E129 errors
Fixed all pep8 E129 errors and changed tox.ini to no longer
ignore them.

Change-Id: I0b06d99ce1d473b79a4cfdd173baa4f02e653847
5 years ago
Brian Haley c3b83a9ca6 Fix all pep8 E265 errors
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them.  Also removed an N536 comment missed from a
previous change.

Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
5 years ago
Brian Haley 90cd939047 Fix W503 pep8 warnings
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.


Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
5 years ago
Zuul a2c1d336b9 Merge "SR-IOV agent should specify host when requesting devices info" 5 years ago
Ihar Hrachyshka a1b49c796e Don't define polling_interval option in two places
The option is currently defined in two places - as a generic agent
option and as a sriov agent specific one. This is a problem now that
oslo.config tracks location where options are defined
(I6dec1e09dcab203c6287f9c56c866f220a42f850) because when unit test
runner imports all modules, it registers the option from those two
separate locations, and since locations are different, it's considered
the options are duplicate, which results in the following error:

DuplicateOptError: duplicate option: polling_interval

To fix the issue, I removed the definition of the option from sriov
module and made sure the common agent options are registered whenever
the sriov module is imported.

Change-Id: I485532692f4b3a5804885e0325a8352fe6b1aa65
Closes-Bug: #1759345
5 years ago
Boden R ef93f7e7f0 use common agent topics from neutron-lib
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.


Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
5 years ago
Boden R 95f1e03446 use plugin constants from neutron-lib
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.


Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
6 years ago