Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
some remaining ones in miscellaneous directories.
Also cleanup any remaining code that I missed in this
series, or has changed since I started.
Trivialfix
Change-Id: I17b4779020a7bfb369c3e721ab6638cd4a6ab50c
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/plugins.
Trivialfix
Change-Id: Id9138652f5f07ef12fa682e182fe210019e8f975
The neutron.plugins.ml2.drivers.agent.config module registers options
commonly used by the ml2 agents but in fact it is used only by linux
bridge agent and macvtap agent.
This change makes all ml2 agents use that base module consistently in
individual config modules.
Change-Id: Ib3ec8a8eaf347721bb06f092a0887e62f3a6bffd
Importing some modules lead to registering config options that may
collide with config options from a project that calls the import. This
patch wraps the side effect that registers config options into a
function that needs to be called in case the caller wants to register
the options.
This solution is also not perfect as it guards the common options to be
registered only once even if the function is called multiple times. This
is to solve problems in unittests, ideally we should always call the
function just once even in our testing suites.
Resolves-Bug: #1968606
Change-Id: Ic1532eb8de887ff1b1085206df11f53e22f7f524
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
[1] changed the signature of set_device_rate to be alligned with how
pyroute2 expects rate parameters, some method which calls
set_device_rate needs to be changed also to use the new dict format for
calling it.
[1]: https://review.opendev.org/q/Ibbb6d938355440c42850812e368224b76b1fce19
Related-Bug: #1962844
Closes-Bug: #1968206
Change-Id: I7256747f45308b011edbb0f8c802439ea277423d
"ip link" commands allow to define VF rates independently.
That means, first "rate" (max BW) can be set and in a second
command "min" (min BW) (check LP bug description).
However Pyroute2 command to set the VF rates requires to set both.
If one value is missing ("min_tx_rate", "max_tx_rate"), the library
sets this value to 0; in other words, the value is deleted.
The Pyroute2 structures are built depending on the parameter names.
In this case, {'vf': {'rate': ...}} will create a "vf_rate" [1]
nla structure, that requires "min_tx_rate" and "max_tx_rate".
This is part of the full structure passed to the "iproute" library
[2].
This is an example of code that only sets the "max_tx_rate" for
the 15th VF of "enp196s0f0":
$ from neutron.plugins.ml2.drivers.mech_sriov.agent import pci_lib
pci = pci_lib.PciDeviceIPWrapper("enp196s0f0")
pci.set_vf_rate(15, {'max_tx_rate': 10})
The "msg" [3] (structure passed to "iproute" library) is this:
https://paste.opendev.org/show/b2FZBOebGOCHMrYhPr6X/. The
"min_tx_rate" is set to the default value 0.
This patch reads first the existing rates ("min_tx_rate",
"max_tx_rate") and populates the command parameters accordingly.
[1]a9564dff8e/pyroute2.core/pr2modules/netlink/rtnl/ifinfmsg/__init__.py (L712-L717)
[2]c8d9d92544/ip/ipaddress.c (L454-L470)
[3]a9564dff8e/pyroute2.core/pr2modules/iproute/linux.py (L1499)
Closes-Bug: #1962844
Change-Id: Ibbb6d938355440c42850812e368224b76b1fce19
The base class "MechanismDriver" now has a property called
"connectivity". This patch overrides the default value in the
in-tree drivers.
The method "_check_drivers_connectivity" now uses this property
that is available in all drivers.
Depends-On: https://review.opendev.org/c/openstack/neutron-lib/+/826501
Closes-Bug: #1959125
bp boot-vm-with-unaddressed-port
Related-Bug: #1821058
Change-Id: I91734835b07d804365b46adfb26e984557107d80
Check new segments added to OVN mechanism driver. This mechanism
driver does not inherit from ``SimpleAgentMechanismDriverBase``
because OVN has no agents (NOTE 1). However OVN mechanism driver
requires an OVN controller service in each deployed chassis. This
is how OVN driver knows the hosts using this backend.
The segments, attached to an external network (connected to a
physical network), can be mapped to a host if ther is a chassis (OVN
controller agent) in this host.
NOTE 1: OVN provides agent API compatibility, presenting the
controller and the metadata services as agents. But OVN mechanism
driver has no control or provides any information (via RPC) to
those services.
Closes-Bug: #1949967
Change-Id: I570b2251da8a4a25d155ba838346b24afafd727f
The mechanism driver support VNIC types validation is done now in the
"SimpleAgentMechanismDriverBase" class __init__ method. If a subclass
needs to administratively prohibit any VNIC type supported by default,
"vnic_type_prohibit_list" must be passed to the base class __init__
call.
Related-Bug: #1578989
Change-Id: Ic25a8a7c716b4980ad2542b44519f77c6fdad309
The rpc_response_max_timeout parameter is used in comminucation over
messaging queue, thus should be available for sriov-agent which
communicate with neutron-server over messaging queue.
Change-Id: Ie6ae31e40488fd8f3d43e83b25e536a7dd9d938c
Closes-Bug: #1930996
SR-IOV agent can handle ports with same MAC address (located in
different networks). The agent can retrieve, from the system, the
MAC address and the PCI slot; because the PCI slot is unique per
port in the same host, this parameter is used to match with the
Neutron port ID stored in the database (published via RPC).
RPC API bumped to version 1.9.
Closes-Bug: #1791159
Change-Id: Id8c3e0485bebc55c778ecaadaabca1c28ec56205
Currently neutron uses socket.gethostname() to determine hypervisor
names, but this implementation is not fully compatible with libvirt
driver which uses canonical name for hypervisor name.
This incompatibility causes an issue with root resource provider
detection if a deployment uses FQDNs as canonicanl names.
This change introduces the resource_provider_default_hypervisor option,
so that users can override the hypervisor name by the single option(*1)
instead of setting two list options(*2). This is especially useful if
the deployment has multiple bridges or interfaces.
(*1)
[OVS]
resource_provider_bandwidths=br-data1:1024:1024,br-data2:1024:1024,\
br-data3:1024,1024,br-data4,1024:1024
resource_provider_default_hypervisor=compute0.mydomain
(*2)
[OVS]
resource_provider_bandwidths=br-data1:1024:1024,br-data2:1024:1024,\
br-data3:1024,1024,br-data4,1024:1024
resource_provider_hypervisors=br-data1:compute0.mydomain,br-data2:\
compute0.mydomain,br-data3:compute0.mydomain,br-data4:compute0.mydomain
Related-Bug: #1926693
Change-Id: I692219200535df3af1265248e88c96947e4d8f9d
Some NICs do not support ip-link vf "min_tx_rate" parameter. This is
not an API issue (ip-link or Pyroute2); the parameter is already
supported. The error is returned by the specific NIC driver.
A sanity check is implemented, reading the SR-IOV configured devices
("physical_device_mappings") and the excluded VFs ("exclude_devices").
Change-Id: If70de0a7112777ac4011ad42af0ac98969765011
Closes-Bug: #1918464
Add support in SR-IOV mechanism driver for VNIC type
"accelerator-direct".
This VNIC type refers to any device that supports hardware
acceleration of any kind and is provided by Cyborg [1].
NOTE: "accelerator-direct-physical" is not supported yet.
[1]https://wiki.openstack.org/wiki/Cyborg
Change-Id: I529e6a2a445a140dca7686976efeefcd13d333f0
Closes-Bug: #1909100
There is no real reason we should be using some of the
terms we do, they're outdated, and we're behind other
open-source projects in this respect. Let's switch to
using more inclusive terms in all possible places.
Change-Id: I99913107e803384b34cbd5ca588451b1cf64d594
Now that we use setproctitle for neutron-server workers (and
neutron-keepalived-state-change), this has the side effect of changing
the process name for agents, impacting some monitoring systems. More
details in launchpad bug.
This patch fixes it by setting the name with setproctitle to:
agent name (original process name).
Also use the newly introduced name constants to replace existing
hardcoded uses.
Change-Id: I74c3a4d3e9f833752571a75f196560cd45529385
Closes-Bug: #1881297
Recent changes in some versions of iproute2 CLI output (v4.18),
have invalidated the regular expression used to parse the
"ip link" output.
To solve this problem and avoid future ones, pyroute2 is used to
retrieve the virtual functions information and set the VF attributes
(spoofcheck, min_tx_rate, max_tx_rate and link_state).
pyroute2 extended the "ip link" support to retrieve this information,
adding "ext_mask=1" in the get command. If no virtual functions are
present in this particular network interface, the added method,
"get_link_vfs", will return an empty list.
The set commands can return a "InterfaceOperationNotSupported" in
case the operation is not supported. For min_tx_rate, if the driver
does not support to set a minimum bandwidth, an "InvalidArgument"
(from a pyroute2.NetlinkError(22)) exception will be raised.
Change-Id: I680da4f64bd114f1caecaaeedbf8a4b1915a0849
Closes-Bug: #1878042
When the SR-IOV agent starts, print information about the Placement
resource providers: bandwidth, inventory default configuration and
hypervisor names.
Change-Id: Ib34f7a4f850301b2a8bf739c9d8d5e73bcc1bc48
Related-Bug: #1578989
This commit modifies the way sriov agent retrieves mac
addresses for macvtap assigned VFs to use the effective
mac taken from the macvtap net device instead of the
administrative mac set on the PF.
This commit rids sriov agent from depending on hypervisor/nova
to set both administrative and effective mac for macvtap ports.
Related-Bug: #1841067
Change-Id: Id499bcc49d27f13f7f03481922a3383b4a255da1
Previously we assumed that we can look up the resource provider (created
by nova) to be used as the parent of the agent and physical NIC resource
provider tree by the name set in the config option DEFAULT.host. This
assumption was wrong.
While nova-compute's DEFAULT.host and neutron-agent's DEFAULT.host
must match for port binding to work, the root resource provider created
by nova does not belong to the compute host (where nova-compute runs)
but it belongs to the compute nodes (i.e. hypervisors). Actually there
may be multiple compute nodes managed by a single nova-compute (think
of ironic). Plus the value of DEFAULT.host and the compute node's ID
may be different even when nova-compute manages a hypervisor on the
same host because of various deployment considerations. For example
when tripleo does not manage the undercloud (so a libvirt hypervisor
returns the plain hostname), but the same tripleo enforces it's host
naming conventions in nova's and neutron's DEFAULT.host settings.
This change enables neutron to use the hypervisor name to locate the
root of the resource provider tree.
We introduce a new configuration option for
(1) ovs-agent: resource_provider_hypervisors, for example:
[ovs]
bridge_mappings = physnet0:br-physnet0,...
resource_provider_bandwidths = br-physnet0:10000000:10000000,...
resource_provider_hypervisors = br-physnet0:hypervisor0,...
(2) sriov-agent: resource_provider_hypervisors, for example:
[sriov_nic]
bridge_mappings = physnet1:ens5,...
resource_provider_bandwidths = ens5:10000000:10000000,...
resource_provider_hypervisors = ens5:hypervisor1,...
For both agents 'resource_provider_hypervisors' values default to
socket.gethostname() for each key in resource_provider_bandwidths.
We try to not block later developments in which one neutron
agent may manage devices on multiple hosts. That's why we allow
the each physdev to be associated with a different hypervisor.
But here we do not try to solve the problem that the natural physdev
identifiers may not be unique accross multiple hosts. We leave solving
this problem to whoever wants to implement an agent handling devices of
multiple hosts.
(3) We extend report_state message's configurations field alike:
{
'bridge_mappings': {'physnet0': 'br-physnet0'},
'resource_provider_bandwidths': {
'br-physnet0': {'egress': 10000000, 'ingress': 10000000}},
'resource_provider_hypervisors': {'br-physnet0': 'hypervisor0'},
...
}
(4) In neutron-server we use
report_state.configurations.resource_provider_hypervisors.PHYSDEV
when selecting parent resource provider for agent and physdev
RP-tree. When not available in the message we fall back to using
report_state.host as before.
Since we only changed the free-format configurations field of the
report_state message rpc version is not bumped and we expect this
change to be backported to stein and train.
Change-Id: I9b08a3a9c20b702b745b41d4885fb5120fd665ce
Closes-Bug: #1853840
This reverts commits:
9a022e7d7bd760416998
As the issue is relevant only for old kernels and almost 4 years
have passed since this fix was introduced, it is safe to undo the
workaround to use ip link show command for determining whether or not
a VF is assigned with macvtap(and followup patch that fixed excessive
use of ip link show commans).
reverting this commit will benefit the agent by reducing the amount of
ip link calls.
While it is possible to perform a revert-per-commit, this change should
really go in as one commit for clarity and avoid introducing a
performance degradation in case one is merged without the other.
Conflicts:
neutron/plugins/ml2/drivers/mech_sriov/agent/eswitch_manager.py
neutron/plugins/ml2/drivers/mech_sriov/agent/pci_lib.py
neutron/tests/unit/plugins/ml2/drivers/mech_sriov/agent/test_eswitch_manager.py
neutron/tests/unit/plugins/ml2/drivers/mech_sriov/agent/test_pci_lib.py
Conflicts details:
In eswitch_manager: merge tool was unable to properly revert
is_assigned_vf() due to changes in later patches.
In pci_lib: conflicts with later changes to is_macvtap_assigned()
and link_show() that are no longer required and revert
of IPCommandDeviceError definition.
In unit tests: conflicts due to newly added tests and reverting the
IPCommandDeviceError exception definition.
Sorting out mocks as link_show() method was removed
Related-bug: #1523083
Change-Id: I04ea8eb63de07a6e1e51c2790c5920b086b8542c
Added "connectivity" parameter to mech driver vif_details. The default
value is "legacy". The in-tree drivers (Linux Bridge, Open vSwitch, SR-IOV
and MacVTap) have "l2" connectivity.
Change-Id: I45480986fc89e0b2f475ee0ceb13d8742fb8c530
Related-Bug: #1821058
Check for configured and actual number of VFs to prevent
device registaration with 0 VFs.
Closes-Bug: #1831622
Change-Id: Ie699d245f8ae2fc1d16b96432d2962788d9dba57
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Add support for binding activate and deactivate callbacks
in SR-IOV mechanism driver agent.
this work item is required for supporting VM live-migration
with SR-IOV in nova with libvirt as hypervisor.
This commits implemets the following:
- Implement binding_activate() and binding_deactivate() methods in agent.
- Configure agent to listen on the relevant topics.
- RPC version bump on agend side.
- When processing an activated binding treat it as a newly added port.
- Deactivated binding will trigger a NO-OP as once a binding is
deactivated it will be removed by Hypervisor.
Closes-Bug: #1815361
Change-Id: I6b7195e08ed8932cfc2785b66627de2854ead85d
If the SR-IOV agent fails to send its report_state
to the server, it logs an exception:
Failed reporting state!: MessagingTimeout: Timed out...
If it then tries a second time and succeeds it just
goes on happily. It would be nice if it logged that
it had success on the subsequent attempt so someone
looking at the logs know it recovered.
Change-Id: I0782599a4f0ffe48c2dd4fa2abc9fe5111715c0c
get_mappings for machanism drivers return back different dicts for
different mechanism drivers. By adding get_standard_device_mappings
method SRIOV and OVS mechanism drivers can return a mapping dict in a
form like this one:
{'physnet_name': ['device_or_bridge_1', 'device_or_bridge_2']}
Change-Id: Ieddd9b3a4f3d7269aafc373f040c55b025f5c201
Related-Bug: #1578989
Now supported_vnic_types is hardcoded to the mechanism drivers, but that
can depend on several factors, like type of the NIC, admin decision,
etc.
With this patch we put the right to decide which vnic types are
supported for ovs agent into the hands of the admin, by allowing
blacklisting items from the mechanism driver specific list.
Background: http://eavesdrop.openstack.org/meetings/neutron_qos/2018/
neutron_qos.2018-07-31-15.00.log.html#l-58
Change-Id: Iad9e2e966df53b4164d2a56a93215c69825b5241
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
Enable QoS minimum-bandwidth rule for ovs and sriov backends for both
ingress and egress directions.
Add qos-bw-limit-direction as supported extension to qos-plugin, and
add create/delete/update_minimum_bandwidth and
delete_minimum_bandwidth_ingress empty methods to sriov/ovs/linuxbridge
extension drivers.
Change-Id: I6eb21ccf0400ea9adae90ff0bf97e08cdb09b8eb
Depends-On: https://review.openstack.org/609368
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
Sometimes due to NIC driver incorrect behavior, VFs might be
missing in 'ip link show' output. This may lead to a VM boot
failure as agent will just skip such missing devices.
Make the agent do a resync in case a newly added device
'disappears' during processing, which should cause a MAC to
get assigned.
Co-authored-by: Oleg Bondarev <obondarev@mirantis.com>
Change-Id: I148b5a025fc388821fd1269d02908cc8ce1882fe
Closes-bug: #1784484
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them. Also removed an N536 comment missed from a
previous change.
Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
The option is currently defined in two places - as a generic agent
option and as a sriov agent specific one. This is a problem now that
oslo.config tracks location where options are defined
(I6dec1e09dcab203c6287f9c56c866f220a42f850) because when unit test
runner imports all modules, it registers the option from those two
separate locations, and since locations are different, it's considered
the options are duplicate, which results in the following error:
DuplicateOptError: duplicate option: polling_interval
To fix the issue, I removed the definition of the option from sriov
module and made sure the common agent options are registered whenever
the sriov module is imported.
Change-Id: I485532692f4b3a5804885e0325a8352fe6b1aa65
Closes-Bug: #1759345
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.
NeutronLibImpact
Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
Brian Haley