MechanismDriverContext has an attribute _plugin_context, which carries
the current context with it. This is used by many ml2 drivers, as it is
the only way for them to get the current context. We now make this a
public API by adding a property to MechanismDriverContext that returns
_plugin_context as a read-only attribute.
Change-Id: If9b05655286f42081cf26c90c563429ca2e63244
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/plugins.
Trivialfix
Change-Id: Id9138652f5f07ef12fa682e182fe210019e8f975
If subnet is attached to a segment we want to endure that the bind
will happen on it.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: I56b22820d29b2d57faf28a2f9b685ab0b2c924b4
The base class "MechanismDriver" now has a property called
"connectivity". This patch overrides the default value in the
in-tree drivers.
The method "_check_drivers_connectivity" now uses this property
that is available in all drivers.
Depends-On: https://review.opendev.org/c/openstack/neutron-lib/+/826501
Closes-Bug: #1959125
bp boot-vm-with-unaddressed-port
Related-Bug: #1821058
Change-Id: I91734835b07d804365b46adfb26e984557107d80
Check new segments added to OVN mechanism driver. This mechanism
driver does not inherit from ``SimpleAgentMechanismDriverBase``
because OVN has no agents (NOTE 1). However OVN mechanism driver
requires an OVN controller service in each deployed chassis. This
is how OVN driver knows the hosts using this backend.
The segments, attached to an external network (connected to a
physical network), can be mapped to a host if ther is a chassis (OVN
controller agent) in this host.
NOTE 1: OVN provides agent API compatibility, presenting the
controller and the metadata services as agents. But OVN mechanism
driver has no control or provides any information (via RPC) to
those services.
Closes-Bug: #1949967
Change-Id: I570b2251da8a4a25d155ba838346b24afafd727f
The mechanism driver support VNIC types validation is done now in the
"SimpleAgentMechanismDriverBase" class __init__ method. If a subclass
needs to administratively prohibit any VNIC type supported by default,
"vnic_type_prohibit_list" must be passed to the base class __init__
call.
Related-Bug: #1578989
Change-Id: Ic25a8a7c716b4980ad2542b44519f77c6fdad309
Provisioning Blocks [1] was introduced to manage composite
object status. Port object is the one Neutron sets provisioning
block during the port processing life cycle. Here is the compute
port (VM's NIC port) processing procedure:
1. nova creates port
2. the 'openvswitch' mechinism driver inserts provisioning block
for this port
3. nova calls related interface to plug the device
4. L2-agent sets the flows (or rules/devices) for the port and
call update_device_list to neutron-server
5. neutron-server try to set port status to ACTIVE
6. neutron-server notify nova that "vif-plugged" success
This works fine for VM with its ports. But for neutron service port,
like router_gateway, router_interface and dhcp, it is unnecessary.
Because there is no dependency among neutron resources. Neutron
just knows that the ports had been set properly. And another thing
is, for most of these internal service port, there is no need of
DHCP, security group or port security.
So for neutron internal service ports, the procesure can be:
1. neutron L3/DHCP/X related service plugin creates port
2. no provisioning_block
3. L3/DHCP/X related agent plug the port
4. L2-agent sets the flows (or rules/devices) for the port and
call update_device_list to neutron-server
5. neutron-server sets port status to ACTIVE directly, then done!
This patch will set neutron *AgentMechanismDrver (including built-in
drivers: linuxbridge, macvtap, sriov, openvswitch) to skip inserting
the provisioning_block for Neutron internal service ports.
[1] https://docs.openstack.org/neutron/latest/contributor/internals/provisioning_blocks.html
Closes-Bug: #1930432
Change-Id: Iaf7788bf0cba19a693cbf456f98e50d7b5de9e41
Each mechanism driver loaded in the ML2Plugin can provide what
extensions are supported. By default, any mech driver can support
all extensions.
This extension filtering is done during the plugin initialization.
If any requested extension is not supported, it will be silently
removed from the loaded extensions.
Depends-On: https://review.opendev.org/#/c/743519
Related-Bug: #1888829
Change-Id: Id4809d4f3c4e13a93f9160df3618f8602b57437c
There is no real reason we should be using some of the
terms we do, they're outdated, and we're behind other
open-source projects in this respect. Let's switch to
using more inclusive terms in all possible places.
Change-Id: I99913107e803384b34cbd5ca588451b1cf64d594
With python 3.x, classes can use the metaclass= logic
to not require usage of the six library.
One step in removing all of six usage from neutron.
Change-Id: I2f815e412d9a96eb5faf2b3bb3a1e393a9db9309
Added "connectivity" parameter to mech driver vif_details. The default
value is "legacy". The in-tree drivers (Linux Bridge, Open vSwitch, SR-IOV
and MacVTap) have "l2" connectivity.
Change-Id: I45480986fc89e0b2f475ee0ceb13d8742fb8c530
Related-Bug: #1821058
In patch [1] there was added usage of context parameter in method
get_vif_type in openvswitch mech_driver.
This exposed that previously "wrong" context was passed to this method
in _update_segmentation_id() method in ml2 plugin and that caused
raising AttributeError as "Context" object didn't have attribute
'current'.
This patch adds new method "get_supported_vif_type" to mechanism
drivers and this method don't need context to return what vif_types
are supported regarding agent type.
[1] https://review.opendev.org/#/c/658784/
Change-Id: Ic6c738db28208e5009f78bb52598eb3c141f639f
Related-Bug: #1832985
In case of Smart NIC vNIC type neutron should mimic nova-compute
that plug the port to the ovs bridge.
Extend the Neutron OVS mechanism driver and Neutron OVS Agent to bind
the Neutron port for the baremetal host with Smart NIC. This will allow
the Neutron OVS Agent to configure the pipeline of the OVS running on
the Smart NIC and leverage the pipeline features such as: VXLAN,
Security Groups and ARP Responder.
Story: #2003346
Closes-Bug: #1785608
Change-Id: I6d520d3bac2e9ceb30b5b6197c6eb0f958cc3659
Rehomed provider_network_attribute_updates_supported to
plugins.ml2.api.MechanismDriver, in order to be inherited in other
mechanism drivers.
Related-Bug: #1823865
Change-Id: I9be175aa2334111a17d8fec874d2331e23923837
In the ML2 plugin, allow to update the segmentation ID of a single
provider network.
A new method in the "SimpleAgentMechanismDriverBase" is added:
"provider_network_attribute_updates_supported". This method returns,
if implemented in the specific agent, which network attributes can
be updated on a live network with ports bound to this network back-end.
By default, an empty list is returned.
Partial-Bug: #1806052
Change-Id: I2595335d6fbc51562b070f14eaeaadf49cf7c418
Drive the choice of mechanism driver during binding as inferred from
the resource provider allocated by nova and as told to neutron via the
port's binding:profile.
As discussed on a neutron qos irc meeting some time ago
this patch introduces a new assumption on bind_port() implementations.
That is an implementation of bind_port() in any mech driver supporting
Guaranteed Minimum Bandwidth bind_port() must not have a non-idempotent
side effect. Because the last binding level will be redone for a 2nd
time with a narrowed down list of mechanism drivers. And if the 2nd call
does not give the same result as the first all kind of weird things can
happen.
Change-Id: I2b7573ec6795170ce45a13d5d0ad7844fb85182d
Depends-On: https://review.openstack.org/574781
Depends-On: https://review.openstack.org/635160
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
Now supported_vnic_types is hardcoded to the mechanism drivers, but that
can depend on several factors, like type of the NIC, admin decision,
etc.
With this patch we put the right to decide which vnic types are
supported for ovs agent into the hands of the admin, by allowing
blacklisting items from the mechanism driver specific list.
Background: http://eavesdrop.openstack.org/meetings/neutron_qos/2018/
neutron_qos.2018-07-31-15.00.log.html#l-58
Change-Id: I63e562e2eccc5b02c1c767d6a2c28cb803131e99
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.
NeutronLibImpact
Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
The ml2 MechanismDriver is now in neutron-lib along with its associated
constants. This patch switches over to the lib versions of those, but
leaves a shim of the MechanismDriver that just ref's the driver from
lib. This shim allows our broad consumer base of the driver to switch
over at their leisure.
NeutronLibImpact
Change-Id: I99e3de6d933a1bb341394f85415fb07306a82a01
This moves the get_vif_type hook point from OVS into the mech agent
base so other mechanism drivers can also return different VIF types
depending on the context of the binding.
This will be used by linux bridge in change
I23c5faaeab69aede1fd038a36f4a0b8f928498ce.
Change-Id: I3b695909c954158df90f436a7ed259890977d25a
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
When multiple mech drivers are loaded, it's very common to
have on not find agents on a host, we shouldn't emit a warning
in this condition. The mechanism manager will log an error if
there is an actual failure to bind a port.
Change-Id: Icb643f6ffc6699579394d1e5d42f7bbfdcad6de4
Closes-Bug: #1622559
Sometimes an object requires multiple disjoint actors to complete
a set of tasks before the status of the object should be transitioned
to ACTIVE. The main example of this is when a port is being created.
The L2 agent has to do its business to wire up the VIF, but at the same
time the DHCP agent has to setup the DHCP reservation. This led to
Nova booting the VM when the L2 agent was done even though the DHCP
agent may have been nowhere near ready.
This patch introduces a provisioning blocks mechansim that allows the
entities to be tracked that need to be involved to make a transition
to ACTIVE happen. See the devref in the dependent patch for a high-level
view of how this works.
The ML2 code is updated to use this new mechanism to prevent updating
the port status to ACTIVE without both the DHCP agent and L2 agent
reporting that the port is ready.
The DHCP RPC API required a version bump to allow the port ready
notification.
This also adds a devref doc for the provisioning_blocks
module with a high-level overview of how it works in addition
to a detailed description of how it is used specifically with
ML2, the L2 agents, and the DHCP agents.
Closes-Bug: #1453350
Change-Id: Id85ff6de1a14a550ab50baf4f79d3130af3680c8
When binding a port fails, it's really hard right now to
understand the reasons of the failure, unless the log level
is set to debug. In addition to that many log statement
are duplicated.
The goal of this patch is to solve these problems.
Closes-bug: #1543510
Change-Id: I262973126e8ca5af2b20a3a5bc35dea4251c0a78
Currently neutron DCHP scheduler assumes that that every server running
a dhcp-agent can reach every network. Typically the scheduler can
wrongly schedule a vlan network on a dhcp-agent that has no reachability
to the network it's supposed to serve (ex: network's physical_network
not supported).
Typically such usecase can append if:
* physical_networks are dedicated to a specific service and we don't
want to mix dnsmasqs related to different services (for
isolation/configuration purpose),
* physical_networks are dedicated to a specific rack (see example
diagram http://i.imgur.com/NTBxRxk.png), the rack interconnection can
be handled outside of neutron or inside when routed-networks will be
supported.
This change makes the DHCP scheduler network reachability aware by
querying plugin's filter_hosts_with_network_access method.
This change provides an implementation for ML2 plugin delegating host
filtering to its mechanism drivers: it aggregates the filtering done by
each mechanism or disables filtering if any mechanism doesn't overload
default mechanism implementation[1] (for backward compatibility with
out-of-tree mechanisms). Every in-tree mechanism overloads the default
implementation: OVS/LB/SRIOV mechanisms use their agent mapping to filter
hosts, l2pop/test/logger ones return empty set (they provide to "L2
capability").
This change provides a default implementation[2] for other plugins
filtering nothing (for backward compatibility), they can overload it to
provide their own implementation.
Such host filtering has some limitations if a dhcp-agent is on a host
handled by multiple l2 mechanisms with one mechanism claiming network
reachability but not the one handling dhcp-agent ports. Indeed the
host is able to reach the network but not dhcp-agent ports! Such
limitation will be handled in a follow-up change using host+vif_type
filtering.
[1] neutron.plugin.ml2.driver_api.MechanismDriver.\
filter_hosts_with_network_access
[2] neutron.db.agents_db.AgentDbMixin.filter_hosts_with_network_access
Closes-Bug: #1478100
Co-Authored-By: Cedric Brandily <zzelle@gmail.com>
Change-Id: I0501d47404c8adbec4bccb84ac5980e045da68b3
The previous message was misleading because it made it
sound like port binding was being attempted even though
the agent is dead. However, the actual logic is that
binding will be completely skipped if the agent is dead.
This patch updates the message to make that clear and also
provides the port ID as part of the warning so operators
without debugging enabled can see which ports failed.
Change-Id: Ic5031ad8fb06062e9d12b87430d89fc37eb6cde0
Closes-Bug: #1522192
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
Port binding failure is an error and should be reported as such.
Additionally, if a port binding fails on a host due to missing
bridge mappings, it's currently quite a mystery to find out. This
should be logged instead of requiring users to debug code. Not
everyone enjoys debugging Python, as weird as that is!
I refactored out the common code in check_segment_for_agent
in order to make logging more robust for all agent-based mechanism
drivers. The OVS and LB mech drivers already log due to a bridge
mappings mismatch and the other agent based mech drivers
will now log as well.
Closes-Bug: #1377710
Change-Id: I451a0763908adcd845721e8cda7f3189da6c8b81
The following ML2 driver API changes are required to support
hierarchical port binding:
* Add segments_to_bind PortContext property containing the list of
network segments with which a mechanism driver should try to bind
the port. All mechanism drivers that bind ports should now use this
property in place of network.network_segments, which contains only
the network's static segments.
* Replace several PortContext properties (bound_segment, bound_driver,
original_bound_segment, original_bound_driver) with new properties
(binding_levels, top_bound_segment, bottom_bound_segment,
original_binding_levels, original_top_bound_segment,
original_bottom_bound_segment) in order to represent hierarchical
bindings.
* Add stubbed-out continue_binding() method to PortContext, allowing
mechanism drivers to partially bind the port.
All existing drivers and unit tests are updated accordingly.
The DB schema changes and logic required for hierarchical port binding
will be implemented in dependent patches.
Gerrit Spec: https://review.openstack.org/#/c/139886/
Partially-implements: blueprint ml2-hierarchical-port-binding
Change-Id: Icb1a016f4661e427cb6cfa3452802ba5e64b7124
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
All the existing LOG.info, LOG.warning, LOG.error and LOG.critical
messages should have _LI, _LW, _LE and _LC respectively.
Also, debug level log shouldn't be translated.
This patch set will cover the ml2 directory under neutron/plugins.
Partial-Bug: #1320867
Change-Id: I9d78d23bbc14e7c536c6ddf2dc4f52c67faeb667
The ML2 plugin now calls the bind_port() operation on the registered
mechanism drivers outside of any enclosing DB transaction. Ports are
created or updated in one transaction, then a binding is established
if possible, and finally a second transaction commits the binding
result.
With [re]binding moved outside the DB transaction that triggered it,
it is now possible that multiple threads or processes will
concurrently try to bind the same port, or that the port will be
updated between transactions. Concurrent attempts to bind the same
port are allowed to proceed, which results are used is resolved in the
second transaction, and binding is retried if necessary.
Improvements to the Cisco Nexus driver and unit tests from Rich Curran
needed due to the binding changes are also included.
Closes-Bug: 1276391
Closes-Bug: 1335226
Change-Id: I65dafc330d6e812dad0667d2383858504d0ba299
This commit enables the H301 hacking rule by removing all the
multiple imports in a single line from neutron.
Partial-Bug: #1291032
Change-Id: I7ba7f82fb36a433d73190eb3d568b6961ccb57c6
The API implemented by ML2 mechanism drivers included three methods
related to port binding that were called within DB transactions, but
that could potentially involve communication with controllers or
devices that should not be done within transactions. A subsequent
patch will move the calls to bind_port() outside of tranactions. This
patch eliminates the other two methods from the MechanismDriver API.
The validate_port_binding() method was previously called on the bound
mechanism driver to check whether an existing binding was still valid,
so that the port could be rebound if something changed. But since nova
has no way to handle changes to binding:vif_type or
binding:vif_details after a port is initially plugged, this turned out
not to be useful, so the method has been removed from the
MechanismDriver API. Now, once a port is successfully bound, the
binding remains until the port is deleted or any of it's
binding:host_id, binding:vnic_type, or binding:profile attribute
values are changed.
The unbind_port() method was previously called on the bound mechanism
driver as an existing binding was removed. This method was not used by
any existing mechanism drivers, and was redundant with the
update_port_precommit() and update_port_postcommit() methods that are
called on all mechanism drivers when an existing binding is removed,
so this method has also been removed from the driver API.
Eliminating the unbind_port() call allows the binding details to be
made available via the PortContext in delete_port_postcommit() calls,
completing the resolution of bug 1276395.
Closes-bug: 1276395
Partial-bug: 1276391
Change-Id: I70fb65b478373c4f07f5273baa097fc50e5ba2ef
Sebastian Lohff
