This is an initial patch in a series that, by using an event framework,
cleans up the relationship between ML2 and L3, so that they are no longer
tightly coupled. A follow-up will take address the coupling during the
port delete operation.
The newly introduced notification hooks not only benefit the L3 service
plugin, but any other plugin that has an interest in knowing about port
events.
Long term, the notification bits can move in a more 'common' place so that
other plugins can take advantage of them, but as mentioned in a parent patch,
the perestroika is not quite there yet.
Related-blueprint: services-split
Related-blueprint: plugin-interface-perestroika
Change-Id: I6527b1cb53a71a1f68329a7a3b1878094558f8c2
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
Sentinel patch to establish whether commit [1] is at
fault of the sudden spike of DBDeadlock errors.
[1] d6a55c1736
Change-Id: I021e94e322f9b5eca665ea3aede41e034d3047cc
Closed-Bug: #1426543
Description:
Stale GRE and VXLAN tunnel endpoints persists in neutron db this should be
deleted from the database. Also, if local_ip of L2 agent changes the
stale tunnel ports and flows persists on br-tun on other Compute Nodes and
Network Nodes for that remote ip this should also be removed.
Implementation
Plugin changes:
The plugin side changes are covered in following patch-set
https://review.openstack.org/#/c/121000/.
Agent changes:
Added tunnel_delete rpc for removing stale ports and flows from br-tun.
tunnel_sync rpc signature upgrade to obtain 'host'.
Added testcases for TunnelRpcCallbackMixin().
This patch-set agent deals with agent side changes.
Closes-Bug: #1179223
Closes-Bug: #1381071
Closes-Bug: #1276629
Co-Authored-By: Aman Kumar <amank@hp.com>
Co-Authored-By: phanipawan <ppawan@hp.com>
Change-Id: I291992ffde5c3ab7152f0d7462deca2e4ac1ba3f
When port is being deleted via API it is not removed
from ovsdb and corresponding iptables chains remain
even though the port does not exist.
This patch adds a notification for the ovs neutron agent,
upon which the port is deleted from ovsdb.
Co-Authored-By: Akash Gangil <akashg1611@gmail.com>
Closes-Bug: #1333365
Change-Id: Iccda3bee98d561ef3a06d0317d3d68d6b1dfb76b
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
The following ML2 driver API changes are required to support
hierarchical port binding:
* Add segments_to_bind PortContext property containing the list of
network segments with which a mechanism driver should try to bind
the port. All mechanism drivers that bind ports should now use this
property in place of network.network_segments, which contains only
the network's static segments.
* Replace several PortContext properties (bound_segment, bound_driver,
original_bound_segment, original_bound_driver) with new properties
(binding_levels, top_bound_segment, bottom_bound_segment,
original_binding_levels, original_top_bound_segment,
original_bottom_bound_segment) in order to represent hierarchical
bindings.
* Add stubbed-out continue_binding() method to PortContext, allowing
mechanism drivers to partially bind the port.
All existing drivers and unit tests are updated accordingly.
The DB schema changes and logic required for hierarchical port binding
will be implemented in dependent patches.
Gerrit Spec: https://review.openstack.org/#/c/139886/
Partially-implements: blueprint ml2-hierarchical-port-binding
Change-Id: Icb1a016f4661e427cb6cfa3452802ba5e64b7124
For some reason sometimes at the same time for the same port
delete_port and update_device_down commands have been executed.
This arise StaleDataError in update_device_down. In other situations
these commands are executed one after another and the error doesn't
appear.
This errors does not show anything broken, but to avoid confusion
it will be skipped.
Closes-bug: #1405379
Change-Id: I4e5a3d1f1866b9cfba44d3cb7ccc5b2dee1d9633
Convert several places in the code that were using the RpcCallback
compatibility class. This class eased the transition to
oslo.messaging. This patch drops the usage of this class in favor of
direct usage of the appropriate oslo.messaging API.
Part of blueprint drop-rpc-compat.
Change-Id: I955958e836635d43dabc7b39d050c1418b18cf8f
Remove usage of the RpcProxy compatibility class from the ml2
AgentNotifierApi. The equivalent oslo.messaging APIs are now used
instead. A couple of other mixin APIs had to be converted at the same
time.
Note that there is one very minor functional change here. The base
rpc version is set to '1.0' now instead of '1.1'. The right pattern
to use is to always set the base to be N.0. Any method that needs a
newer version should specify it.
Part of blueprint drop-rpc-compat.
Change-Id: I640568e2d73c9eb7a9505db640dc1427a1ae2abe
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
All the existing LOG.info, LOG.warning, LOG.error and LOG.critical
messages should have _LI, _LW, _LE and _LC respectively.
Also, debug level log shouldn't be translated.
This patch set will cover the ml2 directory under neutron/plugins.
Partial-Bug: #1320867
Change-Id: I9d78d23bbc14e7c536c6ddf2dc4f52c67faeb667
The dvr vmarp table update notification was being called inside
of the delete_port transaction in ML2, which can cause a yield
and lead to the glorious mysql/eventlet deadlock.
This patch moves it outside the transaction and adjusts it to
use an existing port dictionary rather than re-looking it up since
the port is now gone from the DB by the time it is called.
Closes-Bug: #1377241
Change-Id: I0b4dac61e49b2a926353f8478e421cd1a70be038
Follow-up patch of RPC refactoring of bug 1359416.
It addresses minor comments in the above patch series.
Change-Id: I2d6268db777f0f73fda61a5a7d0967a91bcb292b
Closes-Bug: #1359416
RPC has a version of itself. In Neutron a plugin implements
several RPC interface, so a single RPC version doesn't work.
In Mixin callback class approach, RPC versioning depends on
each plugin implementation and it makes harder to maintain
RPC version appropriately. This patch series replaces mixin
RPC callback of server side with a separate class.
This commit handles server-side callback of security group
RPC interface.
* The server-side callback of Security group RPC is moved to
api/rpc/handler and db/securitygroups_rpc_base now only
contains a mixin class to add agent-based security group
implementation with db operations.
* get_port_from_device method in server-side callback class
is moved to a mixin class of plugin implementation
(SecurityGroupServerRpcMixin) because it involves DB lookup
and is tightly coupled with plugin implementation rather
than RPC interface definition.
Most unit tests for SGServerRpcCallBackTestCase were skipped
in the base class before, but now they are no longer skipped.
The following items will be planned in later patches
to avoid drastic changes in a single patch.
* Merge security group RPC API and agent callback classes in
agent/securitygroups_rpc into api/rpc/handlers/securitygroup_rpc
* Remove completely duplicated db access code in get_port_from_device
and get_port_and_sgs
Partial-Bug: #1359416
Change-Id: Ia6535217d2e3b849a95667c1b53dd09675002892
RPC has a version of itself. In Neutron a plugin implements
several RPC interface, so a single RPC version doesn't work.
In Mixin callback class approach, RPC versioning depends on
each plugin implementation and it makes harder to maintain
RPC version appropriately. This patch series replaces mixin
RPC callback of server side with a separate class.
This commit handles server-side callback of DVR ML2 RPC interface.
Partial-Bug: #1359416
Change-Id: I1b6383f7b0af5d9aed18eda3a15f21d3504d0347
RPC has a version of itself. In Neutron a plugin implements
several RPC interface, so a single RPC version doesn't work.
In Mixin callback class approach, RPC versioning depends on
each plugin implementation and it makes harder to maintain
RPC version appropriately. This patch series replaces mixin
RPC callback of server side with a separate class.
This commit handles server-side callback of dhcp-agent RPC interface.
DHCP-agent server-side callback class is moved from db/ to
api/rpc/handlers because it doesn't involve any db operations
and defining all RPC interfaces in a single place sounds reasonable.
Note that moving other DHCP-agent related RPC interface class
to api/rpc/handlers can be done in a separate patch as this patch
focuses on reorganizing the server-side RPC callback class.
Partial-Bug: #1359416
Change-Id: Ifb2a1bc0b7971995aae2856c9d4cd88c6dbc22d6
Currently, DVR router namespaces are created only
when there is a valid VM port on the compute
node, or for the gateway-port on the service node.
But when an LBaaS VIP port is created the l3 agent
does not create a DVR namespace to service the VIP port.
This fix enables DVR namespaces to be created to
service the LBaaS VIP port.
Also, this fix enables L2 Agent running in DVR
mode, to add-in OVS rules to enable packets to
be routed to such LBaaS VIP Ports which are
resident on DVR routed interfaces.
Therefore, with this fix both East-West and
North-South traffic will be serviced by DVR
for LBaas VIP Ports.
DocImpact
Authored-by: Swaminathan Vasudevan <swaminathan.vasudevan@hp.com>
Co-Authored-By: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Change-Id: I698b971d50721fb0512a11569f7d3139d0d456f3
Closes-Bug: #1356464
This will be used by ofagent to route packets to local VMs.
Partially-implements: blueprint ofagent-merge-bridges
Change-Id: Icaa26448a370dd75d09e95ed439df5b486a862cf
An agent's request to update the ARP entry for a VM port
may come after a deletion request has been processed,
resulting in a PortNotFound exception being raised.
This patch takes care of this condition. A test has
been added, which required a minor refactoring of the
test case class, in order to accommodate the use of
side effects for the objects being mocked.
Closes-bug: #1356120
Change-Id: I40d635bcf47c683663cb4dedf20323902dff2c7f
Without making this call conditional, every l3plugin that
integrates with the ML2 plugin will need to implement this
method and this must not be necessary.
Closes-bug: #1349638
Change-Id: Ie9ba3bad4152810f5bfa530be54be70139cebc0c
This patch introduces the changes necessary to
support DVR at Layer 2 with ML2, and L2pop.
Partially-implements: blueprint neutron-ovs-dvr
Change-Id: I709d877e7df83728868e68e023e6a37bb2e11704
Authored-by: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
This set of changes introduces the ML2 mechanism driver for SR-IOV capable NIC based switching.
Please see the blueprint for more information.
The review is submitted in two parts:
- Part 1 (this part)
The Mechanism Driver to support port binding for SR-IOV virtual functions of
SRIOV capable switching NICs, such as Mellanox ConnectX Family.
Use configurable list of supported NIC vendor and product PCI ids to filter
devices to bind the SR-IOV port.
Use configurable agent_required option to require SRIOV L2 Agent
for bort binding.
- Part2
The SRIOV NIC Based L2 Agent.
Partially implements: blueprint ml2-sriov-nic-switch
Change-Id: If3d28df2a8ffe72ae512ddae076f42fca936cff7
Background:
ML2 plugin sometimes uses truncated port uuids.
For example, in the case of ofagent and linuxbridge,
if port id is 804ceaa1-0e3e-11e4-b537-08606e7f74e7,
an agent would send "tap804ceaa1-0e" to the plugin.
ML2 plugin's _device_to_port_id() would restore it to
"804ceaa1-0e". While it's still truncated, ML2 plugin's
get_port() handles that by using "startswith".
The recently merged DVR change (https://review.openstack.org/#/c/102332/)
assumes that port_id is always a complete uuid (it's the case
for openvswitch) and fails to handle the above mentioned case.
This commit fixes the regression.
Change-Id: I9c0845be606969068ab5d13c0165e76760378500
Closes-Bug: #1343750
This patch introduces the RPC contract changes
required for both the server (plugin) and agent
to propagate and retrieve additional information
about Distributed Routers, like MAC addresses
and Port Bindings.
Partially-implements: blueprint neutron-ovs-dvr
Change-Id: I04a2ee5fceea79d2786c799178f8dd1675925a39
Authored-by: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
The ML2 plugin now calls the bind_port() operation on the registered
mechanism drivers outside of any enclosing DB transaction. Ports are
created or updated in one transaction, then a binding is established
if possible, and finally a second transaction commits the binding
result.
With [re]binding moved outside the DB transaction that triggered it,
it is now possible that multiple threads or processes will
concurrently try to bind the same port, or that the port will be
updated between transactions. Concurrent attempts to bind the same
port are allowed to proceed, which results are used is resolved in the
second transaction, and binding is retried if necessary.
Improvements to the Cisco Nexus driver and unit tests from Rich Curran
needed due to the binding changes are also included.
Closes-Bug: 1276391
Closes-Bug: 1335226
Change-Id: I65dafc330d6e812dad0667d2383858504d0ba299
Allow to get multiple devices details instead of just one
This change introduces a new method in the rpc api.
blueprint bulk-get-device-details
Change-Id: I8497256d7f4f2fb48b5cb792e35aaedf63f129fc
Removed an __init__ method from a mixin class that
made mixing with other classes fragile and inflexible.
This replaces it with an explicit setup method.
This allows the ML2 RPCCallbacks class to correctly
inherit the common RpcCallback class.
Closes-Bug: #1332041
Change-Id: I36cb7dcf57147468f252d61f846b2b28dd77c8ff
Most of this code will probably stay with us for quite some time, so
let's make things easier and consider them as our way of doing RPC.
blueprint oslo-messaging
Change-Id: Iaf353b23f9c54b82d1e02a6bd5a5960cec827c88
Now that we don't have a special dispatcher class and we pass a list of
endpoints to corresponding functions instead, those methods are
unneeded.
blueprint oslo-messaging
Change-Id: If2b187fd8e553594212264f34b51b5b99c4630b2
Now that all preparations are done, actually port the code to use
oslo.messaging. This patch does as little as possible. Follow up patches
that refactor and cleanup the code and configuration files, will be
merged later. The reason for this is to make the patch as slim as
possible, to make review process more smooth and concentrated.
Details:
* neutron/common/rpc.py:
- added init() and cleanup() to set global RPC layer state.
- added utility functions: get_server(), get_client(), get_notifier()
that wrap up oslo.messaging API a bit, enforcing eventlet executor
and setting serializer, among other things.
- removed PluginRpcDispatcher, instead introduced PluginRpcSerializer
to use as a default serializer for API callbacks.
* neutron/common/rpc_compat.py:
- emulated incubator RPC layer behaviour thru previously introduced
stub classes (RpcCallback, RpcProxy, ...) using new oslo.messaging
API.
- switched to using new oslo.messaging exception types.
* neutron/service.py:
- expect multiple RPC listeners that are of MessageHandlingServer
type, not GreenThread.
* neutron/common/config.py:
- initialize RPC layer in init()
* setup.cfg:
- added entry points for old notifier drivers to retain backward
compatibility.
* neutron/tests/...:
- introduced fake_notifier to replace impl_fake.
- faked out consume_in_thread() to avoid starting RPC listeners when
running unit tests.
- used 'fake' transport driver.
- made sure neutron.test.* exceptions are caught.
- initialize and clean up RPC layer for each test case.
* Ported all affected code from using neutron.openstack.common.notifier
API to oslo.messaging.Notifier.
* rpc.set_defaults() was renamed to rpc.set_transport_defaults()
* other changes not worth mentioning here.
blueprint oslo-messaging
DocImpact
Change-Id: I5a91c34df6e300f2dc46217b1b16352fcc3039fc
This class will be used to create proper self.target with appropriate
API version once we migrate to oslo.messaging.
blueprint oslo-messaging
Change-Id: I1fb5eb0aaac0d115fd84630e58b333e695ad4f5f
This class is intended for easy and iterative migration from
oslo-incubator RPC implementation to oslo.messaging. It will adapt
oslo.messaging API to old oslo-rpc interfaces.
Migrated all affected classes to the new RpcProxy class.
blueprint oslo-messaging
Change-Id: I1563c13b470b61eebc982ee81efee47732943d48
the fdb_remove rpc message is sent when the status
of the port goes to BUILD, that is when the new host
send a get_device_details which means that it owns
the migrated port. The fdb_add message will be sent
as soon as the new host send update_device_up
Closes bug: #1237841
Change-Id: Ibdc7768d8db922b7e6eb9dc505382168cbb8e55d
This commit adds support for currently provided Mellanox Plugin
embedded switch functionality as part of the VPI (Ethernet/InfiniBand)
HCA as an ML2 MechanismDriver.
MechanismDriver adds support for VNIC_DIRECT and VNIC_MACVTAP vnic types.
MechanismDriver provides configurable default vif_type for neutron port created
with default VNIC_NORMAL vnic type till nova api support for vnic_type is available.
Implements blueprint mlnx-ml2-support
Change-Id: I16ad318f095b7af879e1b99dcc7f5f9e92facd2b
if host is set in the rpc message update_device_up/down sent by the agent,
the port status will be changed only if the port is bound to the host.
Change-Id: I0e607c734fbebf0b69f83c3bbd3e25a9783672dc
Closes-Bug: #1224967
This patch initiates the blueprint l2-population
Implemented as a ml2 Mechanism driver.
OVS & LinuxBridge drivers will be added as dependencies.
Rebased on ML2 Portbinding.
Change-Id: Ia2345aa262ec791c9f38b6e41e1e4b46f69cadac
The ml2 plugin uses mechanism drivers to determine which network
segment and what VIF driver to use for a port. Mechanism drivers
supporting the openvswitch, linuxbridge, and hyperv agents are
added. The binding:host attribute is set on ports belonging to the
dhcp and l3 agents so that they can be bound.
To use with devstack until it is updated, set
"Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch,linuxbridge" in localrc.
The hyperv L2 agent does not currently implement the agents_db RPC,
and will therefore not work with its ml2 mechanism driver. This issue
will be tracked as a bug to be fixed in a separate merge.
implements blueprint: ml2-portbinding
Change-Id: Icb9c70d8b0d7fcb34b57adc760bb713b740e5dad
This patch add the type_driver GRE to enable the creation of GRE
tunnels with the OVS agent.
No Endpoints ID are managed. Only Endpoint IP are stored in DB,
and this IP is proposed as an endpoint ID for the OVS agent.
It also fixes the bug 1201471.
Implements: blueprint ml2-gre
Change-Id: I1a33a4bd3ebc4c97eecf17a59ce16b8c2066ec66
Previously when starting the dhcp agent the sync_state() process would
be extremely expensive as it would query quantum server for each network.
In order to improve performance a get_active_networks_info() was added
so this information could be retrieved in one query rather than doing a
query for each active network.
The second part of this patch optimizes the logic to avoid calling
get_dhcp_port(). Previously, this method was called once for each network
which makes a call to get_subnets() and get_ports() unnecessarily as
the dhcp agent can determine itself if it needs to update a port or create a
port for dhcp.
This patch also threads the inital sync process and maintains backwards
compatibility with the previous rpc api.
There was also a trivial change to the nvp_plugin where filters are assumed to
be a dict.
implements blueprint improve-dhcp-agent-performance
Change-Id: I3b631057f595250dad76516faa9b421789f60953
This change renames everything to Neutron while providing backwards
compatible adjustments for Grizzly configuration files.
implements blueprint: remove-use-of-quantum
Change-Id: Ie7d07ba7c89857e13d4ddc8f0e9b68de020a3d19
Jenkins