In CommonAgentLoop class, there is logic to detect tap device is changed
locally or not by comparing timestamp with previous.
Sometimes timestamp value could be None depending on the timing (see bug/1781129)
But current _get_devices_locally_modified logic can not detect local
change from None to something because _get_devices_locally_modified
function don't always compare if previous timestamp value was None.
In order not to miss updated device always, better not to use dict.get() to
know previous iteration have timestamp or not.
As part of the implementation of multiple port bindings , add binding
activation support to the linux bridge agent. This will enable the
execution with linux bridge agents of the complete sequence of steps
outlined in  during an instance migration:
1) Create inactive port bindings for destination host
2) Migrate the instance to the destination host and plug its VIFs
3) Activate the port bindings in the destination host
4) Delete the port bindings for the source host
The neutron.common.topics module was rehomed into neutron-lib with
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
This commit adds common_agent_extension class which is agent API
for L2 extension drivers used e.g. by Linuxbridge agent.
This is necessary to be able to use instance of iptables_manager
used in firewall driver also in L2 extension drivers (like qos).
This patch refactors little bit iptables_manager code to make possible
to initialize e.g. mangle or nat table on demand, even if iptables
is created as "state_less"
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.
Part of the work we did while rehoming the callback modules introduced
the notion of payload objects  to replace the unstructured kwargs
used today. When using payloads event sources need to use publish()
rather than notify() to trigger the callback(s) and pass along a payload
object (if needed).
This patch begins to move us onto the payload objects by updating
BEFORE_INIT and AFTER_INIT event types to use the payloads.
NB: This change needs to be in sync with consumers using the events
herein. Once publish() is used with payloads, callback functions must
also define the payload kwarg to follow suit. Therefore such consumers
need to depend on this patch.
In Pike, the agent side of security_groups_provider_updated()
RPC code was changed to a NOOP when the provider rules were
changed to be static, https://review.openstack.org/#/c/432506
Now that we're in Queens we can deprecate it.
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Catch delete port extension failures like we do with normal port
removal to avoid interrupting the deletion processing for other
The callback modules have been available in neutron-lib since commit 
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need .
Libvirt does not set the MTU of the tap device it creates when creating
a bridge interface. It also does not set the MTU of the bridge itself.
This cannot be fixed on the Nova side since libvirt doesn't appear to
have support for setting MTUs on bridges until version 3x.
This results in a situation where the first VM tap interface attached to
a bridge will always have an MTU of 1500. The Neutron agent will then
add in VLAN/VXLAN interfaces with the correct MTU; however, the bridge
MTU will still be pinned to the smallest interface MTU attached to it.
This breaks jumbo frames until all small MTU tap devices are removed
from the bridge.
This patch explicitly sets the MTU on tap devices to match the network
MTU when processing the device.
We will have to carry this workaround until we stop Nova from
plugging taps into bridges or until we drop support for older
libvirts on the Nova side and have it set the MTU.
This bug was introduced by change
I53c0eb57da956b36f09731d25db989719e9bc9dc which reverted automatic
setting of tap MTUs to match those of the physical device.
Refactoring Neutron configuration options for agent common config to be
in neutron/conf/agent/common. This will allow centralization of all
configuration options and provide an easy way to import.
Commit Ifb70a118cef48c3c4cd313e22e907aa47bc51ad0 intended to remove some
classes, but without DeprecationWarnings. To prevent from possible
problems, if it would be now removed, added Warnings and included info,
about future removal.
Pass the host into get_devices_details_list on the linux bridge
agent so the debug logs on the server side don't show "host None".
This is mainly just for cosmetics and consistency with the OVS agent
since the only thing the host is really used for on the server side
is special treatment of DVR ports, which does not currently apply
to linux bridge.
This is the agent-side implementation of vlan-aware-vms for
the Linux Bridge agent. It implements the feature using
Whenever subports are required, the linux bridge trunk driver
will create vlan devices off of the parent port device following
the same naming scheme as normal ports. This allows the normal
agent loop to see these VLAN ports and wire them like any other
port so the trunk logic doesn't have to concern itself with things
like firewall rules, anti-spoofing, or encapsulation onto the
How to try:
* enable the Linux Bridge mech driver and install the Linux Bridge agent
* enable the 'trunk' service plugin
* make a port, turn it into a trunk, attach it to a VM (or boot a VM with it)
* add subports and configure your guest on the corresponding VLAN
* don't forget these subports have security groups so add allow rules!
Partially-implements: blueprint vlan-aware-vms
The agent code is enhanced to allow the trunk agent-side counterpart
to be activated seamlessly by means or local registry notifications.
Some integration with the server side is provided by loading the
RPC agent-side skeleton. Basic unit testing provides some coverage.
More effective functional and system coverage will be provided once
everything comes together.
Partially-implements: blueprint vlan-aware-vms
Co-Authored-By: Adolfo Duarte <email@example.com>
Refactoring neutron ml2 plugin agent driver config opts to be in
neutron/conf/plugins/ml2/drivers/agent so that all the
configuration options for agent drivers reside in a centralized
location. This simplifies the process of looking up the agent driver
config opts and provides an easy way to import.
Some of the agent code may want to receive events whenever
a device is added/updated/deleted. Currently the only way
to do that is to modify the loop directly or create an
extension. Neither of these options are good if it's code
we don't want users to disable (the extensions) and it's
specific to one agent (so we don't want to modify the loop).
Partially-Implements: blueprint vlan-aware-vms
If a device is removed in the middle of
treat_devices_added_updated it can cause failures which
will force all of the other unaffected devices to be
reprocessed. When they are reprocessed it will cause
the ports on the server side to go back into BUILD which
can cause failures in tests expecting ACTIVE ports.
This patch adds an exception catcher that examines if the
port is still present. If it's missing, it suppresses the
exception since the device will be treated as a removed
device on the next iteration.
This change generalizes agent extension code so that all agents can take
advantage of a common mechanism.
Co-Authored-By: Margaret Frances <firstname.lastname@example.org>
Partially-Implements: blueprint l3-agent-extensions
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who wants to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request
* Add initialization of osprofiler at start of service
Currently that includes oslo.messaging notifer instance creation
to send Ceilometer backend notifications.
Neutron client change: Ic11796889075b2a0e589b70398fc4d4ed6f3ef7c
Co-authored-by: Ryan Moats <email@example.com>
DocImpact Add devref and operator documentation on how to use this
This option was originally intended to deprecate in Mitaka
and remove in Newton, but we missed to announce it in Mitaka
release note. It looks better to deprecate it in Newton release
and remove it in Ocata release.
resource_versions were included into agent state reports recently to
support rolling upgrades (commit 97a272a892)
The downside is that it brought additional processing when handling state
reports on server side: update of local resources versions cache and
more seriously rpc casts to all other servers to do the same.
All this led to a visible performance degradation at scale with hundreds
of agents constantly sending reports. Under load (rally test) agents
may start "blinking" which makes cluster very unstable.
In fact there is no need to send and update resource_versions in each state
report. I see two cases when it should be done:
1) agent was restarted (after it was upgraded);
2) agent revived - which means that server was not receiving or being able
to process state reports for some time (agent_down_time). During that
time agent might be upgraded and restarted.
So this patch makes agents include resource_versions info only on startup.
After agent revival server itself will update version_manager with
resource_versions taken from agent DB record - this is to avoid
version_manager being outdated.
* Detect neutron.i18n import (neutron._i18n is recommended)
* Check builtins _ usage
* 'builtins = _' in tox.ini is no longer required.
* Introduce hacking rule doctest framework.
Newly added check_builtins_gettext() hacking check takes
token as argument. It is not a good idea to pass a tokenized
line manually. Instead it is reasonable to use docstring based
tests used in hacking repo.
During a quick instance rebuild on Nova, it may remove a tap
interface and then plug it in within the agent polling window.
In this scenario the agent will not realize a device has changed
and will therefore not ask the server for info an subsequently
update its status. This will prevent the notification from being
sent back to Nova that the vif plugging has finished so the
VM will never resume state.
This adds a new timestamp collection method to the common agent
manager interface for devices that is used by the common agent
loop to determine if a device has locally changed.
The linux bridge implementation of it checks the timestamps on the
tap interface's 'bridge' directory, which will change whenever
the tap is added to bridge.
There is a new QoS extension driver for the linuxbridge agent
being added. This driver provides support for QoS configuring
on the linuxbridge agent.
This patch introduces two new config options for Linuxbridge agent:
kernel_hz - HZ value of host kernel,
tbf_latency - value of latency in tbf qdisc to calculate size of queue.
Co-Authored-By: vikram.choudhary <firstname.lastname@example.org>
DocImpact Update agent configuration to show settings related to
QoS and bandwidth limiting
This is the second patch to allow upgrades on RPC versioned
This enables resource version notifications from agents to all
neutron servers via fanout for updating the version sets in
memory, and via agent status updates for DB storage, so any
neutron server can retrieve such information at boot.
Removed hard-coded "LinuxBridge" from logging message in the common
agent since it could also be used by other agents and changed it to
log the agent type instead. This should allow logs to reflect
which agent type is actually using the common agent.
Removed unused/leftover constants from the common agent.
This protection should always be enabled unless its explicitly
shutoff via the port security extension via the API. The primary
reason it was a config option was because it was merged at the end
of Kilo development so it wasn't considered stable. Now that it
has been enabled by default for all of Liberty and the development
of Mitaka, it's a good idea to just get rid of the option completely.
DocImpact: Remove references to prevent_arp_spoofing and replace
with pointer to port security extension for disabling
The goal is to extract the common agent code from the linuxbridge agent
to share this code with other agents (e.g. sriov and new macvtap ).
This is a first step into the direction of a so called modular l2
Therefore all linuxbridge implementation specifics are moved into the
LinuxBridgeManager class. The manager class will be passed as argument
into the common agent loop instead of instantiating it in its
constructor. In addition the network_maps and the updated_devices map
has been moved into the rpc class.
A clear manager interface has been defined for the communication
between the common agent loop and the impl specific manager class.
In a follow up patchset, the common agent loop will be moved into a
new file. This has not yet happened to simplify tracking the code
changes during review.