With python 3.x, classes can use the metaclass= logic
to not require usage of the six library.
One step in removing all of six usage from neutron.
Change-Id: I2f815e412d9a96eb5faf2b3bb3a1e393a9db9309
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
If an agent using the common agent code fails to send
its report_state to the server, it logs an exception:
Failed reporting state!: MessagingTimeout: Timed out...
If it then tries a second time and succeeds it just
goes on happily. It would be nice if it logged that
it had success on the subsequent attempt so someone
looking at the logs know it recovered.
Change-Id: Ie94e909fba07e5eb91b57160947edc9b86428fe5
In CommonAgentLoop class, there is logic to detect tap device is changed
locally or not by comparing timestamp with previous.
Sometimes timestamp value could be None depending on the timing (see bug/1781129)
But current _get_devices_locally_modified logic can not detect local
change from None to something because _get_devices_locally_modified
function don't always compare if previous timestamp value was None.
In order not to miss updated device always, better not to use dict.get() to
know previous iteration have timestamp or not.
Change-Id: Ib0361ad5c281f88558e8e048cfec588b9f9b1de4
Closes-Bug: #1781129
As part of the implementation of multiple port bindings [1], add binding
activation support to the linux bridge agent. This will enable the
execution with linux bridge agents of the complete sequence of steps
outlined in [1] during an instance migration:
1) Create inactive port bindings for destination host
2) Migrate the instance to the destination host and plug its VIFs
3) Activate the port bindings in the destination host
4) Delete the port bindings for the source host
[1] https://review.openstack.org/#/c/309416/
Change-Id: I2c937cc0a551e5ce0e8534c4dd4384ec2ca92da1
Partial-Bug: #1580880
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them. Also removed an N536 comment missed from a
previous change.
Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
This commit adds common_agent_extension class which is agent API
for L2 extension drivers used e.g. by Linuxbridge agent.
This is necessary to be able to use instance of iptables_manager
used in firewall driver also in L2 extension drivers (like qos).
This patch refactors little bit iptables_manager code to make possible
to initialize e.g. mangle or nat table on demand, even if iptables
is created as "state_less"
Change-Id: I3b66e49b7f176124e8aea3eb96d0d465f1ab1ea0
Closes-Bug: #1736674
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.
NeutronLibImpact
Change-Id: Ie4bcffccf626a6e1de84af01f3487feb825f8b65
Part of the work we did while rehoming the callback modules introduced
the notion of payload objects [1] to replace the unstructured kwargs
used today. When using payloads event sources need to use publish()
rather than notify() to trigger the callback(s) and pass along a payload
object (if needed).
This patch begins to move us onto the payload objects by updating
BEFORE_INIT and AFTER_INIT event types to use the payloads.
NB: This change needs to be in sync with consumers using the events
herein. Once publish() is used with payloads, callback functions must
also define the payload kwarg to follow suit. Therefore such consumers
need to depend on this patch.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: I9194c7857f10392149159071cda8e080e93adc10
In Pike, the agent side of security_groups_provider_updated()
RPC code was changed to a NOOP when the provider rules were
changed to be static, https://review.openstack.org/#/c/432506
Now that we're in Queens we can deprecate it.
Change-Id: Ie018ff653633d3524f0e80c5e172a5d01bdad437
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
This was deprecated over a year ago in [1] so let's
get rid of it to clean up some code.
1. Ib63ba8ae7050465a0786ea3d50c65f413f4ebe38
Change-Id: I6039fb7e743c5d9a1a313e3c174ada36c9874c70
Catch delete port extension failures like we do with normal port
removal to avoid interrupting the deletion processing for other
ports.
Related-Bug: #1697937
Change-Id: I42d86a86965d30e14c85ce975681a6b82be41ecc
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
Libvirt does not set the MTU of the tap device it creates when creating
a bridge interface. It also does not set the MTU of the bridge itself.
This cannot be fixed on the Nova side since libvirt doesn't appear to
have support for setting MTUs on bridges until version 3x.
This results in a situation where the first VM tap interface attached to
a bridge will always have an MTU of 1500. The Neutron agent will then
add in VLAN/VXLAN interfaces with the correct MTU; however, the bridge
MTU will still be pinned to the smallest interface MTU attached to it.
This breaks jumbo frames until all small MTU tap devices are removed
from the bridge.
This patch explicitly sets the MTU on tap devices to match the network
MTU when processing the device.
We will have to carry this workaround until we stop Nova from
plugging taps into bridges[1] or until we drop support for older
libvirts on the Nova side and have it set the MTU.
This bug was introduced by change
I53c0eb57da956b36f09731d25db989719e9bc9dc which reverted automatic
setting of tap MTUs to match those of the physical device.
1. I23c5faaeab69aede1fd038a36f4a0b8f928498ce
Closes-Bug: #1684038
Change-Id: Ia245a3e22339fce026901e24a82e836c8b27cc28
Refactoring Neutron configuration options for agent common config to be
in neutron/conf/agent/common. This will allow centralization of all
configuration options and provide an easy way to import.
Partial-Bug: #1563069
Change-Id: Iebac0cdd3bcfd0135349128921b7ad7a1a939ab8
Needed-By: Ib676003bbe909b5a9013a3178b12dbe291d936af
Commit Ifb70a118cef48c3c4cd313e22e907aa47bc51ad0 intended to remove some
classes, but without DeprecationWarnings. To prevent from possible
problems, if it would be now removed, added Warnings and included info,
about future removal.
Change-Id: Iacb93abc363bf638efc3acb5b29c02a7508bc43a
Related-Bug: #1444112
Pass the host into get_devices_details_list on the linux bridge
agent so the debug logs on the server side don't show "host None".
This is mainly just for cosmetics and consistency with the OVS agent
since the only thing the host is really used for on the server side
is special treatment of DVR ports, which does not currently apply
to linux bridge.
Change-Id: I700fa26982bdb087cf7ea4b3eb69aec2f2e099c8
Closes-Bug: #1622566
This is the agent-side implementation of vlan-aware-vms for
the Linux Bridge agent. It implements the feature using
vlan subinterfaces.
Whenever subports are required, the linux bridge trunk driver
will create vlan devices off of the parent port device following
the same naming scheme as normal ports. This allows the normal
agent loop to see these VLAN ports and wire them like any other
port so the trunk logic doesn't have to concern itself with things
like firewall rules, anti-spoofing, or encapsulation onto the
physical network.
How to try:
* enable the Linux Bridge mech driver and install the Linux Bridge agent
* enable the 'trunk' service plugin
* make a port, turn it into a trunk, attach it to a VM (or boot a VM with it)
* add subports and configure your guest on the corresponding VLAN
* don't forget these subports have security groups so add allow rules!
Partially-implements: blueprint vlan-aware-vms
Change-Id: I688d5b25885c1c3938185467b15502ccf65cf935
The agent code is enhanced to allow the trunk agent-side counterpart
to be activated seamlessly by means or local registry notifications.
Some integration with the server side is provided by loading the
RPC agent-side skeleton. Basic unit testing provides some coverage.
More effective functional and system coverage will be provided once
everything comes together.
Partially-implements: blueprint vlan-aware-vms
Co-Authored-By: Adolfo Duarte <adolfo.duarte@hpe.com>
Change-Id: Id70553e8980593f99548a4d2b0a78355933f7c2c
Refactoring neutron ml2 plugin agent driver config opts to be in
neutron/conf/plugins/ml2/drivers/agent so that all the
configuration options for agent drivers reside in a centralized
location. This simplifies the process of looking up the agent driver
config opts and provides an easy way to import.
Change-Id: I673c10dfa35014fea129f7030139a89bfc20b272
Partial-Bug: #1563069
Some of the agent code may want to receive events whenever
a device is added/updated/deleted. Currently the only way
to do that is to modify the loop directly or create an
extension. Neither of these options are good if it's code
we don't want users to disable (the extensions) and it's
specific to one agent (so we don't want to modify the loop).
Partially-Implements: blueprint vlan-aware-vms
Change-Id: I3468c7f46cc1b4000cdd537e8f216d207a14727a
If a device is removed in the middle of
treat_devices_added_updated it can cause failures which
will force all of the other unaffected devices to be
reprocessed. When they are reprocessed it will cause
the ports on the server side to go back into BUILD which
can cause failures in tests expecting ACTIVE ports.
This patch adds an exception catcher that examines if the
port is still present. If it's missing, it suppresses the
exception since the device will be treated as a removed
device on the next iteration.
Closes-Bug: #1605090
Change-Id: Ia774a7c1578f5aca71f3b706e47819b3fdc1cce2
This change generalizes agent extension code so that all agents can take
advantage of a common mechanism.
Co-Authored-By: Margaret Frances <margaret_frances@cable.comcast.com>
Partially-Implements: blueprint l3-agent-extensions
Change-Id: I9380343c09d28eec67077c9e6d77c33a195e516b
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who wants to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request
* Add initialization of osprofiler at start of service
Currently that includes oslo.messaging notifer instance creation
to send Ceilometer backend notifications.
Neutron client change: Ic11796889075b2a0e589b70398fc4d4ed6f3ef7c
Co-authored-by: Ryan Moats <rmoats@us.ibm.com>
Depends-On: I5102eb46a7a377eca31375a0d64951ba1fdd035d
Closes-Bug: #1335640
DocImpact Add devref and operator documentation on how to use this
APIImpact
Change-Id: I7fa2ad57dc5763ce72cba6945ebcadef2188e8bd
This option was originally intended to deprecate in Mitaka
and remove in Newton, but we missed to announce it in Mitaka
release note. It looks better to deprecate it in Newton release
and remove it in Ocata release.
Change-Id: Iad466abbb0716da77801315d1a8766ba2f7c06b9
Closes-Bug: #1546010
resource_versions were included into agent state reports recently to
support rolling upgrades (commit 97a272a892)
The downside is that it brought additional processing when handling state
reports on server side: update of local resources versions cache and
more seriously rpc casts to all other servers to do the same.
All this led to a visible performance degradation at scale with hundreds
of agents constantly sending reports. Under load (rally test) agents
may start "blinking" which makes cluster very unstable.
In fact there is no need to send and update resource_versions in each state
report. I see two cases when it should be done:
1) agent was restarted (after it was upgraded);
2) agent revived - which means that server was not receiving or being able
to process state reports for some time (agent_down_time). During that
time agent might be upgraded and restarted.
So this patch makes agents include resource_versions info only on startup.
After agent revival server itself will update version_manager with
resource_versions taken from agent DB record - this is to avoid
version_manager being outdated.
Closes-Bug: #1567497
Change-Id: I47a9869801f4e8f8af2a656749166b6fb49bcd3b
* Detect neutron.i18n import (neutron._i18n is recommended)
* Check builtins _ usage
* 'builtins = _' in tox.ini is no longer required.
* Introduce hacking rule doctest framework.
Newly added check_builtins_gettext() hacking check takes
token as argument. It is not a good idea to pass a tokenized
line manually. Instead it is reasonable to use docstring based
tests used in hacking repo.
Change-Id: Ib7464658fc4c8a6f1b03af6ab46f0bd3ee0bfb18
During a quick instance rebuild on Nova, it may remove a tap
interface and then plug it in within the agent polling window.
In this scenario the agent will not realize a device has changed
and will therefore not ask the server for info an subsequently
update its status. This will prevent the notification from being
sent back to Nova that the vif plugging has finished so the
VM will never resume state.
This adds a new timestamp collection method to the common agent
manager interface for devices that is used by the common agent
loop to determine if a device has locally changed.
The linux bridge implementation of it checks the timestamps on the
tap interface's 'bridge' directory, which will change whenever
the tap is added to bridge.
Closes-Bug: #1531862
Change-Id: If172470e907848556b6a8aff13520f94245919bb
There is a new QoS extension driver for the linuxbridge agent
being added. This driver provides support for QoS configuring
on the linuxbridge agent.
This patch introduces two new config options for Linuxbridge agent:
kernel_hz - HZ value of host kernel,
tbf_latency - value of latency in tbf qdisc to calculate size of queue.
Co-Authored-By: vikram.choudhary <vikram.choudhary@huawei.com>
Change-Id: I457ca2569b5d4a916ba09e71040505cd0ad3257b
Closes-Bug: #1500012
Closes-Bug: #1550514
DocImpact Update agent configuration to show settings related to
QoS and bandwidth limiting
This is the second patch to allow upgrades on RPC versioned
objects callbacks.
This enables resource version notifications from agents to all
neutron servers via fanout for updating the version sets in
memory, and via agent status updates for DB storage, so any
neutron server can retrieve such information at boot.
Closes-Bug: #1535247
Change-Id: I67c1323267aaf7e49f4a359ff50b94e52dba4380
Removed hard-coded "LinuxBridge" from logging message in the common
agent since it could also be used by other agents and changed it to
log the agent type instead. This should allow logs to reflect
which agent type is actually using the common agent.
Removed unused/leftover constants from the common agent.
Change-Id: I2e790bf052f18f7314bfd1677fde65b96b2a5edd
Closes-Bug: #1551219
This protection should always be enabled unless its explicitly
shutoff via the port security extension via the API. The primary
reason it was a config option was because it was merged at the end
of Kilo development so it wasn't considered stable. Now that it
has been enabled by default for all of Liberty and the development
of Mitaka, it's a good idea to just get rid of the option completely.
DocImpact: Remove references to prevent_arp_spoofing and replace
with pointer to port security extension for disabling
security features.
Change-Id: Ib63ba8ae7050465a0786ea3d50c65f413f4ebe38
The goal is to extract the common agent code from the linuxbridge agent
to share this code with other agents (e.g. sriov and new macvtap [1]).
This is a first step into the direction of a so called modular l2
agent.
Therefore all linuxbridge implementation specifics are moved into the
LinuxBridgeManager class. The manager class will be passed as argument
into the common agent loop instead of instantiating it in its
constructor. In addition the network_maps and the updated_devices map
has been moved into the rpc class.
A clear manager interface has been defined for the communication
between the common agent loop and the impl specific manager class.
In a follow up patchset, the common agent loop will be moved into a
new file. This has not yet happened to simplify tracking the code
changes during review.
[1] https://bugs.launchpad.net/neutron/+bug/1480979
Change-Id: Ia71f5a403b7029f8cc591f83df91ab2d3916f3f8
Partial-Bug: #1468803
Partial-Bug: #1480979
Brian Haley