In the ML2/OVS when igmp_snooping is enabled but there is no
external querier multicast traffic will stop working after few minutes
as packets will not be flooded to tunnel/external bridges.
So this patch sets "mcast-snooping-disable-flood-unregistered" option
of the br-int to False (default value) even when igmp_snooping is
enabled in the neutron-ovs-agent's config file.
Additionally it configures "mcast-snooping-flood-reports" and
"mcast-snooping-flood" on patch ports in br-int to True.
That way we can provide best effort snooping: multicast isolation where
IGMP queriers are available and flood everywhere else?
Closes-Bug: #1884723
Change-Id: Iefa0044dba9e92592295a79448e5d57d9e14a40b
In case when IPv6 addresses are used for GRE tunnels, tunnel
type set for the openvswitch interface should be "ip6gre" instead of
"gre" which was set so far.
This patch changes that so now Neutron configures correct GRE tunnel
types.
Change-Id: I557af0bcafac4583ad9726c9bf707cf1fb92ffc5
Closes-Bug: #1904564
When creating the OVS manager, define the command timeout
(CONF.OVS.ovsdb_timeout) and inactivity probe time
(CONF.OVS.of_inactivity_probe)
NOTE: CONF.OVS.of_inactivity_probe is defined in seconds but the
parameter should be passed to ovs-vsctl in milliseconds [1].
[1]http://www.openvswitch.org/support/dist-docs/ovs-vswitchd.conf.db.5.txt
Depends-On: https://review.opendev.org/#/c/720785
Change-Id: I8ed1fc85c2f78710bf6589ba3deca518471339b8
Closes-Bug: #1868686
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This completes removal of all references to "import mock",
changing to "from unittest import mock" in fullstack and
functional tests.
Added a hacking check to enforce it in future patches.
Change-Id: Ifcaf1c21bea0ec3c35278e49cecc90a101a82113
There is a bug in OVS 2.12 where it's impossible to change protocol on
a bridge. This patch should be reverted once OVS is fixed. More
information about the bug at [1].
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1782834
Related-Bug: #1852221
Change-Id: I1ead1eee48a0c56193f20797ab35be36a0458270
Neutron-ovs-agent can now enable IGMP snooping in integration bridge
if config option "igmp_snooping_enable" in OVS section in config will
be set to True.
It will also set mcast-snooping-disable-flood-unregistered=true
so flooding of multicast packets to all unregistered ports will be
disabled also.
Both changes are applied on integration bridge.
Change-Id: I12f4030a35d10d1715d3b4bfb3ed5efb9aa28f2b
Closes-Bug: #1840136
By default number of MAC addresses which ovs stores in memory
is quite low - 2048.
Any eviction of a MAC learning table entry triggers revalidation.
Such revalidation is very costly so it cause high CPU usage by
ovs-vswitchd process.
To workaround this problem, higher value of mac-table-size
option can be set for bridge. Then this revalidation will happen
less often and CPU usage will be lower.
This patch adds config option for neutron-openvswitch-agent to allow
users tune this setting in bridges managed by agent.
By default this value is set to 50000 which should be enough for most
systems.
Change-Id: If628f52d75c2b5fec87ad61e0219b3286423468c
Closes-Bug: #1775797
Agent OVS interface code adds ports without a vlan tag,
if neutron-openvswitch-agent fails to set the tag, or takes
too long, the port will be a trunk port, receiving
traffic from the external network or any other port
sending traffic on br-int.
Also, those kinds of ports are triggering a code path
on the ovs-vswitchd revalidator thread which can eventually
hog the CPU of the host (that's a bug under investigation [1])
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1558336
Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>
Change-Id: I024bbbdf7059835b2f23c264b48478c71633a43c
Closes-Bug: 1767422
Adding ability to set DSCP field in OVS tunnels outer header, or
inherit it from the inner header's DSCP value for OVS and linuxbridge.
Change-Id: Ia59753ded73cd23019605668e60cfbc8841e803d
Closes-Bug: #1692951
For OVS based DPDK ports ingress bandwidth limit is now implemented
using egress-policer qos type.
Additionally limit values are set in other_config of QoS because there
is no queue used in this case.
This patch moves also helper methods used to conversion between
bytes and bits and between bits and kilobits to neutron.common.utils
to be able to use it also in ovs_lib module.
Change-Id: I94d1e8dfb82df5c602476db8aaa884ae91fecd7f
Closes-Bug: #1724729
ovsdb maps accept strings as values only. This patch converts integer to
be passed to ovsdb in case vxlan_udp_port config value is used.
Change-Id: Idba77939a80d80a4bc9625d10c8b37b23b91b9c5
Closes-bug: #1742931
This module located in neutron.agent.ovsdb.native.idlutils is
deprecated and used only in functional test of ovs_lib module.
Now it's removed and proper module ovsdbapp.backend.ovs_idl.idlutils
is used instead.
Change-Id: I6174a08a07e273e2180ef907adb1da48b518d579
OVS can hold only one tunnel with same endpoints. Some tests had
hardcoded values for both tunnel endpoints which made them unable to run
in parallel manner.
This patch takes always exclusive address using resource allocator.
Change-Id: If81296d54656551b24917d561f235edb96a6c2df
Closes-bug: #1697533
Add support for QoS ingress bandwidth limiting in
openvswitch agent.
It uses default ovs QoS policies on bandwidth limiting
mechanism.
DocImpact: Ingress bandwidth limit in QoS supported by
Openvswitch agent
Change-Id: I9d94e27db5d574b61061689dc99f12f095625ca0
Partial-Bug: #1560961
n8g-sfc currently has its own variant of OVSBridge to allow the use
of priority in a delete_flows call
This change is meant to make this available outside n8g-sfc and
simplify n8g-sfc code.
This change adds a 'strict' boolean parameter to mod_flow and delete_flows
that results in ovs-ofctl to be run with --strict for del-flows and
mod-flows actions. When strict is set, the use of priority is allowed
and hence not rejected anymore.
Note that for batched actions in a deffered bridge, we disallow mixing
calls with strict and without strict, which can't be translated in one CLI
call.
Needed-By: I3bf939590dd43bff685f133bff86eb7e9068de91
Change-Id: I289d546780f10dc1002ab6bc2e1b38c9ef2d728f
With this change delete_flows will only remove flows matching the default
cookie of the bridge.
The uninstall_flows implementation in the native bridge is also modified
to touch only the flows with the bridge cookie.
To still allow deletion of all cookies, cookie=COOKIE_ANY is introduced
as a special value, and used in the agent code in the places where the
intent is indeed to clean all flows whatever their cookie is.
Partial-Bug: #1557620
Change-Id: Idd0531cedda87224531cb8fb6a912ccd0f1554d5
If the vsctl_timeout > OVS's inactivity probe interval and a
transaction execution time exceeds the probe interval, OVS will
disconnect and the transaction will return TRY_AGAIN and most
likely repeat failing until the vsctl_timeout is reached. This
change ensures that the "failsafe" creation of the manager also
sets the inactivity probe to the vsctl_timeout value.
Currently the patch doesn't override the probe_interval on an
existing Manager since it is possible that connection is used by
outside applications and it theoretically should be handled at
deployment.
Related-Bug: #1627106
Change-Id: I76fa0a0cf04a166edf062086fceb2fd90960ad6b
With this change calling delete_flows with no kwargs will (instead
of resulting in calling "ovs-ofctl <action> <bridge> -", which does
nothing with no flow spec given on stdin) result in calling
"ovs-ofctl <action> <bridge>", which will delete all flows.
This aligns the behavior of delete_flows with the behavior currently seen by
all callers for the same method shadowed by the implementations in
OpenFlowSwitchMixin classes.
Change-Id: Ic0449acb3a0d4915ce025300d6f3c507a3cd8e48
Closes-Bug: 1658019
When updating an ovsdb set-type column, the existing code does
the following:
1. Read the existing column value
2. Call verify() to cause a write failure if something else
modifies the column before we commit
3. Append the value to our local copy of the column
4. Assign the local value to the object.column to trigger
__setattr__ to write the value to the database
If verify() fails, which it *very* often does in a test
environment or a busy system, ovsdb-server will respond with a
TRY_AGAIN response which will cause the whole process to start
over.
In the 2.6 cycle, Row.addvalue()/delvalue() methods were added
which use OVDB's native 'mutate' methods to handle adding/deleting
from a set/map-type column. Using these means calling verify() is
no longer required and things will proceed *much* more efficiently.
Bug #1627106 where we get frequent TimeoutExceptions appears to be
related. Eliminating the frequent TRY_AGAIN responses, in my local
testing, also eliminates the TimeoutExceptions. This doesn't mean
that the underlying issue is resolved, but it may be avoided.
Related-Bug: #1627106
Change-Id: I26c7731f5dbd3bd2955dbfa18a7c41517da63e6e
This change modifies the behavior of OVS native and ovs-ofctl bridge
implementations so that instead of configuring the bridge only for the
required OVS protocol version, they add the required version to the
already configured versions.
To achieve this, an add_protocols method is added to the OVSBridge
class, relying on the OVSDB add_db_attribute added in
Ib6ce75846f9b13c1c33f0ced5ccc619ee7860dc1, with the behavior of
making the provided set of versions supported in addition to already
configured ones.
It is aimed to be a cleaner solution to bug 1622644 than the quickfix merge
from I4475865c4f83cb9f3e12c709af752bc490692ca3 .
After this change, the set_protocols method appears useless and is
hence marked for future removal.
Depends-On: I4386aa293f9b18d2e17b4a80d9c7da4b9b46f3c9
Change-Id: Id5ac7e6431c97fc70d8404b16f89533b6f270eee
Related-Bug: 1622644
The current existing agent erases already set ovsdb managers
entries. In some use cases, cloud admin sets ovsdb managers.
eg, for SDN controllers or monitoring purpose. Neutron agent
shouldn't unconditionally erase the existing ovsdb managers.
This patch implements a new api add_manager (along with
get_manager and remove_manager) to the ovsdb api which will
allow us to configure a manager on a switch without overriding
the existing managers.
Closes-Bug: #1614766
Change-Id: Ibf9bd02fac3070d166546cac478ef984e4e43f28
Co-Authored-By: sridhargaddam <sgaddam@redhat.com>
Co-Authored-By: Terry Wilson <twilson@redhat.com>
The native interface chokes when doing db_add on an object that
was created within the same transaction due to the column we are
modifying not yet existing. This patch adds defaults for the
column.
Change-Id: I4386aa293f9b18d2e17b4a80d9c7da4b9b46f3c9
Closes-Bug: #1642764
On the following:
b.set_db_attribute('Port', pname, 'other_config', {'a': 'b'})
b.set_db_attribute('Port', pname, 'other_config', {'c': 'd'})
will produce different results between the vsctl and native OVSDB
implementations. vsctl will merge the values into a single dict
and native will overwrite the dict.
This patch makes the native implementation mirror vsctl.
Related-Bug: #1630920
Change-Id: Ie7680a301b8b3ee8e5654666e2aea78ecbd07a12
It is at times useful to append to an OVSDB row in a single
transaction instead of doing separate transactions to pull back a
value, then another to modify and append to it. This patch adds
API.db_add(self, table, record, column, *values)
to the OVSDB API and adds it to the vsctl and native
implementations. It follows the format of ovs-vsctl's 'add'
command which works on both sets and maps. For map columns, its
behavior is to set a key in a column if it is not already set. For
sets, it appends to the current values stored in the column.
For maps, values can be passed as
(..., {'key': value}, {'key2': value2, ...)
or
(..., {'key': value, 'key2': value})
Change-Id: Ib6ce75846f9b13c1c33f0ced5ccc619ee7860dc1
Enable the OVSDB API implementations to handle referencing a newly
created object within the same transaction. For vsctl, this is via
the --id=@name construct. For native, this is via storing the
created row as the result, then replacing the result when the
transaction completes. This uses an api.Command object passed as
part of a column value in a db_set/db_create operation as a
reference to that object in a transaction. For example, do:
with self.ovsdb.transaction() as txn:
queue = txn.add(self.ovsdb.db_create("Queue", ...))
qos = txn.add(self.ovsdb.db_create("QoS", queues={0: queue}))
port = txn.add(self.ovsdb.db_set("Port", pname, ('qos', qos)))
instead of having to do 5 separate transactions to: create a
queue, find the queue, create the QoS entry, find the QoS entry,
and finally to update the port with the QoS entry.
Change-Id: I1781794958af1483dabc0f5d17f2df6fea828564
Closes-Bug: #1615105
neutron-sanity-check tool was importing neutron.tests.base module, which
may be not present on some systems (f.e. RDO splits neutron/tests/
subtree in a separate python-neutron-tests package). It made the tool
not usable in some setups.
https://bugzilla.redhat.com/show_bug.cgi?id=1374282
This is not the first time when we by mistake import from
neutron.tests.* and break distributions. It's time to stop it by
proactively forbidding that pattern via a new hacking check.
Some functions were moved from neutron.tests.base to
neutron.common.utils to fulfill the need requirement. They were moved
using debtcollector, no current consumers should be affected.
Closes-Bug: #1621782
Change-Id: I790777ddcbd1b02218b3db54ae3d5c931d72d4fa
As per the recent gate failures (see bug), it appears
OVSLibTestCase.test_db_find_column_type_list is not isolated
and thus its usage of ovsdb's db_list() and db_find() occasionally
obtain different results.
This patch adds the db_list() and db_find() operations within the
test case to run in a transaction so that we get a single snapshot
of the db results.
In addition this patch undoes the changes from patch set 1 as the
initial changes do not appear to address the issue at hand.
Change-Id: I312076edb6e11f21347831843758894e11d6f56c
Closes-Bug: #1592546
A bunch of functional tests fail because of non implemented
x != [] operation in idlutils.condition_match() and
wrong condition passed to db_find() in OVSFW test.
This patch addresses the issue by implementing lists
comparison in native.idlutils and fixing the call to
db_find() in OVSFW test.
A functional test for OVSDB API's db_find() has been
added to ensure that querying a list column gives the same
result both with vsctl and native ovsdb_interface; unit
test for idlutils.condition_match() with corner cases has
been added as well.
Change-Id: Ia93fb925b8814210975904a453249f15f3646855
Closes-bug: #1578233
Remove IPv4 restriction for local_ip configuration statement.
Check for IP version mismatch of local_ip and remote_ip before creating
tunnel.
Create hash of remote IPv6 address for OVS interface/port name with least
posibility for collissions.
Fix existing tests that fail because of the added check for IP version
and subsequently valid IP addresses in _setup_tunnel_port.
DocImpact
Change-Id: I9ec137ef8c688b678a0c61f07e9a01382acbeb13
Closes-Bug: #1525895
Before this patch, get_ports_attributes would call
get_port_name_list. In the case of an empty bridge,
get_port_name_list would return an empty list and feed that
in to ovsdb.db_list, thereby returning all ports instead of
no ports.
Change-Id: I5a91028c59d71943b7cef86a94fa6ab1cc3d667c
Closes-Bug: #1499821
Per [1] we are using a better way to keep tunnel connectivity,
so reset_bridge isn't used anymore. Bug in [2] was caused by
using method reset_bridge which will delete and recreate bridge.
For [1] makes method reset_bridge deprecated, it makes sense to
remove this method, and make [2] no longer produce.
[1] https://review.openstack.org/#/c/182920
[2] Related-bug: #1332450
Change-Id: I155f66a37b8d4081126467fe576e8315c2d5560c
It seems that the Queue + QoS + linux-htb implementation was really
limiting ingress by default. So this patch switches the implementation
to the ovs ingress_policing_rate and ingress_policing_burst parameters
of the Interface table.
Later in time we may want to revise this, to make TC & queueing possible,
but this is good enough for egress limiting.
Also, removed the _update_bandwidth_limit del+set on OvS QoS driver for
the bandwidth limit rule update, since that's not needed anymore.
Change-Id: Ie802a235ae19bf679ba638563ac7377337448f2a
Partially-Implements: ml2-qos
* Created a 'resources' subdir and moved all fixture files
to it.
* Split ML2ConfigFixture to the server-side ml2 configuration
fixture, and the OVS agent configuration fixture.
* Neutron process logs were using H:M:S format as their file name,
but when starting multiple agents of the same type my machine
was fast enough to do that in the same second so that different
processes were outputting to the same log file. No good!
Added ms to the log name format. I also changed the log time
from UTC to local timezone.
* Renamed and moved 'FullstackFixture' to neutron/tests/fullstack/
resources/environment.Environment
* Added a 'Host' abstraction that groups agents that report with
the same 'host' value. Hosts may be interconnected by the
environment via shared bridges.
* The 'Environment' class will accept global
attributes (This will be later filled with stuff like tunneling,
l2pop or other environment-level flags), and in this patch accepts
a list of host attributes (Configuration that may differ between
hosts like the l3 agent mode [legacy, dvr, dvr_snat]).
* Made OVS agent and L3 agent fixtures expose their bridges
so that I could interconnect them.
* Added a super simple L3 HA test to show that this entire thing
works.
Change-Id: Ie64de9f35bd6ab7cbad494061613ecf5e0ccd806
OVSBridge was inheriting db_list from BaseOVS, which was
returning the information of all the ports on the machine,
not only the ones belonging to the bridge.
The OVSNeutronAgent was using that method with the assumption
that ports were filtered by bridge.
To avoid confusion, this patch add a new method to OVSBridge
get_ports_attributes to query the info for all the ports
belonging to the bridge.
db_list is removed from BaseOVS since that method is already
available in ovsdb/api.py
ovs_lib methods that use db_list are refactored accordingly.
Co-Authored-By: Assaf Muller <amuller@redhat.com>
Change-Id: I2ce6d232744f48ba7fc0f824a7db32e3655bc2aa
Closes-Bug: 1473199
In OVS, ports don't equal to interfaces when a bond port created. This patch
add the new API get_iface_name_list to get the interfaces' name, and it's
supplementary to the current get_port_name_list API.
Change-Id: I29c220e099b8dcf78248e2d660c435578bb2932d
Partial-Bug: #1460494
Add support for the if_exists flag to the OVS native
db list command.
Closes-Bug: #1470742
Closes-Bug: #1470894
Change-Id: Ife48d99c145cfab7f0f5523f4cdfd33492085355
Add infrastructure needed for the implementations
(CLI and native) and add API to ovs_lib
Add functional tests for ovs_lib
blueprint ml2-ovs-qos-with-bwlimiting
Change-Id: Ided0740548987ca91f1549f251c7906e6449f91d
During startup, the agent was making many calls per port
to read information about the current VLAN, external ID, etc.
This resulted in hundreds of calls just to read information about
a relatively small number of ports.
This patch addresses that by converting a few key functions to
lookup information for all of the ports at once.
Performance improvement on dev laptop for 250 ports from agent
start to port ACTIVE status:
before: 1m21s
after: 1m06s
Closes-Bug: #1460233
Change-Id: Ic80c85a07fee3e5651dc19819c6cebdc2048dda7
The current API only support setting a bridge fail mode
to secure, this patch allow the user to set it to 'standalone'
as well
Change-Id: If7e6532dc7f8527c35834a37144ea4386fe1b861
Closes-Bug: #1458924
The list of controllers returned by the ovsdb server can be in any order,
therefore we can't assert likes for likes. Assert the sorted lists instead.
Change-Id: Ice3bb8cc0b3da70f8c9aae50d8cdae2b474ff49b
Closes-bug: #1448202
Add an API to change controller connection mode to 'out-of-band', feature
which might be useful for many projects using Openflow controller with OVS
Change-Id: If93f6858f4eed05f5f1d9bdb1667838d80c490cd
Closes-Bug: #1433208
This change defines for OVS, LinuxBridge and veth[1] bridge and port
fixture classes in order to handle bridge and port setUp/cleanUp.
It allows to simplify BaseOVSLinuxTestCase[2] and remove
BaseBridgeTestCase[2].
[1] veth backend simulates a bridge with a veth
[2] in neutron.tests.functional.agent.linux.base
Change-Id: If34c9a8fb6fa584fb1e30173ec619d1aac9701f9
Adds an option to setup OVS rules that will prevent
ports attached to the agent from sending any ARP responses
that contain an IP address not belonging to the port
(in fixed IPs or allowed_address_pairs).
It is disabled by default and requires an OVS version that
can match on ARP fields. If it is too old, traffic will
still flow but it won't have ARP spoofing protection.
There is a sanity check to verify that ARP header matching
is supported.
This prevention is specific to OVS so it will not help with
other plugins that use the reference iptables filtering. A
non-OVS-specific general approach will require something like
the ebtables integration in Ibc6d3d520c1383cf7e00f4bdeb7853a41ac4b14b.
Details:
A new table is added for ARP spoofing prevention. All ARP traffic
on the local switching table is sent to this spoofing table.
The spoofing table will allow all ARP requests because we aren't
interested in them. It will then install an ARP response allow rule
for each IP address the port is assigned. All other ARP responses are
dropped.
DocImpact
SecurityImpact
Partial-Bug: #1274034
Change-Id: I7c079b779245a0af6bc793564fa8a560e4226afe
ovs-vsctl get/set/clear/list can use a record_id that is not an
index on the table being queried. For example, the Controller table
can be queried by a bridge name.
This patch implements the lookup table that ovs-vsctl uses to do
these lookups.
Change-Id: I1983c48c5839df016046ba2596c7c4affa1ebe00
Closes-Bug: 1435567
Slawek Kaplonski
Rodolfo Alonso Hernandez
Brian Haley
Jakub Libosvar
Miguel Angel Ajo
Ali Sanhaji
Sławek Kapłoński
Thomas Morin
Terry Wilson
Isaku Yamahata
Ihar Hrachyshka
Boden R
Inessa Vasilevskaya
Frode Nordahl
Bhagyashri Shewale
lzklibj
Assaf Muller
Miguel Angel Ajo
rossella
Yalei Wang
Kevin Benton
Gal Sagie
armando-migliaccio
Cedric Brandily