Commit Graph

65 Commits (1c2e10f8595d2286bd9bec513bc5a346a84a6f7c)

Author SHA1 Message Date
Brian Haley cf37563c83 Remove deprecated vsctl ovsdb_interface api
This was deprecated in https://review.openstack.org/#/c/503070/
so remove all the vsctl-related code, leaving just the native
ovsdb api.

Also removed renamed ovs_vsctl_timeout value, which was changed
to ovsdb_timeout in https://review.openstack.org/#/c/518391/

Change-Id: I50dfcea3deb41df1bd01fd06b76522453a6ba50b
5 years ago
Slawek Kaplonski 1f8378e0ac [OVS] Add mac-table-size to be set on each ovs bridge
By default number of MAC addresses which ovs stores in memory
is quite low - 2048.

Any eviction of a MAC learning table entry triggers revalidation.
Such revalidation is very costly so it cause high CPU usage by
ovs-vswitchd process.

To workaround this problem, higher value of mac-table-size
option can be set for bridge. Then this revalidation will happen
less often and CPU usage will be lower.
This patch adds config option for neutron-openvswitch-agent to allow
users tune this setting in bridges managed by agent.
By default this value is set to 50000 which should be enough for most
systems.

Change-Id: If628f52d75c2b5fec87ad61e0219b3286423468c
Closes-Bug: #1775797
5 years ago
Miguel Angel Ajo 88f5e11d8b Avoid agents adding ports as trunk by default.
Agent OVS interface code adds ports without a vlan tag,
if neutron-openvswitch-agent fails to set the tag, or takes
too long, the port will be a trunk port, receiving
traffic from the external network or any other port
sending traffic on br-int.

Also, those kinds of ports are triggering a code path
on the ovs-vswitchd revalidator thread which can eventually
hog the CPU of the host (that's a bug under investigation [1])

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1558336

Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>
Change-Id: I024bbbdf7059835b2f23c264b48478c71633a43c
Closes-Bug: 1767422
5 years ago
Zuul ff24fc0727 Merge "Fix ingress bw limit for OVS DPDK ports" 5 years ago
Ali Sanhaji 6bf0788da0 Adding DSCP mark and inheritance in OVS and LB tunnels outer header
Adding ability to set DSCP field in OVS tunnels outer header, or
inherit it from the inner header's DSCP value for OVS and linuxbridge.

Change-Id: Ia59753ded73cd23019605668e60cfbc8841e803d
Closes-Bug: #1692951
5 years ago
Zuul cea9e0bfcb Merge "ovs-lib: Pass string as udp port to ovsdb" 5 years ago
Zuul 6e9e610afb Merge "Remove usage of deprecated idlutils module" 5 years ago
Sławek Kapłoński 1be8574352 Fix ingress bw limit for OVS DPDK ports
For OVS based DPDK ports ingress bandwidth limit is now implemented
using egress-policer qos type.
Additionally limit values are set in other_config of QoS because there
is no queue used in this case.

This patch moves also helper methods used to conversion between
bytes and bits and between bits and kilobits to neutron.common.utils
to be able to use it also in ovs_lib module.

Change-Id: I94d1e8dfb82df5c602476db8aaa884ae91fecd7f
Closes-Bug: #1724729
5 years ago
Jakub Libosvar 622a137974 ovs-lib: Pass string as udp port to ovsdb
ovsdb maps accept strings as values only. This patch converts integer to
be passed to ovsdb in case vxlan_udp_port config value is used.

Change-Id: Idba77939a80d80a4bc9625d10c8b37b23b91b9c5
Closes-bug: #1742931
5 years ago
Sławek Kapłoński ab4143e0ed Remove usage of deprecated idlutils module
This module located in neutron.agent.ovsdb.native.idlutils is
deprecated and used only in functional test of ovs_lib module.
Now it's removed and proper module ovsdbapp.backend.ovs_idl.idlutils
is used instead.

Change-Id: I6174a08a07e273e2180ef907adb1da48b518d579
5 years ago
Sławek Kapłoński 71995b27f2 Remove deprecated method OVSBridge.set_protocols()
This method is deprecated since Ocata and marked to remove in
Queens release.

Change-Id: I1e280d7ad9a966eb576e9c8ca5882b35cb37bad8
5 years ago
Jakub Libosvar 330ebbb1b1 functional-tests: Make addresses for tunneling unique
OVS can hold only one tunnel with same endpoints. Some tests had
hardcoded values for both tunnel endpoints which made them unable to run
in parallel manner.

This patch takes always exclusive address using resource allocator.

Change-Id: If81296d54656551b24917d561f235edb96a6c2df
Closes-bug: #1697533
6 years ago
Sławek Kapłoński 2d0d1a2d76 Add support for ingress bandwidth limit rules in ovs agent
Add support for QoS ingress bandwidth limiting in
openvswitch agent.
It uses default ovs QoS policies on bandwidth limiting
mechanism.

DocImpact: Ingress bandwidth limit in QoS supported by
           Openvswitch agent

Change-Id: I9d94e27db5d574b61061689dc99f12f095625ca0
Partial-Bug: #1560961
6 years ago
Thomas Morin 5981bf416d OVSBridge: add --strict to allow priority in delete_flows
n8g-sfc currently has its own variant of OVSBridge to allow the use
of priority in a delete_flows call

This change is meant to make this available outside n8g-sfc and
simplify n8g-sfc code.

This change adds a 'strict' boolean parameter to mod_flow and delete_flows
that results in ovs-ofctl to be run with --strict for del-flows and
mod-flows actions.  When strict is set, the use of priority is allowed
and hence not rejected anymore.

Note that for batched actions in a deffered bridge, we disallow mixing
calls with strict and without strict, which can't be translated in one CLI
call.

Needed-By: I3bf939590dd43bff685f133bff86eb7e9068de91
Change-Id: I289d546780f10dc1002ab6bc2e1b38c9ef2d728f
6 years ago
Thomas Morin d761d26225 delete_flows shall only touch flows with the bridge cookie
With this change delete_flows will only remove flows matching the default
cookie of the bridge.

The uninstall_flows implementation in the native bridge is also modified
to touch only the flows with the bridge cookie.

To still allow deletion of all cookies, cookie=COOKIE_ANY is introduced
as a special value, and used in the agent code in the places where the
intent is indeed to clean all flows whatever their cookie is.

Partial-Bug: #1557620
Change-Id: Idd0531cedda87224531cb8fb6a912ccd0f1554d5
6 years ago
Terry Wilson 1698bee770 Set OVS inactivity_probe to vsctl_timeout when adding manager
If the vsctl_timeout > OVS's inactivity probe interval and a
transaction execution time exceeds the probe interval, OVS will
disconnect and the transaction will return TRY_AGAIN and most
likely repeat failing until the vsctl_timeout is reached. This
change ensures that the "failsafe" creation of the manager also
sets the inactivity probe to the vsctl_timeout value.

Currently the patch doesn't override the probe_interval on an
existing Manager since it is possible that connection is used by
outside applications and it theoretically should be handled at
deployment.

Related-Bug: #1627106
Change-Id: I76fa0a0cf04a166edf062086fceb2fd90960ad6b
6 years ago
Thomas Morin fcde09462d Fix OVSBridge.delete_flows when called with no args
With this change calling delete_flows with no kwargs will (instead
of resulting in calling "ovs-ofctl <action> <bridge> -", which does
nothing with no flow spec given on stdin) result in calling
"ovs-ofctl <action> <bridge>", which will delete all flows.

This aligns the behavior of delete_flows with the behavior currently seen by
all callers for the same method shadowed by the implementations in
OpenFlowSwitchMixin classes.

Change-Id: Ic0449acb3a0d4915ce025300d6f3c507a3cd8e48
Closes-Bug: 1658019
6 years ago
Terry Wilson 26766963c6 Clean up ovsdb-native's use of verify()
When updating an ovsdb set-type column, the existing code does
the following:

1. Read the existing column value
2. Call verify() to cause a write failure if something else
   modifies the column before we commit
3. Append the value to our local copy of the column
4. Assign the local value to the object.column to trigger
   __setattr__ to write the value to the database

If verify() fails, which it *very* often does in a test
environment or a busy system, ovsdb-server will respond with a
TRY_AGAIN response which will cause the whole process to start
over.

In the 2.6 cycle, Row.addvalue()/delvalue() methods were added
which use OVDB's native 'mutate' methods to handle adding/deleting
from a set/map-type column. Using these means calling verify() is
no longer required and things will proceed *much* more efficiently.

Bug #1627106 where we get frequent TimeoutExceptions appears to be
related. Eliminating the frequent TRY_AGAIN responses, in my local
testing, also eliminates the TimeoutExceptions. This doesn't mean
that the underlying issue is resolved, but it may be avoided.
Related-Bug: #1627106

Change-Id: I26c7731f5dbd3bd2955dbfa18a7c41517da63e6e
6 years ago
Thomas Morin 271a4ffd6d OVS: merge the required OpenFlow version rather than replace
This change modifies the behavior of OVS native and ovs-ofctl bridge
implementations so that instead of configuring the bridge only for the
required OVS protocol version, they add the required version to the
already configured versions.

To achieve this, an add_protocols method is added to the OVSBridge
class, relying on the OVSDB add_db_attribute added in
Ib6ce75846f9b13c1c33f0ced5ccc619ee7860dc1, with the behavior of
making the provided set of versions supported in addition to already
configured ones.

It is aimed to be a cleaner solution to bug 1622644 than the quickfix merge
from I4475865c4f83cb9f3e12c709af752bc490692ca3 .

After this change, the set_protocols method appears useless and is
hence marked for future removal.

Depends-On: I4386aa293f9b18d2e17b4a80d9c7da4b9b46f3c9
Change-Id: Id5ac7e6431c97fc70d8404b16f89533b6f270eee
Related-Bug: 1622644
6 years ago
Jenkins d78903dd60 Merge "ovsdb: don't erase existing ovsdb managers" 7 years ago
Isaku Yamahata 7d42176853 ovsdb: don't erase existing ovsdb managers
The current existing agent erases already set ovsdb managers
entries. In some use cases, cloud admin sets ovsdb managers.
eg, for SDN controllers or monitoring purpose. Neutron agent
shouldn't unconditionally erase the existing ovsdb managers.

This patch implements a new api add_manager (along with
get_manager and remove_manager) to the ovsdb api which will
allow us to configure a manager on a switch without overriding
the existing managers.

Closes-Bug: #1614766
Change-Id: Ibf9bd02fac3070d166546cac478ef984e4e43f28
Co-Authored-By: sridhargaddam <sgaddam@redhat.com>
Co-Authored-By: Terry Wilson <twilson@redhat.com>
7 years ago
Terry Wilson 2c7ef566af Handle db_add in transaction for new objects
The native interface chokes when doing db_add on an object that
was created within the same transaction due to the column we are
modifying not yet existing. This patch adds defaults for the
column.

Change-Id: I4386aa293f9b18d2e17b4a80d9c7da4b9b46f3c9
Closes-Bug: #1642764
7 years ago
Jenkins b81b1a001d Merge "set_db_attribute differs between vsctl and native" 7 years ago
Terry Wilson cb48d15466 set_db_attribute differs between vsctl and native
On the following:

b.set_db_attribute('Port', pname, 'other_config', {'a': 'b'})
b.set_db_attribute('Port', pname, 'other_config', {'c': 'd'})

will produce different results between the vsctl and native OVSDB
implementations. vsctl will merge the values into a single dict
and native will overwrite the dict.

This patch makes the native implementation mirror vsctl.

Related-Bug: #1630920
Change-Id: Ie7680a301b8b3ee8e5654666e2aea78ecbd07a12
7 years ago
Terry Wilson 135a3fdc17 Add db_add to OVSDB API
It is at times useful to append to an OVSDB row in a single
transaction instead of doing separate transactions to pull back a
value, then another to modify and append to it. This patch adds

API.db_add(self, table, record, column, *values)

to the OVSDB API and adds it to the vsctl and native
implementations. It follows the format of ovs-vsctl's 'add'
command which works on both sets and maps. For map columns, its
behavior is to set a key in a column if it is not already set. For
sets, it appends to the current values stored in the column.

For maps, values can be passed as
  (..., {'key': value}, {'key2': value2, ...)
or
 (..., {'key': value, 'key2': value})

Change-Id: Ib6ce75846f9b13c1c33f0ced5ccc619ee7860dc1
7 years ago
Jenkins 605cdce696 Merge "Handle uuid references within an ovsdb transaction" 7 years ago
Terry Wilson 1f4f806584 Handle uuid references within an ovsdb transaction
Enable the OVSDB API implementations to handle referencing a newly
created object within the same transaction. For vsctl, this is via
the --id=@name construct. For native, this is via storing the
created row as the result, then replacing the result when the
transaction completes. This uses an api.Command object passed as
part of a column value in a db_set/db_create operation as a
reference to that object in a transaction. For example, do:

with self.ovsdb.transaction() as txn:
    queue = txn.add(self.ovsdb.db_create("Queue", ...))
    qos = txn.add(self.ovsdb.db_create("QoS", queues={0: queue}))
    port = txn.add(self.ovsdb.db_set("Port", pname, ('qos', qos)))

instead of having to do 5 separate transactions to: create a
queue, find the queue, create the QoS entry, find the QoS entry,
and finally to update the port with the QoS entry.

Change-Id: I1781794958af1483dabc0f5d17f2df6fea828564
Closes-Bug: #1615105
7 years ago
Ihar Hrachyshka 31e1aeb66b Forbid importing neutron.tests.* from outside tests subtree
neutron-sanity-check tool was importing neutron.tests.base module, which
may be not present on some systems (f.e. RDO splits neutron/tests/
subtree in a separate python-neutron-tests package). It made the tool
not usable in some setups.

https://bugzilla.redhat.com/show_bug.cgi?id=1374282

This is not the first time when we by mistake import from
neutron.tests.* and break distributions. It's time to stop it by
proactively forbidding that pattern via a new hacking check.

Some functions were moved from neutron.tests.base to
neutron.common.utils to fulfill the need requirement. They were moved
using debtcollector, no current consumers should be affected.

Closes-Bug: #1621782
Change-Id: I790777ddcbd1b02218b3db54ae3d5c931d72d4fa
7 years ago
Boden R 319bc525b4 isolate test_db_find_column_type_list
As per the recent gate failures (see bug), it appears
OVSLibTestCase.test_db_find_column_type_list is not isolated
and thus its usage of ovsdb's db_list() and db_find() occasionally
obtain different results.

This patch adds the db_list() and db_find() operations within the
test case to run in a transaction so that we get a single snapshot
of the db results.

In addition this patch undoes the changes from patch set 1 as the
initial changes do not appear to address the issue at hand.

Change-Id: I312076edb6e11f21347831843758894e11d6f56c
Closes-Bug: #1592546
7 years ago
Jakub Libosvar 7803175840 functional: Use assertItemsEqual for db_find outputs
Change-Id: I3fc0fbecebb811fda669600173fb7c0832848935
Closes-Bug: 1596585
7 years ago
Inessa Vasilevskaya bffc5f062c functional: fix OVSFW failure with native OVSDB api
A bunch of functional tests fail because of non implemented
x != [] operation in idlutils.condition_match() and
wrong condition passed to db_find() in OVSFW test.
This patch addresses the issue by implementing lists
comparison in native.idlutils and fixing the call to
db_find() in OVSFW test.

A functional test for OVSDB API's db_find() has been
added to ensure that querying a list column gives the same
result both with vsctl and native ovsdb_interface; unit
test for idlutils.condition_match() with corner cases has
been added as well.

Change-Id: Ia93fb925b8814210975904a453249f15f3646855
Closes-bug: #1578233
7 years ago
Frode Nordahl 773394a188 OVS: Add support for IPv6 addresses as tunnel endpoints
Remove IPv4 restriction for local_ip configuration statement.

Check for IP version mismatch of local_ip and remote_ip before creating
tunnel.

Create hash of remote IPv6 address for OVS interface/port name with least
posibility for collissions.

Fix existing tests that fail because of the added check for IP version
and subsequently valid IP addresses in _setup_tunnel_port.

DocImpact

Change-Id: I9ec137ef8c688b678a0c61f07e9a01382acbeb13
Closes-Bug: #1525895
7 years ago
Bhagyashri Shewale 88e899f7a0 Fix module's import order
Made corrections in import order for built-in, third party and
project specific modules as per OpenStack import standards [1].

[1] http://docs.openstack.org/developer/hacking/#import-order-template

Change-Id: I899deefd6ee4732d6c0afd17a5afbe42b0fa37ba
7 years ago
lzklibj 3491cbc0c5 Unify assertEqual for empty usages
Update previous assertEqual(observed, *empty) usages to
assertEqual(*empty*, observed).

This patch aslo update hacking check for assertEqual with
empty types.

Change-Id: I981277618f92254a5beb9d3308a317d8c14e125c
8 years ago
Assaf Muller 9c3e3a0fce Handle empty bridge case in OVSBridge.get_ports_attributes
Before this patch, get_ports_attributes would call
get_port_name_list. In the case of an empty bridge,
get_port_name_list would return an empty list and feed that
in to ovsdb.db_list, thereby returning all ports instead of
no ports.

Change-Id: I5a91028c59d71943b7cef86a94fa6ab1cc3d667c
Closes-Bug: #1499821
8 years ago
lzklibj a13f5afcc8 Remove unused ovs_lib method reset_bridge
Per [1] we are using a better way to keep tunnel connectivity,
so reset_bridge isn't used anymore. Bug in [2] was caused by
using method reset_bridge which will delete and recreate bridge.
For [1] makes method reset_bridge deprecated, it makes sense to
remove this method, and make [2] no longer produce.

[1] https://review.openstack.org/#/c/182920
[2] Related-bug: #1332450

Change-Id: I155f66a37b8d4081126467fe576e8315c2d5560c
8 years ago
Miguel Angel Ajo d56fea0a39 Fix the low level OVS driver to really do egress
It seems that the Queue + QoS + linux-htb implementation was really
limiting ingress by default. So this patch switches the implementation
to the ovs ingress_policing_rate and ingress_policing_burst parameters
of the Interface table.

Later in time we may want to revise this, to make TC & queueing possible,
but this is good enough for egress limiting.

Also, removed the _update_bandwidth_limit del+set on OvS QoS driver for
the bandwidth limit rule update, since that's not needed anymore.

Change-Id: Ie802a235ae19bf679ba638563ac7377337448f2a
Partially-Implements: ml2-qos
8 years ago
Ihar Hrachyshka 0a0b4c53c0 Merge remote-tracking branch 'origin/feature/qos' into merge-branch
Change-Id: I7a78ea4a8b3a03ef2013d41f9788e554f73c990b
8 years ago
Assaf Muller 0de917e09b Enable fullstack multinode tests, add L3 HA test exemplar
* Created a 'resources' subdir and moved all fixture files
  to it.
* Split ML2ConfigFixture to the server-side ml2 configuration
  fixture, and the OVS agent configuration fixture.
* Neutron process logs were using H:M:S format as their file name,
  but when starting multiple agents of the same type my machine
  was fast enough to do that in the same second so that different
  processes were outputting to the same log file. No good!
  Added ms to the log name format. I also changed the log time
  from UTC to local timezone.
* Renamed and moved 'FullstackFixture' to neutron/tests/fullstack/
  resources/environment.Environment
* Added a 'Host' abstraction that groups agents that report with
  the same 'host' value. Hosts may be interconnected by the
  environment via shared bridges.
* The 'Environment' class will accept global
  attributes (This will be later filled with stuff like tunneling,
  l2pop or other environment-level flags), and in this patch accepts
  a  list of host attributes (Configuration that may differ between
  hosts like the l3 agent mode [legacy, dvr, dvr_snat]).
* Made OVS agent and L3 agent fixtures expose their bridges
  so that I could interconnect them.
* Added a super simple L3 HA test to show that this entire thing
  works.

Change-Id: Ie64de9f35bd6ab7cbad494061613ecf5e0ccd806
8 years ago
Ihar Hrachyshka cc0ae6dd49 Merge remote-tracking branch 'origin/feature/qos' into merge-branch
Change-Id: I683102e617202e0ffc953a0d3cc179879f8faf82
8 years ago
rossella e32e74553f Introduce get_ports_attributes in OVSBridge
OVSBridge was inheriting db_list from BaseOVS, which was
returning the information of all the ports on the machine,
not only the ones belonging to the bridge.
The OVSNeutronAgent was using that method with the assumption
that ports were filtered by bridge.
To avoid confusion, this patch add a new method to OVSBridge
get_ports_attributes to query the info for all the ports
belonging to the bridge.
db_list is removed from BaseOVS since that method is already
available in ovsdb/api.py
ovs_lib methods that use db_list are refactored accordingly.

Co-Authored-By: Assaf Muller <amuller@redhat.com>

Change-Id: I2ce6d232744f48ba7fc0f824a7db32e3655bc2aa
Closes-Bug: 1473199
8 years ago
Ihar Hrachyshka fcc5d5bcf7 Merge remote-tracking branch 'origin/feature/qos' into merge-branch
Change-Id: I1c1fd593235fda1cdd053980f50eff21ca9011b6
8 years ago
Yalei Wang b4e42a3418 Add new ovs DB API to inquire interfaces name list in a bridge
In OVS, ports don't equal to interfaces when a bond port created. This patch
add the new API get_iface_name_list to get the interfaces' name, and it's
supplementary to the current get_port_name_list API.

Change-Id: I29c220e099b8dcf78248e2d660c435578bb2932d
Partial-Bug: #1460494
8 years ago
Ihar Hrachyshka 2fed2617cd Merge remote-tracking branch 'origin/feature/qos' into merge-branch
Change-Id: I7f2342d62634f5b4af3a083cc1aaff46efe28519
8 years ago
Jenkins 67ceaa4840 Merge "OVS native DBListcommand if_exists support" 8 years ago
Ihar Hrachyshka 8a0e11143c Merge remote-tracking branch 'origin/master' into merge-branch
Change-Id: I114225ad6fa52ae5a085719251f273f3de4d72ef
8 years ago
Kevin Benton 55cb8e4026 OVS native DBListcommand if_exists support
Add support for the if_exists flag to the OVS native
db list command.

Closes-Bug: #1470742
Closes-Bug: #1470894
Change-Id: Ife48d99c145cfab7f0f5523f4cdfd33492085355
8 years ago
Gal Sagie be1d242fa3 Add Create/Destroy API to OVS QoS BW Limiting
Add infrastructure needed for the implementations
(CLI and native) and add API to ovs_lib
Add functional tests for ovs_lib

blueprint ml2-ovs-qos-with-bwlimiting

Change-Id: Ided0740548987ca91f1549f251c7906e6449f91d
8 years ago
Kevin Benton 4dc68ea88b Read vif port information in bulk
During startup, the agent was making many calls per port
to read information about the current VLAN, external ID, etc.
This resulted in hundreds of calls just to read information about
a relatively small number of ports.

This patch addresses that by converting a few key functions to
lookup information for all of the ports at once.

Performance improvement on dev laptop for 250 ports from agent
start to port ACTIVE status:
   before: 1m21s
   after: 1m06s

Closes-Bug: #1460233
Change-Id: Ic80c85a07fee3e5651dc19819c6cebdc2048dda7
8 years ago
Gal Sagie 7ea278087c OVS_LIB support API for setting fail mode 'standalone'
The current API only support setting a bridge fail mode
to secure, this patch allow the user to set it to 'standalone'
as well

Change-Id: If7e6532dc7f8527c35834a37144ea4386fe1b861
Closes-Bug: #1458924
8 years ago