Because of broken check why one port has similar address to the other
one, the first one by mistake could be set as virtual.
Example:
 port_1: 10.11.1.100
 port_2: 10.11.1.10

On create of port_2 it is set as 'virtual', but shouldn't.

This patch fixes that bug by using common function
utils.get_ovn_port_addresses().

Cherry-picked from Neutron: https://review.opendev.org/#/c/732690/

Change-Id: Ia4b986146c77edf0616015380359e37233df80fc
Closes-Bug: #1881759
(cherry picked from commit e6023ecb48)

Fix functional jobs by installing missing dependencies

Six is needed for the OVS/OVN compilation and tox to run the tests
itself.

Update OVS/OVN to version 2.12 in order to have possibility to
compile it under Ubuntu Bionic.

There is no need to compile OVS kernel module for functional tests,
so disable it.

Change-Id: Ia0ba2e78d4f97294a93ff127d9098978af21ba53
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit b568395182)

We override ovsdbapp's Backend class to change from using a class
variable for the ovsdb_connection to using an instance variable.
This means we have to override several different methods/properties.
This could be simplified in ovsdbapp by making ovsdb_connection a
property, which in ovsdbapp access a class variable, and in neutron
it is overridden to to store on the instance, so there would be
less to override.

ovsdbapp is adding user-definable indexed columns so things like
lookups by name will be faster. Due to the way it is architected,
the place to generate the indexes would be just before the
connection starts. Since methods like start_connection are
classmethods in ovsdbapp, and instance methods in neutron, this is
hard to do cleanly w/o ovsdb_connection being a property. Without
this change, neutron would fail if that property was defined. With
the change, it should pass with both old and new code.

Change-Id: Idd184807de24c79dd892046f01e3acdd1168ca2d
(cherry picked from commit c05bac8c40)

As a result of d92e71c2 the sync util can fail because it tries to
look up the ovsdb connection from the mechanism_manager under 'ovn'
but the sync util registers under 'ovn-sync'.

Change-Id: I6998b08672761a501e427aab5d581064d9425dde
Closes-Bug: #1876752
(cherry picked from Neutron commit 18dd0d4e)

Add test to ensure that logical router port associated
with an external network is created and removed as
needed: test_logical_router_port_creation. With that,
the test exercises the code-path where a gateway is
removed before a new one is set. In terms of OpenStack
workflow, that would look like:

  openstack router set --external-gateway $EXT_NET_ID $RTR
  openstack router unset --external-gateway $RTR
  openstack router set --external-gateway $EXT_NET_ID2 $RTR

Also changed test_gateway_chassis_with_bridge_mappings
to include docstring and have a lower bound count to the
mocked objects it uses.

Change-Id: Ib3edc101e9ee0fb53386fac0ae7f272ab3b3646b
Signed-off-by: Flavio Fernandes <flaviof@redhat.com>
Related-Bug: #1843485
Closes-Bug: #1844652
(cherry picked from commit 5cffa92daa)
(cherry picked from commit b9ab95036b)
(cherry picked from commit f3577c72f1)

This patch is making the transaction from the _delete_port() method in
OVNClient more resilient to errors where elements from that transaction
have already been deleted by another change in the database.

Prior to this patch, a few places could potentially raise RowNotFound
which would abort the whole transaction and would leave the a stable
port for being cleaned up after by the maintenance thread. This patch
tries to catch those exceptions that could potentially fail the
transaction.

Change-Id: I8fd1d1485269d23529a19085bd4aa4c6c74f5f91
Partial-Bug: #1874733
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit 8b234d8786)

When a metadata request arrives to the metadata agent, OVN
will need to figure out which port it corresponds to and it'll do
based on the network ID and IP address which should be a unique pair.

If it's not unique or it doesn't exist, then there is an error
and an ERROR trace is logged.

Change-Id: Id83c2899a03af98e32b0be7ba6ad661e646f07bc
Related-Bug: 1874733
Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
(cherry picked from Neutron commit 591adfee971906dc86e9357903f164d01657ed65)

The delete_port() method from OVNClient has a potential problem of
leaving stale ports when RowNotFound is raised from the process to
delete the port from the OVN database. Since the exception is not
granular enough, the RowNotFound could be raised from other objects that
are part of the same transaction (such as ACLs, DNS entries, etc...)
resulting in the revision for the port being deleted even tho the port
is still in the database.

Instead of giving a pass on the RowNotFound exception, this patch is
logging the error and re-raising it without deleting the revision.

(cherry-picked from Neutron I25b93b7c080403fc38365b638e4e03298b447d0f)

Conflicts:
       networking_ovn/tests/unit/ml2/test_mech_driver.py

Change-Id: Ic0d1234b032a58ce370e1703edc259b584099238
Partial-Bug: #1874733
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>

The networking-ovn has no master anymore, it has been removed at
https://review.opendev.org/#/c/715174/ because the code has now moved
to another repository (openstack/neutron).

Due to this, the release note job will fail and should be removed as
stated in a comment by Andreas at https://review.opendev.org/#/c/704898/

Change-Id: I13c055a8d971dd059ba2ace6f35bcc708b7654af
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>

It is possible to re-use the mech driver ovsdb connections in the
ovn l3 plugin, saving the overhead of two db connections/in-memory
copies of the db per process.

Closes-Bug: #1864548
(cherry picked from Neutron I022dea485f42cf76c4cec67ee43eed9a3770ec9c)

Change-Id: I5fdd80f91ccb23edf6430045d812043e7d3df382
(cherry picked from commit 632dee5d66)

This patch adds support for "virtual" port type following the work in
core OVN [0].

Currently there are two main usages for this type of port:

* Octavia: For creating the logical port for the virtual IP.
* VRRP [1]

Upon adding an IP address to the allowed_address_pairs field of the
Neutron's port, networking-ovn will look if that IP matches with the IP
of another existing port in the same network. If so, networking-ovn will
updating the matching port accordingly setting its type to "virtual"
and adding the required options in the OVN database.

The patch also accounts for other situations such as:

* Creating the VIP port after the parents (the ones with the IP in the
  allowed_address_pairs field) are created.

* When updating removing/adding allowed_address_pairs' the virtual
  ports are also updated.

* When deleting a parent port the virtual ports are also updated.

The code removes the type "virtual" from a virtual port whenever there's
no parents left (in case of deletion or editing allowed_address_pairs)
making it an ordinary port again.

The patch also keeps the logic introduced by
33fd553158 for version of OVN which does
not support the virtual port type (> 2.12) making it backward compatible.

[0]
054f4c85c4
[1]
https://docs.catalystcloud.io/tutorials/deploying-highly-available-instances-with-keepalived.html

Had to fix a bug in a previous cherry-pick that broke a port_type
check in ovn_client.py, https://review.opendev.org/#/c/703195/

Closes-Bug: #1840449
Related-Bug: #1789686
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit 5e72ea104c)

Conflicts:
    networking_ovn/common/ovn_client.py
    networking_ovn/common/utils.py
    networking_ovn/ovsdb/commands.py
    networking_ovn/tests/functional/test_mech_driver.py
    networking_ovn/tests/unit/fakes.py
    networking_ovn/tests/unit/ml2/test_mech_driver.py

Change-Id: I0b01b764413d178759a43028428c212014d3aa80

mech_driver.OVNMechanismDriver "_ovn_client" is not a class member but
a read-only property and can't be assigned.

Backported from Neutron (Ussuri): https://review.opendev.org/#/c/715375

Closes-Bug: #1869342

Change-Id: I98f7deaeaad80c840cfccc0dfc3f6f97de05e088
(cherry picked from commit e32e750b35)
(cherry picked from commit c46d2c6c5b)
(cherry picked from commit b81dc20945)

Only reschedule gateways/update segments when things have changed
that would require those actions.

This code has been cherry-picked from neutron:
https://review.opendev.org/#/c/705660

Co-Authored-By: Terry Wilson <twilson@redhat.com>

Conflicts:
      networking_ovn/common/utils.py
      networking_ovn/ovsdb/ovsdb_monitor.py
      networking_ovn/tests/unit/l3/test_l3_ovn.py
      networking_ovn/tests/unit/ovsdb/test_ovsdb_monitor.py

Change-Id: I62f53dbd862c0f38af4a1434d453e97c18777eb4
Closes-bug: #1861510
Closes-bug: #1861509

The update_acls code can potentially iterate over the same ACL
twice. This temporary workaround silently ignores an attempt to
delete the same row twice in the same transaction, since that
should be safe. Ultimately, refactoring the ACL code to use sets
and possibly handle the fact that an ACL could be referenced from
multiple rows should be done.

Change-Id: I259c92116f7d3186ae5af45f1407052eb57ac0ba
Related-bug: #1857016
(cherry-picked from Neutron I895eaf4006583fedc2657a4eb527df1ff992c5bc)

Master branch now has requirements incompatible with stable branches
constraints [0], so this switches to last tag compatible.

Longer term fix will be to run rally in a virtualenv, but this is lower
priority than other tasks. This can be revisited at the time, though for
older branches we will probably stick with pinned version

[0] 5776e015f1

Change-Id: I54fb021dec49724f17e7b5a330e09bcc638310d6
Closes-Bug: #1868691

In networking-ovn we map subnets to OVN DHCP_Options
rows (where we store the MTU), but from neutron API perspective,
the MTU is applied per network. So we need update all subnets
thats belong to network to propagade correctly MTU for network.

Closes-Bug: #1816449
Change-Id: Ieddc16e607bccef84316f9ee908e884539c7ed39
(cherry picked from commit 87ecdd8e0b)
(cherry picked from commit 017c49ecd5)

We recently merged this test and it its very unstable.
The fix for it was merged for OVN branch-2.12 recently [1].

For now we don't plan to bump OVN to 2.12 in stable/queens, because
of it lest skip this test.

[1] https://patchwork.ozlabs.org/patch/1185708/

Change-Id: I4b9d899db13d7d8071330fe92966a8632f0ab08b

When Load Balancer and its member has FIP assigned
and environment is configured to use DVR the member
FIP needs to be centralized. It is current core OVN
limitation, that should be solved in [1].

This patch adds this mechanism to OVN Client.
It covers cases:

1) FIP association on port that is a member of
   some LB - make it centralized.
2) FIP association on LB VIP - find a members
   FIPs and centralized them.
3) The reverse of each of the above cases.

Related-Bug: #1860662

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1793897

Conflicts:
    networking_ovn/common/ovn_client.py
    networking_ovn/common/constants.py
    networking_ovn/tests/unit/l3/test_l3_ovn.p

Change-Id: I254f0ac28f7585b699a8238e01ffb37dd70282ef
(cherry picked from commit 57ac38921e)

If a floating ip is associated to a port, it will be stored as
Logical_Switch_Port.external_ids:neutron:port_fip=FIP

OVN octavia driver will use this information to associate the floating ip
to the load balancer vip if the port is a virtual ip port created by
OVN octavia driver.

Signed-off-by: Numan Siddique <nusiddiq@redhat.com>
Co-Authored-By: Maciej Józefczyk <mjozefcz@redhat.com>
(cherry picked from commit c8e8cdbb4b)

Conflicts:
    networking_ovn/tests/unit/fakes.py

Change-Id: Ia63a375e360fb7b1ee31e4d9f96b38d9f31f4096

The virtualenv now is created with python3 by default.
Because of it we need to bump os-log-merger to the
version that supports py3.

Change-Id: Ifc0f188122931bc89ae8f90096b4bcc7303ec080
Closes-Bug: #1863220
(cherry picked from commit bfcc2a39b6)

This patch reverts [0].

The code wasn't accounting for VLAN provider networks, as stated in the
bug #1842988, DVR won't work if the provider network (where the FIP is
created) is VLAN.

There was also an incosistency in how the external_mac was set for the
VLAN networks. Upon creating the FIP the code was checking for the
network type and not setting the external_mac attribute in case the
network was VLAN type. But, if the port went down and up again (e.g if
you reboot the VM) the event handler that set/unset the external_mac [1]
wasn't check for the type. This is how people worked around the DVR
problem (as stated in bug #1842988).

For more information see bug #1842988.

[0]
c5aef51edc
[1]
eda5d7f80d/networking_ovn/ml2/mech_driver.py (L794-L800)

Closes-Bug: #1842988
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>

(Cherry-picked from Neutron Ifb795626dc9c2ac4f0104f491dd38c9b4cc902c9)

Change-Id: I3e9e734b757f67d3e4ab1c5a02f8e7f9f0c9aa2e
(cherry picked from commit f9b1dffada)

For now while updating FIP  check if port or logical_ip
has changed and only then we deleted the NAT entry.

Unfortunately each time when FIP update occurs the
method _create_or_update_floatingip() is used. It first deletes
LSP pointed by FIP and adds it again along with new NAT entries.
Based on author comment this actions are required.

So if we don't update FIP with logical_ip or new port_id,
like update a description, the NAT entries gets duplicated.

Since all is wrapped withing a transaction and to not wait for
proper fix (this code need sa refactor based on commments with NAT
external_id column) I think thats safe just to delete the NAT entry
in such situation like described above.

(cherry picked from commit 45ae9dfb7d)

Change-Id: Iea532e2a02b7992305d1b90aa040e064901c340c
Related-Bug: #1859977

It looks like sometimes, the device_id for ports created
by Neutron DHCP Agent can be in the form of:

- dhcp-$hostuuid-$networkid
- 'reserved_dhcp_port' (DEVICE_ID_RESERVED_DHCP_PORT)

Current code is only taking the first form into account when
skipping Neutron DHCP Agent ports. This patch is changing it
to include both forms.

Closes-Bug: #1848521
Co-Authored-By: Lucas Alvares Gomes <lucasagomes@gmail.com>
Change-Id: Ifbfc551ac68dcc5d3d39a155f7642f2f2d9272c4
Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
(cherry picked from commit 15bf8f265b)

This is a tweak to changes done to fix bug 1834637. Specifically,
this change addresses scaling. The previous gerrit change had
modifications to all OVN sub-ports performed as a single
transaction. That did not account for race-condition on neutron
DB queries, which leads to timeouts under heavy loads.

Another cleanup done by this change is to fold an additional
update on neutron db into ovn trunk driver. That saves
update from doing another database transaction, thus making
it faster. The no longer needed function in mech_driver was
called _update_subport_host_if_needed

By breaking the iteration into multiple transactions, the
change in time is marginal:

  Service-level agreement
  NeutronTrunks :: neutron.create_trunk

  from 34.2 sec to 35.6 for 50%ile
  from 35.6 sec to 36.1 for 95%ile

This patch doesn't go to master networking-ovn. It
has been  migrated to master Neutron [1].

[1]: https://review.opendev.org/#/c/701646/

Depends-On: https://review.opendev.org/#/c/695693/
Change-Id: I7de3ac2a7cf8869ead8ab5fbb34a9861a96d3a0c
Closes-Bug: #1834637
Co-authored-by: Maciej Józefczyk <mjozefcz@redhat.com>
(cherry picked from commit c418dd720b)

There's a special case in the update_port_precommit() method to handle
the case where an existing port is being added to a router. The method
will try to create an entry in the ovn_revision_numbers table but, if
that entry is already present a DBDuplicateEntry exception will be
raised and the whole update method will fail.

The update_port_precommit() is the only method at present that have this
special handler, for all other resources creating a new record in the
ovn_revision_numbers table happens at the create_*_precommit() methods
(this include create_port_precommit() as well).

This patch is fixing the problem by adding a new parameter to the
reate_initial_revision() method called "may_exist". When set to True the
code will not raise an DBDuplicateEntry if the record already exists. By
default this method is False.

Closes-Bug: #1800875
Change-Id: Iddd89725e617b22c35e30ae7f20a7f87de296cdb
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit b5642d93a6)

Prior to this patch, the "unknown" address was being set the logical
switch ports solely based on whether the port security was enabled or
disabled. That's not how it's intended to work.

With this patch the "unknown" address is only set to the normal logical
switch ports, those which types are "router", "localnet" or "localport"
won't be affected.

The maintenance task was updated to correct this behavior for existing
ports (the maintenance was suppose to be removed in the T cycle, this
patch bumps it to U cycle instead due to this change).

Closes-Bug: #1838535
Related-Bug: #1815270

Conflicts:
   networking_ovn/common/ovn_client.py
   networking_ovn/common/constants.py

Change-Id: I3c01bd7d1685c8a7e13a55e545e98baf19e9a0f9
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit 6a89dbfe47)

The following patch provides L3HA Rescheduling of gateways when chassis
are added/removed. It reschedules the gateway ports when a new chassis
is added/old one removed. However, the number of chassis where a
gateway can be hosted is limited by the constant MAX_GW_CHASSIS.

Co-Authored-By: Maciej Józefczyk <mjozefcz@redhat.com>

Conflicts:
   networking_ovn/tests/functional/base.py
   networking_ovn/tests/functional/test_router.py
   networking_ovn/l3/l3_ovn_scheduler.py

Change-Id: I0d96efe4ceef4168039930738285c19d5c003951
Closes-Bug: #1762691
(cherry picked from commit 12070403db)

This patch fixes wrong path which prevents doc from being correctly
built.

Change-Id: I865da811157aa41bf0c7f1cf1089b93cb373707e
Closes-Bug: #1795883
Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
(cherry picked from commit 993ef63b66)

When a port's port security is disabled then the port should be allowed to use
any mac address. And for this to work, OVN expects 'unknown' address
to be added into the Logical_Switch_Port.addresses column. This patch
adds this 'unknown' address if the port security is disabled.

Change-Id: I5f25f4d4b2acccada3ab2944e9cfd2461149ef3e
Closes-bug: #1815270
Co-Authored-By: Lucas Alvares Gomes <lucasagomes@gmail.com>
(cherry picked from commit ef5e9d31ec)

This patch is adding an external_id to the Logical_Router_Port
rows including the Neutron network ID.

The reason for this is that if we consume Octavia from Stein+ in
Rocky/Queens, this external id is missing and the OVN Octavia
driver (octavia_api) will not be able to apply the Load Balancers
to all the Logical Switches and Logical Routers where needed.

Note: This is a partial cherry-pick from
https://review.opendev.org/#/c/668397/ since Octavia itself
does not exist in stable/queens but can be made to work
with neutron's stable/queens.

Change-Id: I4a0614afda8d2c2b8ccbb2583bc339a232fc802e
Co-Authored-By: Daniel Alvarez <dalvarez@redhat.com>
Closes-Bug: #1855328
Related-Bug: #1794260
(cherry picked from commit aa4268a094)

Prior to this patch, metadata agent was fetching the OVN bridge
from the settings but this could lead to inconsistencies if the
config is different from the actual OVSDB configuration as
ovn-controller would be using one bridge and Metadata agent
another one.

In order to eliminate these inconsistencies, this patch is
changing the behavior of the agent so that it reads the OVN
bridge from OVSDB as ovn-controller does. Also, when the
agent is restarted, if the bridge has changed, it'll plug all
the VETH pairs to the right bridge and unplug it from the older
one (this is handy for the migration from ML2/OVS where an
intermediate OVS bridge is used temporarily).

The config option is not removed by this patch but ignored to
avoid issues when it doesn't match OVSDB configuration.

 Conflicts:
	networking_ovn/agent/metadata/agent.py
	networking_ovn/tests/functional/test_metadata_agent.py

Change-Id: I33f3509b8e5fe0398a27d35923e46f0409a1935d
Closes-Bug: #1799216
Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
(cherry picked from commit 3e5d9213aa)
(cherry picked from commit 950a53748c)

Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.

To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:

from 101 sec to 35.6 for 95%ile
from 99 sec to 34.2 for 50%ile

The same thing has been done for Trunk deletion.

This reverts commit 2e0832f7b8

[0] https://github.com/openstack/rally-openstack/blob/master/rally_openstack/scenarios/neutron/trunk.py#L37

Note: While this is a back-merge from Rocky, special care was
needed to account for the fact that in Queens release there
is only one port binding for a db_port. Thus no iteration over
db_port.bindings is used.

Change-Id: I6b659cbede25f271fa3b6a1c9e72019694ab6608
Closes-Bug: #1834637
Related-Bug: #1845479
Co-authored-by: Maciej Józefczyk <mjozefcz@redhat.com>
(cherry picked from commit 62eb828186)