openstack
/
neutron-specs
Code Issues Proposed changes

Add spec for coexistence support for OVN Interconnect RFE 

Related-bug: #2027742
Change-Id: I4915d482358f01e81e4b48986f18f733b9d93642
This commit is contained in:
Roberto Bartzen Acosta 2023-08-11 11:11:11 -03:00
parent 8a0eaebf9a
commit 0e8d530ba1
1 changed files with 529 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

529
specs/2024.1/ml2ovn-coexistence-support-ovn-interconnect.rst Normal file
View File

@ -0,0 +1,529 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
============================================================
ML2/OVN - Coexistence Support for OVN Interconnect Resources
============================================================
https://bugs.launchpad.net/neutron/+bug/2027742
Currently two individual Neutron deployments using ML2/OVN are separate and may
only communicate using normal provider networks. However OVN supports the
feature OVN interconnect, that allows multiple separate OVN deployments to be
connected together. The interconnection is implemented using a normal overlay
(just like the one between compute nodes) and can therefor be created easily
in a large scale (something that is not necessarily the case for provider
networks).
Nowadays the Neutron OVN db sync command is actively interfering with
OVN interconnect by removing resources from the northbound database.
The goal of this spec is to remove this interference and allow operators to
use OVN interconnect. Is out of the scope of this spec to integrate OVN
interconnect directly to Neutron (however this might be part of a future spec).
Background: OVN Interconnect
============================
The OVN Interconnect allows multiple clusters to be interconnected at Layer 3
level. This can be useful for deployments with multiple Availability
Zones (AZs) that needs to allowing connectivity between workloads in separate
zones.
In the case of layer 3 interconnection, the logical routers on each cluster /
availability zone can be connected via transit overlay networks. The transit
network is an abstract representation of the interconnect layer, and the
element responsible for this intercluster visibility is the Transit Switch.
These interconnection switches are created in a global database and replicated
to each AZ via ovn-ic daemon. So, basically, a logical router in one AZ is
connected to a Logical router in another AZ via Transit switch. More details
can be found in the ovn-architecture manpage [1]_.
The reference design using two separated OVN clusters (north and south) is
described below::
+---------------------------------------------------------------------+
|OVN north cluster |
| +------------------------------------------+ |
| |OVN central node | |
| +---------------+ | +-------------+ +-------------+ | |
| |network node | | |ovsdb-server | |ovsdb-server | | |
| |ovn-controller +---------+southbound | |northbound | | |
| | | | | | | | | |
| +---------------+ | | | | | | |
| | | | | | | |
| +---------------+ | | | | | | |
| |compute node | | | | | | | |
| |ovn-controller +---------+ | | | | |
| | | | | | | | | |
| +---------------+ | +------+------+ +-------+-----+ | |
| | | | | |
| | +---+---------------+---+ | |
| | | | | |
| | +----------+-----+ +-----+---------+ | |
| | |ovn-ic | |ovn-northd | | |
| | | | | | | |
| | | | | | | |
| | +-------+--------+ +---------------+ | |
| | | | |
| +----------|-------------------------------+ |
| | |
+-----------------------------------|---------------------------------+
|
+-----------------------+---------------------+
|Global DB - Transit switches |
|ovsdb-server: northbound IC, southbound IC |
| |
+-----------------------+---------------------+
|
+-----------------------------------|---------------------------------+
|OVN south cluster | |
| +----------|-------------------------------+ |
| | | | |
| | +-------+--------+ +---------------+ | |
| | |ovn-ic | |ovn-northd | | |
| | | | | | | |
| | | | | | | |
| | +----------+-----| +-----+---------+ | |
| | | | | |
| | +---+---------------+---+ | |
| | | | | |
| +---------------+ | +------+------+ +-------+-----+ | |
| |network nodeer | | |ovsdb-server | |ovsdb-server | | |
| |ovn-controller +---------+southbound | |northbound | | |
| | | | | | | | | |
| +---------------+ | | | | | | |
| | | | | | | |
| +---------------+ | | | | | | |
| |compute node | | | | | | | |
| |ovn-controller +---------+ | | | | |
| | | | | | | | | |
| +---------------+ | +-------------+ +-------------+ | |
| |OVN central node | |
| +------------------------------------------+ |
+---------------------------------------------------------------------+
The following example will outline how OVN Interconnect works within the scope
of OVN. It therefor follows the naming of OVN and not that of Neutron (should
the two disagree).
Example setup::
+-------------------------------------------------------------+
| Logical_Switch |
| ts1 |
+----------+----------------------------------+---------------+
| |
+-------+-----------+ +-------+-----------+
|Logical_Switch_Port| |Logical_Switch_Port|
|lsp_ts1_lr1 | |lsp_ts1_lr2 |
+-------+-----------+ +-------+-----------+
| |
+-------+------------+ +-------+------------+
|Logical_Router_Port | |Logical_Router_Port |
|lrp_lr1_ts1 | |lrp_lr2_ts1 |
|172.24.0.10/24 | |172.24.0.20/24 |
+-------+------------+ +-------+------------+
| |
+-------+------------+ +-------+------------+
|Logical_Router | |Logical_Router |
|lr1 | |lr2 |
+-------+------------+ +-------+------------+
| |
+-------+------------+ +-------+------------+
|Logical_Router_Port | |Logical_Router_Port |
|lrp_lr1_ls1 | |lrp_lr2_ls2 |
|192.168.0.1/24 | |192.168.1.1/24 |
+-------+------------+ +-------+------------+
| |
+-------+------------+ +-------+------------+
|Logical_Switch_Port | |Logical_Switch_Port |
|lsp_ls1_lr1 | |lsp_ls2_lr2 |
+-------+------------+ +-------+------------+
| |
+-------+------------+ +-------+------------+
|Logical_Switch | |Logical_Switch |
|ls1 | |ls2 |
+-------+------------+ +-------+------------+
| |
+-------+------------+ +-------+------------+
|Logical_Switch_Port | |Logical_Switch_Port |
|lsp_ls1_vm1 | |lsp_ls2_vm2 |
+--------------------+ +--------------------+
The example above is a logical representation of the elements involved in the
interconnection process. On each side we have an OVN cluster/AZ with its local
managed resources: LSP for VMs, LS, LSP connecting the VM to the router and
the LR. What does OVN interconnect add to a standard topology to make this
works? A connection between the Tenant logical router and a Transit Switch.
The global database of the OVN IC dynamically replicates the TS between all
members of the interconnect domain (clusters/AZs), and what needs to be done
is basically to add this dynamically created TS in the OVN Northbound database
with the Tenant logical router.
Ownership of resources
======================
With the usage of OVN Interconnect Neutron is no longer the only owner of
resources in each OVN deployment.
We therefor need to first define which component owns which kind of resources.
The resources will be listed below for the left OVN deployment from the
example above.
Resources owned by Neutron
--------------------------
* lr1
* lrp_lr1_ls1
* lsp_ls1_lr1
* ls1
* lsp_ls1_vm1
All of these represent the:
* Tenant Network (ls1)
* Port of VMs or other things (lsp_ls1_vm1)
* Router of the Tenant (lr1, lrp_lr1_ls1, lsp_ls1_lr1)
Resources owned by ovn-ic
-------------------------
* ts1
* lsp_ts1_lr2 (created in the left OVN deployment)
* potentially Logical_Router_Static_Routes attached to lr1
(if route learning is enabled)
The resources owned by ovn-ic are dynamically created and removed by the
ovn-ic daemon when the process syncronizes the Northbound database between
interconnect domain elements.
resources owned by the operator
-------------------------------
* lsp_ts1_lr1
* lrp_lr1_ts1
The resources to connect the transit switch to the router of the user need to
be created by the operator (manually or with some kind of automation).
Managing these resources in Neutron is out of scope of this spec.
Problem Description
===================
The above setup can already be created by an operator.
However the `neutron-ovn-db-sync` tool will remove the resources owned by
ovn-ic and the operator, as Neutron does not know about them.
For resources created by ovn-ic this can be fixed by checking specific fields
on these resources:
* Logical_Switch we ignore all resources with the field
`other_config:interconn-ts` with any value
* Logical_Router_Static_Route we ignore all resources with the field
`external_ids:ic-learned-route` with any value
* Logical_Switch_Port we ignore all resources where the field `type`
is set to `remote`
These fields are automatically set by ovn-ic, so we can rely on them being
there.
For the resources owned by the operator such predefined options do not exist.
We therefor define a new key `external_ids:neutron-ovn-db-sync-ignore`.
We ignore all resources that have this key set with any value.
For resources created by operator with the new key in external_ids this can be
fixed by checking specific fields on these resources:
* Logical_Switch_Port we ignore all resources with the field
`external_ids:neutron-ovn-db-sync-ignore` with any value
* Logical_Router_Port we ignore all resources with the field
`external_ids:neutron-ovn-db-sync-ignore` with any value
Proposed Change
===============
To solve the problem described above, the proposal is to introduce a new filter
rule for the ovn-ic and operator resources created in the OVN Northbound
database.
This implementation is being named to as coexistence support for OVN
interconnect because it is out of scope any type of ovn-ic integration as part
of Neutron. The proposal of this implementation is the creation of filters in
the checking of the resources created by the ovn-ic and operator, and it is the
basis for any future implementation that intends to integrate the OVN
interconnect to Neutron.
There are pros and cons to completelly integrating OVN interconnect into
Neutron. From a technical perspective, the operator can use the coexistence
support proposed in this RFE to manage their interconnect resources in a
large-scale and multi-tenant deployment.
To implement the coexistence support the `neutron-ovn-db-sync` tool needs to
know about the ovn-ic and the operator interconnection resources. The main idea
of this proposal is described bellow.
For resources created by ovn-ic the proposed solution implement the support by
checking specific fields on these resources:
* Logical_Switch
`other_config:interconn-ts` with any value
OVN interconnection dynamically create logical switches in the OVN NB database
with the `interconn-ts` key in other_config register.
.. code::
_uuid : d312baed-5201-493f-9072-cb842ad28e2d
acls : []
copp : []
dns_records : []
external_ids : {}
forwarding_groups : []
load_balancer : []
load_balancer_group : []
name : admin-tenant1
other_config : {interconn-ts=admin-tenant1,
mcast_flood_unregistered="false", mcast_snoop="false",
requested-tnl-key="16711685"}
ports : [16c5229e-242a-427c-9d0e-3a20a284aef2,
c8cd4a0c-6f39-4497-916b-ce5f387d5a98,
d6f00ea0-d457-4f82-8438-c0d67f4eeff5,
d8f88fde-b030-4e5b-8c58-e72116194d51]
qos_rules : []
* Logical_Switch_Port
`type` field is set to `remote`
OVN interconnection dynamically create logical switch ports in the OVN NB
Logical_Switch table with the field `type` set to `remote`.
.. code::
switch d312baed-5201-493f-9072-cb842ad28e2d (admin-tenant1)
port admin-rt1-tenant1
type: router
router-port: rt1-admin-tenant1
port admin-rt2-tenant1
type: remote
addresses: ["aa:aa:aa:aa:ab:02 169.254.100.12/24 fd00::2/64"]
port admin-rt3-tenant1
type: remote
addresses: ["aa:aa:aa:aa:ab:03 169.254.100.13/24 fd00::3/64"]
port admin-k8s-tenant1
type: remote
addresses: ["aa:aa:aa:aa:cc:11 169.254.100.21/24 fd00::a/64"]
The Neutron method `sync_networks_ports_and_dhcp_opts` is the part of the
db_sync tool that perform the sync of networks and ports between Neutron and
OVN NB database. It is important to check for this case the Logical_Switch with
`interconn-ts` key in other_config register and the Logical_Switch_Port with
the field `type` set to `remote`, and keep these resources without removing
them.
* Logical_Router_Static_Route
`external_ids:ic-learned-route` with any value
OVN interconnection dynamically create Static Routes in the OVN NB
Logical_Router entries referenced to the table Logical_Router_Static_Route
with the `ic-learned-route` key in external_ids register.
Neutron managed Logical_Router example:
.. code::
_uuid : 078fd69b-f4c7-4469-a900-918d0a229bd1
copp : []
enabled : true
external_ids : {"neutron:availability_zone_hints"="",
"neutron:gw_port_id"=
"3f56ab75-549a-40e7-8383-e8688bc7478f",
"neutron:revision_number"="11",
"neutron:router_name"=rt1}
load_balancer : []
load_balancer_group : []
name : neutron-477cf920-21e3-46e5-8c8f-7b8caef7f549
nat : [af87c0b1-1b59-4583-8d2e-867ddf2fbb8d,
f8fa54eb-b4eb-4165-919a-d296e391247c,
ffa2ad54-7a81-4cbe-a8a6-5f798734f773]
options : {always_learn_from_arp_request="false",
dynamic_neigh_routers="true"}
policies : []
ports : [1d17c9a5-26bc-4960-b75f-07be40197955,
3f6eef14-53e9-4191-98a6-6102e1ec8f63,
6a8bbf7b-4bf6-46a8-b0be-631154b87446,
d26d298f-a0e4-497b-afad-a41f6aeeaf5e,
f744a1c3-4a09-490b-b8a8-49e4bc8f0da9]
static_routes : [097e7fed-81d3-4000-bbf1-c0c60810eb50,
0c7801a0-957c-4467-a882-827101bb8e82,
2a3c92c8-d367-435c-8449-b5d558b3782d,
3ff858bb-785f-452b-8a65-e5d158257a48,
45018d5d-d603-469f-b8c6-830ae65dd559,
86ca33a3-869d-4eb3-9777-869a43863883,
ace748d0-608e-49d9-878d-0312be7027f5,
bb5e806a-2827-4355-a727-1ca3d27db358,
c8f52e38-db5b-41cf-adf6-60fc3d41933b,
eb340708-abd2-4840-a07a-28320099882e]
ovn-ic learned static routes example (IPv4 and IPv6):
.. code::
_uuid : 45018d5d-d603-469f-b8c6-830ae65dd559
bfd : []
external_ids : {ic-learned-route=
"7d9b877c-b975-4bf0-8255-8d30b6ea6852"}
ip_prefix : "2801:80:3ea0:823::/64"
nexthop : "fd00::3"
options : {origin=connected}
output_port : []
policy : []
route_table : ""
_uuid : 097e7fed-81d3-4000-bbf1-c0c60810eb50
bfd : []
external_ids : {ic-learned-route=
"8f795f14-1dd4-4a43-9bcc-ac1f1ae6194f"}
ip_prefix : "10.0.2.1/24"
nexthop : "169.254.100.13"
options : {origin=connected}
output_port : []
policy : []
route_table : ""
The Neutron method `sync_routers_and_rports` is the part of the db_sync tool
that perform the sync of Logical_Router between Neutron and OVN NB database.
This method gets the list of the static routes related to a Logical_Router. It
is important to check for this case the routes with `ic-learned-route` in the
external_ids register and keep these resources without removing them.
For resources created by operator the proposed solution implement the support
by checking specific fields on these resources:
* Logical_Switch_Port
`external_ids:neutron-ovn-db-sync-ignore` with any value
OVN interconnection dynamically create the Logical_Switch (TS) in the OVN NB.
The operator needs to create a Logical_Switch_Port to link TS with the tenant
Logical_Router.
The Logical_Switch_Port created by operator needs to set the new key
`neutron-ovn-db-sync-ignore` in the external_ids register with any value. In
the example below, the value was filled with `true` but it could have any
value because the only relevant information is the key contained in the
external_ids register:
.. code::
ovn-nbctl set Logical_Switch_Port <ID> external_ids:
"neutron-ovn-db-sync-ignore"="true"
Operator managed Logical_Switch_Port example (with db sync ignore key):
.. code::
_uuid : 16c5229e-242a-427c-9d0e-3a20a284aef2
addresses : [router]
dhcpv4_options : []
dhcpv6_options : []
dynamic_addresses : []
enabled : []
external_ids : {neutron-ovn-db-sync-ignore="true"}
ha_chassis_group : []
mirror_rules : []
name : admin-rt1-tenant1
options : {requested-tnl-key="9", router-port=rt1-admin-tenant1}
parent_name : []
port_security : []
tag : []
tag_request : []
type : router
up : true
* Logical_Router_Port
`external_ids:neutron-ovn-db-sync-ignore` with any value
The Logical_Router_Port created by operator needs to set the new key
`neutron-ovn-db-sync-ignore` in the external_ids register with any value. In
the example below, the value was filled with `true` but it could have any
value because the only relevant information is the key contained in the
external_ids register:
.. code::
ovn-nbctl set Logical_Router_Port <ID> external_ids:
"neutron-ovn-db-sync-ignore"="true"
Operator managed Logical_Router_Port example (with db sync ignore key):
.. code::
_uuid : 6a8bbf7b-4bf6-46a8-b0be-631154b87446
enabled : []
external_ids : {neutron-ovn-db-sync-ignore="true"}
gateway_chassis : [b4487769-273e-4ba9-abd5-4743ff987f74]
ha_chassis_group : []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "aa:aa:aa:aa:ab:01"
name : rt1-admin-tenant1
networks : ["169.254.100.11/24", "fd00::1/64"]
options : {}
peer : []
For the Logical_Switch_Port and Logical_Router_Port created by operator, the
db_sync tool needs to ignore these resources when syncronizing between Neutron
and OVN NB database. It is important to check for this case the LSP and LRP
with `neutron-ovn-db-sync-ignore` in the external_ids register and keep these
resources without removing them.
DB Impact
---------
None
Rest API Changes
----------------
None
OVN driver changes
------------------
Update `neutron-ovn-db-sync` as described above.
Out of Scope
============
* Integrating ovn-interconnect in any way;
* Managing the Logical_Switch_Port and Logical_Router_Port between the Neutron
created Logical_Router and the ovn-ic created Logical_Switch;
Implementation
==============
Assignee(s)
-----------
* Primary assignees:
Felix Huettner <felix.huettner@mail.schwarz>
Roberto Bartzen Acosta <rbartzen@gmail.com>
Work Items
----------
* Add exclusion to `neutron-ovn-db-sync`
* Implement relevant unit and functional tests using the existing facilities
in Neutron.
Testing
=======
* Unit/functional tests.
References
==========
.. [1] https://www.ovn.org/support/dist-docs/ovn-architecture.7.html