openstack/ovn-bgp-agent
Code Issues Proposed changes

Add new driver ovn_stretched_l2_bgp_driver

Browse Source 
This driver allows to announce tenant networks with
an address scope via the virtual router IP.

For this to work, all routers in the respective L2
network must be reachable to each other and the
ovn-bgp-agent/frr needs an IP in this network to
talk to its BGP peer.

The following changes have been made:
- To filter which networks are announced via which
  agent/BGP session, we added a filter on the
  OpenStack address scope
- Networks are announced instead of VM IPs
- Add SubnetRouterUpdateEvent to handle updates of
  lrp ports

Depends-on: https://review.opendev.org/c/openstack/neutron/+/861719

Change-Id: I6e48c7e056ba2101ad670ab54c96e072459c5e65
This commit is contained in:
Luca Czesla 2022-10-18 14:22:32 +00:00
parent c833607821
commit d59c61caa5
14 changed files with 2370 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

304
doc/source/contributor/bgp_mode_stretched_l2_design.rst Normal file
View File

@ -0,0 +1,304 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
Convention for heading levels in Neutron devref:
======= Heading 0 (reserved for the title in a document)
------- Heading 1
~~~~~~~ Heading 2
+++++++ Heading 3
''''''' Heading 4
(Avoid deeper levels because they do not render well.)
====================================================
OVN BGP Agent: Design of the stretched L2 BGP Driver
====================================================
Purpose
-------
The main reason for adding the L2 BGP driver is to announce networks via BGP
that are not masqueraded (SNAT disabled) and can communicate directly via
routing. The driver requires that all routers to be announced are in a L2
provider network and that the BGP Neighbor and Speaker are also part of this
network. The whole tenant networks are announced via the external gateway IP
of the router, instead of as /32 (or /128 for IPv6). This means that the
tenant networks can be routed directly via the router IP and failover can run
completely via BFD in OVN. No additional BGP announcements are needed incase
of failover of routers, but only ARP/GARP in the respective L2 network.
The resulting routes are the same on all instances of the ovn-bgp-agent and
are not bound to the machine the agent is running on. For redundancy reasons
it is recommended to run multiple instances.
Overview
--------
The OVN BGP Agent is a Python-based daemon that can run on any node. However,
it is recommended to run the L2 BGP driver on the gateway node since all
requirements are already met there, e.g. connectivity to the L2 provider
network. The driver connects to the OVN SouthBound database (OVN SB DB) for
all information and responds to events via it. It uses a VRF to create the
routes locally and FRR to announce them to the BGP peer. The VRF is completely
isolated and is not used for anything else than announcing routes via FRR.
The tenant routers/networks cannot be reached from the VRF either.
.. note::
Note it is only intended for the N/S traffic, the E/W traffic will work
exactly the same as before, i.e., VMs are connected through geneve
tunnels.
The agent provides a multi-driver implementation that allows you to configure
it for specific infrastructure running on top of OVN, for instance OpenStack
or Kubernetes/OpenShift.
This simple design allows the agent to implement different drivers, depending
on what OVN SB DB events are being watched (watchers examples at
``ovn_bgp_agent/drivers/openstack/watchers/``), and what actions are
triggered in reaction to them (drivers examples at
``ovn_bgp_agent/drivers/openstack/XXXX_driver.py``, implementing the
``ovn_bgp_agent/drivers/driver_api.py``).
A common driver API is defined exposing the next methods:
- ``expose_ip`` and ``withdraw_ip``: used to expose/withdraw IPs/Networks for
OVN ports.
- ``expose_subnet``, ``update_subnet`` and ``withdraw_subnet``: used to
expose/withdraw subnets through the external router gateway ip.
OVN SB DB Events
~~~~~~~~~~~~~~~~
The watcher associated to this BGP driver detect the relevant events on the
OVN SB DB to call the driver functions to configure BGP and linux kernel
networking accordingly.
The BGP watcher detects OVN Southbound Database events at the ``Port_Binding``
and ``Load_Balancer`` tables. It creates new event classes named
``PortBindingChassisEvent`` and ``OVNLBMemberEvent``, that all the events
watched for BGP use as the base (inherit from).
The driver react specifically to the following events:
- ``PortBindingChassisCreatedEvent``: Detects when a port of type
``""`` (empty double-qoutes), ``virtual``, or ``chassisredirect`` gets
attached to the OVN chassis where the agent is running. This is the case for
VM or amphora LB ports on the provider networks, VM or amphora LB ports on
tenant networks with a FIP associated, and neutron gateway router ports
(CR-LRPs). It calls ``expose_ip`` driver method to perform the needed
actions to expose it.
- ``PortBindingChassisDeletedEvent``: Detects when a port of type
``""`` (empty double-quotes), ``virtual``, or ``chassisredirect`` gets
detached from the OVN chassis where the agent is running. This is the case
for VM or amphora LB ports on the provider networks, VM or amphora LB ports
on tenant networks with a FIP associated, and neutron gateway router ports
(CR-LRPs). It calls ``withdraw_ip`` driver method to perform the needed
actions to withdraw the exposed BGP route.
- ``SubnetRouterAttachedEvent``: Detects when a patch port gets created.
This means a subnet is attached to a router. If this port is associated to
a cr-lrp port, the subnet will get announced.
- ``SubnetRouterDetachedEvent``: Same as previous one, but for the deletion
of the port. It calls ``withdraw_subnet``.
- ``SubnetRouterUpdateEvent``: Detects when a subnet/IP is added to an
existing patch port. This can happen when multiple subnets are generated
from an address pool and added to the same router.
It calls ``update_subnet``.
Driver Logic
~~~~~~~~~~~~
The stretched L2 BGP driver is responsible for announcing all tenant networks
that match the corresponding address scope (if used for filtering subnets).
If the config option ``address_scopes`` is not set, all tenant networks will
be announced via the corresponding provider network router IP.
BGP Advertisement
+++++++++++++++++
The OVN BGP Agent is in charge of triggering FRR (IP routing protocol
suite for Linux which includes protocol daemons for BGP, OSPF, RIP,
among others) to advertise/withdraw directly connected routes via BGP.
To do that, when the agent starts, it ensures that:
- FRR local instance is reconfigured to leak routes for a new VRF. To do that
it uses ``vtysh shell``. It connects to the existing FRR socket (
``--vty_socket`` option) and executes the next commands, passing them through
a file (``-c FILE_NAME`` option):
.. code-block:: ini
LEAK_VRF_TEMPLATE = '''
router bgp {{ bgp_as }}
address-family ipv4 unicast
import vrf {{ vrf_name }}
exit-address-family
address-family ipv6 unicast
import vrf {{ vrf_name }}
exit-address-family
router bgp {{ bgp_as }} vrf {{ vrf_name }}
bgp router-id {{ bgp_router_id }}
address-family ipv4 unicast
redistribute kernel
exit-address-family
address-family ipv6 unicast
redistribute kernel
exit-address-family
'''
- There is a VRF created (the one leaked in the previous step) by default
with name ``bgp_vrf``.
- There is a dummy interface type (by default named ``bgp-nic``), associated to
the previously created VRF device.
Then, to expose the tenant networks as they are created (or upon
initialization or re-sync), since the FRR configuration has the
``redistribute kernel`` option enabled, the only action needed to
expose/withdraw the tenant networks is to add/remove the routes in
the ``bgp_vrf_table_id`` table. Then it relies on Zebra to do the BGP
advertisement, as Zebra detects the addition/deletion of the routes in the
table and advertises/withdraw the route. In order to add these routes we have
to make the Linux kernel believe that it can reach the respective router IPs.
For this we use link-local routes pointing to the interface of the VRF. If we
use the provider network ``111.111.111.0/24``, a router with the IP
``111.111.111.17/24`` on the gateway port and the tenant subnet ``192.168.0.0/24``,
the route would be added like this (same logic applies to IPv6):
.. code-block:: ini
$ ip route add 111.111.111.0/24 dev bgp-nic table 10
$ ip route add 192.168.0.0/24 via 111.111.111.17 table 10
.. note::
The link-local route for the provider network is also announced and is
only removed when no router to be announced has a gateway port on the
network. Since all BGP peers should also be on this network, the BGP
neighbor will prefer its connected route over the announced link-local
route.
On the BGP neighbor side, the route should look like this:
.. code-block:: ini
$ ip route show
192.168.0.0/24 via 111.111.111.17
Driver API
++++++++++
The BGP driver needs to implement the ``driver_api.py`` interface with the
following functions:
- ``expose_ip``: Creates the routes for all tenant networks and announces
them via FRR. If no subnets are connected to this port, nothing is
announced.
- ``withdraw_ip``: Removes all routes for the tenant networks and withdraws
them from FRR.
- ``expose_subnet``: Announces the tenant network via the router IP if this
router has an external gateway port.
- ``withdraw_subnet``: Withdraws the tenant network if this
router has an external gateway port.
- ``update_subnet``: Does the same as ``expose_subnet`` / ``withdraw_subnet``
and is called when a subnet is added or removed from the port.
Agent deployment
~~~~~~~~~~~~~~~~
The agent can be deployed anywhere as long as it is in the respective L2
network that is to be announced. In addition, OVS agent must be installed on
the machine (from which it reads SB DB address) and it must be possible to
connect to the Southbound Database. The L2 network can be filtered via the
address scope, so it is not necessary that the agent has access to all L2
provider networks, but only the one in which it is to peer. Unlike the
``ovn_bgp_driver``, it announces all routes regardless of which chassis they
are on.
As an example of how to start the OVN BGP Agent on the nodes, see the commands
below:
.. code-block:: ini
$ python setup.py install
$ cat bgp-agent.conf
# sample configuration that can be adapted based on needs
[DEFAULT]
debug=True
reconcile_interval=120
driver=ovn_stretched_l2_bgp_driver
address_scopes=2237917c7b12489a84de4ef384a2bcae
$ sudo bgp-agent --config-dir bgp-agent.conf
....
Note that the OVN BGP Agent operates under the next assumptions:
- A dynamic routing solution, in this case FRR, is deployed and
advertises/withdraws routes added/deleted to/from the vrf routing table.
A sample config for FRR is:
.. code-block:: ini
frr version 7.0
frr defaults traditional
hostname cmp-1-0
log file /var/log/frr/frr.log debugging
log timestamp precision 3
service integrated-vtysh-config
line vty
debug bgp neighbor-events
debug bgp updates
router bgp 64999
bgp router-id 172.30.1.1
neighbor pg peer-group
neighbor 172.30.1.2 remote-as 64998
address-family ipv6 unicast
redistribute kernel
neighbor pg activate
neighbor pg route-map IMPORT in
neighbor pg route-map EXPORT out
exit-address-family
address-family ipv4 unicast
redistribute kernel
neighbor pg activate
neighbor pg route-map IMPORT in
neighbor pg route-map EXPORT out
exit-address-family
route-map EXPORT deny 100
route-map EXPORT permit 1
match interface bgp-nic
route-map IMPORT deny 1
line vty
Limitations
-----------
- TBD

1
doc/source/contributor/index.rst
View File

@ -7,4 +7,5 @@
bgp_mode_design
evpn_mode_design
bgp_mode_stretched_l2_design

9
ovn_bgp_agent/config.py
View File

@ -81,6 +81,10 @@ agent_opts = [
default=4789,
help='The UDP port used for EVPN VXLAN communication. By '
'default 4789 is being used.'),
cfg.BoolOpt('clear_vrf_routes_on_startup',
help='If enabled, all routes are removed from the VRF table'
'(specified by bgp_vrf_table_id option) at startup.',
default=False),
cfg.StrOpt('bgp_nic',
default='bgp-nic',
help='The name of the interface used within the VRF '
@ -94,6 +98,11 @@ agent_opts = [
help='The Routing Table ID that the VRF (bgp_vrf option) '
'should use. If it does not exist, this table will be '
'created.'),
cfg.ListOpt('address_scopes',
default=None,
help='Allows to filter on the address scope. Only networks'
' with the same address scope on the provider and'
' internal interface are announced.'),
]
root_helper_opts = [

3
ovn_bgp_agent/constants.py
View File

@ -50,3 +50,6 @@ OVS_PATCH_PROVNET_PORT_PREFIX = 'patch-provnet-'
LINK_UP = "up"
LINK_DOWN = "down"
SUBNET_POOL_ADDR_SCOPE4 = "neutron:subnet_pool_addr_scope4"
SUBNET_POOL_ADDR_SCOPE6 = "neutron:subnet_pool_addr_scope6"

550
ovn_bgp_agent/drivers/openstack/ovn_stretched_l2_bgp_driver.py Normal file
View File

@ -0,0 +1,550 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import collections
import dataclasses
import ipaddress
import threading
from oslo_concurrency import lockutils
from oslo_config import cfg
from oslo_log import log as logging
from ovn_bgp_agent import constants
from ovn_bgp_agent.drivers import driver_api
from ovn_bgp_agent.drivers.openstack.utils import frr
from ovn_bgp_agent.drivers.openstack.utils import ovn
from ovn_bgp_agent.drivers.openstack.utils import ovs
from ovn_bgp_agent.drivers.openstack.watchers import bgp_watcher as watcher
from ovn_bgp_agent.utils import linux_net
CONF = cfg.CONF
LOG = logging.getLogger(__name__)
OVN_TABLES = ["Port_Binding", "Chassis", "Datapath_Binding"]
@dataclasses.dataclass(frozen=True, eq=True)
class HashedRoute:
network: str
prefix_len: int
dst: str
class OVNBGPStretchedL2Driver(driver_api.AgentDriverBase):
def __init__(self):
self.ovn_local_cr_lrps = {}
self.vrf_routes = set()
self.ovn_routing_tables_routes = collections.defaultdict()
self.allowed_address_scopes = set(CONF.address_scopes or [])
self.propagated_lrp_ports = {}
self._sb_idl = None
self._post_fork_event = threading.Event()
@property
def sb_idl(self):
if not self._sb_idl:
self._post_fork_event.wait()
return self._sb_idl
@sb_idl.setter
def sb_idl(self, val):
self._sb_idl = val
def start(self):
self.ovs_idl = ovs.OvsIdl()
self.ovs_idl.start(CONF.ovsdb_connection)
# Ensure FRR is configured to leak the routes
# NOTE: If we want to recheck this every X time, we should move it
# inside the sync function instead
frr.vrf_leak(
CONF.bgp_vrf,
CONF.bgp_AS,
CONF.bgp_router_id,
template=frr.LEAK_VRF_KERNEL_TEMPLATE,
)
LOG.debug("Setting up VRF %s", CONF.bgp_vrf)
linux_net.ensure_vrf(CONF.bgp_vrf, CONF.bgp_vrf_table_id)
# Create OVN dummy device
linux_net.ensure_ovn_device(CONF.bgp_nic, CONF.bgp_vrf)
# Clear vrf routing table
if CONF.clear_vrf_routes_on_startup:
linux_net.delete_routes_from_table(CONF.bgp_vrf_table_id)
self.chassis = self.ovs_idl.get_own_chassis_name()
self.ovn_remote = self.ovs_idl.get_ovn_remote()
LOG.debug("Loaded chassis %s.", self.chassis)
if self.allowed_address_scopes:
LOG.debug("Configured allowed address scopes: %s",
", ".join(self.allowed_address_scopes))
events = ()
for event in self._get_events():
event_class = getattr(watcher, event)
events += (event_class(self),)
self._post_fork_event.clear()
# TODO(lucasagomes): The OVN package in the ubuntu LTS is old
# and does not support Chassis_Private. Once the package is updated
# we can remove this fallback mode.
try:
self.sb_idl = ovn.OvnSbIdl(
self.ovn_remote,
chassis=self.chassis,
tables=OVN_TABLES + ["Chassis_Private"],
events=events,
).start()
except AssertionError:
self.sb_idl = ovn.OvnSbIdl(
self.ovn_remote,
chassis=self.chassis,
tables=OVN_TABLES,
events=events,
).start()
# Now IDL connections can be safely used
self._post_fork_event.set()
def _get_events(self):
return set(
[
"SubnetRouterAttachedEvent",
"SubnetRouterUpdateEvent",
"SubnetRouterDetachedEvent",
"PortBindingChassisCreatedEvent",
"PortBindingChassisDeletedEvent",
]
)
@lockutils.synchronized("bgp")
def sync(self):
self.ovn_local_cr_lrps = {}
self.ovn_routing_tables_routes = collections.defaultdict()
self.vrf_routes = set()
self.propagated_lrp_ports = {}
LOG.debug("Syncing current routes.")
# Get all current exposed routes
vrf_routes = linux_net.get_routes_on_tables([CONF.bgp_vrf_table_id])
for cr_lrp_port in self.sb_idl.get_cr_lrp_ports():
if not cr_lrp_port.mac or len(cr_lrp_port.mac[0].split(" ")) <= 1:
continue
self._expose_cr_lrp(cr_lrp_port.mac[0].split(" ")[1:], cr_lrp_port)
# remove all left over routes
delete_routes = []
for route in vrf_routes:
r = HashedRoute(
network=route.dst,
prefix_len=route.dst_len,
dst=route.gateway if route.gateway else None)
if r not in self.vrf_routes:
delete_routes.append(route)
linux_net.delete_ip_routes(delete_routes)
def _add_route(self, network, prefix_len, dst=None):
LOG.debug("Adding BGP route for Network %s/%d via %s",
network, prefix_len, dst)
linux_net.add_ip_route(
self.ovn_routing_tables_routes,
network,
CONF.bgp_vrf_table_id,
CONF.bgp_nic,
vlan=None,
mask=prefix_len,
via=dst)
r = HashedRoute(
network=network,
prefix_len=prefix_len,
dst=dst)
self.vrf_routes.add(r)
LOG.debug("Added BGP route for Network %s/%d via %s",
network, prefix_len, dst)
def _del_route(self, network, prefix_len, dst=None):
LOG.debug("Deleting BGP route for Network %s/%d via %s",
network, prefix_len, dst)
linux_net.del_ip_route(
self.ovn_routing_tables_routes,
network,
CONF.bgp_vrf_table_id,
CONF.bgp_nic,
vlan=None,
mask=prefix_len,
via=dst)
r = HashedRoute(
network=network,
prefix_len=prefix_len,
dst=dst)
if r in self.vrf_routes:
self.vrf_routes.remove(r)
LOG.debug("Deleted BGP route for Network %s/%d via %s",
network, prefix_len, dst)
def _address_scope_allowed(self, scope1, scope2, ip_version):
if not self.allowed_address_scopes:
# No address scopes to filter on => announce everything
return True
if scope1[ip_version] != scope2[ip_version]:
# Not the same address scope => don't announce
return False
if scope1[ip_version] not in self.allowed_address_scopes:
# This address scope does not match => don't announce
return False
return True
def _get_addr_scopes(self, port):
return {
constants.IP_VERSION_4: port.external_ids.get(
constants.SUBNET_POOL_ADDR_SCOPE4
),
constants.IP_VERSION_6: port.external_ids.get(
constants.SUBNET_POOL_ADDR_SCOPE6
),
}
@lockutils.synchronized("bgp")
def expose_subnet(self, ip, row):
cr_lrp = self.sb_idl.is_router_gateway_on_any_chassis(row.datapath)
if not cr_lrp:
return
self._ensure_network_exposed(row, cr_lrp.logical_port)
@lockutils.synchronized("bgp")
def update_subnet(self, old, row):
cr_lrp = self.sb_idl.is_router_gateway_on_any_chassis(row.datapath)
if not cr_lrp or not cr_lrp.mac or len(cr_lrp.mac[0].split(" ")) <= 1:
return
current_ips = row.mac[0].split(" ")[1:]
previous_ips = (
old.mac[0].split(" ")[1:]
if old.mac or len(old.mac[0].split(" ")) > 1
else []
)
add_ips = list(
filter(lambda ip: ip not in previous_ips, current_ips))
delete_ips = list(
filter(lambda ip: ip not in current_ips, previous_ips))
self._update_network(row, cr_lrp.logical_port, add_ips, delete_ips)
@lockutils.synchronized("bgp")
def withdraw_subnet(self, ip, row):
port_info = self.propagated_lrp_ports.get(row.logical_port)
if not port_info:
return
self._withdraw_subnet(port_info, port_info["cr_lrp"])
gateway = self.ovn_local_cr_lrps.get(port_info["cr_lrp"])
if gateway and row.logical_port in gateway["lrp_ports"]:
gateway["lrp_ports"].remove(row.logical_port)
self.propagated_lrp_ports.pop(row.logical_port)
def _withdraw_subnet(self, port_info, cr_lrp):
gateway = self.ovn_local_cr_lrps.get(cr_lrp)
if not gateway:
# If we dont have it cached then its either not existing or
# or we got an event while starting up which then the sync
# function can fix.
return
gateway_ips = gateway["ips"]
subnets = [
ipaddress.ip_network(subnet)
for subnet in port_info["subnets"]]
for gateway_ip in gateway_ips:
for subnet in subnets:
if gateway_ip.version != subnet.version:
continue
self._del_route(
network=str(subnet.network_address),
prefix_len=subnet.prefixlen,
dst=str(gateway_ip.ip))
# Check if can delete the link-local route
exposed_routes = linux_net.get_exposed_routes_on_network(
[CONF.bgp_vrf_table_id],
gateway_ip.network)
if not exposed_routes:
self._del_route(
network=str(gateway_ip.network.network_address),
prefix_len=gateway_ip.network.prefixlen)
@lockutils.synchronized('bgp')
def withdraw_ip(self, ips, row, associated_port=None):
if not (row.type == constants.OVN_CHASSISREDIRECT_VIF_PORT_TYPE and
row.logical_port.startswith("cr-")):
return
self._withdraw_cr_lrp(ips, row)
def _withdraw_cr_lrp(self, ips, row):
if self.allowed_address_scopes:
# Validate address scopes
address_scopes = self.ovn_local_cr_lrps[row.logical_port][
"address_scopes"]
if not any([
scope in self.allowed_address_scopes
for scope in address_scopes.values()]):
return
# Check if there are networks attached to the router,
# and if so, remove them locally
lrp_ports = self.ovn_local_cr_lrps[row.logical_port]["lrp_ports"]
for lrp_logical_port in lrp_ports:
port_info = self.propagated_lrp_ports.get(lrp_logical_port)
if not port_info:
continue
# withdraw network
self._withdraw_subnet(port_info, row.logical_port)
self.propagated_lrp_ports.pop(lrp_logical_port, None)
self.ovn_local_cr_lrps.pop(row.logical_port, None)
@lockutils.synchronized("bgp")
def expose_ip(self, ips, row, associated_port=None):
if not (row.type == constants.OVN_CHASSISREDIRECT_VIF_PORT_TYPE and
row.logical_port.startswith("cr-")):
return
self._expose_cr_lrp(ips, row)
def _expose_cr_lrp(self, ips, row):
LOG.debug("Adding BGP route for CR-LRP Port %s", row.logical_port)
# Keeping information about the associated network for
# tenant network advertisement
self.ovn_local_cr_lrps[row.logical_port] = {
"ips": [ipaddress.ip_interface(ip) for ip in ips],
"address_scopes": {},
"lrp_ports": set(),
}
if self.allowed_address_scopes:
# Validate address scopes
patch_port = row.logical_port.split("cr-lrp-")[1]
port = self.sb_idl.get_port_by_name(patch_port)
if not port:
LOG.error("Patchport %s for CR-LRP %s missing, skipping.",
patch_port, row.logical_port)
return
address_scopes = self._get_addr_scopes(port)
self.ovn_local_cr_lrps[row.logical_port][
"address_scopes"] = address_scopes
if not any([
scope in self.allowed_address_scopes
for scope in address_scopes.values()]):
return
# Check if there are networks attached to the router,
# and if so, add the needed routes
lrp_ports = self.sb_idl.get_lrp_ports_for_router(row.datapath)
for lrp in lrp_ports:
if (
lrp.chassis or
not lrp.logical_port.startswith("lrp-") or
"chassis-redirect-port" in lrp.options.keys()
):
continue
# expose network
self._ensure_network_exposed(lrp, row.logical_port)
def _update_network(self, router_port, gateway_port, add_ips, delete_ips):
gateway = self.ovn_local_cr_lrps.get(gateway_port)
if not gateway:
# If we dont have it cached then its either not existing or
# or we got an event while starting up which then the sync
# function can fix.
return
gateway_ips = gateway["ips"]
if not router_port.mac or len(router_port.mac[0].split(" ")) <= 1:
return
# get all ips from the router port
ips_to_add = [ipaddress.ip_interface(ip) for ip in add_ips]
ips_to_delete = [ipaddress.ip_interface(ip) for ip in delete_ips]
for router_ip in ips_to_add + ips_to_delete:
if router_ip in gateway_ips:
return
address_scopes = None
if self.allowed_address_scopes:
patch_port = router_port.logical_port.split("lrp-")[1]
port = self.sb_idl.get_port_by_name(patch_port)
if not port:
LOG.error("Patchport %s for CR-LRP %s missing, skipping.",
patch_port, gateway_port)
return
address_scopes = self._get_addr_scopes(port)
# if we should filter on address scopes and this port has no
# address scopes set we do not need to go further
if not any(address_scopes.values()):
return
subnets = set()
for gateway_ip in gateway_ips:
for router_ip in ips_to_add:
if gateway_ip.version != router_ip.version:
continue
if not self._address_scope_allowed(
gateway["address_scopes"],
address_scopes,
router_ip.version):
continue
# Add link-local route
self._add_route(
network=str(gateway_ip.network.network_address),
prefix_len=gateway_ip.network.prefixlen)
# add route for the tenant network pointing to the
# gateway ip
self._add_route(
network=str(router_ip.network.network_address),
prefix_len=router_ip.network.prefixlen,
dst=str(gateway_ip.ip))
subnets.add(str(router_ip.network))
for router_ip in ips_to_delete:
if gateway_ip.version != router_ip.version:
continue
if not self._address_scope_allowed(
gateway["address_scopes"],
address_scopes,
router_ip.version):
continue
self._del_route(
network=str(router_ip.network.network_address),
prefix_len=router_ip.network.prefixlen,
dst=str(gateway_ip.ip))
# We only need to check this if we really deleted a route for
# a tenant network
if ips_to_delete:
# Check if can delete the link-local route
exposed_routes = linux_net.get_exposed_routes_on_network(
[CONF.bgp_vrf_table_id],
gateway_ip.network
)
if not exposed_routes:
self._del_route(
network=str(gateway_ip.network.network_address),
prefix_len=gateway_ip.network.prefixlen)
self.ovn_local_cr_lrps[gateway_port]["lrp_ports"].add(
router_port.logical_port)
self.propagated_lrp_ports[router_port.logical_port] = {
"cr_lrp": gateway_port,
"subnets": subnets
}
def _ensure_network_exposed(self, router_port, gateway_port):
gateway = self.ovn_local_cr_lrps.get(gateway_port)
if not gateway:
# If we dont have it cached then its either not existing or
# or we got an event while starting up which then the sync
# function can fix.
return
gateway_ips = gateway["ips"]
if not router_port.mac or len(router_port.mac[0].split(" ")) <= 1:
return
# get all ips from the router port
router_ips = [
ipaddress.ip_interface(ip)
for ip in router_port.mac[0].split(" ")[1:]]
for router_ip in router_ips:
if router_ip in gateway_ips:
return
address_scopes = None
if self.allowed_address_scopes:
patch_port = router_port.logical_port.split("lrp-")[1]
port = self.sb_idl.get_port_by_name(patch_port)
if not port:
LOG.error("Patchport %s for CR-LRP %s missing, skipping.",
patch_port, gateway_port)
return
address_scopes = self._get_addr_scopes(port)
# if we have address scopes configured and none of them matches
# for this port, we can skip further processing
if not any(address_scopes.values()):
return
subnets = set()
for gateway_ip in gateway_ips:
for router_ip in router_ips:
if gateway_ip.version != router_ip.version:
continue
if not self._address_scope_allowed(
gateway["address_scopes"],
address_scopes,
router_ip.version):
continue
# Add link-local route
self._add_route(
network=str(gateway_ip.network.network_address),
prefix_len=gateway_ip.network.prefixlen)
# add route for the tenant network pointing to the
# gateway ip
self._add_route(
network=str(router_ip.network.network_address),
prefix_len=router_ip.network.prefixlen,
dst=str(gateway_ip.ip))
subnets.add(str(router_ip.network))
if subnets:
self.ovn_local_cr_lrps[gateway_port]["lrp_ports"].add(
router_port.logical_port)
self.propagated_lrp_ports[router_port.logical_port] = {
"cr_lrp": gateway_port,
"subnets": subnets
}
@lockutils.synchronized("bgp")
def expose_remote_ip(self, ip_address):
raise NotImplementedError()
@lockutils.synchronized("bgp")
def withdraw_remote_ip(self, ip_address):
raise NotImplementedError()

26
ovn_bgp_agent/drivers/openstack/utils/frr.py
View File

@ -70,6 +70,28 @@ router bgp {{ bgp_as }} vrf {{ vrf_name }}
'''
LEAK_VRF_KERNEL_TEMPLATE = '''
router bgp {{ bgp_as }}
address-family ipv4 unicast
import vrf {{ vrf_name }}
exit-address-family
address-family ipv6 unicast
import vrf {{ vrf_name }}
exit-address-family
router bgp {{ bgp_as }} vrf {{ vrf_name }}
bgp router-id {{ bgp_router_id }}
address-family ipv4 unicast
redistribute kernel
exit-address-family
address-family ipv6 unicast
redistribute kernel
exit-address-family
'''
def _get_router_id():
output = ovn_bgp_agent.privileged.vtysh.run_vtysh_command(
@ -96,7 +118,7 @@ def _run_vtysh_config_with_tempfile(vrf_config):
f.close()
def vrf_leak(vrf, bgp_as, bgp_router_id=None):
def vrf_leak(vrf, bgp_as, bgp_router_id=None, template=LEAK_VRF_TEMPLATE):
LOG.info("Add VRF leak for VRF %s on router bgp %s", vrf, bgp_as)
if not bgp_router_id:
bgp_router_id = _get_router_id()
@ -104,7 +126,7 @@ def vrf_leak(vrf, bgp_as, bgp_router_id=None):
LOG.error("Unknown router-id, needed for route leaking")
return
vrf_template = Template(LEAK_VRF_TEMPLATE)
vrf_template = Template(template)
vrf_config = vrf_template.render(vrf_name=vrf, bgp_as=bgp_as,
bgp_router_id=bgp_router_id)
_run_vtysh_config_with_tempfile(vrf_config)

26
ovn_bgp_agent/drivers/openstack/utils/ovn.py
View File

@ -165,13 +165,18 @@ class OvsdbSbOvnIdl(sb_impl_idl.OvnSbApiIdlImpl, Backend):
rows = self.db_list_rows('Port_Binding').execute(check_error=True)
return [r for r in rows if r.chassis and r.chassis[0].name == chassis]
def get_cr_lrp_ports(self):
return self.db_find_rows(
"Port_Binding",
("type", "=", constants.OVN_CHASSISREDIRECT_VIF_PORT_TYPE),
).execute(check_error=True)
def get_cr_lrp_ports_on_chassis(self, chassis):
rows = self.db_find_rows(
'Port_Binding',
('type', '=', constants.OVN_CHASSISREDIRECT_VIF_PORT_TYPE)
).execute(check_error=True)
return [r.logical_port for r in rows
if r.chassis and r.chassis[0].name == chassis]
return [
r.logical_port
for r in self.get_cr_lrp_ports()
if r.chassis and r.chassis[0].name == chassis
]
def get_cr_lrp_nat_addresses_info(self, cr_lrp_port_name, chassis, sb_idl):
# NOTE: Assuming logical_port format is "cr-lrp-XXXX"
@ -222,6 +227,15 @@ class OvsdbSbOvnIdl(sb_impl_idl.OvnSbApiIdlImpl, Backend):
except IndexError:
pass
def is_router_gateway_on_any_chassis(self, datapath):
port_info = self.get_ports_on_datapath(
datapath, constants.OVN_CHASSISREDIRECT_VIF_PORT_TYPE)
try:
if port_info and port_info[0].chassis[0].name:
return port_info[0]
except IndexError:
pass
def get_lrp_port_for_datapath(self, datapath):
for row in self.get_ports_on_datapath(
datapath, constants.OVN_PATCH_VIF_PORT_TYPE):

34
ovn_bgp_agent/drivers/openstack/watchers/bgp_watcher.py
View File

@ -175,6 +175,40 @@ class SubnetRouterAttachedEvent(base_watcher.PortBindingChassisEvent):
self.agent.expose_subnet(ip_address, row)
class SubnetRouterUpdateEvent(base_watcher.PortBindingChassisEvent):
def __init__(self, bgp_agent):
events = (self.ROW_UPDATE,)
super(SubnetRouterUpdateEvent, self).__init__(
bgp_agent, events)
def match_fn(self, event, row, old):
# This will match if the mac field has changed between old and row.
# This can happen when you have multiple subnets in the same network,
# those will be added/removed to/from the same lrp-port in the mac
# field.
# Format:
# mac = [ff:ff:ff:ff:ff:ff subnet1/cidr subnet2/cidr [...]]
try:
# single and dual-stack format
if (not self._check_ip_associated(row.mac[0]) and
not self._check_ip_associated(old.mac[0])):
return False
return (
not row.chassis and
row.logical_port.startswith("lrp-") and
"chassis-redirect-port" not in row.options.keys() and
old.mac != row.mac
)
except (IndexError, AttributeError):
return False
def run(self, event, row, old):
if row.type != constants.OVN_PATCH_VIF_PORT_TYPE:
return
with _SYNC_STATE_LOCK.read_lock():
self.agent.update_subnet(old, row)
class SubnetRouterDetachedEvent(base_watcher.PortBindingChassisEvent):
def __init__(self, bgp_agent):
events = (self.ROW_DELETE,)

1243
ovn_bgp_agent/tests/unit/drivers/openstack/test_ovn_stretched_l2_bgp_driver.py Normal file
View File

File diff suppressed because it is too large Load Diff

22
ovn_bgp_agent/tests/unit/drivers/openstack/utils/test_ovn.py
View File

@ -291,6 +291,28 @@ class TestOvsdbSbOvnIdl(test_base.TestCase):
def test_is_router_gateway_on_chassis_not_on_chassis(self):
self._test_is_router_gateway_on_chassis(match=False)
def _test_is_router_gateway_on_any_chassis(self, match=True):
if match:
ch = fakes.create_object({'name': 'chassis-0'})
else:
ch = fakes.create_object({'name': ''})
port = '39c38ce6-f0ea-484e-a57c-aec0d4e961a5'
with mock.patch.object(self.sb_idl, 'get_ports_on_datapath') as m_dp:
row = fakes.create_object({'logical_port': port, 'chassis': [ch]})
m_dp.return_value = [row, ]
ret = self.sb_idl.is_router_gateway_on_any_chassis('fake-dp')
if match:
self.assertEqual(row, ret)
else:
self.assertIsNone(ret)
def test_is_router_gateway_on_any_chassis(self):
self._test_is_router_gateway_on_any_chassis()
def test_is_router_gateway_on_chassis_not_on_any_chassis(self):
self._test_is_router_gateway_on_any_chassis(match=False)
def _test_get_lrp_port_for_datapath(self, has_options=True):
peer = '75c793bd-d865-48f3-8f05-68ba4239d14e'
with mock.patch.object(self.sb_idl, 'get_ports_on_datapath') as m_dp:

81
ovn_bgp_agent/tests/unit/drivers/openstack/watchers/test_bgp_watcher.py
View File

@ -391,6 +391,87 @@ class TestSubnetRouterAttachedEvent(test_base.TestCase):
self.agent.expose_subnet.assert_not_called()
class TestSubnetRouterUpdateEvent(test_base.TestCase):
def setUp(self):
super(TestSubnetRouterUpdateEvent, self).setUp()
self.chassis = '935f91fa-b8f8-47b9-8b1b-3a7a90ef7c26'
self.agent = mock.Mock(chassis=self.chassis)
self.event = bgp_watcher.SubnetRouterUpdateEvent(self.agent)
def test_match_fn(self):
row = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'],
options={})
self.assertTrue(self.event.match_fn(mock.Mock(), row, mock.Mock()))
def test_match_fn_not_single_or_dual_stack(self):
row = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff'],
options={})
old = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff'],
options={})
self.assertFalse(self.event.match_fn(mock.Mock(), row, old))
def test_match_fn_not_lrp(self):
row = utils.create_row(chassis=[], logical_port='fake-lp',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'],
options={})
self.assertFalse(self.event.match_fn(mock.Mock(), row, mock.Mock()))
def test_match_fn_chassis_redirect(self):
row = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'],
options={'chassis-redirect-port': True})
self.assertFalse(self.event.match_fn(mock.Mock(), row, mock.Mock()))
def test_match_fn_chassis_set(self):
row = utils.create_row(chassis=[mock.Mock()], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'],
options={})
self.assertFalse(self.event.match_fn(mock.Mock(), row, mock.Mock()))
def test_match_fn_mac_not_changed(self):
row = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'],
options={})
old = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'],
options={})
self.assertFalse(self.event.match_fn(mock.Mock(), row, old))
def test_match_fn_mac_changed(self):
row = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'],
options={})
old = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16 10.10.1.17'],
options={})
self.assertTrue(self.event.match_fn(mock.Mock(), row, old))
def test_match_fn_index_error(self):
row = utils.create_row(chassis=[], logical_port='lrp-fake',
mac=[], options={})
self.assertFalse(self.event.match_fn(mock.Mock(), row, mock.Mock()))
def test_run(self):
row = utils.create_row(type=constants.OVN_PATCH_VIF_PORT_TYPE,
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'])
old = utils.create_row(type=constants.OVN_PATCH_VIF_PORT_TYPE,
mac=['aa:bb:cc:dd:ee:ff'])
self.event.run(mock.Mock(), row, old)
self.agent.update_subnet.assert_called_once_with(old, row)
def test_run_wrong_type(self):
row = utils.create_row(type='coxinha',
mac=['aa:bb:cc:dd:ee:ff 10.10.1.16'])
old = utils.create_row(type='coxinha',
mac=['aa:bb:cc:dd:ee:ff'])
self.event.run(mock.Mock(), row, old)
self.agent.update_subnet.assert_not_called()
class TestSubnetRouterDetachedEvent(test_base.TestCase):
def setUp(self):

63
ovn_bgp_agent/tests/unit/utils/test_linux_net.py
View File

@ -39,6 +39,9 @@ class TestLinuxNet(test_base.TestCase):
self.dev = 'ethfake'
self.mac = 'aa:bb:cc:dd:ee:ff'
self.bridge = 'br-fake'
self.table_id = 100
self.network = ipaddress.IPv4Network("10.10.1.0/24")
self.network_v6 = ipaddress.IPv6Network("2002:0:0:1234:0:0:0:0/64")
def test_get_ip_version_v4(self):
self.assertEqual(4, linux_net.get_ip_version('%s/32' % self.ip))
@ -238,6 +241,66 @@ class TestLinuxNet(test_base.TestCase):
self.assertEqual([self.ip, self.ipv6], ret)
def test_get_exposed_routes_on_network_v4(self):
route0 = mock.MagicMock(
dst=mock.Mock(),
table=self.table_id,
scope=1,
proto=11,
gateway=self.ip,
)
route1 = mock.MagicMock(
dst=mock.Mock(),
table=self.table_id,
scope=1,
proto=11,
gateway=self.ipv6,
)
route2 = mock.MagicMock(
dst=mock.Mock(),
table=self.table_id,
scope=1,
proto=11,
gateway=None,
)
self.fake_ndb.routes.dump.return_value = [route0, route1, route2]
ret = linux_net.get_exposed_routes_on_network(
[self.table_id], self.network
)
self.assertEqual([route0], ret)
def test_get_exposed_routes_on_network_v6(self):
route0 = mock.MagicMock(
dst=mock.Mock(),
table=self.table_id,
scope=1,
proto=11,
gateway=self.ip,
)
route1 = mock.MagicMock(
dst=mock.Mock(),
table=self.table_id,
scope=1,
proto=11,
gateway=self.ipv6,
)
route2 = mock.MagicMock(
dst=mock.Mock(),
table=self.table_id,
scope=1,
proto=11,
gateway=None,
)
self.fake_ndb.routes.dump.return_value = [route0, route1, route2]
ret = linux_net.get_exposed_routes_on_network(
[self.table_id], self.network_v6
)
self.assertEqual([route1], ret)
def test_get_ovn_ip_rules(self):
rule0 = mock.Mock(table=7, dst=10, dst_len=128, family='fake')
rule1 = mock.Mock(table=7, dst=11, dst_len=32, family='fake')

14
ovn_bgp_agent/utils/linux_net.py
View File

@ -282,6 +282,20 @@ def get_exposed_ips_on_network(nic, network):
return exposed_ips
def get_exposed_routes_on_network(table_ids, network):
with pyroute2.NDB() as ndb:
# NOTE: skip bgp routes (proto 186)
return [
r
for r in ndb.routes.dump()
if r.table in table_ids and
r.dst != "" and
r.gateway is not None and
r.proto != 186 and
ipaddress.ip_address(r.gateway) in network
]
def get_ovn_ip_rules(routing_table):
# get the rules pointing to ovn bridges
ovn_ip_rules = {}

3
setup.cfg
View File

@ -1,6 +1,6 @@
[metadata]
name = ovn-bgp-agent
summary = The OVN BGP Agent allows to expose VMs/Containers through BGP on OVN
summary = The OVN BGP Agent allows to expose VMs/Containers/Networks through BGP on OVN
description-file =
README.rst
author = OpenStack
@ -37,6 +37,7 @@ console_scripts =
ovn_bgp_agent.drivers =
ovn_bgp_driver = ovn_bgp_agent.drivers.openstack.ovn_bgp_driver:OVNBGPDriver
ovn_evpn_driver = ovn_bgp_agent.drivers.openstack.ovn_evpn_driver:OVNEVPNDriver
ovn_bgp_stretched_l2_driver = ovn_bgp_agent.drivers.openstack.ovn_stretched_l2_bgp_driver:OVNBGPStretchedL2Driver
oslo.config.opts =
ovnbgpagent = ovn_bgp_agent.config:list_opts