diff --git a/specs/newton/l2-border-gateway-api.rst b/specs/newton/l2-border-gateway-api.rst new file mode 100644 index 0000000..96b7b4c --- /dev/null +++ b/specs/newton/l2-border-gateway-api.rst @@ -0,0 +1,262 @@ + + +============== +L2 Gateway API +============== + +Until this version, the L2GW was meant to connect overlay Neutron network to +bare metal servers via hardware switches. This was done by connecting the +hardware switch to the local Compute Nodes using VXLAN and connecting the bare +metal servers to the hardware switches physical ports. To bind the proper +overlay network to the bare metal server the L2GW uses the overlay network's +segmentation-id as VXLAN tunnel id and VLANs on the hardware switch. + +It is our intention to use this project to also support the connection of the +local overlay networks with remote overlay networks. One possible scenario is to +connect multiple OpenStack clouds in a way that the overlay networks could be +connected and act as local networks that are connected via WAN link +(inter-cloud connection). Currently the L2GW support L2 connection only but this +can also be changed in the future to add L3 connection as well. + +To support the inter-cloud connection, the L2GW API is extended to provide a way +to stretch the network between the local and the remote clouds. + +Note: the proposed API is an extension of the existing API and can be executed +by admin users only, as the previous one. + +================ +New API Commands +================ + +1. Remote Gateway + +Remote Gateway is an entity that connects to a local Gateway using tunnel to +enable connection between local and remote networks. Different networks are +connected using different segmentation id. The segmentation id is implementation +specific. Different implementation can use different tunnel protocol with its +own tunnel key to be used as the segmentation id (VxLAN, GRE, Geneve, Etc.). +Single Remote Gateway can connect to multiple local Gateways and the local +networks that are attached to them. Also Local Gateway can connect to multiple +Remote Gateways using multiple Remote Gateway Connections, each connection to +a different Remote Gateway. + +1.1. Remote Gateway creation: + +usage: l2-remote-gateway-create + +where is a logical name for the remote gateway +and is the IP address of the remote gateway which will be the end +point of the tunnel. + + +1.2. Remote Gateway update: + +usage: l2-remote-gateway-update + +After creating a Remote Gateway, one can use this command to update the +IP address or/and the gateway's logical name. + +1.3. Remote Gateway deletion: + +usage: l2-remote-gateway-delete + +Use this command to delete Remote Gateway using its logical name or UUID. + +1.4. Remote Gateway list: + +usage: l2-remote-gateway-list + +This command lists all the Remote Gateways. A response will look like +the following: + ++--------------------------------------+------+----------+ +| id | name | ipaddr | ++--------------------------------------+------+----------+ +| 63821629-6cfd-44c8-8c4a-14c4b5fe28ea | rgw1 | 10.0.0.1 | ++--------------------------------------+------+----------+ + + +1.5. Remote Gateway show: + +usage: l2-remote-gateway-show + +Get information of specific Remote Gateway using its logical name or UUID. +Example of response: + ++--------+--------------------------------------+ +| Field | Value | ++--------+--------------------------------------+ +| id | 63821629-6cfd-44c8-8c4a-14c4b5fe28ea | +| ipaddr | 10.0.0.1 | +| name | rgw1 | ++--------+--------------------------------------+ + + +2. Remote Gateway Connection + +Remote Gateway Connection is a connection between local and remote networks. +Currently the connections are implemented using VXLAN between the local +and remote switches. + + +2.1 Remote Gateway Connection creation: + +Use this command to create a connection between local and remote network. + +usage: l2-remote-gateway-connection-create + + + + + is UUID or logical name of previously created local gateway. + + is UUID or logical name of tenant network that was +previously added to local gateway using l2-gateway-connection-create command. + + is UUID or logical name of the Remote Gateway that +the remote network is connected to. + +--seg-id is an optional parameter that will be used as tunnel key for the +connection between the local and remote Gateways. This parameter is optional and +in case it will not be used, the tunnel key that will be used is the overlay +network segmentation id. + +--flood is an optional string of True/False values that state if the local +switch should flood unknown MACs or broadcast MAC to the remote connection. + + +2.2 Remote Gateway Connection delete: + +usage: l2-remote-gateway-connection-delete + +use Remote Gateway UUID to delete connection to remote network. + +2.3 Remote Gateway Connection list: + + +usage: l2-remote-gateway-connection-list + +Lists all the connection to Remote Gateways. +example of response: + ++------------+-------------------+-----------------+-----------------+ +| id | l2_gateway_id | network_id | segmentation_id | ++------------+-------------------+-----------------+-----------------+ +| 31c204.... | c8da3e74-fde9-... | d90db94a-2b3... | 9765 | ++------------+-------------------+-----------------+-----------------+ + + +2.4 Remote Gateway Connection show: + +Show information of a single connection to a Remote Gateway using the +connection UUID. + +example: + +l2-remote-gateway-connection-show 31c20418-9cf2-4d47-b17e-d92906e3f248 + ++-----------------+--------------------------------------+ +| Field | Value | ++-----------------+--------------------------------------+ +| id | 31c20418-9cf2-4d47-b17e-d92906e3f248 | +| l2_gateway_id | c8da3e74-fde9-48e3-81f2-5fee756dd9de | +| network_id | d90db94a-2b3c-4415-971f-967e2f52248d | +| segmentation_id | 9765 | +| tenant_id | 84429618ed684296bc48eb120acf57bc | ++-----------------+--------------------------------------+ + + +3. Remote MAC + +The following command could be used by orchestration application to provide +information on remote hosts - their MAC address, IP address +(for ARP suppression) and link them to Remote Gateway Connection for the local +switch to know where to switch the packets to. + +3.1 Remote MAC creation: + +usage: l2-remote-mac-create + + is a tuple of remote host MAC address and optional IP address. +The format is as followed: + +[;] + +Where: + is the MAC address of the remote host in the form of 00:00:00:00:00:00 + IP address of the remote host (optional). + + the UUID of the Remote Gateway Connection that +will lead to the network where the remote host is located. + +REST message format: + +{ + "remote-mac" : { + "mac": "00:11:22:33:44:55', + "ip": "1.2.3.4" + } + +} + + +3.2 Bulk Remote MAC creation + +To send bulk remote MAC creation to the server, the following REST message +should be used: + +{ + "remote-macs": [ + { + "mac": "00:11:22:33:44:55', + "ip": "1.2.3.5" + }, + { + "mac": "00:11:22:33:44:66', + "ip": "1.2.3.6" + } + ] +} + + + +3.3 Remote MAC delete: + +usage: l2-remote-mac-delete + +L2_REMOTE_MAC is the UUID of the MAC address to be deleted. + + +3.4 Remote MAC list: + +usage: l2-remote-mac-list [--remote-connection UUID] + +Lists all the remote MAC addresses. +example of a response: + ++--------------+-------------------+--------------------------------------+ +| uuid | mac | ip_addr | rgw_connection | ++--------------+-------------------+--------------------------------------+ +| b59584eb-... | 00:11:22:33:44:55 | 192.168.10.23 | a60dc097-... | ++--------------+-------------------+--------------------------------------+ + +if the optional remote-connection is provided, only MACs that are configured +on this connection will be displayed. + +3.4 Remote MAC show: + +usage: l2-remote-mac-show + +show information of a specific MAC address using its UUID. +example of response: + + l2-remote-mac-show b59584eb-432a-4dba-9a09-f929e77da0c7 + ++----------------+--------------------------------------+ +| Field | Value | ++----------------+--------------------------------------+ +| ipaddr | 3.3.3.3 | +| mac | 00:11:22:33:44:55 | +| rgw_connection | a60dc097-13d7-4a9a-9842-117440911eb9 | +| uuid | b59584eb-432a-4dba-9a09-f929e77da0c7 | ++----------------+--------------------------------------+ diff --git a/specs/newton/l2-border-gateway-implementation.rst b/specs/newton/l2-border-gateway-implementation.rst new file mode 100644 index 0000000..ae8d0f4 --- /dev/null +++ b/specs/newton/l2-border-gateway-implementation.rst @@ -0,0 +1,254 @@ + +=================================== +Implementation of L2 Border Gateway +=================================== + + + +Problem Description +=================== + +The current implementation of the L2 Gateway connects bare metal servers +connected to hardware switch to OpenStack overlay tenant networks. + +This spec propose an extension of the current functionality to a scenario where +two or more overlay networks running on two or more different OpenStack domains +(such as OpenStack instances, cells or regions) needs to be connected +in Layer 2. + +The name - Border Gateway will be used in order to provide distinction between +the L2GW functionality, which is to connect overlay network to bare metal +servers and the functionality of the Border Gateway, which is to connect two or +more overlay networks that resides on different clouds. + +The diagram below provides high level overview of the system components:: + + +-------------------+ + | | OpenStack A + | | + | Computer Node | +----------------+ + | | | | + | +------------+ | VXLAN Tunnel | Border Gateway | <---------+ + | | OVS | | <--------------> | | | + | +------------+ | +----------------+ | + +-------------------+ | + | + VXLAN | WAN + Tunnel | + | + +-------------------+ | + | | OpenStack B | + | | | + | Computer Node | +----------------+ | + | | | | | + | +------------+ | VXLAN Tunnel | Border Gateway | <---------+ + | | OVS | | <--------------> | | + | +------------+ | +----------------+ + +-------------------+ + + + +Network Segmentation Handling +============================= + +The connection between the Border Gateway and the internal overlay network is +unchanged and done by using the create gateway and create gateway-connection +commands. These commands actually creates logical switch in the physical switch +(L2GW device) and also create a tunnel from the logical switch to the OVS in +each Compute Node. The segmentation of the overlay networks is done using the +overlay network segmentation id. Each cloud manages its own segmentation id +and so the usage of single end-to-end segmentation id, which is translated to +VXLAN tunnel key, is hard to manage or even impossible. + +To overcome this challenge, the Border Gateway uses different tunnel key +internally - to connect to each Compute Nodes, and externally - to connect the +two Border Gateways. When configuring end-to-end connection between the overlay +networks, the admin needs to make sure that the remote gateway connection uses +the same segmentation id on both sides on the WAN link, as the configuration of +the WAN tunnel is done on every Border Gateway separately. + +Following is a diagram that provides overview of the different segmentation id +used by the Border Gateway:: + + OpenStack A + +----------------+ +----------------+ + | Compute Node 1 | ------------------ | Border Gateway |-----------------+ + +----------------+ Seg-ID 123 +----------------+ Seg-ID 456 | + | | | + | Seg-ID 123 | | + +----------------+ | | + | Compute Node 2 | -------------------------+ | + +----------------+ Seg-ID 123 | + | + | + | + OpenStack B | + +----------------+ +----------------+ | + | Compute Node | ------------------ | Border Gateway |-----------------+ + +----------------+ Seg-ID 789 +----------------+ Seg-ID 456 + + +The segmentation id that is used in each cloud between each of the Compute Nodes +and the Border Gateway is handled by Neutron. When setting up gateway connection +using l2-gateway-connection-create command, a tunnel is created between a +logical switch in the Border Gateway and the OpenVSwitch in each Compute Node +with the same segmentation id as the tunnel key. In the diagram above you can +see segmentation ids 123 and 789 that are used for intra-cloud connection for +OpenStack A and OpenStack B respectively. + +When setting up the connection between the Border Gateways, the admin need to +provide segmentation id to be used by each of the gateways to connect the +internal overlay network to the inter-cloud tunnel. + +The inter-cloud connection command, l2-remote-gateway-connection-create, needs +to be run on the two Border Gateway while using the same segmentation id - 456 +on both sides, in addition to the internal overlay network information. + +When configuring the inter-cloud tunnel, the segmentation id is optional and +if not provided the tunnel between the Border Gateways will use the same +segmentation id on this link as the id used for Border Gateway to Compute Node +tunnels. + + +OVSDB Support +============= + +The L2 Gateway project uses OVSDB's HARDWARE VTEP schema to configure the +hardware switches. The previous HARDWARE VTEP schema had a tunnel key parameter +in the Logical_Switch table, which leads to a situation where all tunnels that +connects to this logical switch would use the same tunnel id. The schema needs +to be updated so Physical_Locator table will also have tunnel key parameter +providing the ability to setup different tunnels the uses different tunnel key +while connecting to a single logical switch. + +The tunnel setting will support hierarchical configuration in a way that if the +tunnel key is configured in the Physical_Locator table, it will be used for the +tunnel configuration, and if not, the tunnel key that is configured in the +Logical_Switch table will be used. + + +Tunnel Protocol Support +======================= + +In the existing version, only VXLAN is supported as a tunnel protocol. The +limitation is not a technical one but the fact that the encapsulation_type +field in the Physical_Locator table is an enum with only one value: +vxlan_over_ipv4. In future releases, a multi tunnel protocol support can be +achieved with the different tunnel keys support for intra and inter cloud +tunnels explained above by adding additional values to the encapsulation_type +field. With these modifications, not only the inter-cloud and intra-cloud +tunnels will be able to use other protocols than VXLAN, but the Border Gateway +will be able to use different protocol for different connection (inter or intra +cloud connection). + +Data Model Impact +----------------- + +To support different segmentation id on each tunnel, a new column will be added +to physical_locators Neutron table. + +Two Additional tables will be added to Neutron DB: + +1. l2remotegateways table that will hold remote gateway information + +2. l2remotegatewayconnections table that will hold configuration information +for remote gateway connection. + + +REST API Impact +--------------- + +API commands will be added for the following: + +1. Create/Update/Delete/List/Show Remote Gateway configuration + +2. Create/Delete/List/Show Remote Gateway Connection configuration + +3. Create/Delete/List/Show Remote MAC configuration. This will enable adding +remote host switching information. + +The above commands can be invoked by administrator or by special purpose +process. + +See l2-border-gateway-api.rst document for more detailed information. + + +Security Impact +--------------- + +None. + + +Notifications Impact +-------------------- + +A cast message from the plugin to L2 gateway agents to create connection to +remote gateway for unknown MAC addresses. This will instruct the switch to +forward packets with unknown destination MAC addresses and broadcast +destination MAC to a connection to remote gateway. + +A cast message from the plugin to L2 gateway agents to create remote MAC with +a remote gateway connection to be used for packet forwarding. + + +Performance Impact +------------------ + +None + +IPv6 Impact +----------- + +None + +Dependencies +============ + +* L2 gateway APIs + + +Implementation +============== + +Assignee(s) +----------- + +Ofer Ben-Yacov (oferby) + + +Testing +======= + +Tempest Tests +------------- + +None + +Functional Tests +---------------- + +None + +API Tests +--------- +None + +Documentation Impact +==================== + +User Documentation +------------------ + +Functionality and configuration details will be documented + + +Developer Documentation +----------------------- +OpenStack Neutron wiki needs to be updated. +See here: https://wiki.openstack.org/wiki/Neutron/L2-GW + + +References +========== + +API change request: https://bugs.launchpad.net/networking-l2gw/+bug/1529863