Merge "spec for l3 networking"
This commit is contained in:
commit
fb6f9d92f1
|
@ -0,0 +1,327 @@
|
||||||
|
=================================================
|
||||||
|
A New Layer-3 Networking multi-NS-with-EW-enabled
|
||||||
|
=================================================
|
||||||
|
|
||||||
|
Problems
|
||||||
|
========
|
||||||
|
Based on spec for l3 networking [1], a l3 networking which enables multiple
|
||||||
|
NS traffic along with EW traffic is demonstrated. However, in the
|
||||||
|
aforementioned l3 networking model, the host route will be only valid after
|
||||||
|
DHCP lease time expired and renewed. It may take dhcp_lease_duration for VMs
|
||||||
|
in the subnet to update the host route, after a new pod with external
|
||||||
|
network is added to Tricircle. To solve the problem, this spec is written
|
||||||
|
to introduce a new l3 networking model.
|
||||||
|
|
||||||
|
Proposal
|
||||||
|
========
|
||||||
|
For the networking model in [1], a tenant network is attached to two
|
||||||
|
routers, one for NS traffic, the other for EW traffic. In the new networking
|
||||||
|
model, inspired by combined bridge network [2], we propose to attach the
|
||||||
|
tenant network to one router, and the router takes charge of routing NS
|
||||||
|
and EW traffic. The new networking mode is plotted in Fig. 1. ::
|
||||||
|
|
||||||
|
+-----------------------+ +----------------------+
|
||||||
|
| ext-net1 | | ext-net2 |
|
||||||
|
| +---+---+ | | +--+---+ |
|
||||||
|
|RegionOne | | | RegionTwo | |
|
||||||
|
| +---+---+ | | +----+--+ |
|
||||||
|
| | R1 +------+ | | +--------+ R2 | |
|
||||||
|
| +-------+ | | | | +-------+ |
|
||||||
|
| net1 | | | | net2 |
|
||||||
|
| +------+---+-+ | | | | +-+----+------+ |
|
||||||
|
| | | | | | | | | |
|
||||||
|
| +---------+-+ | | | | | | +--+--------+ |
|
||||||
|
| | Instance1 | | | | | | | | Instance2 | |
|
||||||
|
| +-----------+ | | | | | | +-----------+ |
|
||||||
|
| +----+--+ | | | | ++------+ |
|
||||||
|
| | R3(1) +-+-----------------+--+ R3(2) | |
|
||||||
|
| +-------+ | bridge net | +-------+ |
|
||||||
|
+-----------------------+ +----------------------+
|
||||||
|
|
||||||
|
Figure 1 Multiple external networks with east-west networking
|
||||||
|
|
||||||
|
As shown in Fig. 1, R1 connects to external network (i.e., ext-net1) and
|
||||||
|
ext-net1 is the default gateway of R1. Meanwhile, net1 is attached to R3
|
||||||
|
and R3’s default gateway is the bridge net. Further, interfaces of bridge
|
||||||
|
net are only attached to R1 and R2 which are regarded as local routers.
|
||||||
|
|
||||||
|
In such a scenario, all traffic (no matter NS or EW traffic) flows to R3.
|
||||||
|
For EW traffic, from net1 to net2, R3(1) will forwards packets to the
|
||||||
|
interface of net2 in R3(2) router namespace. For NS traffic, R3 forwards
|
||||||
|
packets to the interface of an available local router (i.e., R1 or R2)
|
||||||
|
which attached to the real external network. As a result, bridge net is
|
||||||
|
an internal net where NS and EW traffic is steered, rather than the real
|
||||||
|
external network of R3.
|
||||||
|
|
||||||
|
To create such a topology, we need to create a logical (non-local) router
|
||||||
|
R3 in the central Neutron. Tricircle central Neutron plugin then creates
|
||||||
|
R3(1) in RegionOne and R3(2) in RegionTwo, as well as the bridge network
|
||||||
|
to inter-connect R3(1) and R3(2). As such, the networking for EW traffic
|
||||||
|
is ready for tenants. To enable NS traffic, real external networks are
|
||||||
|
required to be attached to R3. When explicitly adding the gateway port
|
||||||
|
of each external network to R3, Tricircle automatically creates a local
|
||||||
|
router (e.g. R1) for external network and set the gateway to the local
|
||||||
|
router. Then to connect the local router (e.g. R1) and the non-local
|
||||||
|
router (R3), two interfaces of bridge-net are also created and attached
|
||||||
|
to respect router. The logical topology in central Neutron is plotted
|
||||||
|
in Fig. 2. ::
|
||||||
|
|
||||||
|
ext-net1 ext-net2
|
||||||
|
+---+---+ +---+---+
|
||||||
|
| |
|
||||||
|
+---+---+ +---+---+
|
||||||
|
| R1 | | R2 |
|
||||||
|
+---+---+ +---+---+
|
||||||
|
| |
|
||||||
|
+---+--------------------+---+
|
||||||
|
| bridge-net |
|
||||||
|
+-------------+--------------+
|
||||||
|
|
|
||||||
|
|
|
||||||
|
+-------------+--------------+
|
||||||
|
| R3 |
|
||||||
|
+---+--------------------+---+
|
||||||
|
| net1 net2 |
|
||||||
|
+---+-----+-+ +---+-+---+
|
||||||
|
| |
|
||||||
|
+---------+-+ +--+--------+
|
||||||
|
| Instance1 | | Instance2 |
|
||||||
|
+-----------+ +-----------+
|
||||||
|
|
||||||
|
Figure 2 Logical topology in central Neutron
|
||||||
|
|
||||||
|
To improve the logic of building l3 networking, we introduce routed network to
|
||||||
|
manage external networks in central Neutron. In central Neutron, one routed
|
||||||
|
network is created as a logical external network, and real external networks
|
||||||
|
are stored as segments of the external network. As such, the local routers
|
||||||
|
(e.g., R1 and R2 in Fig. 2) are transparent to users. As a result, when a real
|
||||||
|
external network is created, a local router is created and the external
|
||||||
|
network's gateway is set to the router. Moreover, a port of bridge-net is
|
||||||
|
created and added to the local router.
|
||||||
|
|
||||||
|
The routed network is created as follows: ::
|
||||||
|
|
||||||
|
openstack --os-region-name=CentralRegion network create --share --provider-physical-network extern --provider-network-type vlan --provider-segment 3005 ext-net
|
||||||
|
openstack --os-region-name=CentralRegion network segment create --physical-network extern --network-type vlan --segment 3005 --network ext-net ext-sm-net1
|
||||||
|
openstack --os-region-name=CentralRegion network segment create --physical-network extern --network-type vlan --segment 3005 --network ext-net ext-sm-net2
|
||||||
|
openstack --os-region-name=CentralRegion subnet create --network ext-net --network-segment ext-net1 --ip-version 4 --subnet-range 203.0.113.0/24 net1-subnet-v4
|
||||||
|
openstack --os-region-name=CentralRegion subnet create --network ext-net --network-segment ext-net1 --ip-version 4 --subnet-range 203.0.114.0/24 net2--subnet-v4
|
||||||
|
|
||||||
|
The logical topology exposed to users is plotted in Fig. 3. ::
|
||||||
|
|
||||||
|
ext-net (routed network)
|
||||||
|
+---+---+
|
||||||
|
|
|
||||||
|
|
|
||||||
|
+--------------+-------------+
|
||||||
|
| R3 |
|
||||||
|
+---+--------------------+---+
|
||||||
|
| net1 net2 |
|
||||||
|
+---+-----+-+ +---+-+---+
|
||||||
|
| |
|
||||||
|
+---------+-+ +--+--------+
|
||||||
|
| Instance1 | | Instance2 |
|
||||||
|
+-----------+ +-----------+
|
||||||
|
|
||||||
|
Figure 3 Logical topology exposed to users in central Neutron
|
||||||
|
|
||||||
|
For R3, net1 and net2 should be attached to R3: ::
|
||||||
|
|
||||||
|
openstack --os-region-name=CentralRegion router add subnet R3 <net1's subnet>
|
||||||
|
openstack --os-region-name=CentralRegion router add subnet R3 <net2's subnet>
|
||||||
|
|
||||||
|
The gateway of the ext-net, i.e., the routed network, is set to R3: ::
|
||||||
|
|
||||||
|
openstack --os-region-name=CentralRegion router set <ext-net> R3
|
||||||
|
|
||||||
|
However, a routed network does not have a gateway. Consequently, the command
|
||||||
|
above fails for trying adding the gateway of a routed network to the router,
|
||||||
|
i.e., R3. To ensure the command works, we plan to create a gateway port for
|
||||||
|
the routed network before setting the gateway to a router. Actually, the port
|
||||||
|
is a blank port which does not have an IP, because a routed network is a
|
||||||
|
software entity of multiple segments (i.e., subnets). To make sure the
|
||||||
|
gateways of real external networks can be retrieved, we manage the IPs of
|
||||||
|
gateways in "tags" field of the gateway port.
|
||||||
|
|
||||||
|
This command creates a port of bridget-net and add it to R3, which is plotted in
|
||||||
|
Fig. 2.
|
||||||
|
|
||||||
|
Tricircle central Neutron plugin will automatically configure R3(1), R3(2)
|
||||||
|
and bridge-network as follows:
|
||||||
|
|
||||||
|
For net1 and net2, no host route is needed, so in such an l3 networking
|
||||||
|
model, users are no longer required to wait for DHCP renew to update
|
||||||
|
host route. All traffic is forwarded to R3 by default.
|
||||||
|
|
||||||
|
In R3(1), extra route will be configured: ::
|
||||||
|
|
||||||
|
destination=net2's cidr, nexthop=R3(2)'s interface in bridge-net
|
||||||
|
destination=ext-net1's cidr, nexthop=R1's interface in bridge-net
|
||||||
|
|
||||||
|
In R3(2), extra route will be configured: ::
|
||||||
|
|
||||||
|
destination=net1's cidr, nexthop=R3(1)'s interface in bridge-net
|
||||||
|
destination=ext-net2's cidr, nexthop=R2's interface in bridge-net
|
||||||
|
|
||||||
|
R3(1) and R3(2) will set the external gateway to bridge-net: ::
|
||||||
|
|
||||||
|
router-gateway-set R3(1) bridge-net
|
||||||
|
router-gateway-set R3(2) bridge-net
|
||||||
|
|
||||||
|
Now, north-south traffic of Instance1 and Instance2 work as follows: ::
|
||||||
|
|
||||||
|
Instance1 -> net1 -> R3(1) -> R1 -> ext-net1
|
||||||
|
Instance2 -> net2 -> R3(2) -> R2 -> ext-net2
|
||||||
|
|
||||||
|
Two hops for north-south traffic.
|
||||||
|
|
||||||
|
East-west traffic between Instance1 and Instance2 work as follows: ::
|
||||||
|
|
||||||
|
Instance1 <-> net1 <-> R3(1) <-> bridge-net <-> R3(2) <-> net2 <-> Instance2
|
||||||
|
|
||||||
|
Two hops for cross Neutron east-west traffic.
|
||||||
|
|
||||||
|
The topology with cross Neutron L2 networks except local networks is
|
||||||
|
illustrated in Fig. 4. ::
|
||||||
|
|
||||||
|
+-----------------------+ +-----------------------+
|
||||||
|
| ext-net1 | | ext-net2 |
|
||||||
|
| +---+---+ | | +--+---+ |
|
||||||
|
|RegionOne | | | RegionTwo | |
|
||||||
|
| +---+------+ | | +----------+--+ |
|
||||||
|
| | R1 +---+ | | +---+ R2 | |
|
||||||
|
| +----------+ | | | | +-------------+ |
|
||||||
|
| net1 | | | | net2 |
|
||||||
|
| ++---+ | | | | +-----+ |
|
||||||
|
| | net3 | | | | net4| |
|
||||||
|
| | ++---+ | | | | +--+-+ | |
|
||||||
|
| | | | | net5 | | | | |
|
||||||
|
| | | +-+-----------------------------+-+ | | |
|
||||||
|
| | | | | | net6 | | | | | |
|
||||||
|
| | | | ++-----------------------++ | | | |
|
||||||
|
| | | | | | | | | | | | | |
|
||||||
|
| | | | | | | | | | | | | |
|
||||||
|
| | | | | | | | | | | | | |
|
||||||
|
| | | | | | | | | | | | | |
|
||||||
|
| +----+---+---+--+-+ | | bridge-net | | ++--+---+---+-----+ |
|
||||||
|
| | R3(1) +-+----------------+-+ R3(2) | |
|
||||||
|
| +-----------------+ | | +-----------------+ |
|
||||||
|
+-----------------------+ +-----------------------+
|
||||||
|
|
||||||
|
Figure 4 Multi-NS and cross Neutron L2 networks
|
||||||
|
|
||||||
|
The logical topology in central Neutron for Figure. 4 is plotted in Fig. 5. ::
|
||||||
|
|
||||||
|
ext-net1 ext-net2
|
||||||
|
+---+---+ +--+---+
|
||||||
|
| |
|
||||||
|
+--+-----------+ +---+------------+
|
||||||
|
| R1 | | R2 |
|
||||||
|
+----------+---+ +----+-----------+
|
||||||
|
| |
|
||||||
|
+----------+--------------------------+-----------+
|
||||||
|
| bridge-net |
|
||||||
|
+-----------------------+-------------------------+
|
||||||
|
|
|
||||||
|
+-----------------------+-------------------------+
|
||||||
|
| R3 |
|
||||||
|
+--+----+------+-----------------+---------+----+-+
|
||||||
|
| | | | | |
|
||||||
|
| | | | | |
|
||||||
|
| | | | | |
|
||||||
|
| | +-+--------------------+ | |
|
||||||
|
| | net5 | | |
|
||||||
|
| | +--------------+------+ | |
|
||||||
|
| | net6 | |
|
||||||
|
| +-+---+ +---+-+ |
|
||||||
|
| net3 net2 |
|
||||||
|
+-+---+ +---+-+
|
||||||
|
net1 net4
|
||||||
|
|
||||||
|
Figure 5 Logical topology in central Neutron with cross Neutron L2 network
|
||||||
|
|
||||||
|
By adding networks to R3, EW traffic is routed by R3.
|
||||||
|
|
||||||
|
For net5 in RegionOne, extra route in R3(1) should be added: ::
|
||||||
|
|
||||||
|
destination=net1's cidr, nexthop=<net5-R3-RegionOne-interface's IP>
|
||||||
|
destination=net3's cidr, nexthop=<net5-R3-RegionOne-interface's IP>
|
||||||
|
|
||||||
|
For net5 in RegionTwo, extra route in R3(2) should be added: ::
|
||||||
|
|
||||||
|
destination=net1's cidr, nexthop=<net5-R3-RegionTwo-interface's id>
|
||||||
|
destination=net3's cidr, nexthop=<net5-R3-RegionTwo-interface's IP>
|
||||||
|
|
||||||
|
The east-west traffic between these networks will work as follows::
|
||||||
|
|
||||||
|
net1 <-> R3 <-> net3
|
||||||
|
net1 <-> R3 <-> net5
|
||||||
|
net1 <-> R3 <-> net6
|
||||||
|
net3 <-> R3 <-> net5
|
||||||
|
net3 <-> R3 <-> net6
|
||||||
|
net5 <-> R3 <-> net6
|
||||||
|
|
||||||
|
For NS traffic, the route to external network is already configured,
|
||||||
|
so NS traffic is routed to R1 or R2.
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Part 0: add an option in local.conf to enable the new l3 networking model
|
||||||
|
|
||||||
|
Add an option "ENABLE_HOST_ROUTE_INDEPENDENT_L3_NETWORKING", whose value
|
||||||
|
is TRUE or FALSE, to indicate whether users expect to adopt such new l3
|
||||||
|
networking model.
|
||||||
|
|
||||||
|
Part 1: enable external network creation with transparent (local) router
|
||||||
|
|
||||||
|
This part mainly ensures a real external network is created along with a
|
||||||
|
local router, and set the gateway of the external network to the router.
|
||||||
|
As shown in Fig. 2, when ext-net1 is created, R1 is created, too. And the
|
||||||
|
gateway of ext-net1 is set to R1. Moreover, the local router, e.g. R1, is
|
||||||
|
transparent to users. In other words, users only create external network,
|
||||||
|
while tricircle complete the creation of the local router. As a result,
|
||||||
|
users are unaware of the local routers.
|
||||||
|
|
||||||
|
Part 2: enable routed network and gateway setting process
|
||||||
|
|
||||||
|
This part enables routed network in the central neutron. Meanwhile, this
|
||||||
|
part also needs to complete the process of setting gateway of the routed
|
||||||
|
network to the distributed router, e.g. R3 in Fig. 2. Here since the routed
|
||||||
|
network is a software entity of multiple real external networks, the gateway
|
||||||
|
ip of the routed network is set as NULL. And the gateway ips of real external
|
||||||
|
networks is planned to stored in tag field of the routed network. So this
|
||||||
|
part mainly deal with the blank gateway ip of the routed network when setting
|
||||||
|
gateway to the router.
|
||||||
|
|
||||||
|
Part 3: modify floating ip creation
|
||||||
|
|
||||||
|
In the existing l3 networking, external network and tenant network is
|
||||||
|
connected by a router, so implementing floating ip only needs NAT once.
|
||||||
|
However, in the new l3 networking model, as shown in Fig. 2, external network
|
||||||
|
and tenant network connect two routers, respectively. And the two routers
|
||||||
|
are connected by bridge network. So implementing floating ip needs to be NATed
|
||||||
|
twice. This part mainly deal with such an issue.
|
||||||
|
|
||||||
|
Data Model Impact
|
||||||
|
=================
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
1. Add a new guide for North South Networking via Multiple External Networks
|
||||||
|
with east-west enabled.
|
||||||
|
2. Release notes.
|
||||||
|
|
||||||
|
Reference
|
||||||
|
=========
|
||||||
|
|
||||||
|
.. [1] https://github.com/openstack/tricircle/blob/master/specs/pike/l3-networking-multi-NS-with-EW-enabled.rst
|
||||||
|
.. [2] https://github.com/openstack/tricircle/blob/master/specs/ocata/l3-networking-combined-bridge-net.rst
|
Loading…
Reference in New Issue