Merge "spec for l3 networking"
This commit is contained in:
commit
fb6f9d92f1
|
@ -0,0 +1,327 @@
|
|||
=================================================
|
||||
A New Layer-3 Networking multi-NS-with-EW-enabled
|
||||
=================================================
|
||||
|
||||
Problems
|
||||
========
|
||||
Based on spec for l3 networking [1], a l3 networking which enables multiple
|
||||
NS traffic along with EW traffic is demonstrated. However, in the
|
||||
aforementioned l3 networking model, the host route will be only valid after
|
||||
DHCP lease time expired and renewed. It may take dhcp_lease_duration for VMs
|
||||
in the subnet to update the host route, after a new pod with external
|
||||
network is added to Tricircle. To solve the problem, this spec is written
|
||||
to introduce a new l3 networking model.
|
||||
|
||||
Proposal
|
||||
========
|
||||
For the networking model in [1], a tenant network is attached to two
|
||||
routers, one for NS traffic, the other for EW traffic. In the new networking
|
||||
model, inspired by combined bridge network [2], we propose to attach the
|
||||
tenant network to one router, and the router takes charge of routing NS
|
||||
and EW traffic. The new networking mode is plotted in Fig. 1. ::
|
||||
|
||||
+-----------------------+ +----------------------+
|
||||
| ext-net1 | | ext-net2 |
|
||||
| +---+---+ | | +--+---+ |
|
||||
|RegionOne | | | RegionTwo | |
|
||||
| +---+---+ | | +----+--+ |
|
||||
| | R1 +------+ | | +--------+ R2 | |
|
||||
| +-------+ | | | | +-------+ |
|
||||
| net1 | | | | net2 |
|
||||
| +------+---+-+ | | | | +-+----+------+ |
|
||||
| | | | | | | | | |
|
||||
| +---------+-+ | | | | | | +--+--------+ |
|
||||
| | Instance1 | | | | | | | | Instance2 | |
|
||||
| +-----------+ | | | | | | +-----------+ |
|
||||
| +----+--+ | | | | ++------+ |
|
||||
| | R3(1) +-+-----------------+--+ R3(2) | |
|
||||
| +-------+ | bridge net | +-------+ |
|
||||
+-----------------------+ +----------------------+
|
||||
|
||||
Figure 1 Multiple external networks with east-west networking
|
||||
|
||||
As shown in Fig. 1, R1 connects to external network (i.e., ext-net1) and
|
||||
ext-net1 is the default gateway of R1. Meanwhile, net1 is attached to R3
|
||||
and R3’s default gateway is the bridge net. Further, interfaces of bridge
|
||||
net are only attached to R1 and R2 which are regarded as local routers.
|
||||
|
||||
In such a scenario, all traffic (no matter NS or EW traffic) flows to R3.
|
||||
For EW traffic, from net1 to net2, R3(1) will forwards packets to the
|
||||
interface of net2 in R3(2) router namespace. For NS traffic, R3 forwards
|
||||
packets to the interface of an available local router (i.e., R1 or R2)
|
||||
which attached to the real external network. As a result, bridge net is
|
||||
an internal net where NS and EW traffic is steered, rather than the real
|
||||
external network of R3.
|
||||
|
||||
To create such a topology, we need to create a logical (non-local) router
|
||||
R3 in the central Neutron. Tricircle central Neutron plugin then creates
|
||||
R3(1) in RegionOne and R3(2) in RegionTwo, as well as the bridge network
|
||||
to inter-connect R3(1) and R3(2). As such, the networking for EW traffic
|
||||
is ready for tenants. To enable NS traffic, real external networks are
|
||||
required to be attached to R3. When explicitly adding the gateway port
|
||||
of each external network to R3, Tricircle automatically creates a local
|
||||
router (e.g. R1) for external network and set the gateway to the local
|
||||
router. Then to connect the local router (e.g. R1) and the non-local
|
||||
router (R3), two interfaces of bridge-net are also created and attached
|
||||
to respect router. The logical topology in central Neutron is plotted
|
||||
in Fig. 2. ::
|
||||
|
||||
ext-net1 ext-net2
|
||||
+---+---+ +---+---+
|
||||
| |
|
||||
+---+---+ +---+---+
|
||||
| R1 | | R2 |
|
||||
+---+---+ +---+---+
|
||||
| |
|
||||
+---+--------------------+---+
|
||||
| bridge-net |
|
||||
+-------------+--------------+
|
||||
|
|
||||
|
|
||||
+-------------+--------------+
|
||||
| R3 |
|
||||
+---+--------------------+---+
|
||||
| net1 net2 |
|
||||
+---+-----+-+ +---+-+---+
|
||||
| |
|
||||
+---------+-+ +--+--------+
|
||||
| Instance1 | | Instance2 |
|
||||
+-----------+ +-----------+
|
||||
|
||||
Figure 2 Logical topology in central Neutron
|
||||
|
||||
To improve the logic of building l3 networking, we introduce routed network to
|
||||
manage external networks in central Neutron. In central Neutron, one routed
|
||||
network is created as a logical external network, and real external networks
|
||||
are stored as segments of the external network. As such, the local routers
|
||||
(e.g., R1 and R2 in Fig. 2) are transparent to users. As a result, when a real
|
||||
external network is created, a local router is created and the external
|
||||
network's gateway is set to the router. Moreover, a port of bridge-net is
|
||||
created and added to the local router.
|
||||
|
||||
The routed network is created as follows: ::
|
||||
|
||||
openstack --os-region-name=CentralRegion network create --share --provider-physical-network extern --provider-network-type vlan --provider-segment 3005 ext-net
|
||||
openstack --os-region-name=CentralRegion network segment create --physical-network extern --network-type vlan --segment 3005 --network ext-net ext-sm-net1
|
||||
openstack --os-region-name=CentralRegion network segment create --physical-network extern --network-type vlan --segment 3005 --network ext-net ext-sm-net2
|
||||
openstack --os-region-name=CentralRegion subnet create --network ext-net --network-segment ext-net1 --ip-version 4 --subnet-range 203.0.113.0/24 net1-subnet-v4
|
||||
openstack --os-region-name=CentralRegion subnet create --network ext-net --network-segment ext-net1 --ip-version 4 --subnet-range 203.0.114.0/24 net2--subnet-v4
|
||||
|
||||
The logical topology exposed to users is plotted in Fig. 3. ::
|
||||
|
||||
ext-net (routed network)
|
||||
+---+---+
|
||||
|
|
||||
|
|
||||
+--------------+-------------+
|
||||
| R3 |
|
||||
+---+--------------------+---+
|
||||
| net1 net2 |
|
||||
+---+-----+-+ +---+-+---+
|
||||
| |
|
||||
+---------+-+ +--+--------+
|
||||
| Instance1 | | Instance2 |
|
||||
+-----------+ +-----------+
|
||||
|
||||
Figure 3 Logical topology exposed to users in central Neutron
|
||||
|
||||
For R3, net1 and net2 should be attached to R3: ::
|
||||
|
||||
openstack --os-region-name=CentralRegion router add subnet R3 <net1's subnet>
|
||||
openstack --os-region-name=CentralRegion router add subnet R3 <net2's subnet>
|
||||
|
||||
The gateway of the ext-net, i.e., the routed network, is set to R3: ::
|
||||
|
||||
openstack --os-region-name=CentralRegion router set <ext-net> R3
|
||||
|
||||
However, a routed network does not have a gateway. Consequently, the command
|
||||
above fails for trying adding the gateway of a routed network to the router,
|
||||
i.e., R3. To ensure the command works, we plan to create a gateway port for
|
||||
the routed network before setting the gateway to a router. Actually, the port
|
||||
is a blank port which does not have an IP, because a routed network is a
|
||||
software entity of multiple segments (i.e., subnets). To make sure the
|
||||
gateways of real external networks can be retrieved, we manage the IPs of
|
||||
gateways in "tags" field of the gateway port.
|
||||
|
||||
This command creates a port of bridget-net and add it to R3, which is plotted in
|
||||
Fig. 2.
|
||||
|
||||
Tricircle central Neutron plugin will automatically configure R3(1), R3(2)
|
||||
and bridge-network as follows:
|
||||
|
||||
For net1 and net2, no host route is needed, so in such an l3 networking
|
||||
model, users are no longer required to wait for DHCP renew to update
|
||||
host route. All traffic is forwarded to R3 by default.
|
||||
|
||||
In R3(1), extra route will be configured: ::
|
||||
|
||||
destination=net2's cidr, nexthop=R3(2)'s interface in bridge-net
|
||||
destination=ext-net1's cidr, nexthop=R1's interface in bridge-net
|
||||
|
||||
In R3(2), extra route will be configured: ::
|
||||
|
||||
destination=net1's cidr, nexthop=R3(1)'s interface in bridge-net
|
||||
destination=ext-net2's cidr, nexthop=R2's interface in bridge-net
|
||||
|
||||
R3(1) and R3(2) will set the external gateway to bridge-net: ::
|
||||
|
||||
router-gateway-set R3(1) bridge-net
|
||||
router-gateway-set R3(2) bridge-net
|
||||
|
||||
Now, north-south traffic of Instance1 and Instance2 work as follows: ::
|
||||
|
||||
Instance1 -> net1 -> R3(1) -> R1 -> ext-net1
|
||||
Instance2 -> net2 -> R3(2) -> R2 -> ext-net2
|
||||
|
||||
Two hops for north-south traffic.
|
||||
|
||||
East-west traffic between Instance1 and Instance2 work as follows: ::
|
||||
|
||||
Instance1 <-> net1 <-> R3(1) <-> bridge-net <-> R3(2) <-> net2 <-> Instance2
|
||||
|
||||
Two hops for cross Neutron east-west traffic.
|
||||
|
||||
The topology with cross Neutron L2 networks except local networks is
|
||||
illustrated in Fig. 4. ::
|
||||
|
||||
+-----------------------+ +-----------------------+
|
||||
| ext-net1 | | ext-net2 |
|
||||
| +---+---+ | | +--+---+ |
|
||||
|RegionOne | | | RegionTwo | |
|
||||
| +---+------+ | | +----------+--+ |
|
||||
| | R1 +---+ | | +---+ R2 | |
|
||||
| +----------+ | | | | +-------------+ |
|
||||
| net1 | | | | net2 |
|
||||
| ++---+ | | | | +-----+ |
|
||||
| | net3 | | | | net4| |
|
||||
| | ++---+ | | | | +--+-+ | |
|
||||
| | | | | net5 | | | | |
|
||||
| | | +-+-----------------------------+-+ | | |
|
||||
| | | | | | net6 | | | | | |
|
||||
| | | | ++-----------------------++ | | | |
|
||||
| | | | | | | | | | | | | |
|
||||
| | | | | | | | | | | | | |
|
||||
| | | | | | | | | | | | | |
|
||||
| | | | | | | | | | | | | |
|
||||
| +----+---+---+--+-+ | | bridge-net | | ++--+---+---+-----+ |
|
||||
| | R3(1) +-+----------------+-+ R3(2) | |
|
||||
| +-----------------+ | | +-----------------+ |
|
||||
+-----------------------+ +-----------------------+
|
||||
|
||||
Figure 4 Multi-NS and cross Neutron L2 networks
|
||||
|
||||
The logical topology in central Neutron for Figure. 4 is plotted in Fig. 5. ::
|
||||
|
||||
ext-net1 ext-net2
|
||||
+---+---+ +--+---+
|
||||
| |
|
||||
+--+-----------+ +---+------------+
|
||||
| R1 | | R2 |
|
||||
+----------+---+ +----+-----------+
|
||||
| |
|
||||
+----------+--------------------------+-----------+
|
||||
| bridge-net |
|
||||
+-----------------------+-------------------------+
|
||||
|
|
||||
+-----------------------+-------------------------+
|
||||
| R3 |
|
||||
+--+----+------+-----------------+---------+----+-+
|
||||
| | | | | |
|
||||
| | | | | |
|
||||
| | | | | |
|
||||
| | +-+--------------------+ | |
|
||||
| | net5 | | |
|
||||
| | +--------------+------+ | |
|
||||
| | net6 | |
|
||||
| +-+---+ +---+-+ |
|
||||
| net3 net2 |
|
||||
+-+---+ +---+-+
|
||||
net1 net4
|
||||
|
||||
Figure 5 Logical topology in central Neutron with cross Neutron L2 network
|
||||
|
||||
By adding networks to R3, EW traffic is routed by R3.
|
||||
|
||||
For net5 in RegionOne, extra route in R3(1) should be added: ::
|
||||
|
||||
destination=net1's cidr, nexthop=<net5-R3-RegionOne-interface's IP>
|
||||
destination=net3's cidr, nexthop=<net5-R3-RegionOne-interface's IP>
|
||||
|
||||
For net5 in RegionTwo, extra route in R3(2) should be added: ::
|
||||
|
||||
destination=net1's cidr, nexthop=<net5-R3-RegionTwo-interface's id>
|
||||
destination=net3's cidr, nexthop=<net5-R3-RegionTwo-interface's IP>
|
||||
|
||||
The east-west traffic between these networks will work as follows::
|
||||
|
||||
net1 <-> R3 <-> net3
|
||||
net1 <-> R3 <-> net5
|
||||
net1 <-> R3 <-> net6
|
||||
net3 <-> R3 <-> net5
|
||||
net3 <-> R3 <-> net6
|
||||
net5 <-> R3 <-> net6
|
||||
|
||||
For NS traffic, the route to external network is already configured,
|
||||
so NS traffic is routed to R1 or R2.
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Part 0: add an option in local.conf to enable the new l3 networking model
|
||||
|
||||
Add an option "ENABLE_HOST_ROUTE_INDEPENDENT_L3_NETWORKING", whose value
|
||||
is TRUE or FALSE, to indicate whether users expect to adopt such new l3
|
||||
networking model.
|
||||
|
||||
Part 1: enable external network creation with transparent (local) router
|
||||
|
||||
This part mainly ensures a real external network is created along with a
|
||||
local router, and set the gateway of the external network to the router.
|
||||
As shown in Fig. 2, when ext-net1 is created, R1 is created, too. And the
|
||||
gateway of ext-net1 is set to R1. Moreover, the local router, e.g. R1, is
|
||||
transparent to users. In other words, users only create external network,
|
||||
while tricircle complete the creation of the local router. As a result,
|
||||
users are unaware of the local routers.
|
||||
|
||||
Part 2: enable routed network and gateway setting process
|
||||
|
||||
This part enables routed network in the central neutron. Meanwhile, this
|
||||
part also needs to complete the process of setting gateway of the routed
|
||||
network to the distributed router, e.g. R3 in Fig. 2. Here since the routed
|
||||
network is a software entity of multiple real external networks, the gateway
|
||||
ip of the routed network is set as NULL. And the gateway ips of real external
|
||||
networks is planned to stored in tag field of the routed network. So this
|
||||
part mainly deal with the blank gateway ip of the routed network when setting
|
||||
gateway to the router.
|
||||
|
||||
Part 3: modify floating ip creation
|
||||
|
||||
In the existing l3 networking, external network and tenant network is
|
||||
connected by a router, so implementing floating ip only needs NAT once.
|
||||
However, in the new l3 networking model, as shown in Fig. 2, external network
|
||||
and tenant network connect two routers, respectively. And the two routers
|
||||
are connected by bridge network. So implementing floating ip needs to be NATed
|
||||
twice. This part mainly deal with such an issue.
|
||||
|
||||
Data Model Impact
|
||||
=================
|
||||
|
||||
None
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
1. Add a new guide for North South Networking via Multiple External Networks
|
||||
with east-west enabled.
|
||||
2. Release notes.
|
||||
|
||||
Reference
|
||||
=========
|
||||
|
||||
.. [1] https://github.com/openstack/tricircle/blob/master/specs/pike/l3-networking-multi-NS-with-EW-enabled.rst
|
||||
.. [2] https://github.com/openstack/tricircle/blob/master/specs/ocata/l3-networking-combined-bridge-net.rst
|
Loading…
Reference in New Issue