Spec for Neutron DVR support in Fuel
Implements blueprint neutron-dvr-deployment Change-Id: I666c1d1de9e128147174abc0a178a41828ae5d93
This commit is contained in:
Oleg Bondarev
parent
b4b03b8730
commit
af52304690
372
specs/7.0/neutron-dvr-deployment.rst
Normal file
372
specs/7.0/neutron-dvr-deployment.rst
Normal file
@ -0,0 +1,372 @@
|
|||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
===================
|
||||||
|
Neutron DVR support
|
||||||
|
===================
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/fuel/+spec/neutron-dvr-deployment
|
||||||
|
|
||||||
|
Neutron Distributed Virtual Router implements the L3 Routers across the
|
||||||
|
compute nodes, so that tenants intra VM communication will occur without
|
||||||
|
hitting the controller node. (East-West Routing)
|
||||||
|
|
||||||
|
Also Neutron Distributed Virtual Router implements the Floating IP namespace
|
||||||
|
on every compute node where the VMs are located. In this case the VMs with
|
||||||
|
FloatingIPs can forward the traffic to the external network without reaching
|
||||||
|
the controller node. (North-South Routing)
|
||||||
|
|
||||||
|
Neutron Distributed Virtual Router provides the legacy SNAT behavior for
|
||||||
|
the default SNAT for all private VMs. SNAT service is not distributed,
|
||||||
|
it is centralized and the service node will host the service.
|
||||||
|
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
|
||||||
|
Currently Neutron L3 Routers are deployed on specific Nodes (controller nodes)
|
||||||
|
where all the compute traffic will flow through.
|
||||||
|
|
||||||
|
* Problem 1: Intra VM traffic flows through the controller node
|
||||||
|
|
||||||
|
In this case even VMs traffic that belong to the same tenant on a different
|
||||||
|
subnet has to hit the controller node to get routed between the subnets.
|
||||||
|
This would affect performance and scalability.
|
||||||
|
|
||||||
|
* Problem 2: VMs with FloatingIP also receive and send packets through
|
||||||
|
the controller node routers
|
||||||
|
|
||||||
|
Today FloatingIP (DNAT) translation is done on the controller node and
|
||||||
|
also the external network gateway port is available only at the controller.
|
||||||
|
So any traffic that goes to the external network from the VM will
|
||||||
|
have to go through the controller node. In this case the controller node
|
||||||
|
becomes a single point of failure and also the traffic will heavily load
|
||||||
|
the controller node. This would affect the performance and scalability.
|
||||||
|
|
||||||
|
|
||||||
|
Proposed change
|
||||||
|
===============
|
||||||
|
|
||||||
|
The proposal is to distribute L3 Routers across compute nodes when required
|
||||||
|
by VMs. This implies having external network access on each compute node.
|
||||||
|
|
||||||
|
In this case there will be enhanced L3 Agents running on each and every
|
||||||
|
compute node (This is not a new agent, this is an updated version of the
|
||||||
|
existing L3 Agent). Based on the configuration in the L3 Agent.ini file,
|
||||||
|
the enhanced L3 Agent will behave in legacy (centralized router) mode or as
|
||||||
|
a distributed router mode.
|
||||||
|
|
||||||
|
Also the FloatingIP will have a new namespace created on the specific
|
||||||
|
compute node where the VM is located (this is done by L3 agent itself).
|
||||||
|
Each Compute Node will have one new namespace for FloatingIP per external
|
||||||
|
network that will be shared among the tenants. Additional namespace and
|
||||||
|
external gateway port will also be created on each compute node for the
|
||||||
|
external traffic to flow through, in case there are VMs with floating ips
|
||||||
|
residing on this node. This port will consume additional IP address from
|
||||||
|
external network.
|
||||||
|
|
||||||
|
Default SNAT functionality will still be centralized and will be running on
|
||||||
|
controller nodes.
|
||||||
|
|
||||||
|
The Metadata agent will be distributed as well and will be hosted on all
|
||||||
|
compute nodes and the Metadata Proxy will be hosted on all the distributed
|
||||||
|
routers.
|
||||||
|
|
||||||
|
This implementation is specific to ML2 with OVS driver.
|
||||||
|
All three type of segmentation are supported: GRE, VXLAN, VLAN.
|
||||||
|
|
||||||
|
Constraints and Limitations
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
* No Distributed SNAT
|
||||||
|
|
||||||
|
Neutron Distributed Virtual Router provides the legacy SNAT behavior for the
|
||||||
|
default SNAT for all private VMs. SNAT service is not distributed,
|
||||||
|
it is centralized and the service node will host the service.
|
||||||
|
Thus current DVR architecture is not fully fault tolerant - outbound traffic
|
||||||
|
for VMs without floating IPs is still going through one L3_agent node and
|
||||||
|
is still prone to failures of a single node.
|
||||||
|
|
||||||
|
* Only with ML2-OVS/L2-pop
|
||||||
|
|
||||||
|
DVR feature is supported only by ML2 plugin with OVS mechanism driver. If
|
||||||
|
using tunnel segmentation (VXLAN, GRE) L2 population mechanism should be
|
||||||
|
enabled as well.
|
||||||
|
|
||||||
|
* OVS and Kernel versions
|
||||||
|
|
||||||
|
Proper operation of DVR requires OpenvSwitch 2.1 or newer and VXLAN requires
|
||||||
|
kernel 3.13 or newer.
|
||||||
|
|
||||||
|
* No bare metal support
|
||||||
|
|
||||||
|
Distributed routers rely on local l3 agent (residing on compute node) for
|
||||||
|
address translation, so for bare metal instances only legacy routers should
|
||||||
|
be used.
|
||||||
|
|
||||||
|
Deployment impact
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
* Architecture changes
|
||||||
|
|
||||||
|
* Neutron L3 and metadata agents will be deployed on all compute nodes and
|
||||||
|
managed by Upstart. Agents deployment scheme on controller nodes is not
|
||||||
|
changed.
|
||||||
|
|
||||||
|
* All compute nodes require bridge to external network
|
||||||
|
|
||||||
|
* Fuel Library related changes
|
||||||
|
|
||||||
|
* Update Neutron Puppet module to support DVR-related options (L3 agent mode,
|
||||||
|
L2 population, distributed router option). This step will be done as a part
|
||||||
|
of blueprint upgrade-openstack-puppet-modules, when all necessary changes
|
||||||
|
will be synced from puppet-neutron project
|
||||||
|
|
||||||
|
* Update Cloud Networking related Puppet modules to deploy Neutron L3 and
|
||||||
|
metadata agents on compute nodes with appropriate configuration. This step
|
||||||
|
will likely require changes in Granular deployment to execute Neutron
|
||||||
|
agents related granulars on compute nodes
|
||||||
|
|
||||||
|
* update Horizon related Puppet modules to add an ability to use Neutron DVR
|
||||||
|
options (create either centralized or distributed routers)
|
||||||
|
|
||||||
|
* Fuel Web related changes
|
||||||
|
|
||||||
|
* When Neutron DVR is enabled, a network scheme with external bridges on all
|
||||||
|
compute nodes should be generated. astute.yaml possible examples:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
Compute nodes:
|
||||||
|
-----------------
|
||||||
|
network_scheme:
|
||||||
|
endpoints:
|
||||||
|
br-ex:
|
||||||
|
IP: none
|
||||||
|
-----------------
|
||||||
|
quantum_settings:
|
||||||
|
DVR: true
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
Controller nodes:
|
||||||
|
-----------------
|
||||||
|
network_scheme:
|
||||||
|
endpoints:
|
||||||
|
br-ex:
|
||||||
|
IP:
|
||||||
|
- 172.16.0.3/24
|
||||||
|
----------------
|
||||||
|
quantum_settings:
|
||||||
|
DVR: true
|
||||||
|
----------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Data model impact
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
REST API impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
No FUEL REST API changes.
|
||||||
|
|
||||||
|
Upgrade impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
The upgrade path from legacy to distributed router is supported. It's a 3
|
||||||
|
step process:
|
||||||
|
|
||||||
|
* neutron router-update router1 --admin_state_up=False
|
||||||
|
|
||||||
|
* neutron router-update router1 --distributed=True
|
||||||
|
|
||||||
|
* neutron router-update router1 --admin_state_up=True
|
||||||
|
|
||||||
|
distributed->legacy upgrade is not officially supported in Kilo but it may
|
||||||
|
work, just needs to be tested.
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Notifications impact
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other end user impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
Inter VM traffic between the tenant subnets doesn't need to reach the router
|
||||||
|
in the controller node to get routed and will be routed locally from the
|
||||||
|
compute node. This would increase the performance substantially.
|
||||||
|
|
||||||
|
Also the Floating IP traffic for a VM from a Compute Node will directly hit
|
||||||
|
the external network from the compute node, instead of going through the router
|
||||||
|
on the controller node.
|
||||||
|
|
||||||
|
Dataplane testing results from 25 bare metal nodes env show significant
|
||||||
|
performance improvement for both East-West and North-South (with floating IPs)
|
||||||
|
scenarios.
|
||||||
|
|
||||||
|
Plugin impact
|
||||||
|
-------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other deployer impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Developer impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Infrastructure impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
obondarev
|
||||||
|
|
||||||
|
Other contributors:
|
||||||
|
skolekonov (DE)
|
||||||
|
kkuznetsova (QA)
|
||||||
|
tnurlygayanov (QA)
|
||||||
|
|
||||||
|
Mandatory design reviewers:
|
||||||
|
svasilenko
|
||||||
|
vkuklin
|
||||||
|
sgolovatiuk
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
* Patch fuel-lib to enable DVR by default
|
||||||
|
|
||||||
|
* this will enable DVR testing at early stage
|
||||||
|
|
||||||
|
* Scale testing
|
||||||
|
|
||||||
|
* Rally scenarios
|
||||||
|
|
||||||
|
* Shaker scenarios
|
||||||
|
|
||||||
|
* debug
|
||||||
|
|
||||||
|
* bug fixing/backport from upstream
|
||||||
|
|
||||||
|
* Patch fuel-web to add ability to enable/disable DVR
|
||||||
|
|
||||||
|
* disable DVR by default
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
This will likely depend on enabling l2-population for tunneling which is a
|
||||||
|
separate effort. However we will not wait but enable l2 pop as part of DVR
|
||||||
|
effort if needed.
|
||||||
|
|
||||||
|
It also correlates with blueprint upgrade-openstack-puppet-modules as all
|
||||||
|
required changes might be already in master in upstream manifests.
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
Manual Acceptance Tests
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
* On an environment with DVR enabled check that created router has
|
||||||
|
“distributed “ attribute set to True via Horizon or CLI
|
||||||
|
|
||||||
|
* Boot a VM on a subnet connected to DVR router. Check external connectivity.
|
||||||
|
|
||||||
|
* Assign Floating IP to the VM. Check external connectivity. Ensure VM is
|
||||||
|
reachable from external network.
|
||||||
|
|
||||||
|
* Boot a second VM on a different subnet connected to the same router. Ensure
|
||||||
|
inter-subnet connectivity (both VM can reach each other)
|
||||||
|
|
||||||
|
Scale
|
||||||
|
-----
|
||||||
|
|
||||||
|
* Environment with DVR enabled should pass all tests currently run on Scale
|
||||||
|
Lab with no significant performance degradation
|
||||||
|
|
||||||
|
* No additional Rally scenarios are needed to test specifics of DVR.
|
||||||
|
|
||||||
|
HA/Destructive Tests
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
All existing HA/destructive tests should pass on env with DVR enabled.
|
||||||
|
Additional scenarios should include:
|
||||||
|
|
||||||
|
* East-West HA Test
|
||||||
|
|
||||||
|
* Have several VM from different subnets running on different compute nodes.
|
||||||
|
The subnets should be connected to each other and to an external network by
|
||||||
|
a DVR router
|
||||||
|
|
||||||
|
* Shutdown all controllers of the environment
|
||||||
|
|
||||||
|
* Inter-subnet connectivity should be preserved: VMs from different
|
||||||
|
subnets/compute nodes should still be able to reach each other
|
||||||
|
|
||||||
|
* No dataplane downtime is expected
|
||||||
|
|
||||||
|
* North-South HA Test
|
||||||
|
|
||||||
|
* Have a VM with Floating IP running on a subnet connected to an external
|
||||||
|
network by a DVR router
|
||||||
|
|
||||||
|
* Shutdown all controllers of the environment.
|
||||||
|
|
||||||
|
* External connectivity should be preserved: VMs should still be able to
|
||||||
|
reach external network
|
||||||
|
|
||||||
|
* No dataplane downtime is expected
|
||||||
|
|
||||||
|
Data Plane Tests with Shaker
|
||||||
|
----------------------------
|
||||||
|
Shaker scenarios should be run on a bare-metal environment with DVR enabled.
|
||||||
|
Significant increase in performance is expected for east-west and north-south
|
||||||
|
(with Floating IPs) topologies. Some of the results were already obtained
|
||||||
|
(see "Performance Impact" section of the this doc)
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
Ability to enable DVR support in Neutron should be documented in
|
||||||
|
Fuel Deployment Guide.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/fuel/+spec/neutron-dvr-deployment
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/fuel/+spec/upgrade-openstack-puppet-modules
|
||||||
Loading…
Reference in New Issue
Block a user