openstack/neutron
Code Issues Proposed changes

doc: OpenVSwitch firewall internal design, improvements (+ newline fix)

Browse Source 
* explain 'ingress' and 'egress'
* reword a few explanations
* provide table numbers as well as table names
* capitalize protocol names: IP, ARP, TPC, etc.
* reword explanations for tables acting as integration
  points with L2 other extensions
* nits

This change also includes a newline removal in the related portion
of the code; this newline separates constants that are in fact related,
and its removal had been agreed upon to be handle as a follow-up
change to I60d299275effd9ef35c8007773d3c9fcabfa50fa.

Change-Id: Ied215b8942bfffa1f2af2f569d0e4a110a64a364
This commit is contained in:
Thomas Morin 2018-09-04 15:02:41 +02:00 committed by Thomas Morin
parent eaff4be1c1
commit fdaf24bead
2 changed files with 143 additions and 83 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

225
doc/source/contributor/internals/openvswitch_firewall.rst
View File

@ -30,10 +30,45 @@ Class ``SGPortMap`` was created to keep state consistent, and maps from ports
to security groups and vice-versa. Every port and security group is represented to security groups and vice-versa. Every port and security group is represented
by its own object encapsulating the necessary information. by its own object encapsulating the necessary information.
Note: Open vSwitch firewall driver uses register 5 for marking flow .. note::
related to port and register 6 which defines network and is used for conntrack
zones.
Open vSwitch firewall driver uses ``register 5`` for identifying the port
related to the flow and ``register 6`` which identifies the network, used
in particular for conntrack zones.
Ingress/Egress Terminology
--------------------------
In this document, the terms ``ingress`` and ``egress`` are relative to
a VM instance connected to OVS (or a netns connected to OVS):
* ``ingress`` applies to traffic that will ultimately go into a VM (or into
a netns), assuming it is not dropped
* ``egress`` applies to traffic coming from a VM (or from a netns)
::
. .
_______|\ _______|\
\ ingress \ \ ingress \
/_______ / /_______ /
|/ .-----------------. |/
' | | '
| |-----------( netns interface )
( non-VM, non-netns )---| OVS |
( interface: phy, patch ) | |------------( VM interface )
. | | .
/|________ '-----------------' /|________
/ egress / / egress /
\ ________\ \ ________\
\| \|
' '
Note that these terms are used differently in OVS code and documentation, where
they are relative to the OVS bridge, with ``ingress`` applying to traffic as
it comes into the OVS bridge, and ``egress`` applying to traffic as it leaves
the OVS bridge.
Firewall API calls Firewall API calls
------------------ ------------------
@ -139,39 +174,39 @@ Rules example with explanation:
------------------------------- -------------------------------
The following example presents two ports on the same host. They have different The following example presents two ports on the same host. They have different
security groups and there is icmp traffic allowed from first security group to security groups and there is ICMP traffic allowed from first security group to
the second security group. Ports have following attributes: the second security group. Ports have following attributes:
:: ::
Port 1 Port 1
- plugged to the port 1 in OVS bridge - plugged to the port 1 in OVS bridge
- ip address: 192.168.0.1 - IP address: 192.168.0.1
- mac address: fa:16:3e:a4:22:10 - MAC address: fa:16:3e:a4:22:10
- security group 1: can send icmp packets out - security group 1: can send ICMP packets out
- allowed address pair: 10.0.0.1/32, fa:16:3e:8c:84:13 - allowed address pair: 10.0.0.1/32, fa:16:3e:8c:84:13
Port 2 Port 2
- plugged to the port 2 in OVS bridge - plugged to the port 2 in OVS bridge
- ip address: 192.168.0.2 - IP address: 192.168.0.2
- mac address: fa:16:3e:24:57:c7 - MAC address: fa:16:3e:24:57:c7
- security group 2: - security group 2:
- can receive icmp packets from security group 1 - can receive ICMP packets from security group 1
- can receive tcp packets from security group 1 - can receive TCP packets from security group 1
- can receive tcp packets to port 80 from security group 2 - can receive TCP packets to port 80 from security group 2
- can receive ip packets from security group 3 - can receive IP packets from security group 3
- allowed address pair: 10.1.0.0/24, fa:16:3e:8c:84:14 - allowed address pair: 10.1.0.0/24, fa:16:3e:8c:84:14
``table 0`` contains a low priority rule to continue packets processing in |table_0| contains a low priority rule to continue packets processing in
``table 60`` aka TRANSIENT table. ``table 0`` is left for use to other |table_60| aka TRANSIENT table. |table_0| is left for use to other
features that take precedence over firewall, e.g. DVR. The only requirement is features that take precedence over firewall, e.g. DVR. The only requirement is
that after feature is done with its processing, it needs to pass packets for that after such a feature is done with its processing, it needs to pass packets
processing to the TRANSIENT table. This TRANSIENT table distinguishes the for processing to the TRANSIENT table. This TRANSIENT table distinguishes the
traffic to ingress or egress and loads to ``register 5`` value identifying port ingress traffic from the egress traffic and loads into ``register 5`` a value
traffic. identifying the port (for egress traffic based on the switch port number, and
Egress flow is determined by switch port number and ingress flow is determined for ingress traffic based on the network id and destination MAC address);
by destination mac address. ``register 6`` contains port tag to isolate ``register 6`` contains a value identifying the network (which is also the
connections into separate conntrack zones. OVSDB port tag) to isolate connections into separate conntrack zones.
:: ::
@ -183,16 +218,17 @@ connections into separate conntrack zones.
table=60, priority=90,dl_vlan=0x284,dl_dst=fa:16:3e:8c:84:14 actions=load:0x2->NXM_NX_REG5[],load:0x284->NXM_NX_REG6[],resubmit(,81) table=60, priority=90,dl_vlan=0x284,dl_dst=fa:16:3e:8c:84:14 actions=load:0x2->NXM_NX_REG5[],load:0x284->NXM_NX_REG6[],resubmit(,81)
table=60, priority=0 actions=NORMAL table=60, priority=0 actions=NORMAL
Following ``table 71`` implements arp spoofing protection, ip spoofing The following table, |table_71| implements ARP spoofing protection, IP spoofing
protection, allows traffic for obtaining ip addresses (dhcp, dhcpv6, slaac, protection, allows traffic related to IP address allocations (dhcp, dhcpv6,
ndp) for egress traffic and allows arp replies. Also identifies not tracked slaac, ndp) for egress traffic, and allows ARP replies. Also identifies not
connections which are processed later with information obtained from tracked connections which are processed later with information obtained from
conntrack. Notice the ``zone=NXM_NX_REG6[0..15]`` in ``actions`` when obtaining conntrack. Notice the ``zone=NXM_NX_REG6[0..15]`` in ``actions`` when obtaining
information from conntrack. It says every port has its own conntrack zone information from conntrack. It says every port has its own conntrack zone
defined by value in ``register 6``. It's there to avoid accepting established defined by the value in ``register 6`` (OVSDB port tag identifying the network).
traffic that belongs to different port with same conntrack parameters. It's there to avoid accepting established traffic that belongs to different
port with same conntrack parameters.
The very first rule in ``table 71`` is a rule removing conntrack information The very first rule in |table_71| is a rule removing conntrack information
for a use-case where Neutron logical port is placed directly to the hypervisor. for a use-case where Neutron logical port is placed directly to the hypervisor.
In such case kernel does conntrack lookup before packet reaches Open vSwitch In such case kernel does conntrack lookup before packet reaches Open vSwitch
bridge. Tracked packets are sent back for processing by the same table after bridge. Tracked packets are sent back for processing by the same table after
@ -218,7 +254,7 @@ solicitation and neighbour advertisement.
table=71, priority=95,icmp6,reg5=0x2,in_port=2,icmp_type=135 actions=resubmit(,94) table=71, priority=95,icmp6,reg5=0x2,in_port=2,icmp_type=135 actions=resubmit(,94)
table=71, priority=95,icmp6,reg5=0x2,in_port=2,icmp_type=136 actions=resubmit(,94) table=71, priority=95,icmp6,reg5=0x2,in_port=2,icmp_type=136 actions=resubmit(,94)
Following rules implement arp spoofing protection Following rules implement ARP spoofing protection
:: ::
@ -241,7 +277,7 @@ instances.
table=71, priority=70,udp,reg5=0x2,in_port=2,tp_src=67,tp_dst=68 actions=resubmit(,93) table=71, priority=70,udp,reg5=0x2,in_port=2,tp_src=67,tp_dst=68 actions=resubmit(,93)
table=71, priority=70,udp6,reg5=0x2,in_port=2,tp_src=547,tp_dst=546 actions=resubmit(,93) table=71, priority=70,udp6,reg5=0x2,in_port=2,tp_src=547,tp_dst=546 actions=resubmit(,93)
Flowing rules obtain conntrack information for valid ip and mac address Flowing rules obtain conntrack information for valid IP and MAC address
combinations. All other packets are dropped. combinations. All other packets are dropped.
:: ::
@ -257,14 +293,14 @@ combinations. All other packets are dropped.
table=71, priority=0 actions=drop table=71, priority=0 actions=drop
``table 72`` accepts only established or related connections, and implements |table_72| accepts only established or related connections, and implements
rules defined by the security group. As this egress connection might also be an rules defined by security groups. As this egress connection might also be an
ingress connection for some other port, it's not switched yet but eventually ingress connection for some other port, it's not switched yet but eventually
processed by ingress pipeline. processed by the ingress pipeline.
All established or new connections defined by security group rule are All established or new connections defined by security group rules are
``accepted``, which will be explained later. All invalid packets are dropped. ``accepted``, which will be explained later. All invalid packets are dropped.
In case below we allow all icmp egress traffic. In the case below we allow all ICMP egress traffic.
:: ::
@ -273,10 +309,10 @@ In case below we allow all icmp egress traffic.
table=72, priority=50,ct_state=+inv+trk actions=resubmit(,93) table=72, priority=50,ct_state=+inv+trk actions=resubmit(,93)
Important on the flows below is the ``ct_mark=0x1``. Such value have flows that Important on the flows below is the ``ct_mark=0x1``. Flows that
were marked as not existing anymore by rule introduced later. Those are were marked as not existing anymore by rule introduced later will value this
typically connections that were allowed by some security group rule and the value. Those are typically connections that were allowed by some security group
rule was removed. rule and the rule was removed.
:: ::
@ -293,7 +329,7 @@ allowed.
table=72, priority=50,ct_state=-new-est+rel-inv,ct_zone=644,ct_mark=0,reg5=0x1 actions=resubmit(,94) table=72, priority=50,ct_state=-new-est+rel-inv,ct_zone=644,ct_mark=0,reg5=0x1 actions=resubmit(,94)
table=72, priority=50,ct_state=-new-est+rel-inv,ct_zone=644,ct_mark=0,reg5=0x2 actions=resubmit(,94) table=72, priority=50,ct_state=-new-est+rel-inv,ct_zone=644,ct_mark=0,reg5=0x2 actions=resubmit(,94)
In the following flows are marked established connections that weren't matched In the following, flows are marked established connections that weren't matched
in the previous flows, which means they don't have accepting security group in the previous flows, which means they don't have accepting security group
rule anymore. rule anymore.
@ -305,10 +341,10 @@ rule anymore.
table=72, priority=40,ct_state=+est,reg5=0x2 actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[])) table=72, priority=40,ct_state=+est,reg5=0x2 actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
table=72, priority=0 actions=drop table=72, priority=0 actions=drop
In following ``table 73`` are all detected ingress connections sent to ingress In following |table_73| are all detected ingress connections sent to ingress
pipeline. Since the connection was already accepted by egress pipeline, all pipeline. Since the connection was already accepted by egress pipeline, all
remaining egress connections are sent to normal flood'n'learn switching remaining egress connections are sent to normal flood'n'learn switching
(table 94). in |table_94|.
:: ::
@ -322,8 +358,8 @@ remaining egress connections are sent to normal flood'n'learn switching
table=73, priority=80,reg5=0x2 actions=resubmit(,94) table=73, priority=80,reg5=0x2 actions=resubmit(,94)
table=73, priority=0 actions=drop table=73, priority=0 actions=drop
``table 81`` is similar to ``table 71``, allows basic ingress traffic for |table_81| is similar to |table_71|, allows basic ingress traffic for
obtaining ip address and arp queries. Note that vlan tag must be removed by obtaining IP address and ARP queries. Note that vlan tag must be removed by
adding ``strip_vlan`` to actions list, prior to injecting packet directly to adding ``strip_vlan`` to actions list, prior to injecting packet directly to
port. Not tracked packets are sent to obtain conntrack information. port. Not tracked packets are sent to obtain conntrack information.
@ -353,11 +389,11 @@ port. Not tracked packets are sent to obtain conntrack information.
table=81, priority=80,ct_state=+trk,reg5=0x2 actions=resubmit(,82) table=81, priority=80,ct_state=+trk,reg5=0x2 actions=resubmit(,82)
table=81, priority=0 actions=drop table=81, priority=0 actions=drop
Similarly to ``table 72``, ``table 82`` accepts established and related Similarly to |table_72|, |table_82| accepts established and related
connections. In this case we allow all icmp traffic coming from connections. In this case we allow all ICMP traffic coming from
``security group 1`` which is in this case only ``port 1``. ``security group 1`` which is in this case only ``port 1``.
The first four flows match on the ip addresses, and the The first four flows match on the IP addresses, and the
next two flows match on the icmp protocol. next two flows match on the ICMP protocol.
These six flows define conjunction flows, and the next two define actions for These six flows define conjunction flows, and the next two define actions for
them. them.
@ -404,15 +440,15 @@ the security group 2. These flows are from the following security group rules,
:: ::
- can receive tcp packets from security group 1 - can receive TCP packets from security group 1
- can receive tcp packets to port 80 from security group 2 - can receive TCP packets to port 80 from security group 2
and these rules have been processed by ``merge_port_ranges`` into: and these rules have been processed by ``merge_port_ranges`` into:
:: ::
- can receive tcp packets to port != 80 from security group 1 - can receive TCP packets to port != 80 from security group 1
- can receive tcp packets to port 80 from security group 1 or 2 - can receive TCP packets to port 80 from security group 1 or 2
before translating to flows so that there is only one matching flow before translating to flows so that there is only one matching flow
even when the TCP destination port is 80. even when the TCP destination port is 80.
@ -429,10 +465,10 @@ flows, but the corresponding security group rules have different
remote group IDs. Unlike the above TCP example, there's no convenient remote group IDs. Unlike the above TCP example, there's no convenient
way of expressing ``protocol != TCP`` or ``icmp_code != 1``. So the way of expressing ``protocol != TCP`` or ``icmp_code != 1``. So the
OVS firewall uses a different priority than the previous TCP flows so OVS firewall uses a different priority than the previous TCP flows so
as not to mix up them. as not to mix them up.
The mechanism for dropping connections that are not allowed anymore is the The mechanism for dropping connections that are not allowed anymore is the
same as in ``table 72``. same as in |table_72|.
:: ::
@ -449,40 +485,53 @@ same as in ``table 72``.
table=82, priority=0 actions=drop table=82, priority=0 actions=drop
Note: Conntrack zones on a single node are now based on network to which port is .. note::
plugged in. That makes a difference between traffic on hypervisor only and
east-west traffic. For example, if port has a VIP that was migrated to a port on
different node, then new port won't contain conntrack information about previous
traffic that happened with VIP.
Using OpenFlow in conjunction with OVS firewall Conntrack zones on a single node are now based on the network to which
----------------------------------------------- a port is plugged in. That makes a difference between traffic on hypervisor
only and east-west traffic. For example, if a port has a VIP that was
migrated to a port on a different node, then the new port won't contain
conntrack information about previous traffic that happened with VIP.
There are three tables where packets are sent once they get through the OVS OVS firewall integration points
firewall pipeline. The tables can be used by other mechanisms using OpenFlow -------------------------------
that are supposed to work with the OVS firewall. Packets sent to ``table 91``
are considered accepted by the egress pipeline, and will be processed so that
they are forwarded to their destination by being submitted to a NORMAL action
that results in Ethernet flood/learn processing. Note that ``table 91`` merely
resubmits to ``table 94``that contains the actual NORMAL action; this allows
to have``table 91`` be a single places where the NORMAL action can be overriden
by other components (currently used by ``networking-bagpipe`` driver for
``networking-bgpvpn``).
Packets sent to ``table 92`` were processed by the ingress filtering pipeline. There are three tables where packets are sent once after going through the OVS
As packets from the ingress filtering pipeline were injected to its firewall pipeline. The tables can be used by other mechanisms that are supposed
destination, ``table 92`` receives copies of those packets and therefore to work with the OVS firewall, typically L2 agent extensions.
default action is ``drop``. Finally, packets sent to ``table 93`` were filtered
by the firewall and should be dropped. Default action is ``drop`` in this
table.
In regard to the performance perspective, please note that only the first Egress pipeline
accepted packet of each connection session will go to ``table 91`` and ~~~~~~~~~~~~~~~
``table 92``.
Packets are sent to |table_91| and |table_94| when they are considered accepted
by the egress pipeline, and they will be processed so that they are forwarded
to their destination by being submitted to a NORMAL action, that results in
Ethernet flood/learn processing.
Two tables are used to differentiate between the first packets of a connection
and the following packets. This was introduced for performance reasons to
allow the logging extension to only log the first packets of a connection.
Only the first accepted packet of each connection session will go to |table_91|
and the following ones will go to |table_94|.
Note that |table_91| merely resubmits to |table_94| that contains the actual
NORMAL action; this allows to have a single place where the NORMAL action can
be overridden by other components (currently used by ``networking-bagpipe``
driver for ``networking-bgpvpn``).
Ingress pipeline
~~~~~~~~~~~~~~~~
The first packet of each connection accepted by the ingress pipeline is sent
to |table_92|. The default action in this table is DROP because at this point
the packets have already been delivered to their destination port. This
integration point is essentially provided for the logging extension.
Packets are sent to |table_93| if processing by the ingress filtering
concluded that they should be dropped.
Upgrade path from iptables hybrid driver Upgrade path from iptables hybrid driver
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ----------------------------------------
During an upgrade, the agent will need to re-plug each instance's tap device During an upgrade, the agent will need to re-plug each instance's tap device
into the integration bridge while trying to not break existing connections. One into the integration bridge while trying to not break existing connections. One
@ -505,3 +554,15 @@ firewall rules and finally remove the drop rules for the instance.
use the OVS firewall, and instances from other nodes can be live-migrated to use the OVS firewall, and instances from other nodes can be live-migrated to
it. Once the first node is evacuated, its firewall driver can be then be it. Once the first node is evacuated, its firewall driver can be then be
switched to the OVS driver. switched to the OVS driver.
.. |table_0| replace:: ``table 0`` (LOCAL_SWITCHING)
.. |table_60| replace:: ``table 60`` (TRANSIENT)
.. |table_71| replace:: ``table 71`` (BASE_EGRESS)
.. |table_72| replace:: ``table 72`` (RULES_EGRESS)
.. |table_73| replace:: ``table 73`` (ACCEPT_OR_INGRESS)
.. |table_81| replace:: ``table 81`` (BASE_INGRESS)
.. |table_82| replace:: ``table 82`` (RULES_INGRESS)
.. |table_91| replace:: ``table 91`` (ACCEPTED_EGRESS_TRAFFIC)
.. |table_92| replace:: ``table 92`` (ACCEPTED_INGRESS_TRAFFIC)
.. |table_93| replace:: ``table 93`` (DROPPED_TRAFFIC)
.. |table_94| replace:: ``table 94`` (ACCEPTED_EGRESS_TRAFFIC_NORMAL)

1
neutron/plugins/ml2/drivers/openvswitch/agent/common/constants.py
View File

@ -75,7 +75,6 @@ OVS_FIREWALL_TABLES = (
ACCEPTED_EGRESS_TRAFFIC_TABLE = 91 ACCEPTED_EGRESS_TRAFFIC_TABLE = 91
ACCEPTED_INGRESS_TRAFFIC_TABLE = 92 ACCEPTED_INGRESS_TRAFFIC_TABLE = 92
DROPPED_TRAFFIC_TABLE = 93 DROPPED_TRAFFIC_TABLE = 93
ACCEPTED_EGRESS_TRAFFIC_NORMAL_TABLE = 94 ACCEPTED_EGRESS_TRAFFIC_NORMAL_TABLE = 94
# --- Tunnel bridge (tun_br) # --- Tunnel bridge (tun_br)