Since iptables-restore doesn't support --dport with protocol vrrp,
it errors out setting the security groups on the hypervisor.
Marking this a partial fix, since we need a change to prevent
adding those incompatible rules in the first place, but this
patch will stop the bleeding.
Change-Id: If5e557a8e61c3aa364ba1e2c60be4cbe74c1ec8f
Partial-Bug: #1818385
(cherry picked from commit 8c213e4590)
The firewall won't attempt on update to initialize port in case
port hasn't been initialized by sg_agent yet. This fixes a race where update
rpc call arrives between wiring tap device with integration bridge and
firewall initialization.
Change-Id: Ice0667df606ae23061acebceea23ab6e49dadbcf
Closes-bug: #1740885
(cherry picked from commit ed57c3de42)
Dnsmasq driver used by dhcp agent has restart() method which is
calling disable() and then enable() dnsmasq process again.
What can be observed in functional tests from time to time it may
happen that start dnsmasq process will be called before old process
is really down. That leads to error that IP address to which
dnsmasq wants to bind is already in use and it fails to start.
This patch adds possibility to call disable() method with block flag
set to True. In such case driver will ensure in disable() method that
process is really not active.
This blocking disable() is used in restart() method now.
Change-Id: I419a451633badbc3d32edcee1945fca3e3d9f6be
Closes-Bug: #1811126
(cherry picked from commit d471a85931)
Currently any dhcp agent instance will work as an open resolver. For
deployments using publicly routed addresses for tenant networks, this
allows the agent being abused in dDoS attacks, see [1].
By setting the `--local-service` option dnsmasq will filter DNS queries
and reply only to queries from directly attached networks.
[1] https://bugs.launchpad.net/neutron/+bug/1501206
Conflicts:
neutron/cmd/sanity_check.py
Closes-Bug: 1501206
Change-Id: I76d810aad2ce0f15a88bd798963012fa0efca74e
(cherry picked from commit 0fce3ca2c1)
AsyncProcess.stop() method has now additional parameter
kill_timeout. If this is set to some value different than
None, eventlet.green.subprocess.Popen.wait() will be called
with this timeout, so TimeoutExpired exception will be raised
in case if process will not be killed for this "kill_timeout"
time.
In such case process will be killed "again" with SIGKILL signal
to make sure that it is gone.
This should fix problem with failing fullstack tests, when
ovs_agent process is sometimes not killed and test timeout was
reached in this wait() method.
Conflicts:
neutron/agent/linux/async_process.py
Change-Id: I1e12255e5e142c395adf4e67be9d9da0f7a3d4fd
Closes-Bug: #1798472
(cherry picked from commit 9b23abbdb6)
Merge the system protocol assignments into the iptables name
to protocol mapping array, IPTABLES_PROTOCOL_NAME_MAP, so that
systems with updated or new values in /etc/protocols can
successfully install iptables rules.
This was done as an IptablesFirewallDriver() instance mapping
since there is typically only a single instance per-agent,
and it also allows us to more easily unit test it.
Conflicts:
neutron/tests/unit/agent/linux/test_iptables_firewall.py
Change-Id: Ib73def4e2a9e3644462fdee312768382fcb800a5
Closes-Bug: #1783378
(cherry picked from commit 034db863a0)
It turns out that in environments with a big number of VMs, sometimes
the neutron dhcp agent fails to read the dhcp lease file because some
lines with the ipv4/ipv6 entries don't have enough fields and causes the
dhcp agent to fail.
When this happens the agent calls sync_state to
fully resync the agent state, that causes a serious performance problems
in scale environments.
We need to be more robust reading the file to handle these exceptions.
Co-authored-by: stephen-ma
Partial-Bug: #1788556
Change-Id: Ia681a5e929df5bf8c97ae9445876c306c34061b5
(cherry picked from commit 8a3ff8a19e)
Without this flag, dnsmasq prefers to ask the servers that
are known to be up, rather than hitting servers that are either
down or known to be broken. This greatly reduces responsivity
impact of broken upstream servers.
Closes-Bug: #1746000
Change-Id: Ieee4dafc578c3bda0935fcdb80faad6c342a10e9
(cherry picked from commit d3c69dc4f2)
Sometimes calls to dhcp_release(6) do not result in removal
of a lease from the leases file, for example, when the release
packet is not received by dnsmasq. Trying more than once is
recommended in this case.
Instead of blindly trying some number of times, we monitor
the lease file contents, and retry the dhcp_release(6) call
when an entry still remains. This is possible since
dhcp_release(6) is being run from the DHCP server itself.
We try three times and wait 0.3 seconds between tries.
We also now check for any stale leases in the leases file
that are unknown to neutron, also trying to remove them.
Change-Id: Ic1864f7efbc94db1369ac7f3e2879fda86f95a11
Closes-bug: #1764481
Closes-bug: #1783908
(cherry picked from commit fab032b426)
Patch [1] added configuration of forward rule for trusted ports in
iptables firewall driver.
This patch fixes issue with many "duplicate iptables rule detected"
warning messages due to try to add such forward rule each time when
trusted port is updated.
Now such rule is added only once for port.
[1] https://review.openstack.org/#/c/525607/
Change-Id: Ib816887f07f16b6ac865bb81d0f27f12d0b47dfb
Closes-Bug: #1754770
(cherry picked from commit 8be0c2a551)
There was missing mock of
ipv6_utils.is_enabled_and_bind_by_default() in BridgeLibTest
unit tests and that cause failing some of tests from this module
when tests are running on host with disabled IPv6.
Now it's mocked and tests are running properly and are
testing what they should test.
Closes-Bug: #1773818
Change-Id: I9144450ce85e020c0e33c5214a2178acbbbf5f54
(cherry picked from commit 8930d33c71)
SG with hybrid-iptables driver uses per port conntrack zones.
FWaaS port security uses per network conntrack zones based on
local vlans assigned by ovs l2 agent.
In case both SG iptables-hybrid driver and FWaaS port security is enabled,
there is a posibility of iptables-hybrid and OVS based FWaaS driver
allocating overlapping zone and creating security holes.
This patch changes the zone allocation range for iptables and
hybrid_iptables driver to 4097 - 65535. While OVS based
port security driver can use zones based on local vlan range 1 - 4096
Closes-Bug: #1745642
Change-Id: I4d51637ed1de8fe85b4982a03410d4a3f637ea3f
(cherry picked from commit 9a620f6ea5)
Ironic is sending 'client-id' as a number (61) to meet rfc4776 [1],
due to this change Neutron DHCP should support this option as well.
[1] 228a2a7885
Closes-Bug: #1770932
Change-Id: I9728354d5f9e08a0dc23900b2bc22b4a0aedb737
(cherry picked from commit 59bc19c14a)
If a distributed router has interfaces on multiple tenant networks, with
'fast exit' functionality policy based rules are created in qrouter
namespace for every tenant network subnet and 'from <cidr>' is included
into an 'ip rule' command invocation.
When a port on a tenant network is deleted 'from <cidr>' part is not
included and a first rule matching specified parameters gets deleted.
For example with the following layout
ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.100.0/24 lookup 16
80000: from 192.168.200.0/24 lookup 16
and neutron l3 agent will use this command
ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip -4 rule\
del priority 80000 table 16 type unicast
and 192.168.100.0/24 rule will get deleted even if you actually removed
a port on 192.168.200.0.
This results in an extra rule present and not cleaned up and the right
rule removed. It is only recreated if a router is disabled and enabled
again.
additional changes:
1) Floating IP rules are identified by priority only as implemented
currently - for this reason this change adds fixed_ip to the rule
removal code. Rule priorities are 32-bit values in iproute2 so,
in theory, those should be not be used to cover IPv6.
2) IP protocol information for 'from all' rules is currently
derived from link-local address IP version. The same approach
is preserved by using version-specific /0 addresses without
changing the API provided by ip_lib.
Change-Id: I0ea6dddd26e17771be223a1fbdf21792c90f3e9c
Closes-Bug: #1759956
(cherry picked from commit 81db328b2d)
Allowed_address_pair IP when associated with a network port will
inherit the services MAC.
Right now the ARP entry is updated with the last MAC that it is
associated with. But when allowed_address_pair IPs are used in
the context of VRRP the MAC keeps switching between the MASTER
and SLAVE. VRRP instance sends out GARP, but the ARP entry in the
router namespace is not getting updated based on the GARP.
This might cause the VRRP IP and the service using the IP to fail.
Since we having been adding the ARP entry with NUD state as
PERMANENT, the ARP entries are set for ever and does not adopt the
GARP sent out by the VRRP instance.
This will cause instances associated with DVR routers to have a
service interruption.
So the proposed patch will add the ARP entry for the Allowed address
pair with NUD for 'REACHABLE'.
This allows the Allowed_address_pair IP MAC to be updated on the
fly.
Change-Id: I43c3471f5d259e8c2ee1685398a06a4680c0bfcd
Closes-Bug: #1608400
(cherry-picked from commit fbe308bdc1)
When a security group rule is created with icmp/ipv6-icmp alias such as
protocol number 1(ICMP), 58(ICMPv6) or string icmpv6(legacy name for
ipv6-icmp) as its protocol along with ICMP/ICMPv6 message type
specified, _port_arg will generate a wrong str for iptables/ip6tables.
Change-Id: Iae01b9a0da34797a5f061a110f06e18be9bbec5a
Closes-Bug: #1743552
(cherry picked from commit 0efe1aec18)
When the support of protocol numbers in security groups
was fixed in [1], it introduced two deficiencies in the
iptables code:
- it was missing some protocols, for example, 'icmp', 'tcp'
and 'udp', so when rules were added by number we did not
use their name as iptables expects
- it used a dictionary to map numbers to names, but protocol
numbers are stored as strings (i.e. '1' != 1)
Updated the iptables number mapping dict to have all
currently-known values, even those that are already well-known
and should have been using a string instead of a number.
Also changed the iptables number mapping dict to use
strings as the keys instead of numbers, since that's
what will be passed from the security group code.
Removed IPTABLES_PROTOCOL_MAP as it lives in neutron-lib,
and accidentally snuck-in in [1].
[1] I5895250b47ddf664d214cf085be693c3897e0c87
Change-Id: I6b7575eb531b4f35579960c3feb47000cd259b86
Closes-Bug: 1719711
(cherry picked from commit 37bd42e4f5)
This commit adds common_agent_extension class which is agent API
for L2 extension drivers used e.g. by Linuxbridge agent.
This is necessary to be able to use instance of iptables_manager
used in firewall driver also in L2 extension drivers (like qos).
This patch refactors little bit iptables_manager code to make possible
to initialize e.g. mangle or nat table on demand, even if iptables
is created as "state_less"
Change-Id: I3b66e49b7f176124e8aea3eb96d0d465f1ab1ea0
Closes-Bug: #1736674
(cherry picked from commit cbee0f9f88)
If callers of get_devices_with_ip(), or
device.addr.list(to=address) pass an ip_cidr, it
could match any ip_cidr in that range on the interface.
Callers need to pass the IP without the prefix portion in
order to match it exactly. Added a helper utility to
strip the cidr part from a ip_cidr.
Determined the unit test for this can't actually check
this case since we are mocking the return value from
/sbin/ip, so modified it to just make sure the dict
is correct.
Added a functional test that adds two IP addresses in
the same IP range to verify that we actually filter
correctly when a 'to=IP' is specified.
Change-Id: I3a95b3bb72a43f322ad23892d8959398aac22a1c
Closes-bug: #1728080
(cherry picked from commit 7b8289253c)
The OpenFlow spec says packets shouldn't match against multiple flows
at the same priority or the result is undefined. In ovsfw, 8 priority
levels are needed to comply with this rule.
Note: unlike overlapping TCP port ranges cases, the current version
of OVS seems to handle this case magically.
Change-Id: I6deaee8dbe81453285b1fc685282952bc9456949
Closes-bug: #1708092
(cherry picked from commit 4ac4c22a64)
The ovsfw code generated multiple flows with the same or overlapping
match fields and different actions=conjunction(nnn,2/2) flows.
Merge such flows and generate only one flow with
actions=conjunction(mmm,2/2),conjunction(nnn,2/2) so that filtering
are correctly performed.
Change-Id: I0cd325b02f35e103606595b8b124010fff8dc397
Partial-bug: #1708092
(cherry picked from commit 237ec30ca9)
The default wait-interval for iptables-restore when
using -w is 1 second between tries. On a busy system
that could mean we timeout before we get the lock. Try
5 times per second instead by using -W 200000.
Change-Id: I8307db20187516be781e37c191d8f09a9a8e3dc3
Related-bug: #1712185
(cherry picked from commit 46081445d6)
In the case where we called iptables-restore with a
-w argument and it succeeded, we should short-circuit
future calls to always use -w, instead of trying
without it, just to fall-back to using it on failure.
While analyzing some l3-agent log files I have seen
lots of "Perhaps you want to use the -w option?",
followed by a call with -w, followed by not using it
the next time. Changing this can save one failing
call to iptables-restore.
Change-Id: Icac99eb1d43648c64b6beaee0d6201f990eacb51
Related-bug: #1712185
(cherry picked from commit 6c50ad5858)
In this case, it's an expected error, and we retry again with -w.
Related-Bug: #1712185
Change-Id: I97bf3032b5cebcbce51a3b3de6cb128ca342bd87
(cherry picked from commit 2f0ffa998a)
Similarly to filtered ports this patch caches so called trusted ports to
avoid processing in case of unknown port is passed down to firewall
driver. The cached ofport is used for removal as the cache reflects
currently installed flows.
The patch also catches exception caused by inconsistency coming from
ovsdb.
Closes-bug: #1707339
Change-Id: I15cdb28072835fcb8c37ae4b56fc8754375a807c
(cherry picked from commit 1d80c960f6)
port_range_min/max should be read as ICMP type/code for ICMP(V6).
Change-Id: I65157037038d11464902c0311eb4c54b84f60c72
Closes-bug: #1708358
(cherry picked from commit b82db0a9f2)
ovsfw ignored port_ranges when a SG rule protocol was sctp or given
in a number rather than a token. This commit fixes that.
Change-Id: I6c810a152990246d42d98c3673c4b5ee126ebb4b
Closes-bug: #1708580
(cherry picked from commit effa12889b)
When user create network with isolated subnet, dhcp agent will
create md-proxy with vrouter id. This will conflict with then md-proxy
created by l3 agent. This patch updated dhcp agent start md-proxy with
vrouter id only when the network has metadata subnet.
Change-Id: I3288327bf9d0cdf759a6fdf365d1289e8b7442db
Closes-Bug: #1703059
(cherry picked from commit 13eea520b5)
Upstream iptables added support for -w ('wait') argument to
iptables-restore. It makes the command grab a 'xlock' that guarantees
that no two iptables calls will mess a table if called in parallel.
[This somewhat resembles what we try to achieve with a file lock we
grab in iptables manager's _apply_synchronized.]
If two processes call to iptables-restore or iptables in parallel, the
second call risks failing, returning error code = 4, and also printing
the following error:
Another app is currently holding the xtables lock. Perhaps you want
to use the -w option?
If we call to iptables / iptables-restore with -w though, it will wait
for the xlock release before proceeding, and won't fail.
Though the feature was added in iptables/master only and is not part of
an official iptables release, it was already backported to RHEL 7.x
iptables package, and so we need to adopt to it. At the same time, we
can't expect any underlying platform to support the argument.
A solution here is to call iptables-restore with -w when a regular call
failed. Also, the patch adds -w to all iptables calls, in the iptables
manager as well as in ipset-cleanup.
Since we don't want to lock agent in case current xlock owner doesn't
release it in reasonable time, we limit the time we wait to ~1/3 of
report_interval, to give the agent some time to recover without
triggering expensive fullsync.
In the future, we may be able to get rid of our custom synchronization
lock that we use in iptables manager. But this will require all
supported platforms to get the feature in and will take some time.
Closes-Bug: #1712185
Change-Id: I94e54935df7c6caa2480eca19e851cb4882c0f8b
(cherry picked from commit a521bf0393)
Current ovsfw implementation does not take care of the different
MACs in allowed_address_pairs with the VM's MAC.
This patch use the following method to fix this issue:
1. Do not check dl_src in table=72 because table=71 has checked
dl_src for Egress.
2. Add all allowed MACs in table=0 and table=73 for Ingress.
3. Do not check dl_dst in table=82 because this check has done
in table=0 and table=73.
4. Delete allowed MACs in table=0 and table=73 when needed.
Change-Id: Iad59096f0c9855ebfd4a0d5b447e73b443d66c1d
Closes-Bug: #1697593
The patch relies on the fact that traffic not going from instance
(and thus port not managed by firewall) is tagged. Traffic coming from
the instance is not tagged and thus net register is used for marking
such traffic. These two approaches make matching rules unique even if
two ports from different networks share its' mac addressess.
Traffic coming from trusted ports is marked with network in registry
so firewall can decide later to which network traffic belongs.
Closes-bug: #1626010
Change-Id: Ia05d75a01b0469a0eaa82ada67b16a9481c50f1c
This change moves the strip_vlan action from RULES_INGRESS/82 to
TRANSIENT_TABLE after the point where the traffic from local VM ports
has been moved to BASE_EGRESS. A reason for this move is that strip_vlan
is only needed for traffic *not* coming from VM ports and coming on a
patch port from br-*, and that it is hence simpler do do the strip_vlan
in TRANSIENT_TABLE rather than in mutliple places that also happen to be
common with traffic from local VMs.
This change also addresses another need:
I16a35b5d6c54901899d24fc94bd3438c1f1be05e results in add_flow being
possibly done with an Openflow version higher than OF1.0. The
strip_action as currently done is not compatible with OF>1.0, because
later versions require matching on dl_vlan first (the "strip vlan if
there is one" behavior of OF1.0 is not supported anymore). For this
reason this change adds a match on dl_vlan for the strip_vlan rule.
Change-Id: I76ee34a614237bbc99989ce9c1b96a30456be282
Commit ce8a0b2b7d introduces a TRANSIENT
table where all traffic local to br-int is sent after it's been
preprocessed by other features using openflow. This patch adopts the
table.
Change-Id: Ic66c186ab73bad6fcd133f2b9d15e07fd0eebb33
Related-bug: #1696983
Introduce vxlan.{udp_srcport_min, udp_srcport_max and udp_dstport}
for setting the port range used for VXLAN communication.
Change-Id: I97999988090eee7aee9b533ac1dad2de95b29cbe
Closes-Bug: #1483853
DocImpact: vxlan.{udp_srcport_min, udp_srcport_max and udp_dstport}
can be used to set UDP port numbers used for VXLAN in LinuxBridge
agent.
In reviews we usually check import grouping but it is boring.
By using flake8-import-order plugin, we can avoid this.
It enforces loose checking so it sounds good to use it.
This flake8 plugin is already used in tempest.
Note that flake8-import-order version is pinned to avoid unexpected
breakage of pep8 job.
Setup for unit tests of hacking rules is tweaked to disable
flake8-import-order checks. This extension assumes an actual file exists
and causes hacking rule unit tests.
Change-Id: Ib51bd97dc4394ef2b46d4dbb7fb36a9aa9f8fe3d
To complete the DHCP support for non-local routed networks via relay
agent the dhcp agent must configure the netmask in the --dhcp-range
option for IPv4 networks. This was previously not required becuase
dnsmasq can determine it from the local interface information for
local networks.
DNSMASQ(8)
For directly connected networks (ie, networks on which the machine
running dnsmasq has an interface) the netmask is optional: dnsmasq
will determine it from the interface configuration. For networks
which receive DHCP service via a relay agent, dnsmasq cannot
determine the netmask itself, so it should be specified, otherwise
dnsmasq will have to guess, based on the class (A, B or C) of the
network address. The broadcast address is always optional. It is
always allowed to have more than one dhcp-range in a single subnet.
Change-Id: I5d609e47a3463c11338dd38aedebfb4a6822f503
Closes-Bug: #1699754
Zuul