Passing newline to the dnsmasq may cause security issues, especially
that in case of Neutron that dhcp options' values are controlled by
cloud users.
This patch removes everything what is after first newline character
in the dhcp_extra_opt's values before passing them to dnsmasq.
Closes-Bug: #1939733
Change-Id: Ifeaf258f0b5ea86f25620ac4116d618980a7272e
(cherry picked from commit df891f0593)
Because the DHCP agent does not know the IAID (identity association
identifier) of assigned IPv6 addresses it's not possible to generate the
lease file including IPv6 leases. Because of this IPv6 addresses are
excluded when generating the lease file in case of DHCP agent restarts.
This causes DHCPv6 clients to fail to RENEW their lease and to go
through a full address discovery cycle with possible short connectivity
disruption.
This commit copies the existing IPv6 leaes from an already existing
lease file if present. While this does not allow for DHCP agent
failover, this is still better than just skipping the IPv6 addresses.
A lease file without the IPv6 addresses is still generated if an agent
is migrated to a different host.
This commit complements the fix implemented in
Ib1b2f284ab81f1c4af7b08b5257b45a3f6e79c3e which just skips the IPv6
leases as otherwise the lease file would be invalid and all leases would
be lost. It does not change the behavior for still valid IPv4 leases.
With this issue fixed an additional fix is required to not loose DHCPv6
leases when the agent restarts dnsmasq. Currently the DHCP agent
regenerates all configuration files on restart. This means that DHCPv6
leases are lost as they can't be regenerated. This changes the agent to
only delete the config files if the agent's ports are also removed.
Closes-Bug: #1722126
Related-Change: Ib1b2f284ab81f1c4af7b08b5257b45a3f6e79c3e
This backport contains some test fixes to make the tests work with
Python 2.7 and the PEP8 N322 check.
Changed file: neutron/tests/unit/agent/linux/test_dhcp.py
Change-Id: I40761b30563749251b9d74731bbe7a80a124da89
(cherry picked from commit 6bc1c00d66)
Neighbor Advertisments are used to inform other machines of the MAC
address to use to reach an IPv6. This commits prevents VMs from
pretending they are assigned IPv6 they should not use.
It also prevents sending UDP packets with spoofed IP or MAC even using
DHCP(v6) request ports.
Co-authored-by: David Sinquin <david.sinquin@gandi.net>
Closes-bug: #1902917
Conflicts:
neutron/agent/linux/openvswitch_firewall/firewall.py
Change-Id: Iffb6643359562487414460f5a7e19a7fae9f935c
(cherry picked from commit ca7822e210)
Ports with device_owner like:
* floating_ip,
* DHCP,
* some types of router ports, like: HA interface interface,
don't need to be configured in the dnsmasq file.
So there is no need to reload dnsmasq every time when such port is
added/updated to the network.
This patch adds skip in such case which should improve load on the
Neutron DHCP agent.
Conflicts:
neutron/agent/linux/dhcp.py
Closes-Bug: #1913269
Change-Id: I63221507713b941c261cdf88781133149da8ab8d
(cherry picked from commit e4bbeee206)
When a HA router instance changes the state (active, backup), the
GW interface is set to up or down. As reported in the bug, while
keepalived is configuring the interface, the interface disappears
and appears again from the kernel namespace, as seen in the udev
messages.
This patch is a workaround until the real issue is addressed (if
possible), retrying the interface configuration for a small period
of time.
Related-Bug: #1916024
Change-Id: I8ced69f4f8e7d7c73da130a57e89e9d66590390b
(cherry picked from commit 662f483120)
Both files cannot be deleted with the default permissions because
those files are created by the "root" user.
Conflicts:
neutron/agent/metadata/driver.py
neutron/tests/functional/agent/l3/framework.py
neutron/tests/unit/agent/dhcp/test_agent.py
Change-Id: I73dd37b3104fac8d3172f520f71cffd85d040c4b
Closes-Bug: #1907695
(cherry picked from commit 0a0f647ea0)
As reported in LP#1896734, there is a limit in the size of information
that can be transmitted in one single message between an application
and the privsep daemon. The read socket buffer is limited in size;
a message exceeding this size will generate an exception.
In order to limit the amount of information to be sent, this patch
improves the performance of "get_devices_with_ip". In the previous
implementation, the whole list of network devices from a namespace
was retrieved. In some environments, the list of devices could be
so big that the list returned by "privileged.get_link_devices" can
exceed the read buffer size (as reported in the LP bug when the
OVS agent tries to retrieve the list of IP addresses in the system).
Now the function calls "privileged.get_ip_addresses", that returns
a much smaller list. This patch is also reducing the number of system
calls to just one; the previous implementation was retrieving first
the devices link information list (that method was returning a much
bigger blob) and then, per device, retrieving the IP address
information.
Conflicts:
neutron/tests/functional/agent/linux/test_ip_lib.py
Change-Id: I97ada62484023b9833ed12afd68eb4c8d337fd1f
Related-Bug: #1896734
(cherry picked from commit a6cdf273c0)
Before that patch it was possible to send ICMPv6 packets like e.g.
neutron_lib.constants.ICMPV6_TYPE_MLD_QUERY,
neutron_lib.constants.ICMPV6_TYPE_RS,
neutron_lib.constants.ICMPV6_TYPE_NS,
neutron_lib.constants.ICMPV6_TYPE_NA
And that could cause some security issues as instance could advertise
that it owns IPv6 address which really don't belong to it.
Now rules in table=71 which allows that traffic are "per mac/ipaddress"
and are allowed only for fixed ips allocated to port and port's
allowed_address_pairs.
Closes-Bug: #1902917
Change-Id: I4749fdc6a6cabd253b971bf4010ff76f5593c59c
(cherry picked from commit 4b5bcff64c)
In case when security group is removed from port, connections
established to/from port which were hitting some of the removed SG
rules will be marked with CT_MARK_INVALID in the conntrack table.
Due to that such existing connections will be stopped.
But then if SG will be added again to the port, those conntrack entries
marked as invalid will not be removed and those hang connections will
not be revived.
To fix that, this patch adds deletion of the conntrack entries marked
with CT_MARK_INVALID and related to the port of which SG are
updated/added.
Closes-Bug: #1915530
Change-Id: I84718b8ac4d8d6e39426e1c4485c2d7fe28185dd
(cherry picked from commit 1e220d7136)
Reinitialize conj_ip_manager when OVS is restarted.
Closes-Bug: #1912651
Change-Id: I5b5cf563f0c3d6ea352303f76323177cffc57c6e
(cherry picked from commit 412160b97f)
To stop a "keepalived" process, a SIGTERM signal is sent. This patch
checks if the process is not present in the system. If the instance
is still running after a short time, a SIGKILL signal is sent.
Conflicts:
neutron/tests/unit/agent/linux/test_keepalived.py
Change-Id: I2e784ea7e00c145135288bf309bb34ce311ac15c
Closes-Bug: #1908057
(cherry picked from commit 2618acff51)
Currently when deleting a remote-group's member IPs, the deleted IPs'
conjunctive flows are not cleaned up in OF tables. This is because
the conjunctive flows' cookies don't match with the OVSBridge default
cookie used by the delete flow method. This patch fixed the issue by
using an ANY cookie that can always match with the cookies of the
conjunctive flows.
Change-Id: I74916acf8311989dca267f23261ec4cf449a6abf
Closes-Bug: 1907491
(cherry picked from commit f4b64e519c)
Formatting error causes exception in certain circumstances; fix is
trivial.
Co-Authored-By: captainmish@gmail.com
Change-Id: I2bd7f2f3d798228fee6ba26d0e1a32e3033e9fc1
Closes-Bug: #1898789
(cherry-picked from commit 24cd8598dc)
In RULES_INGRESS_TABLE table 82 there is a rule for allow established and
related connections. The current rule sends the packet directly to the dest
port without doing a mac learning. This is causing ovs to age out the dest mac
of the remote VM and causing the rule to be changed in flood rule. For the normal
case it fine as they try to avoid high cpu. ovs hardware offload reduce cpu usage
by moving some of the packet processing to nic and flood rule is not offloaded,
therefore it prefre to use the NORMAL action to avoid the flood rule.
We also keep the same logic as today when using explicitly_egress_direct=True
which avoid NORMAL action in the entire pipeline.
Closes-Bug: #1897637
Change-Id: I9b611d62be5d0529e8b35e3d8280baa5be54bc2b
(cherry picked from commit 8fc80b7e13)
In "IpAddrCommand.list" method, the "scope" parameter is a string
("link", "site", "global" or "host"). This method will retrieve all
devices with an IP address calling "ip_lib.get_devices_with_ip".
Since [1], "ip_lib.get_devices_with_ip" makes the conversion of
"scope" string parameter to pyroute2 format (see
"pyroute2.netlink.rtnl.rtscopes"). The list command should skip then
the previous conversion.
Closes-Bug: #1899141
[1]https://review.opendev.org/#/c/747406/
Change-Id: I55a0f4341b328af52ea3bd758a72f633fbe3abcb
(cherry picked from commit 0705699e60)
(cherry picked from commit f0849bb0d2)
(cherry picked from commit cd0526dce8)
Passing --conf-file= with no value has no effect on the dnsmasq
process. Intended effect here is for the default system dnsmasq.conf
file not to be read and included in configuring the process. For
that to happen some value has to be passed to --conf-file. Passing
/dev/null will invoke the desired outcome to skip the system
default conf file.
Closes-Bug: #1896945
Change-Id: I22570a44f84d14a792633747c04d7426ab231009
(cherry picked from commit 704576e54e)
Currently codes only support assocate tunnel network and vlan network
to DVR router. This patch add codes that make the flat network assocate
to DVR router and make it work fine.
The patch also remove two unused constant entries: 'FLAT_VLAN_ID' and
'LOCAL_VLAN_ID'
Change-Id: I7d792ce288d96548298f169748565266a130bd86
Closes-Bug: #1876092
(cherry picked from commit cd721a7dcb)
In case when L3 agent is hosting routers which have got subnets
with Prefix Delegation enabled, agent couldn't properly handle
IpAddressAlreadyExists exception raised when pd module tries to
configure link local IPv6 addresses.
Now this is fixed and L3 agent can restart without problems in such
case.
Change-Id: Icc995f7b2b465921e41342711d17539f16ead0ce
Closes-Bug: #1892362
(cherry picked from commit 81d375d39a)
Patch [1] added option "no_track" to the keepalived's config file which
is generated by L3 agent in HA mode.
This was added to handle properly keepalived 2.x and interfaces which
are in DOWN state in the backup nodes.
But this "no_track" option is not compatible with keepalived 1.x series
which is available e.g. on Ubuntu 18.04.
As there is no easy way to check automatically if keepalived supports or
not this config flag, this patch introduces new config option
"keepalived_use_no_track".
If this config option will be set to False, neutron L3 agent will not
add "no_track" to the keepalived's config.
As master branch is moving to gate on Ubuntu 20.04 where keepalived 2.x
is already available, this new config option default value is set to
True.
[1] https://review.opendev.org/#/c/721799/
Conflicts:
neutron/conf/agent/l3/config.py
Change-Id: I2dfdb9f56de28d56ca0f240ff34fa7c3a12e339b
Closes-Bug: #1890400
(cherry picked from commit 7abe0ee34c)
This method is using ip_lib.get_devices_with_ip function to get
IP addresses with scope "link".
Unfortunatelly this method wasn't translating scope names to the pyrout2
values and due to that wasn't returning correct IP addresses.
Now this is fixed and correctl link local IPv6 addresses are returned.
Change-Id: Ia41c1bc627ad2ce89d658ff1fdedee802f6dfa15
Closes-Bug: #1892489
(cherry picked from commit a4e04a7f8b)
When any port in the OVS agent is using a security groups (SG) and
this SG is removed, is marked to be deleted. This deletion process
is done in [1].
The SG deletion process consists on removing any reference of this SG
from the firewall and the SG port map. The firewall removes this SG in
[2].
The information of a SG is stored in:
* ConjIPFlowManager.conj_id_map = ConjIdMap(). This class stores the
conjunction IDS (conj_ids) in a dictionary using the following keys:
ConjIdMap.id_map[(sg_id, remote_sg_id, direction, ethertype,
conj_ids)] = conj_id_XXX
* ConjIPFlowManager.conj_ids is a nested dictionary, built in the
following way:
self.conj_ids[vlan_tag][(direction, ethertype)][remote_sg_id] = \
set([conj_id_1, conj_id_2, ...])
This patch stores all conjuntion IDs generated and assigned to the
tuple (sg_id, remote_sg_id, direction, ethertype). When a SG is
removed, the deletion method will look for this SG in the new storage
variable created, ConjIdMap.id_map_group, and will mark all the
conjuntion IDs related to be removed. That will cleanup those rules
left in the OVS matching:
action=conjunction(conj_id, 1/2)
[1]118930f03d/neutron/agent/linux/openvswitch_firewall/firewall.py (L731)
[2]118930f03d/neutron/agent/linux/openvswitch_firewall/firewall.py (L399)
Conflicts:
neutron/tests/unit/agent/linux/openvswitch_firewall/test_firewall.py
Change-Id: I63e446a30cf10e7bcd34a6f0d6ba1711301efcbe
Related-Bug: #1881157
(cherry picked from commit 0eebd002cc)
(cherry picked from commit ed22f7a2ff)
When add allowed-address-pair 0.0.0.0/0 to one port, it will
unexpectedly open all others' protocol under same security
group. IPv6 has the same problem.
The root cause is the openflow rules calculation of the
security group, it will unexpectedly allow all IP(4&6)
traffic to get through.
For openvswitch openflow firewall, this patch adds a source
mac address match for the allowed-address-pair which has
prefix lenght 0, that means all ethernet packets from this
mac will be accepted. It exactly will meet the request of
accepting any IP address from the configured VM.
Test result shows that the remote security group and
allowed address pair works:
1. Port has 0.0.0.0/0 allowed-address-pair clould send any
IP (src) packet out.
2. Port has x.x.x.x/y allowed-address-pair could be accepted
for those VMs under same security group.
3. Ports under same network can reach each other (remote
security group).
4. Protocol port number could be accessed only when there
has related rule.
Closes-bug: #1867119
Change-Id: I2e3aa7c400d7bb17cc117b65faaa160b41013dde
(cherry picked from commit 00298fe6e8)
The need for this change stems from following issues:
1) When ovs_use_veth = False with ovs-dpdk issue with ovs
was observed - after vswitch restart interface is not comming up.
Meaning ovs-dpdk uses ovs internal ports and it is not able to bring
them up on restart.
2) When ovs_use_veth = True and ovs-dpkd is used, packets sent with
incorrect checksum due to the fact that ovs-dpdk does not do checksum
calculations for veth interface.
This commit allows to use second option and resolve checksum issue by
disabling checksum offload.
Closes-Bug: #1832021
Related-Bug: #1831935
Change-Id: Iecce8d2c6c2c46718cc1020c6e8f914cd4560e4b
(cherry picked from commit 11838a2bc5)
The rate value is converted to bytes per second before being
sent to Pyroute2, but it used the wrong value for the calculations.
This resulted in incorrect rates.
It should be multiplied by 1000 (kbit), not 1024 (Kibit).
The same applies to the burst value (kb).
Change-Id: I70cb1fe651a50b2f6495d7a365a6beb2ba111c6d
Closes-Bug: #1884273
(cherry picked from commit 94d6e38fa0)
(cherry picked from commit 19f60ce728)
This patch fixes:
- The IPv6 tag added in the "host" file if is supported in
dnsmasq. That shifts all other parameters in the register.
- IPv6 registers can have more than one IP address; in this
case, the method "_read_hosts_file_leases" should return a
tuple per IP address.
Change-Id: I4d0bc1eb9448366d8f1b2dacc9c5c2e4e6958253
Closes-Bug: #1884105
(cherry picked from commit 8eb4955bb6)
(cherry picked from commit 68706b556b)
Recent changes in some versions of iproute2 CLI output (v4.18),
have invalidated the regular expression used to parse the
"ip link" output.
To solve this problem and avoid future ones, pyroute2 is used to
retrieve the virtual functions information and set the VF attributes
(spoofcheck, min_tx_rate, max_tx_rate and link_state).
pyroute2 extended the "ip link" support to retrieve this information,
adding "ext_mask=1" in the get command. If no virtual functions are
present in this particular network interface, the added method,
"get_link_vfs", will return an empty list.
The set commands can return a "InterfaceOperationNotSupported" in
case the operation is not supported. For min_tx_rate, if the driver
does not support to set a minimum bandwidth, an "InvalidArgument"
(from a pyroute2.NetlinkError(22)) exception will be raised.
Conflicts:
neutron/tests/unit/agent/linux/test_ip_link_support.py
Change-Id: I680da4f64bd114f1caecaaeedbf8a4b1915a0849
Closes-Bug: #1878042
(cherry picked from commit c5d8fd6329)
(cherry picked from commit 5df65489dc)
In the patch [1] we changed definition of the abstract method
"plug" in the LinuxInterfaceDriver class.
That broke e.g. 3rd-party drivers which still don't accept this
new parameter called "link_up" in the plug_new method.
So this patch fixes this to make such legacy drivers to be still working
with the new base interface driver class.
This commit also marks such definition of the plug_new method as
deprecated. Possibility of using it without accepting link_up parameter
will be removed in the "W" release of the OpenStack.
[1] https://review.opendev.org/#/c/707406/
Change-Id: Icd555987a1a57ca0b31fa7e4e830583d6c69c861
Closes-Bug: #1879307
(cherry picked from commit 30d573d5ab)
In dnsmasq 2.81 there is a regression (see [1] for details).
Prior versions of dnsmasq would select a host record where:
a) no address is present in the host record.
b) an address matching address family of the client request
is present in the host record.
dnsmasq 2.81 will also use a host record where a only an address
not matching the address family of the client request is present.
The same issue is also backported to the dnsmasq-2.79-11.el8.x86_64
which is e.g. in RHEL 8.2 and Centos 8.
dnsmasq version 2.81 also adds support for using tag's on host
records. When a dhcpv6 request is received, dnsmasq automatically
sets the tag 'dhcpv6'.
This change adds a runtime check, testing for dnsmasq host entry
tag support. And adds 'tag:dhcpv6' to all IPv6 host records when
dnsmasq supports this.
Adding the tag makes dnsmasq prefer the tagged host for dhcpv6
requests, i.e it's a workaround fix for the regression issue.
[1] http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2020q2/014051.html
Closes-Bug: #1876094
Change-Id: Ie654c84137914226bdc3e31e16219345c2efaac9
(cherry picked from commit f951871430)
In "ip_lib.ensure_device_is_ready", before retrieving the interface
attributes, a check is done to know if the interface exists. In case
it does not exist, the exception "NetworkInterfaceNotFound" will not
be raised and written in the logs.
Conflicts:
neutron/privileged/agent/linux/ip_lib.py
Change-Id: I4b9fd0885d850601717274a5058e042871211bbb
Closes-Bug: #1854723
(cherry picked from commit 8cc2765b5f)
By default, if no metric is defined, the kernel interprets the
highest value (0).
The current implementation, using pyroute2, is a translation from
the CLI command "ip route". This command uses the netlink API to
communicate with the kernel. In IPv6, when the metric value is not
set is translated as 1024 as default [1].
[1]https://access.redhat.com/solutions/3659171
Change-Id: I0c5f9e320bbbf314a2d6a22c515bf903de84cdaf
Related-Bug: #1855759
(cherry picked from commit 7593f95a74)
Patch [1] introduced new mechanism which only brings UP interfaces
on master node of HA router. It works fine with keepalived 1.x
but it is broken when keepalived 2.x was used (e.g. on Centos 8) as
in this new version of keepalived by default all interfaces of VIPs
and routes are tracked, and if one of them is DOWN, keepalived is
going to FAULT state. Because of that router will never be
transitioned to MASTER on any node.
This patch fixes it by adding "no_track" option to all VIPs
and routes in keepalived's config file.
This "no_track" option isn't added to ha interface so this one
is still tracked by keepalived.
[1] https://review.opendev.org/#/c/707406/
Closes-bug: #1874211
Change-Id: Ic16cf83fe1d1576d91047adb2d4f9e07d57185b6
(cherry picked from commit dc9084a8ec)
"keepalived_state_change" monitor does not use eventlet but normal
Python threads. When "send_ip_addr_adv_notif" is called from inside
the monitor, the arping command is never sent because the eventlet
thread does not start. In order to be able to be called from this
process, this method should also have an alternative implementation
using "threading".
"TestMonitorDaemon.test_new_fip_sends_garp" is also modified to
actually test the GARP sent. The test was originally implemented with
only one interface in the monitored namespace.
"keepalived_state_change" sends a GARP when a new IP address is added
in a interface other than the monitored one. That's why this patch
creates a new interface and sets it as the monitor interface. When
a new IP address is added to the other interface, the monitor populates
it by sending a GARP through the modified interface [1].
[1] 8ee34655b8/neutron/agent/l3/keepalived_state_change.py (L90)
Change-Id: Ib69e21b4645cef71db07595019fac9af77fefaa1
Closes-Bug: #1870313
(cherry picked from commit 21935365f2)
Slawek Kaplonski