As reported in LP#1896734, there is a limit in the size of information
that can be transmitted in one single message between an application
and the privsep daemon. The read socket buffer is limited in size;
a message exceeding this size will generate an exception.
In order to limit the amount of information to be sent, this patch
improves the performance of "get_devices_with_ip". In the previous
implementation, the whole list of network devices from a namespace
was retrieved. In some environments, the list of devices could be
so big that the list returned by "privileged.get_link_devices" can
exceed the read buffer size (as reported in the LP bug when the
OVS agent tries to retrieve the list of IP addresses in the system).
Now the function calls "privileged.get_ip_addresses", that returns
a much smaller list. This patch is also reducing the number of system
calls to just one; the previous implementation was retrieving first
the devices link information list (that method was returning a much
bigger blob) and then, per device, retrieving the IP address
information.
Conflicts:
neutron/tests/functional/agent/linux/test_ip_lib.py
Change-Id: I97ada62484023b9833ed12afd68eb4c8d337fd1f
Related-Bug: #1896734
(cherry picked from commit a6cdf273c0)
In "IpAddrCommand.list" method, the "scope" parameter is a string
("link", "site", "global" or "host"). This method will retrieve all
devices with an IP address calling "ip_lib.get_devices_with_ip".
Since [1], "ip_lib.get_devices_with_ip" makes the conversion of
"scope" string parameter to pyroute2 format (see
"pyroute2.netlink.rtnl.rtscopes"). The list command should skip then
the previous conversion.
Closes-Bug: #1899141
[1]https://review.opendev.org/#/c/747406/
Change-Id: I55a0f4341b328af52ea3bd758a72f633fbe3abcb
(cherry picked from commit 0705699e60)
(cherry picked from commit f0849bb0d2)
(cherry picked from commit cd0526dce8)
This method is using ip_lib.get_devices_with_ip function to get
IP addresses with scope "link".
Unfortunatelly this method wasn't translating scope names to the pyrout2
values and due to that wasn't returning correct IP addresses.
Now this is fixed and correctl link local IPv6 addresses are returned.
Change-Id: Ia41c1bc627ad2ce89d658ff1fdedee802f6dfa15
Closes-Bug: #1892489
(cherry picked from commit a4e04a7f8b)
Recent changes in some versions of iproute2 CLI output (v4.18),
have invalidated the regular expression used to parse the
"ip link" output.
To solve this problem and avoid future ones, pyroute2 is used to
retrieve the virtual functions information and set the VF attributes
(spoofcheck, min_tx_rate, max_tx_rate and link_state).
pyroute2 extended the "ip link" support to retrieve this information,
adding "ext_mask=1" in the get command. If no virtual functions are
present in this particular network interface, the added method,
"get_link_vfs", will return an empty list.
The set commands can return a "InterfaceOperationNotSupported" in
case the operation is not supported. For min_tx_rate, if the driver
does not support to set a minimum bandwidth, an "InvalidArgument"
(from a pyroute2.NetlinkError(22)) exception will be raised.
Conflicts:
neutron/tests/unit/agent/linux/test_ip_link_support.py
Change-Id: I680da4f64bd114f1caecaaeedbf8a4b1915a0849
Closes-Bug: #1878042
(cherry picked from commit c5d8fd6329)
(cherry picked from commit 5df65489dc)
In "ip_lib.ensure_device_is_ready", before retrieving the interface
attributes, a check is done to know if the interface exists. In case
it does not exist, the exception "NetworkInterfaceNotFound" will not
be raised and written in the logs.
Conflicts:
neutron/privileged/agent/linux/ip_lib.py
Change-Id: I4b9fd0885d850601717274a5058e042871211bbb
Closes-Bug: #1854723
(cherry picked from commit 8cc2765b5f)
By default, if no metric is defined, the kernel interprets the
highest value (0).
The current implementation, using pyroute2, is a translation from
the CLI command "ip route". This command uses the netlink API to
communicate with the kernel. In IPv6, when the metric value is not
set is translated as 1024 as default [1].
[1]https://access.redhat.com/solutions/3659171
Change-Id: I0c5f9e320bbbf314a2d6a22c515bf903de84cdaf
Related-Bug: #1855759
(cherry picked from commit 7593f95a74)
"keepalived_state_change" monitor does not use eventlet but normal
Python threads. When "send_ip_addr_adv_notif" is called from inside
the monitor, the arping command is never sent because the eventlet
thread does not start. In order to be able to be called from this
process, this method should also have an alternative implementation
using "threading".
"TestMonitorDaemon.test_new_fip_sends_garp" is also modified to
actually test the GARP sent. The test was originally implemented with
only one interface in the monitored namespace.
"keepalived_state_change" sends a GARP when a new IP address is added
in a interface other than the monitored one. That's why this patch
creates a new interface and sets it as the monitor interface. When
a new IP address is added to the other interface, the monitor populates
it by sending a GARP through the modified interface [1].
[1] 8ee34655b8/neutron/agent/l3/keepalived_state_change.py (L90)
Change-Id: Ib69e21b4645cef71db07595019fac9af77fefaa1
Closes-Bug: #1870313
(cherry picked from commit 21935365f2)
IP monitor is a method that is going to be executed in a separate
process, to monitor the IP addresses changes in a namespace.
This method spawns a thread to read from a socket opened by Pyroute2.
The read function is a blocking method that will end only when the
socket is closed. To avoid thread starvation that can happen using
greenthreads, IP monitor will use kernel threads.
This will increase the resources used but will ensure that no message
is lost when reading the monitor socket.
Reduced the number of IPs generated in "test_add_and_remove_multiple_ips"
to shrink the testing time used.
Change-Id: I3fbba2854d40ab0f683443aa30c2a95752345d2e
Closes-Bug: #1849547
(cherry picked from commit 48730d9449)
The gateway IP address in the gateway dictionary returned by
"ip_lib.list_ip_routes" is stored in "via".
"priority" parameter is changed to "metric", to match input and
output parameters.
Change-Id: I67ae473dca8d706f963c3b55b9410f9a79d7f32b
Closes-Bug: #1855759
(cherry picked from commit b99765df8f)
In "NamespaceFixture", before deleting the namespace, this patch
introduces a check to first kill all processes running on it.
Closes-Bug: #1838793
Change-Id: I27f3db33f2e7ab685523fd2d6922177d7c9cb71b
- Add a new property to IPDevice to allow us to identify
the kind of the interface.
This change is required as an out of tree interface driver
which supports operations on a per-physnet basis
needs to be aware of the kind of interface an interface driver
created in order to correlate between an interface driver
and an interface created by it.
Change-Id: Icbdb011a639475f416ca1b98fdf3ce2f52482c7c
Partial-Bug: #1834176
In order to capture all IP address changes, the method reading the
netlink socket will be executed in a parallel thread. Once the
"ip_monitor" method is stopped, this blocking thread will be killed.
A new functional test, "test_add_multiple_ips", is added in order to
stress test this method.
Change-Id: I8f1de4a31f97bab734a33f94c3069444defd870f
Closes-Bug: #1832307
This method allows to track any IP address change in a
namespace. In future patches, this method will replace
the current IP monitor used in the keepalived_state_change
daemon. The current implementation relays in a spawned shell,
executed in root mode, and the output of this shell,
conveniently parsed.
If the passed namespace is not None, this new method must
be executed in privileged mode (root user), but cannot use
privsep because is a blocking function and can exhaust the
number of working threads.
This function should be executed in a parallel thread, returning
the data using the eventlet queue. Pyroute does not implement yet
a non blocking method to retrieve the command output or to know if
the buffer has data. This method, spawned in a greenthread, must be
stopped by killing this thread.
An example of how to use it can be found in the functional tests
implemented in this patch.
Change-Id: I86e4487035d60e1b52e951dd3cd50d6bb54f388b
Related-Bug: #1680183
Now ip_lib.get_devices_info function is implemented using pyroute2,
"vlan_in_use" and "vxlan_in_use" can make use of it.
Change-Id: I82a2c3ea76195b10880cf37bf2229341b995b0ae
Closes-Bug: #1815498
In ip_lib.get_devices_info(), privileged.get_link_devices() can return
devices with links not present in this namespace or not listed. In this
situation, get_devices_info() will always try to find the device to set
the parameter "parent_name", what will trigger an exception.
This patch solves this issue avoiding the population of "parent_name"
if the link device is not present in the devices list.
Change-Id: Ic5c7d9008a11da5c406dc383cfdae2892a3118d8
Closes-Bug: #1815758
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
Added VLAN parent device name and index and VXLAN link device
name and index.
Change-Id: Ib44a63c0648a7b5b07b1021b10e8994002031ce8
Related-Bug: #1804274
Currently the metadata proxy binds to default 0.0.0.0, which does not
add any advantage (metadata requests are not sent to random IP
addresses), and may allow access to cloud information from
third parties.
This changes the generated configuration to bind to METADATA_DEFAULT_IP
address instead.
This is not enabled in other metadata proxy configuration (in the L3
agent), as this would require net.ipv4.ip_nonlocal_bind everywhere
(currently only enabled for DVR) or transparent mode in haproxy (which
requires net.ipv4.ip_nonlocal_bind anyway)
Changed set_ip_nonlocal_bind_for_namespace() to support setting the
value in both the given and root namespace correctly, since it was
only used from inside the neutron codebase according to codesearch.
Change-Id: I388391cf697dade1a163d15ab568b33134f7b2d9
Co-Authored-By: Andrey Arapov <andrey.arapov@nixaid.com>
Closes-Bug: #1745618
The centralized floating IP can exist on the router device
due to some reasons like: uncleaned fip addr, and especially
multiple IP addr adding action: HA router _add_vip() and
Edge router add_ip_address().
This patch catches the IpAddressAlreadyExists error if fip
was already set on the device, and still process next step.
Change-Id: I324f6b96baa0520a0f7ef62a83d81864d7b27999
Closes-Bug: #1811213
In neutron.agent.linux.ip_lib.add_ip_rule, the "table" argument can be
an integer or a string ("default", "main", "local").
This parameter is incorrectly treated in "_make_pyroute2_args".
Change-Id: I0a50af5fe9b1550700e607eb680bb16e5044e8ef
Closes-Bug: #1807128
brctl is being deprecated in some Linux distros, so
change neutron to start using iproute2 commands or
the pyroute2 library where possible.
Added create() to IpLinkCommand class to allow usage
of pyroute2 for bridge creation.
Change-Id: If679e79fa3242ee1cd8610b5525deca35b41c87e
Closes-bug: #1801919
When a namespace gets created, if the fb_tunnels_only_for_init_net
sysctl configuration is set to 0, the fallback tunnels will be
automatically created if the corresponding tunnel is present
in the initial network namespace. Source [0].
This is being observed in some systems where namespaces are not
getting cleaned up due to the presence of such devices as
ip_lib.namespace_is_empty() is returning False.
This patch is adding such devices as per the kernel documentation
to the list of excluded devices by default.
[0] https://www.kernel.org/doc/Documentation/sysctl/net.txt
Closes-Bug: #1797084
Change-Id: I94415a0da5367e2d98d792a5eb4ba3919b838326
Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
This patch contains the l3 agent extension and agent part code.
This patch introduce a new l3 agent extension named "port_forwarding",
to process the binding of the port forwarding resources, manage its own
floatingip configuration on router interface and floatingip status.
Currrently, we support all Neutron Router reference implementations.
This extension uses the period router sync task and PortForwarding OVO
rpc.
* The main idea about this new extension is using the generic router sync
rpc to maintain the host port forwarding resources,
* For a single port forwarding create/update/delete, process it one by one
in smaller scope for forbidding refresh the iptables with a larger
scope frequently.
Partially-Implements: blueprint port-forwarding
Partial-Bug: #1491317
Change-Id: Ic56e67d428f6177099c285a9d1bccabc1e710f2b
In case of HA routers IPv6 forwarding is not disabled by default and
then enabled only on master node.
Before this patch it was done in opposite way, so forwarding was
enabled by default and then disabled on backup nodes.
When forwarding was enabled/disabled for qg- port, MLDv2 packets are
sent and that might lead to temportary packets loss as packets to
FIP were sent to this backup node instead of master one.
Related-Bug: #1771841
Change-Id: Ia6b772e91c1f94612ca29d7082eca999372e60d6
In case when MTU value configured for VXLAN network is
higher than physical device's MTU - VXLAN overhead
then Linuxbridge agent will not be able to create properly
vxlan interface with desired MTU value.
In patch [1] there was introduced validation if MTU of
physical device is big enough to create MTU with required
MTU value but it was working only for IPv4 tunnels as
for IPv6 overhead is bigger.
This patch changes this validation a bit and now it will
works properly for both IPv4 and IPv6 VXLAN tunnels.
[1] https://review.openstack.org/#/c/546291/
Change-Id: Ib707312adb45dae7295884aba4ece6538d330d56
Related-Bug: #1744101
When specifying a value for the 'dstport' option in the [vxlan] section
of the Linux bridge agent, the agent crashes upon start. This is
because the value specified in 'dstport' is incorrectly converted to
a string before being passed to pyroute2 when creating a VXLAN tunnel.
pyroute2 expects an integer value. This patch fixes this bug.
Change-Id: I83131f3bb32253cbc4acec3a1f38bf225223ff8b
Closes-Bug: #1764867
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
If a distributed router has interfaces on multiple tenant networks, with
'fast exit' functionality policy based rules are created in qrouter
namespace for every tenant network subnet and 'from <cidr>' is included
into an 'ip rule' command invocation.
When a port on a tenant network is deleted 'from <cidr>' part is not
included and a first rule matching specified parameters gets deleted.
For example with the following layout
ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.100.0/24 lookup 16
80000: from 192.168.200.0/24 lookup 16
and neutron l3 agent will use this command
ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip -4 rule\
del priority 80000 table 16 type unicast
and 192.168.100.0/24 rule will get deleted even if you actually removed
a port on 192.168.200.0.
This results in an extra rule present and not cleaned up and the right
rule removed. It is only recreated if a router is disabled and enabled
again.
additional changes:
1) Floating IP rules are identified by priority only as implemented
currently - for this reason this change adds fixed_ip to the rule
removal code. Rule priorities are 32-bit values in iproute2 so,
in theory, those should be not be used to cover IPv6.
2) IP protocol information for 'from all' rules is currently
derived from link-local address IP version. The same approach
is preserved by using version-specific /0 addresses without
changing the API provided by ip_lib.
Change-Id: I0ea6dddd26e17771be223a1fbdf21792c90f3e9c
Closes-Bug: #1759956
Rodolfo Alonso Hernandez