In version 4.15 of iproute2 there was added support
for chain index in tc_filter [1].
Such version is available e.g. in Ubuntu 18.04 and it
has to be supported in l3_tc_lib regex to match
properly output of "tc filter" command.
[1] https://lwn.net/Articles/745643/
Closes-bug: #1809497
Change-Id: Id4066b5cff933ccd0dd3c751bf67b5d58af662d1
(cherry picked from commit e788d29458)
Create a method for bulk assignment of IP addresses within the ipam
driver, to support bulk creation of ports.
This also changes the logic for how the window of available IP addresses
to assign from is calculated within the neutrondb IPAM driver. The
Python random module is used to produce a statistically sampled set of
IP addresses out of the set of available IPs; this will facilitate
collission avoidance. When requesting multiple IP addresses the
selection window sized is increased significantly to ensure a larger
number of available IPs, but caps are placed on the amount to make sure
we do not transgress system limits when building pools of IPv6
addresses.
Change-Id: Iad8088eaa261b07153fa358ae34b9a2442bc2a3e
Implements: blueprint speed-up-neutron-bulk-creation
(cherry picked from commit 06e38be42e)
Do not flood the packets to bridge, since we have the
bridge port list, we can add a simple direct flow to
the right port only.
Conflicts:
neutron/agent/linux/openvswitch_firewall/firewall.py
neutron/conf/plugins/ml2/drivers/ovs_conf.py
Closes-Bug: #1732067
Related-Bug: #1841622
Change-Id: I14fefe289a19b718b247bf0740ca9bc47f8903f4
(cherry picked from commit efa8dd0895)
For vlan type network, we add a segment match flow
to the openflow security group ingress table. Then
the packets will be recorded in conntrack table, and
the reply packets can be processed properly.
Conflicts:
doc/source/contributor/internals/openvswitch_firewall.rst
Change-Id: Ieded0654d0ad16235ec923b822dcd842bd7735e5
Closes-Bug: #1831534
(cherry picked from commit aa58542e82)
Master branch now has requirements incompatible with stable branches
constraints [0], so this switches to last tag compatible.
Longer term fix will be to run rally in a virtualenv, but this is lower
priority than other tasks. This can be revisited at the time, though for
older branches we will probably stick with pinned version
[0] 5776e015f1
Change-Id: I9ec8331bae54d191955a843f9d3f8c5537dc37f6
Closes-Bug: #1868691
If a user specifies a header in their request for metadata,
it could override what the proxy would have inserted on their
behalf. Make sure to remove any headers we don't want, and
override something that might be present in the request.
If the agent somehow gets a request with both headers it will
silently drop it.
Change-Id: Id6c103b7bcebe441c27c6049d349d84ba7fd15a6
Closes-bug: #1865036
(cherry picked from commit 5af046fd4e)
During processing of security group rule list API call Neutron will
now ensure that default security group for project given in the filters
or in the context exists.
It is similar to what is done for list of security groups or creation of
new network/port in the project.
Conflicts:
neutron/db/securitygroups_db.py
Change-Id: Id6fee5a752968b356b884d939b708a420016c9bc
Closes-Bug: #1864171
(cherry picked from commit 4739a4febb)
If the DVR+HA router has external gateway, the snat-namespace will be
initialized twice during agent restart. And that ns initialization
function will run many external resource processing actions which will
definitely increase the starting time of L3 agent. This patch addresses
this issue.
Change-Id: I7719491275fa1ebfa7e881366e5cb066e3d4185c
Closes-Bug: #1850779
(cherry picked from commit 7a9d6d2641)
Patch https://review.opendev.org/#/c/697655/ cannot be backported
because it includes an RPC version change. This patch is for the
stable branches.
Currently the ovs agent calls update_device_list with the
agent_restarted flag set only on the first loop iteration. Then the
server knows to send the l2pop flooding entries for the network to
the agent. But when a compute node with many instances on many
networks reboots, it takes time to readd all the active devices and
some may be readded after the first loop iteration. Then the server
can fail to send the flooding entries which means there will be no
flood_to_tuns flow and broadcasts like dhcp will fail.
This patch fixes that by also setting the agent_restarted flag if
the agent has not received the flooding entries for a network.
Change-Id: Iccc4fe4a785ee042fd76a663d0e76a27facd1809
Closes-Bug: #1853613
(cherry picked from commit bc0ab0fcd7)
(cherry picked from commit aee87e72b1)
Port deletion triggers disassociate_floatingips. This patch ensures
that method not only clears the port association for a Floating IP,
but also removes any DNS record associated with it.
Change-Id: Ia6202610c09811f240af35e2523126447bf02ca5
Closes-Bug: #1812168
(cherry picked from commit 4379310846)
Without this mock UT related configure_ipv6 method were failing
e.g. on systems when IPv6 was disabled or on systems where such
check should be done differently, like MacOS.
Change-Id: I6d13ab1db1d5465b2ff6abf6499e0d17e1ee8bbb
(cherry picked from commit c20b5e347d)
During the ha router state change event, the gateway port only
changed the L2 binding host. So l3 agent has the entire gateway
port information. It is not necessary to send a router_update
message to l3 agent again.
Depends-On: https://review.opendev.org/708825/
Closes-Bug: #1795127
Change-Id: Ia332421aff995f42e7a6e6e96b74be1338d54fe1
(cherry picked from commit 452b282412)
If both are run under the same process, and api_workers >= 2, the server
process will instantiate two oslo_service.ProcessLauncher instances
This should be avoided [0], and indeed causes issues on subprocess and
signal handling: killed RPC workers not respawning, SIGHUP on master
process leading to unresponsive server, signal not properly sent to all
child processes, ...
To avoid this, use the wsgi ProcessLauncher instance if it exists
[0] https://docs.openstack.org/oslo.service/latest/user/usage.html#launchers
Change-Id: Ic821f8ca84add9c8137ef712031afb43e491591c
Closes-Bug: #1780139
(cherry picked from commit 13aa00026f)
Security group can have a state of empty ports but non-empty members. So
we need skip the flow update only when members dict is empty.
Change-Id: I429edb3d2dea5fa97441909b4d2c776f97f0516f
Closes-Bug: #1862703
Related-Bug: #1854131
(cherry picked from commit 6dbba8d5ce)
Low port delete priority may lead to duplicate entries in network
cache if IPs are reused frequently.
Also can't find a strict reason why it should be of lower priority.
Change-Id: I55f858d50e636eb9091570b256380330b9ce9cb3
Related-bug: #1862315
Related-bug: #1828423
(cherry picked from commit a0bb5763b2)
Common neutron resource(e.g, Port) consists of:
1. Resource Attributes, e.g: Port.mac_address, etc.
2. Standard Attributes, e.g: created_at, and are shared among all
neutron resources.
The `sort` opt only supports limited attributes. We need to filter
attributes that are defined with `is_sort_key=True` and it's preferred
to explicitly warn CLI & API users of illegal sort keys rather than
just accept without check, pass forward and then hit a internal error
which's quite confusing.
Depends-on: https://review.opendev.org/#/c/660097/
Change-Id: I8d206f909b09f1279dfcdc25c39989a67bff93d5
Closes-Bug: #1659175
(cherry picked from commit 335ac4e2d9)
When I'm trying to introduce a central sort-keys validation within
patch [1] to `get_sorts` method in `neutron.api.api_common` module,
I get blocked by some resource schemas and test cases. After reading
neutron API docs and some inspection, I believe there exists uses of
improper sort keys in test cases and some resource schemes need to
keep aligned with offical documents.
* Schemas of resource SecurityGroups/SG Rules/Segments don't provide
`is_sort_key` flag for their sort key properties claimed in offical
docs as neutron-lib does. See [2] for more details.
* Test cases of resource NetworkSegmentRange use unsupported sort keys,
e.g: physical_network. Replace it with `name` property. See [2] for more
details.
[1] https://review.opendev.org/#/c/653903/
[2] https://developer.openstack.org/api-ref/network/v2/index.html
Conflicts:
neutron/tests/unit/extensions/test_network_segment_range.py
Change-Id: I45a51736e4075e3dbc16827486869d70b659622d
(cherry picked from commit ed3f1087fa)
(cherry picked from commit 874ffe0d6b)
In neutron.agent.linux.openvswitch_firewall.firewall make the method
update_port_filter catch OVSFWTagNotFound and log it to avoid
traceback in log files.
Conflicts:
neutron/agent/linux/openvswitch_firewall/firewall.py
Change-Id: I584d867f0e1c47447cb8790fd715fa01ec902438
Closes-Bug: #1811405
(cherry picked from commit 22f55822aa)
The OVS agent processes the port events in a polling loop. It could
happen (and more frequently in a loaded OVS agent) that the "removed"
and "added" events can happen in the same polling iteration. Because
of this, the same port is detected as "removed" and "added".
When the virtual machine is restarted, the port event sequence is
"removed" and then "added". When both events are captured in the same
iteration, the port is already present in the bridge and the port is
discharted from the "removed" list.
Because the port was removed first and the added, the QoS policies do
not apply anymore (QoS and Queue registers, OF rules). If the QoS
policy does not change, the QoS agent driver will detect it and won't
call the QoS driver methods (based on the OVS agent QoS cache, storing
port and QoS rules). This will lead to an unconfigured port.
This patch solves this issue by detecting this double event and
registering it as "removed_and_added". When the "added" port is
handled, the QoS deletion method is called first (if needed) to remove
the unneded artifacts (OVS registers, OF rules) and remove the QoS
cache (port/QoS policy). Then the QoS policy is applied again on the
port.
NOTE: this is going to be quite difficult to be tested in a fullstack
test.
Conflicts:
neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py
Change-Id: I51eef168fa8c18a3e4cee57c9ff86046ea9203fd
Closes-Bug: #1845161
(cherry picked from commit 50ffa5173d)
(cherry picked from commit 3eceb6d2ae)
(cherry picked from commit 6376391b45)
When trunk port deletion is attempted but fails because the driver does
not permit it, the logs do not contain the driver error message that
specifies the precise rationale for preventing the trunk port deletion.
Log it explicitly.
Change-Id: I7ed1a742849dfce9e65b8eb36566112501fb0e39
OVS agent is a single thread module executed on a os-ken AppManager
context. os-ken uses, by default (and no other implementation is
available today [1]), "eventlet" threads. Those threads are scheduled
manually by the code itself; the context switch is done through
yielding. The easiest way to do this is by executing:
eventlet.sleep()
If the assigned thread is not ready to take the GIL and do not yield
back the executor, other threads will starve and eventually will
timeout.
This patch removes the "sleep" command during the DP retrieval. This
will keep the executor on the current thread and will prevent the
execution timeouts, as seen in the bug related.
[1]1f751b2d7d/os_ken/lib/hub.py
Closes-Bug: #1861269
Change-Id: I19e1af1bda788ed970d30ab251e895f7daa11e39
(cherry picked from commit 740741864a)
For large scale deployment, the dvr router will be installed to
the scheduled DHCP host. This will definitely increase the l3
agent service pressure, especially in large number of concurrent
updates, creation, or agent restart.
This patch adds a config ``host_dvr_for_dhcp`` for the DHCP port
device_owner filter during DVR host query. Then if we set
``host_dvr_for_dhcp = False``, L3-agent will not host the DVR router
namespace in its connected networks' DHCP agent hosts.
Closes-Bug: #1609217
Change-Id: I53e20be9b306bf9d3b34ec6a31e3afabd5a0fd6f
(cherry picked from commit 8f057fb49a)
This is to fix race conditions on neutron server init.
Please see bug for details.
Conflicts:
neutron/db/models/flavor.py
Change-Id: I943e6397319b9a4a7fc1a5b3acb721920ddffb02
Partial-Bug: #1824299
(cherry picked from commit 38daf9eaae)
(cherry picked from commit d7945c60b0)
FirewallDriver.process_trusted_ports" is called with many ports,
"_initialize_egress_no_port_security" retrieves the VIF ports
("Interface" registers in OVS DB), one per iteration, based in the
port_id. Instead of this procedure, if the DB is called only once to
retrieve all the VIF ports, the performance increase is noticeable.
E.g.: bridge with 1000 ports and interfaces.
Retrieving 100 ports:
- Bulk operation: 0.08 secs
- Loop operation: 5.6 secs
Retrieving 1000 ports:
- Bulk operation: 0.08 secs
- Loop operation: 59 secs
Closes-Bug: #1836095
Related-Bug: #1836023
Change-Id: I5b259717c0fdb8991f1df86b1ef4fb8ad0f18e70
(cherry picked from commit ae1d36fa9d)
Slawek Kaplonski