update port may takes an excessive number of seconds
to complete if dvr routers are running on more than 100
compute nodes. This patch tries to save some time by removing
unnecessary calls inside looping through hosts.
Change-Id: Ide740e0c5c43c2d2b842460a37c8ce125da12b28
Closes-Bug: #1830456
(cherry picked from commit 00eb6f26f6)
We have a problem with SNAT with too many connections using the
same source and destination on the network nodes.
In addition we can see in the conntrack table that the who
"instert_failed" increases.
This might be a generic problem with conntrack and linux.
We suspect that we encounter the following "limitation / bug"
in the kernel.
There seems to be a workaround to alleviate this behavior by
setting the -random-fully flag in iptables for port consumption.
This patch fixes the problem by adding the --random-fully to
the SNAT rules.
Conflicts:
neutron/agent/linux/iptables_manager.py
neutron/common/constants.py
neutron/tests/unit/agent/l3/test_agent.py
Change-Id: I246c1f56df889bad9c7e140b56c3614124d80a19
Closes-Bug: #1814002
(cherry picked from commit 30f35e08f9)
It may happen that subnet is connected to dvr router using IP address
different than subnet's gateway_ip.
So in br-tun arp to dvr router's port should be dropped instead of
dropping arp to subnet's gateway_ip (or mac in case of IPv6).
Conflicts:
neutron/tests/unit/plugins/ml2/drivers/openvswitch/agent/test_ovs_neutron_agent.py
Change-Id: Ida6b7ae53f3fc76f54e389c5f7131b5a66f533ce
Closes-bug: #1831575
(cherry picked from commit ae3aa28f5a)
This change fixes an 'RuntimeError: dictionary changed size during
iteration' error that is raised because of different behaviour between
python2 and python3. An earlier attempt [1] to use the six library did
not fix the issue.
[1] https://review.opendev.org/660427
Resolves-Bug: #1829304
Change-Id: Ic4c5790a5d66e10a606327f4f4e4c92611f161d8
(cherry picked from commit a65ae9c81e)
This patch adds a check to determine if the 'segments' service plugin is
enabled. The segment-host mapping db table should only be saved and
updated to the db table if the users configure the 'segments' service
plugin in the config file. The data should be available only in a routed
network resource situation.
Conflicts:
neutron/tests/unit/extensions/test_segment.py
NOTE(s10): conflict is due to If6792d121e7b8e1ab4c7a548982a42e69023da2b
not being in Queens
Change-Id: I65a42aa2129bef696906a18d82575461dc02ba21
Closes-Bug: #1799328
(cherry picked from commit 06ba6a1ace)
(cherry picked from commit 7f8c446d5f)
In functional tests for L3 HA agent, like e.g.
L3HATestFailover.test_ha_router_failover
it may happen that L3 agent will not change ipv6 accept_ra
knob and test fails because it checks that only once just
after router state is change.
This patch fixes that race by adding wait for 60 seconds to
ipv6 accept_ra change.
Conflicts:
neutron/tests/functional/agent/l3/framework.py
Change-Id: I459ce4b791c27b1e3d977e0de9fbdb21a8a379f5
Closes-Bug: #1829889
(cherry picked from commit 62b2f2b1b1)
If security group contains rule(s) which were created by different
user (admin), owner of this security group should see such rules
even if those rules don't belong to him.
This patch changes to use admin_context to get security group rules
in get_security_group() method to achieve that.
Test to cover such case is added in neutron-tempest-plugin repo.
Conflicts:
neutron/db/securitygroups_db.py
Change-Id: I890c81bb6eabc5caa620ed4fcc4dc88ebfa6e1b0
Closes-Bug: #1824248
(cherry picked from commit 1920a37a94)
(cherry picked from commit 252d80058c)
Sometimes when port is created on dhcp agent's side, it may happend
that same port is already in network cache.
Before this patch if port with same IP address was already in cache,
resync was rescheduled because of duplicate IPs found in cache.
Now resync will be scheduled only if duplicate IP address belongs to
port with different MAC address or different id.
Change-Id: I23afbc10725f5dc78e3c63e6e505ef89ba8dc4a5
Closes-Bug: #1824802
(cherry picked from commit 5c433a027d)
New IP command introduced by Ie3fe825d65408fc969c478767b411fe0156e9fbc
requires only privsep initialization. This patch removes the prisep
error FailedToDropPrivileges when executed under neutron-rootwrap.
Closes-Bug: #1823038
Change-Id: I6cde3c9dae7ffdccce49e88c3c79d1c379f291cf
(cherry picked from commit aacd11ab9f)
This change fixes an 'RuntimeError: dictionary changed size during
iteration' error that is raised because of different behaviour between
python2 and python3. We use the six library to ensure that the behavior
is compatible across versions.
Change-Id: I0723ae10825e1e2d86789627895e3286d8c97602
Resolves-Bug: #1829304
(cherry picked from commit 2c1098b3af)
There are some extreme conditions which will result the unbound
router gateway port. Then all the centralized floating IPs will
not be reachable since the gateway port was set to 4095 tag.
This patch adds the HA status to the router related port
processing code path. If it is HA router, the gateway port
will go to the right HA router processing code branch.
Closes-Bug: #1827754
Change-Id: Ida1c9f3a38171ea82adc2f11cb17945d6e2434be
(cherry picked from commit 3d99147e73)
RPC notifier method can sometimes be time-consuming,
this will cause other parallel processing resources
fail to send notifications in time. This patch changes
the notify to asynchronous.
Closes-Bug: #1824911
Change-Id: I3f555a0c78fbc02d8214f12b62c37d140bc71da1
(cherry picked from commit 0f471a47c0)
Some policy rules e.g. for create_port are using rule "network:shared"
in which "shared" field is related to network resource instead of
port directly.
Because of that, "shared" was missing from "target" in policy
enforce module thus validation wasn't working properly for such rule.
This patch fixes it by adding to FieldCheck checker possibility to
get network object and use its "shared" field to validate policy.
Conflicts:
neutron/tests/unit/test_policy.py
Change-Id: I56c99883fce40c37a5ee26e6e661c0cc0783c42f
Closes-Bug: #1808112
(cherry picked from commit 0396912208)
(cherry picked from commit fcfd46b231)
In case of policy rule checks for rules like e.g.
"create_port:fixed_ips:subnet" couldn't be created to be
passed to policy enforcer because policy module could only
create rule checks for subattributes which are dict types.
With this patch checks for such rules can be created also for
attributes which are list of dicts, like e.g. fixed_ips in port
resource.
Conflicts:
etc/policy.json
neutron/tests/etc/policy.json
Change-Id: I02fffe77f57a513d2362df78885d327042bb8095
Closes-Bug: #1822105
(cherry picked from commit 9318fb8bb9)
(cherry picked from commit a238b1bed6)
(cherry picked from commit 73bbfa4315)
This patchset adds missing policy actions to the policy.json
file for several reasons:
1) It signals to operators all the policy actions that are
enforced in the system. With the governance spec [0]
urging projects toward policy in code documentation,
it makes sense to document all policy actions in the
policy.json as Neutron doesn't have policy in code.
2) It is consistent with Neutron's policy enforcement
documentation [1]:
"For each attribute which has been explicitly specified in the
request create a rule matching policy names in the form
<operation>_<resource>:<attribute> rule"
So it makes sense to capture each policy that is enforced,
including all those with these special attributes.
3) Why include "update_router:external_gateway_info" but not
"create_router:external_gateway_info"? This is inconsistent.
4) It makes it difficult to validate Neutron's policy via Patrole
if the policies aren't contained in the policy.json -- how else
is it possible to determine which policies to expect if they
aren't documented anywhere?
[0] https://governance.openstack.org/tc/goals/queens/policy-in-code.html
[1] https://docs.openstack.org/neutron/pike/contributor/internals/policy.html#authorization-workflow
Change-Id: I40f84134f0b56cfd574dfd69e5ebbf6a3fc2b3df
(cherry picked from commit 41fe927c80)
Once HA port is set, it must remain this value no matter
what the server return. Because there is race condition
between l3-agent side sync router info for processing
and server side router deleting.
This patch adds a helper function for every ha_port set
action. If the ha_port is not None, it will always stay
with original value.
Conflicts:
neutron/tests/unit/agent/l3/test_ha_router.py
Closes-Bug: #1826726
Change-Id: I96a088d25048be02a9c5b12c1d087df075b36fc4
(cherry picked from commit 45957f12c8)
(cherry picked from commit 13cb3cd34c)
_get_ports_query add filters after a get_collection. If limits are
passed to applied to the query, then no additional filter is allowed.
This patch extracts an eventual limit argument, to apply it only after
the additional filters.
Change-Id: I83394394860d10e27379efe0356d0fa9c567140e
Closes-Bug: #1826186
(cherry picked from commit cf8f3326be)
In conjunction with the prior fix to only get a subset of fields
when needed, this makes the querying of non-rules SG objects
very very fast.
Before the two fixes, if you have about ten security groups with 2000 rules each:
list all: 14s
list all, just 'id' field: 14s
list one: 0.6s
list one, just 'id' field: 0.6s
With just the previous partial fix:
list all: 14s
list all, just 'id' field: 6s
list one: 0.6s
list one, just 'id' field: 0.2s
Now with this change:
list all: 14s
list all, just 'id' field: 0.04s
list one: 0.6s
list one, just 'id' field: 0.03s
Closes-Bug: #1810563
Change-Id: I15df276ba7dbcb3763ab20b63b26cddf2d594954
(cherry picked from commit 1e9086f6e2)
SQLAlchemy may asynchronously push models out of session cache in which
case we may receive DetachedInstanceError.
In the test case, instead of deepcopying models to compare, compare
each modified attribute independently.
This change also includes conversion from InstrumentedLists to regular
lists when converting model attributes to object fields. The fact that
we were returning InstrumentedLists was always an oversight but it
revealed itself after the modification of the test case that is the
core of this patch.
When converting object fields to db, convert Port's distributed_binding
None value to a empty list to reflect that the relationship of the Port
database model is a list. It was not an issue before the patch because
we were not comparing model attribute for equality but for in-equality
before, and so None was always != [].
Finally, this patch moves a bunch of TODOs to better reflect where they
belong to.
Closes-Bug: #1770452
Change-Id: I42cdf540129bd4470ec1a59345db9845a6198328
(cherry picked from commit 1a8a15f630)
To fix bug 1722584 we inserted a checksum-fill rule for
metadata proxy replies. Recent kernels have disabled
this support for TCP because it was invalid, and
supposedly not doing anything, so let's get ahead of
things and remove the code.
Kernel mailing list discussion is at
https://lore.kernel.org/patchwork/patch/824819/
Partially reverts ed1c3b0217
Change-Id: Ib7cc8f82a91972f17987fb95130edc4069d9423f
Related-bug: #1722584
(cherry picked from commit b1b8a438fe)
1. give each HA failover case an independent vrrp_id
2. give each HA port an independent IP address, so the
interface IPs for router HA ports will be:
169.254.192.100 and 169.254.192.101
169.254.192.102 and 169.254.192.103
169.254.192.104 and 169.254.192.105
169.254.192.106 and 169.254.192.107
VIP of each case will be:
169.254.0.10/24
169.254.0.11/24
169.254.0.12/24
169.254.0.13/24
169.254.0.14/24
Conflicts:
neutron/tests/common/l3_test_common.py
Closes-Bug: #1819160
Change-Id: I1216d96af40449ec16a852cc1f6c4f15c85f4546
(cherry picked from commit c69a87405a)
(cherry picked from commit 2c5957f56d)
(cherry picked from commit c50bdf2329)
For auto-address IPv6 subnets postcommit has update port action
if the net already has ports. This results in
"cannot be called within a transaction" error for bulk IPv6 subnet
create.
Closes-Bug: #1822582
Change-Id: Ia32ec4c11c0793e7df07dcce19c122b3c7f865e1
(cherry picked from commit 14c76d3181)
The transition of a router from distributed to centralized may
mistakenley get its 'ha' attribute updated at the same time. The
side-effect is that the router may become HA enabled unexpectedly.
This patch fixed the mismatched attribute in _update_router_provider
method which addressed the issue cited above.
Closes-Bug: #1780094
Change-Id: Ib00de137692979229d1b7ba033ecff04e9cc9db0
(cherry picked from commit 886e241553)
When two routers are created at the same time, we can't assume the
status of each one. Instead of this, the status of each router is
first checked and then compared to the other router status.
Change-Id: If20a3a414986ea29fbfd50616761c14e5b249b2c
Closes-Bug: #1819160
(cherry picked from commit 8f35331c91)
The test bridge veth pair devices is not up which cause the
VRRP advertisement packet can not pass to each HA port. Then
multiple master router is up. This patch just sets the veth
pair devices up.
Closes-Bug: #1819160
Change-Id: I0e0d0311d73bce83d3c7341e7a0167917818b1ff
(cherry picked from commit 8cc480bd01)
Zuul