rpc_workers can be set < 1 with 'ovn' backend when no
other agent is running apart from ovn agents to
consume these rpc notifications.
Add and apply disable_notifications decorator on
methods which do rpc cast calls to agents, the
decorator makes the caller method execute only
when rpc_workers >=1. This patch not changing
default behavior and utilizes the rpc_workers config option
to enable rpc notification on resources updates only when
rpc_workers >= 1.
Also set rpc_workers=0 in ovn jobs to cover this scenario.
Closes-Bug: #1889737
Closes-Bug: #1992352
Change-Id: I700fe2cd422bc1eb8b5144ec116e7f0a60238419
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/api.
Trivialfix
Change-Id: I1258b04f64a18036407e1d9de9ddca7472af0d11
When segment plugin is enabled, we should return segments details as
they are part of network.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: I1dab155bc812f8764d22e78ebb7d80aaaad65515
After port update, DHCP agent will be notified about changes only if
one of the port's attributes related somehow to the DHCP will change.
Such fields are:
* fixed_ips,
* MAC address,
* dns_domain,
* dns_name,
* dns_assignment,
* extra_dhcp_opts.
In other cases there is no reason to send notifications to the agent.
This will results with less notifications to the DHCP agent and less
possibilities to race condition between DHCP and L2 agents while
switching ports from the DOWN to ACTIVE status and sending notifications
to nova.
Closes-Bug: #1982367
Change-Id: If7990bdec435af76ad2e88fd4ea2bc24a255fd5a
In case when HA router isn't active on any L3 agent,
_ensure_host_set_on_port method shouldn't try to update port's host to
the host from which there was an rpc message sent, as this can be host
on which router is in the "standby" mode.
This method should only update port's host to the router's "active_host"
if there is such active_host found already.
Depends-On: https://review.opendev.org/c/openstack/requirements/+/841489
Closes-Bug: #1973162
Closes-Bug: #1942190
Change-Id: Ib3945d294601b35f9b268c25841cd284b52c4ca3
The agent side codes need consider three scenarios:
1. Non-dvr router. The all related rules are applied in
qrouter-namespace
2. Dvr router with the local agent mode is dvr_no_external.
The all related rules are applied in snat-namespace.
3. Dvr router with the local agent mode is dvr. In this scenario,
The all related rules are applied in fip-namespace.
Change-Id: Ie8729586d318be4a673858021a0116e09e193522
Partial-Bug: #1877301
Host parameter is needed there to filter subnets per segment when
segments plugin is enabled.
When dhcp agent requests informations about networks, and segments
plugin is enabled, subnets which belongs to the network are filtered out
based on the host passed as argument to the get_network_info() method.
But we never passed host to that method, even when we should e.g.
during the full sync of the DHCP agent, when it requests details about
each network.
This patch fixes that issue by passing host parameter to that method.
Closes-Bug: #1958955
Change-Id: Ib5eef501493f6735a47ea085196242a5807c4565
In patch [1] method get_network_info was refactored and that causes
NameError in the DHCP agent when there is "network object passed in
kwargs and there are subnets with segments in network. See related bug
for details.
[1] https://review.opendev.org/c/openstack/neutron/+/820190
Closes-Bug: #1958955
Change-Id: Iad8d85c79f8b11a24b1bb1ca44c776e909b610c3
OVS agent part of Local IP feature was divided into
2 parts to make it easier for reviewers:
1. This patch adds agent extension skeleton and sets
server <-> agent RPC communication mechanism via
push notifications of LocalIPAssociation objects
create/delete. It also shows how the extension would
treat those changes. It may be called extension "frontend".
2. Agent extension flows patch (next one) - deals with OVS
flows and can be called extension "backend".
Partial-Bug: #1930200
Change-Id: I31cb4062b6a21b71c739ab202c60aa7002e4d36e
This parameter, sent by the DHCP agent, is needed to remove the
workaround method "_get_network_lock_id".
The removal of this method will be done in [1] in Y release.
Related-Bug: #1732456
[1]https://review.opendev.org/c/openstack/neutron/+/800967
Change-Id: Ibd7fed33d314e901c69da33f42029f8ea67df98d
This patch switches over to callback payloads for PORT
AFTER_DELETE events.
Some shims were removed.
Change-Id: If69e37b84fe1b027777b1d673b3d08a6651a979e
This reverts commit 062336e59b.
Now, we have proper fix for the system_scope='all' in elevated context
in the neutron-lib so we can revert temporary fix made at the end of the
Wallaby cycle.
Related-Bug: #1920001
Conflicts:
neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py
neutron/common/utils.py
neutron/db/address_group_db.py
neutron/services/segments/db.py
Change-Id: Ife9b647b403bdd76a8a99984ea8858bf95c96bc3
This patch switches the code over to the payload style of callbacks [1]
for PORT AFTER_CREATE events. In addition it adds a branch/shim to the
dhcp_rpc_agent_api to support both payload and kwarg style callbacks.
NeutronLibImpact
[1]
https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html
Change-Id: I25d43d4f8f2390b07e0d11c631f894d88669bbe0
This patch switches the code over to the payload style of callbacks [1]
for ROUTER_INTERFACE events for those that are not using them yet.
The unit tests are also updated where needed to account for the
payload style callbacks and publish() method. In addition, a few
callback methods that use the retry_if_session_inactive() decorator are
separated out from the callback so that the context can still be
passed and detected by retry_if_session_inactive logic.
NeutronLibImpact
[1]
https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html
Change-Id: I8d9f8296952dfb10fcccd6afd72e90a5d4f379eb
This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
removed.
NeutronLibImpact
Change-Id: I889364b5d184d47a79fe6ed604ce13a4b334acfa
Add enable_dhcp, to make a filter to avoid unnecessary
net_info data transfer through rpc.
Change-Id: Ibcef366f5b1f4b7da4f47f1f538a17111da0faa1
Closes-Bug: #1552614
DHCP notification is done after each create/update/delete for
network, subnet and port.
This notification currently has to retrieve network from DB almost
every time, which is a quite heavy DB request and hence affects
performance of port and subnet CRUD.
This patch suggests 2 optimizations:
- do not fetch network if not needed (only fetch when schedule needed)
- for port and subnet AFTER_CREATE event pass network dict from plugin
According to Rally tests these changes improve performance:
- port create ~20%
- port update ~20%
- subnet create ~15%
- port delete and subnet update/delete - not tested
Closes-Bug: #1923161
Change-Id: I0ab836ac09225f4f3ad435e9ceaf315018855d52
In case when enforce_new_defaults is set to True and new policy rules
are used, context.is_admin flag isn't really working as it was with old
rules.
But in case when elevated context is needed, it means that we need
context which has full rights to the system. So we should also set
"system_scope" parameter to "all" to be sure that system scope queries
can be done with such elevated context always.
It is needed e.g. when elevated context is used to get some data from
db. In such case we need to have db query which will not be scoped to
the single project_id and with new defaults to achieve that system_scope
has to be set to "all".
Proper fix for that should be done in neutron-lib and it is proposed
in [1] already but as we are have frozen neutron-lib version for
stable/wallaby already this patch for neutron is temporary fix for that
issue.
We can revert that patch as soon as we will be in Xena development cycle
and [1] will be merged and released.
[1] https://review.opendev.org/c/openstack/neutron-lib/+/781625
Related-Bug: #1920001
Change-Id: I0068c1de09f5c6fae5bb5cd0d6f26f451e701939
Support security group rules with remote_address_group_id in openvswitch
firewall. This change reuses most of the firewall functions handling remote
security groups to also process remote address groups. The conjunctive flows
for a rule with remote_adress_group_id are similar to others with
remote_group_id but have different conj_ids.
Change-Id: I8c69e62ba56b0d3204e9c12df3133126071b92f7
Implements: blueprint address-groups-in-sg-rules
When processing port events (create, update, delete), the port
provisioning (port creation) has priority over the other events [1].
As reported in the related bug, if a port deletion with an IP
address and another port creation with the same IP address arrive
to the DHCP agent, those events can be processed in the same queue.
Because of the creation event priority, even when this event arrived
after the deletion event, it will be processed first. That will
clash with the DHCP agent cache, that contains a port (not deleted
yet) with the same IP address. That will trigger an unwanted resync.
This patch implements a specific logic to store the events in
"ResourceProcessingQueue" (that uses "PriorityQueue" [2]). When
a port event arrives, the event comparison method checks the
(subnet, fixed_ips) tuple set of both elements. If there is a
coincidence, that means those ports are the same or are using
the same IP addreses (the race condition explained in the bug).
In this case, the priority is defined only by the timestamp;
that means the events are processed in order of arrival.
Because the Neutron server do not allow to have two ports in the
same subnet with the same IP address, the order of the events is
guaranteed. In the case explained in the bug, the deletion event
will be processed first.
[1]https://review.opendev.org/c/openstack/neutron/+/626830
[2]https://docs.python.org/3/library/queue.html#queue.PriorityQueue
Closes-Bug: #1913723
Change-Id: I89438feae3c0244f6da5e6a2a035d45b956ac247
This change adds code to retrieve for the agent the security group ids
affected by an update or deletion of an address group.
Also adds event notificatoins to add and remove addresses from address
groups.
Co-authored-by: Hang Yang <hangyang@verizonmedia.com>
Change-Id: I34766b96cb775356664f5e0d48a08a22ac6898e2
Router HA port may be deleted concurrently while the plugin
is trying to update. This patch catches the known exceptions.
Should not `plugin.update_port_statuses` use because:
1. plugin.update_port_statuses will hide all exception
no matter the port exists.
2. The code just needs to catch the port not found error,
but let all other exception raised if port still exists.
Closes-Bug: #1906375
Change-Id: Id5d9c99be3bd6854568d2b1baa86c25c0cfd4756
A partial upgrading of neutron cluster, neutron-server
has a newer version while neutron-agents not, does not
run well after a RPC data structure upgrading. This
patch upgrades the security group related RPC version
between neutron-server and agents. A partial upgrading
neutron cluster will explicitly raise error. The RPC
version should be aligned.
Closes-bug: #1903531
Related-bug: #1867119
Change-Id: I6cb2ba05fa3337be46eb01f2d9f869efa41e4db6
Adds agent side code to enable the OVS agent to receive address groups
from the push notifications cache.
Change-Id: I1f27eccb2a69c553631fdc12d34e9025925844c5
Partial-Bug: #1592028
Adds server side code to enable the ML2 plugin to push notifications for
address groups. A follow up patch adds the corresponding code for the
OVS agent to process the notifications.
Change-Id: I49fc6e41a69f849a14da2ae67d1e4bff561dd4ec
Related-Bug: #1592028
As spotted in Focal testing patch [0], pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 [1]
Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel
[0] https://review.opendev.org/#/c/738163/
[1] https://github.com/PyCQA/pylint/issues/2737
Change-Id: Ie646b7093aa8634fd950c136a0eba9adcf56591c
With python 3.x, classes can use the metaclass= logic
to not require usage of the six library.
One step in removing all of six usage from neutron.
Change-Id: I2f815e412d9a96eb5faf2b3bb3a1e393a9db9309
When add allowed-address-pair 0.0.0.0/0 to one port, it will
unexpectedly open all others' protocol under same security
group. IPv6 has the same problem.
The root cause is the openflow rules calculation of the
security group, it will unexpectedly allow all IP(4&6)
traffic to get through.
For openvswitch openflow firewall, this patch adds a source
mac address match for the allowed-address-pair which has
prefix lenght 0, that means all ethernet packets from this
mac will be accepted. It exactly will meet the request of
accepting any IP address from the configured VM.
Test result shows that the remote security group and
allowed address pair works:
1. Port has 0.0.0.0/0 allowed-address-pair clould send any
IP (src) packet out.
2. Port has x.x.x.x/y allowed-address-pair could be accepted
for those VMs under same security group.
3. Ports under same network can reach each other (remote
security group).
4. Protocol port number could be accessed only when there
has related rule.
Closes-bug: #1867119
Change-Id: I2e3aa7c400d7bb17cc117b65faaa160b41013dde
Low port delete priority may lead to duplicate entries in network
cache if IPs are reused frequently.
Also can't find a strict reason why it should be of lower priority.
Change-Id: I55f858d50e636eb9091570b256380330b9ce9cb3
Related-bug: #1862315
Related-bug: #1828423
When setting up the DHCP agent of a network, the DHCP namespace external
port is configured. If this port already exists and the fixed IP
addresses are correctly configured (in the DHCP subnets range), the port
is used as is.
Sometimes, because of 1627480 or 1841636, the port information is not
correctly retrieved. This patch does not solve it but mitigates the
process of resynchronizing the network DHCP. If the stored DHCP port
does not have the correct information, the agent calls the RPC plugin to
retrieve from the server the DHCP port updated information, including
the fixed IP address and the subnets.
Change-Id: Iff40e7bba645ee12c2001d7ce735a36e0ddc81e9
Related-Bug: #1627480
Related-Bug: #1841636
Save order by in port query when not require fixed_ips,
and save some useless query for dvr subnet mac.
Closes-Bug: #1834308
Change-Id: I6836840edcaa5a21fd2ba9f65ffd24f7e5038fa3
- Added get_networks() RPC call for DHCP agent
- Added get_networks() RPC call for L3 agent
This change is required in order to support out of tree
MultiInterfaceDriver and IPoIBInterfaceDriver interface drivers
as they require information on the network a port is being plugged
to.
These RPCs will be passed as kwargs when loading the relevant
interface driver.
get_networks() keyword args map to the keyword arguments of:
neutron.neutron_plugin_base_v2.NeutronPluginBaseV2.get_networks()
Change-Id: I11d82380aad8655a4fdc9656737b912b16e2859b
Partial-Bug: #1834176
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
yatinkarel
Brian Haley