Commit Graph

1734 Commits (b79842f289f0c9b9fdce9d6cc689538ec61216e7)

Author SHA1 Message Date
Brian Haley b79842f289 Start enforcing E125 flake8 directive
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues.  Didn't think it was going to be
close to 100 files when I started.

Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
4 years ago
Zuul f3935ba3bf Merge "[DHCP] Implement an aging method for deleted_ports" 4 years ago
Zuul 64c8bfbbf0 Merge "Improve "OVSFirewallDriver.process_trusted_ports"" 4 years ago
Rodolfo Alonso Hernandez ae1d36fa9d Improve "OVSFirewallDriver.process_trusted_ports"
FirewallDriver.process_trusted_ports" is called with many ports,
"_initialize_egress_no_port_security" retrieves the VIF ports
("Interface" registers in OVS DB), one per iteration, based in the
port_id. Instead of this procedure, if the DB is called only once to
retrieve all the VIF ports, the performance increase is noticeable.
E.g.: bridge with 1000 ports and interfaces.

Retrieving 100 ports:
- Bulk operation: 0.08 secs
- Loop operation: 5.6 secs

Retrieving 1000 ports:
- Bulk operation: 0.08 secs
- Loop operation: 59 secs

Closes-Bug: #1836095
Related-Bug: #1836023

Change-Id: I5b259717c0fdb8991f1df86b1ef4fb8ad0f18e70
4 years ago
Rodolfo Alonso Hernandez 11380ff5da Register SG opts when testing test_firewall
In some debuggers (Pycharm), when OVSFirewallDriver is loaded,
the project configuration is not fully populated and does not
include some variables, e.g.:

This will generate a configuration exception:
  oslo_config.cfg.NoSuchOptError: no such option SECURITYGROUP in group

By loading the SG groups options before the OVSFirewallDriver class, we
can solve this issue.


Change-Id: I96580635b8a21f68df86b302e528d622ba3cfffc
4 years ago
Rodolfo Alonso Hernandez 5754f00ea8 [DHCP] Implement an aging method for deleted_ports
When a port is deleted, is added to the NetworkCache instance
as a "deleted port". This set of ports is used by the DHCP agent
to mark out as "stalled" any message received with this port
in the payload.

Eventually, each deleted port should be dropped from this set
depending on when was deleted. A timeout of 1 day has been defined
in this patch to drop any older deleted port.

A "loopingcall.FixedIntervalLoopingCall" executes the deleted port
cleanup, with an execution interval of 1 day too.

Co-Authored-By: Rodolfo Alonso Hernandez <>
Partial-Bug: #1732458

Change-Id: I0bca03a67e8b0ff8f508cf59383340a13724e51c
4 years ago
Zuul ba436615b0 Merge "Implement "ip route delete" command using Pyroute2" 4 years ago
Zuul 95034c4b4f Merge "Implement "ip route" commands using Pyroute2" 4 years ago
Zuul e9dce50362 Merge "Minimizing L3 agent QoS extensions lock granularity" 4 years ago
Zuul 412924ae17 Merge "Adjust some HA router log" 4 years ago
Zuul ed1b82531e Merge "Use Pyroute2 "add_tc_qdisc" function in l3_tc_lib" 4 years ago
Zuul 509dabbc6c Merge "Limit max ports per rpc for dhcp_ready_on_ports()" 4 years ago
Rodolfo Alonso Hernandez fb7185bf35 Use Pyroute2 "add_tc_qdisc" function in l3_tc_lib
Change-Id: I67ddf9d9a6bb2d9d2e8ff0b6345a0118ec37d837
Related-Bug: #1492714
4 years ago
Rodolfo Alonso Hernandez b6cbc95dcb Use Pyroute2 "list_tc_qdiscs" function in l3_tc_lib
Change-Id: Ifdccd02411e3c3bae441fc28ab8ed09ff746993c
Related-Bug: #1492714
4 years ago
LIU Yulong 426a5b2833 Adjust some HA router log
In case router is concurrently deleted, so the HA
state change LOG is not necessary. It sometimes
makes us confusing.
Also print the log for the pid of router
keepalived-state-change child process.

Change-Id: Id57dd787c254994af967db17647a3a28925714da
Related-Bug: #1798475
4 years ago
LIU Yulong ab57410ec8 Minimizing L3 agent QoS extensions lock granularity
If agent is concurrently processing large set of resources,
the bottleneck lock will multiply increase processing time
of those resources which have been waiting for the lock for
a long time.

This patch moves the lock to the core cache resource, and
leverage the coordination lock to the resource prcessing
and notification thread functions.

Closes-Bug: #1824911
Change-Id: Id43829b11631727f1a46362ffea5c22d2177fd79
4 years ago
Rodolfo Alonso Hernandez 429c77c574 Implement "ip route delete" command using Pyroute2
Change-Id: I960455d6a9bc1b633d485c42a26b3a254731558e
Related-Bug: #1492714
4 years ago
Sebastian Lohff 76ccdb35d4 Limit max ports per rpc for dhcp_ready_on_ports()
The Neutron dhcp agents reports all ready ports to the Neutron
server via the dhcp_ready_on_ports() rpc call. When the dhcp agent
gets ports ready faster than the server can process them the amount
of ports per rpc call can grow so high (e.g. 10000 Ports) that the
neutron server never has a chance of processing the request before
the rpc timeout kills the request, leading to the dhcp agent
sending the request again, resulting in an endless loop of
dhcp_ready_on_ports() calls. This happens especially on agent startup.

To mitigate this problems we now limit the number of ports sent
per dhcp_ready_on_ports() call.

Closes-bug: #1834257
Change-Id: I407e126e760ebf6aca4c31b9c3ff58dcfa55107f
4 years ago
Rodolfo Alonso Hernandez 0699713609 Implement "ip route" commands using Pyroute2
Commands implemented:
* Add route
* List routes

Related-Bug: #1492714

Change-Id: I5e5e9f6981024317773979d9d2d77db3f5e7ec98
4 years ago
Slawek Kaplonski c195352e70 Remove mock of not existing method in L3 agent UT.
There was mock of ri._get_floatingips_bound_to_host() in
L3 test_agent unit test module. This method was removed long time
ago in [1] so this mock is not necessary anymore.



Change-Id: Ia93cab667f8154663ba62b78bc0329ee16b8202c
4 years ago
Zuul 81ffd710c0 Merge "Revert "Pass network's dns_domain to dnsmasq conf"" 4 years ago
Zuul 6ceba7aa47 Merge "Add ip_monitor command implemented using Pyroute2" 4 years ago
Zuul b7a37b3192 Merge "Optimize the code that fixes the race condition of DHCP agent." 4 years ago
Zuul 1f2f538895 Merge "Add TC filtering for VXLAN traffic" 4 years ago
Rodolfo Alonso Hernandez a477c31a23 Add ip_monitor command implemented using Pyroute2
This method allows to track any IP address change in a
namespace. In future patches, this method will replace
the current IP monitor used in the keepalived_state_change
daemon. The current implementation relays in a spawned shell,
executed in root mode, and the output of this shell,
conveniently parsed.

If the passed namespace is not None, this new method must
be executed in privileged mode (root user), but cannot use
privsep because is a blocking function and can exhaust the
number of working threads.

This function should be executed in a parallel thread, returning
the data using the eventlet queue. Pyroute does not implement yet
a non blocking method to retrieve the command output or to know if
the buffer has data. This method, spawned in a greenthread, must be
stopped by killing this thread.

An example of how to use it can be found in the functional tests
implemented in this patch.

Change-Id: I86e4487035d60e1b52e951dd3cd50d6bb54f388b
Related-Bug: #1680183
4 years ago
Boden R 03ec94ce5e use test tools from neutron-lib
This patch switches the code over to use neutron-lib's test tools module
where appropriate rather than using neutron's.
This includes removing the following functions/classes from neutron and
using them from lib instead:
- get_random_EUI
- get_random_ip_network
- reset_random_seed
- OpenFixture

Change-Id: I0fbfcc7919f1b17b6bb0026fa9b98f157168255e
4 years ago
Zuul c3e611eaf1 Merge "Add kill hooks for external processes" 4 years ago
Slawek Kaplonski 93015527f0 Add kill hooks for external processes
This patch adds possibility to configure kill hooks used to kill
external processes, like dnsmasq or keepalived.

Change-Id: I29dfbedfb7167982323dcff1c4554ee780cc48db
Closes-Bug: #1825943
4 years ago
Slawek Kaplonski db119dfe6c Fix mock of execute module in fdb_population UT
In some of unit tests in test_fdb_population module, there was mocked
neutron.agent.linux.utils.execute function
but in fact used function is neutron.agent.common.utils.execute which in
case of Linux is in fact the same but mock didn't work properly and
execute() was really run during UT.

Now proper function is mocked and it is not executing any commands on
OS where tests are running.

Change-Id: I327c5130a9c63942bd6bfb3a3c5804c88c981034
4 years ago
James Page 8f5020120e Revert "Pass network's dns_domain to dnsmasq conf"
The dns_domain attribute of a network is intended for use
by neutron when creating DNS records in an external DNS
system such as Designate.

By using the networks dns_domain, the configured search
path on booted instances mismatches with the generated
dns assignments for instance ports in the hosts file
for dnsmasq which creates a mismatched forward/reverse
lookup behaviour.

This reverts commit 137a6d6105
and commit 7fdd6adc7a.

Closes-Bug: 1826419
Depends-On: I145144c042b100f7e12a02a8ac7e0fbbe41e984d
Change-Id: I5ff03b5ad8af432a9f7919ef953d7d8c434b93bd
4 years ago
Zuul 3e6376f428 Merge "Add Smart NIC representor port to integration bridge" 4 years ago
Zuul e7629fc1c5 Merge "Support multiple external networks in L3 agent" 4 years ago
Rodolfo Alonso Hernandez 5f99c79a50 Add TC filtering for VXLAN traffic
This new function in neutron.agent.linux.tc_lib creates a TC filter in
a device depending on the VXLAN ID (VNI) and the source MAC address
(usually the VM TAP MAC address). This filter will send all the egress
tunneled traffic from the VM to a TC class in other to shape it (QoS).

Change-Id: Ic04b52bc0aca7a18fa06ea89e981c80d67f42eb1
Related-Bug: #1560963
4 years ago
Miguel Lavalle 0b3f5f429d Support multiple external networks in L3 agent
Change [1] removed the deprecated option external_network_bridge. Per
commit message in change [2], "l3 agent can handle any networks by
setting the neutron parameter external_network_bridge and
gateway_external_network_id to empty". So the consequence of [1] was to
introduce a regression whereby multiple external networks are not
supported by the L3 agent anymore.

This change proposes a new simplified rule. If
gateway_external_network_id is defined, that is the network that the L3
agent will use. If not and multiple external networks exist, the L3
agent will handle any of them.


Change-Id: Idd766bd069eda85ab6876a78b8b050ee5ab66cf6
Closes-Bug: #1824571
4 years ago
Ihar Hrachyshka 4aeec20001 Drop of_interface option
Default value for "of_interface" config option was switched
to "native" in Pike release.
In the same release this option was deprecated to removal.
Now it's time to remove it and force use of "native" driver to
manage openflows.

Change-Id: Ic900209868acfbe3bbb56fabbbf5c4472857e412
Co-Authored-By: Ihar Hrachyshka <>
Co-Authored-By: Slawek Kaplonski <>
4 years ago
Hamdy Khader b4243ad3f7 Add Smart NIC representor port to integration bridge
In case of Smart NIC vNIC type neutron should mimic nova-compute
that plug the port to the ovs bridge.

Extend the Neutron OVS mechanism driver and Neutron OVS Agent to bind
the Neutron port for the baremetal host with Smart NIC. This will allow
the Neutron OVS Agent to configure the pipeline of the OVS running on
the Smart NIC and leverage the pipeline features such as: VXLAN,
Security Groups and ARP Responder.

Story: #2003346
Closes-Bug: #1785608
Change-Id: I6d520d3bac2e9ceb30b5b6197c6eb0f958cc3659
4 years ago
Zuul 8887343782 Merge "Change provider network segmentation ID in OVS agent" 4 years ago
Yang JianFeng 494b65d951 Optimize the code that fixes the race condition of DHCP agent.
The above patchs that resolve the race condition of DHCP agent will
result in neutron-server raise DhcpPortInUse ERROR log. And, the
second patch may result in old dhcp agent create a redundant port.

Closes-Bug: #1829332
Change-Id: If7a7ac2f88ce5b0e799c1104c936735a6cc860aa
4 years ago
Zuul c3bad545f6 Merge "[DHCP] Don't resync network if same port is alredy in cache" 4 years ago
Rodolfo Alonso Hernandez 2bcc178be1 Change provider network segmentation ID in OVS agent
Added the ability to change the segmentation ID of a network
with ports bound to OVS agent. The rules, both in the integration
bridge and the physical bridge, to convert the internal VLAN tag
and the external segmentation ID (external VLAN tag) are deleted
and created again with the new value. The traffic from the tenant
networks will be tagged then with the new segmentation ID.

Added get network details agent RPC call to retrieve the information
of the updated network.

Partial-Bug: #1806052

Change-Id: I69f6f3ef717c3ed40218099b1f389afd3d39bd62
4 years ago
Zuul 1134b2a44d Merge "Use --bind-dynamic with dnsmasq instead of --bind-interfaces" 4 years ago
Zuul 1bc30c915c Merge "Make sure the port still in port map when prepare_port_filter" 4 years ago
Brian Haley 09ee934786 Use --bind-dynamic with dnsmasq instead of --bind-interfaces
Dnsmasq emits a warning when started in most neutron deployments:

dnsmasq[27287]: LOUD WARNING: use --bind-dynamic rather than
    --bind-interfaces to avoid DNS amplification attacks via
    these interface(s)

Since option --bind-dynamic is available since dnsmasq 2.63
( and
we require 2.67, change to use this option instead.

Change-Id: Id7971bd99b04aca38180ff109f542422b1a925d5
Closes-bug: #1828473
4 years ago
Slawek Kaplonski 5c433a027d [DHCP] Don't resync network if same port is alredy in cache
Sometimes when port is created on dhcp agent's side, it may happend
that same port is already in network cache.
Before this patch if port with same IP address was already in cache,
resync was rescheduled because of duplicate IPs found in cache.

Now resync will be scheduled only if duplicate IP address belongs to
port with different MAC address or different id.

Change-Id: I23afbc10725f5dc78e3c63e6e505ef89ba8dc4a5
Closes-Bug: #1824802
4 years ago
Yang Li 82782d3763 Make sure the port still in port map when prepare_port_filter
The current code will remove the port from sg_port_map, but then it
won't be added into the map, when we resize/migrate this instance,
the related openflow won't be deleted, this will cause vm connectivity

Closes-Bug: #1825295
Change-Id: I94ddddda3c1960d43893c7a367a81279d429e469
4 years ago
Rodolfo Alonso Hernandez 9bc45d70c6 Error in "tc_lib._handle_from_hex_to_string" formatting
"tc_lib._handle_from_hex_to_string" should print major and minor values
in hex format, not in decimal format:
  0xMMMMmmmm -> "M:m"
  0x123A456B -> "123A:456B"

Change-Id: I91eb5d9fc58e8233c48b6aabba772cd6ff65a156
Closes-Bug: #1826570
4 years ago
LIU Yulong 45957f12c8 Keep HA ports info for HA router during entire lifecycle
Once HA port is set, it must remain this value no matter
what the server return. Because there is race condition
between l3-agent side sync router info for processing
and server side router deleting.

This patch adds a helper function for every ha_port set
action. If the ha_port is not None, it will always stay
with original value.

Closes-Bug: #1826726
Change-Id: I96a088d25048be02a9c5b12c1d087df075b36fc4
4 years ago
Zuul c8047fb747 Merge "Reorder classless static route (RFC3442) records" 4 years ago
Zuul cd6809e143 Merge "Trivalfix: set the right function to fip qos test" 4 years ago
Zuul b43aad9d75 Merge "Add namespace support for "bridge" commands" 4 years ago