This patch enables to bind a QoS policy to the router gateway,
then in L3 agent side SNAT traffic for the VMs without floating
IPs can be limited under the policy bandwidth rules. This is
suit for all kinds of L3 routers: DVR, DVR with SNAT HA, L3 HA
API update router gateway json:
Partially-Implements blueprint: router-gateway-ip-qos
The bulk port creation scenario requires the ability to generate
multiple MAC addresses for the bulk added ports. This change leverages
the code added in  to make bulk MAC creation available.
Implements: blueprint speed-up-neutron-bulk-creation
For L3 DVR HA router, the centralized floating IP nat rules are not
installed in every HA node snat namespace. So, install the rules to
all the router snat-namespace on every scheduled HA router host.
- This change updates _set_bridge_name to set
the bridge name field in the vif binding details.
- This change adds the integration_bridge name
to the agent configuration report.
On l3-agent restart, prefix delegation subnets weren't always
inserted into the local router_info cache, leading to a missing
ip6tables rule. Add it when the internal network is configured
if the prefix has already been assigned.
async_process.py and ovsdb_monitor.py are now platform
independent, for which reason we can move them to
Note that a few subprojects are using async_process. We'll use
debtcollector so that we don't break those projects, while logging
a deprecation warning.
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8. This ensures future python3 compatibility
and a bit better code styling.
Agent OVS interface code adds ports without a vlan tag,
if neutron-openvswitch-agent fails to set the tag, or takes
too long, the port will be a trunk port, receiving
traffic from the external network or any other port
sending traffic on br-int.
Also, those kinds of ports are triggering a code path
on the ovs-vswitchd revalidator thread which can eventually
hog the CPU of the host (that's a bug under investigation )
Co-Authored-By: Slawek Kaplonski <email@example.com>
Add test validating migration from iptables_hybrid firewall driver to
openvswitch. The test creates simple environment with a single node then
spawns two vms, each has its own security group. Then firewall is
switched and OVS agent is restarted. Connectivity is then validated
again, security groups are removed, tested no traffic is allowed and
then security groups are added back to make sure new firewall driver
works with updates.
Because update operation updates openflow rules three times:
1) New rules with new cookie
2) Delete old rules with old cookie
3) Change new cookie back to old cookie
and the step 2) uses --strict parameter, it's needed to apply rules
before deleting the old rules because --strict parameter cannot be
combined with non-strict. This patch applies openflow rules after
step 1), then --strict rules in step 2 are applied right away and then
rest of delete part from 2) and all new rules from 3) are applied
This patch adds optional interval parameter to Pinger class which sends
more ICMP packets per second in the firewall blink tests to increase a
chance of sending a packet while firewall is in inconsistent state.
The neutron.common.topics module was rehomed into neutron-lib with
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
The EGRESS_DIRECTION and INGRESS_DIRECTION constants live in neutron-lib
now. This patch removes them from neutron and uses lib's version of
Bug 1733649 is now closed with patch
This commit removes decorator which marked fullstack test
This commit removes also some additional logs of tcpdump's
output added only as temporary "debug" solution.
In fullstack test test_dscp_marking_packets two fake hosts are
created, each of them has got one fake instance (called sender and
Instance called "sender" sends ICMP packets to instance called
"receiver". In receiver's namespace tcpdump process is spawned with
filter to match only packets marked with specified DSCP value.
After sender instance successfully pings receiver, tcpdump process
is killed and stdout from it was examined to search logged ICMP
packets which were send from sender's IP to receiver's IP address.
That check was failing sometimes as is described in bug report.
It was failing when tcpdump doesn't capture any packets so there was
nothing on stdout from it. But even in such case tcpdump reports on
stderr summary of packets and it was like below:
0 packets captured
6 packets received by filter
0 packets dropped by kernel
which means that packets matching our filter was received by tcpdump but
were probably not processed yet. See  for more details.
So this patch changes filter used in tcpdump and way how its output is
Now in filter expression there are added src and dst IP addresses also so
it will match only packets send from sender instance to receiver instance.
After that tcpdump's stderr output is examined with regex to check if
X packets received by filter
is there and if X value is different than 0. If so, it means that
tcpdump received packets with wanted DSCP mark and test should pass.
In fullstack test for dscp marking packets tcpdump is used to capture
packets send from one namespace (Fake VM) to another one.
When tcpdump is closed quickly with SIGKILL signal it may not displays
all packets and that might cause an issue with failed dscp marking test.
Now tcpdump will be kill with SIGINT signal so it should display
all captured packets to the stdout.
Also it should now return to stderr summary of all captured/dropped
packets which can be helpful in future debugging of issue with those
tests if that will not solve the problem.
This patch implements the L3 agent side floating IP rate limit.
For all routers, if floating IP has qos rules, the corresponding
TC filters will be added to:
1. for legacy/HA router, the device is qg-device of qrouter-namespace,
aka router gateway in network node.
2. for dvr (HA) router in compute node, the device is rfp-device, the
namespace is qrouter-namespace.
3. for dvr (HA) router in network node, the device is qg-device in
Partially-Implements blueprint: floating-ip-rate-limit
This patch adds additional logging of tcpdump stderr output in
DSCP marking packets tests.
It is required to figure out why those tests are failing sometimes.
Commit I9642ed9b513a43c5558f9611f43227299707284a rehomed the
PROVISIONAL_IPV6_PD_PREFIX constant into neutron-lib. This patch
consumes it removing the constant in neutron and using lib's version
of it instead.
If we set environment for Neutron tests in tox.ini, we may get
type error like:
TypeError: %d format: a number is required, not str
os.environ.get method will get string, not integer. This patch
Change network namespace add/delete/list code to use
pyroute2 library instead of calling /sbin/ip.
Also changed all in-tree callers to use the new calls.
neutron-lib contains the synchronized lockutils decorator as well as
the SYNCHRONIZED_PREFIX global. This patch consumes them from
neutron-lib and removes them from neutron.
/sbin may not be in the regular user's PATH or tools like sysctl/ss
may require root privileges to execute correctly on OpenSUSE, and this
makes net_helpers functions fail with OSError. There is no harm in
running ss or sysctl as root user for these functions and that allows
fullstack/functional tests to operate correctly on OpenSUSE.
The change requires a testcase to inherit from BaseSudoTestCase due
to the new run_as_root=True flag.
Refactoring neutron agent linux and ovsdb config opts
to be in neutron/conf/agent so that all the config options
reside in a centralized location. This simplifies the
process of looking up the config opts and provides an easy
way to import.
The patch relies on the fact that traffic not going from instance
(and thus port not managed by firewall) is tagged. Traffic coming from
the instance is not tagged and thus net register is used for marking
such traffic. These two approaches make matching rules unique even if
two ports from different networks share its' mac addressess.
Traffic coming from trusted ports is marked with network in registry
so firewall can decide later to which network traffic belongs.
Commit ce8a0b2b7d introduces a TRANSIENT
table where all traffic local to br-int is sent after it's been
preprocessed by other features using openflow. This patch adopts the
OVS can hold only one tunnel with same endpoints. Some tests had
hardcoded values for both tunnel endpoints which made them unable to run
in parallel manner.
This patch takes always exclusive address using resource allocator.
RootHelperProcess extends Popen from subprocess and sets all
stdin/stdout/stderr descriptors to PIPE. These descriptors use byte
array by default in Python 3. If universal_newlines  is set for Popen
object, then those descriptors work in text mode.
Fixes "TypeError: unhashable type: 'IPDevice'" in
IPDevice class defines an __eq__() method, which in Python 3 disables
the default __hash__() method (and cannot be used in a set). Use a list
instead as it is enough for the test
Add support for QoS ingress bandwidth limiting in
It uses default ovs QoS policies on bandwidth limiting
DocImpact: Ingress bandwidth limit in QoS supported by
The result later may be concatenated with another str, and it will then
fail with: TypeError: can't concat bytes to str
It's safer to always return a str into test cases.
.write expects a byte string in python3, while we were passing a
str. It worked in py2 but failed in py3 with:
TypeError: memoryview: a bytes-like object is required, not 'str'
It turned out dhcp tests work only because agents are considered dead
after 10 seconds while they report to server every 60 seconds. This led
to calling network resync after agent revival and hiding the fact dhcp
agent is not capable of receiving any amqp messages.
This patch sets the report interval of agents to the half of
agent_down_time on server side and uses eventlet dhcp agent in order to
trigger eventlet monkey patching code.
Eventlet was behind the failure with messages not getting processed. As
 notes: "Note: If the “eventlet” executor is used, the threading and
time library need to be monkeypatched."
Because each port calls dhclient to obtain IP address and each dhclient
instance overwrites /etc/resolv.conf there was added a script that
generates fullstack-dhclient-script from an existing dhclient-script
before starting fulltstack tests. This generated script is passed to
each dhclient process running in fake fullstack machine using -sf
The regular IPTablesFirewall needs zones to support safely
clearly conntrack entries.
In order to support the single bridge use case, the conntrack
manager had to be refactored slightly to allow zones to be
either unique to ports or unique to networks.
Since all ports in a network share a bridge in the IPTablesDriver
use case, a zone per port cannot be used since there is no way
to distinguish which zone traffic should be checked against when
traffic enters the bridge from outside the system.
A zone per network is adequate for the single bridge per network
solution since it implicitly does not suffer from the double-bridge
cross in a single network that led to per port usage in OVS.
This had to adjust the functional firewall tests to use the correct
bridge name now that it's relevant in the non hybrid IPTables case.