Commit Graph

251 Commits (db972238418f87a77b8fdf9f0b32d88b54347f0b)

Author SHA1 Message Date
zhangyuhe d286f9ea1e Always set ovs bridge name in vif:binding-details
- This change updates _set_bridge_name to set
  the bridge name field in the vif binding details.
- This change adds the integration_bridge name
  to the agent configuration report.

Closes-Bug: #1788009
Closes-Bug: #1856152

(cherry picked from commit 995744c576)
Change-Id: I454efcb226745c585935d5bd1b3d378f69a55ca2
3 years ago
LIU Yulong c50bdf2329 Give some HA router case specific resources
1. give each HA failover case an independent vrrp_id
2. give each HA port an independent IP address, so the
interface IPs for router HA ports will be: and and and and
VIP of each case will be:


Closes-Bug: #1819160
Change-Id: I1216d96af40449ec16a852cc1f6c4f15c85f4546
(cherry picked from commit c69a87405a)
(cherry picked from commit 2c5957f56d)
4 years ago
Slawek Kaplonski 570f6086c0 Fix fullstack test_dscp_marking_packets test
Fullstack test test_dscp_marking_packets spawns tcpdump
process inside one fake vm's namespace and then tries to
ping this instance from second one.
After that it checks if tcdump captured any packet marked with
specific DSCP mark.

Ping was done usually only once (1 packet) because
it was done by using vm.wait_until_ping() method, so it could
happen sometimes that ping was send before tcpdump actually
started captuting traffic.
In such case test failed because there was no any packet captured.

This patch changes this by:
1. Start tcpdump async process with block=True, so it should be
   already really started before test will go to the next steps,
2. Send always 10 packets instead of (usually) only one.

In addition this patch adds logging of captured tcpdump's stdout
and stderr streams. It may help debugging issues with this test in
the future.

Change-Id: I23bbde59af0250267843623dde2c5407059d9db2
Closes-Bug: #1818335
(cherry picked from commit 8c2a16796b)
4 years ago
Zuul 8bb898705f Merge "Fix IPv6 prefix delegation issue on agent restart" into stable/rocky 5 years ago
LIU Yulong 2a1cdf01b5 Install centralized floating IP nat rules to all ha nodes
For L3 DVR HA router, the centralized floating IP nat rules are not
installed in every HA node snat namespace. So, install the rules to
all the router snat-namespace on every scheduled HA router host.


Closes-Bug: #1793527
Change-Id: I08132510b3ed374a3f85146498f3624a103873d7
(cherry picked from commit ee7660f593)
5 years ago
Brian Haley b2cd92241f Fix IPv6 prefix delegation issue on agent restart
On l3-agent restart, prefix delegation subnets weren't always
inserted into the local router_info cache, leading to a missing
ip6tables rule.  Add it when the internal network is configured
if the prefix has already been assigned.

Change-Id: Ic045e2763ba2772bcaf037591821501e84e40878
Closes-bug: #1789403
(cherry picked from commit d19dcf1ef2)
5 years ago
IWAMOTO Toshihiro 0f180d8e2e Adopt hacking 1.1.0
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8.  This ensures future python3 compatibility
and a bit better code styling.

Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
5 years ago
Miguel Angel Ajo 88f5e11d8b Avoid agents adding ports as trunk by default.
Agent OVS interface code adds ports without a vlan tag,
if neutron-openvswitch-agent fails to set the tag, or takes
too long, the port will be a trunk port, receiving
traffic from the external network or any other port
sending traffic on br-int.

Also, those kinds of ports are triggering a code path
on the ovs-vswitchd revalidator thread which can eventually
hog the CPU of the host (that's a bug under investigation [1])


Co-Authored-By: Slawek Kaplonski <>
Change-Id: I024bbbdf7059835b2f23c264b48478c71633a43c
Closes-Bug: 1767422
5 years ago
Brian Haley 7cfdf4aa81 Fix all pep8 E129 errors
Fixed all pep8 E129 errors and changed tox.ini to no longer
ignore them.

Change-Id: I0b06d99ce1d473b79a4cfdd173baa4f02e653847
5 years ago
Zuul d49a2beeee Merge "fullstack: Migration from iptables_hybrid to openvswitch" 5 years ago
Jakub Libosvar 75d28cbc73 fullstack: Migration from iptables_hybrid to openvswitch
Add test validating migration from iptables_hybrid firewall driver to
openvswitch. The test creates simple environment with a single node then
spawns two vms, each has its own security group. Then firewall is
switched and OVS agent is restarted. Connectivity is then validated
again, security groups are removed, tested no traffic is allowed and
then security groups are added back to make sure new firewall driver
works with updates.

Change-Id: Idef80c76c1b82be9f1007f17ea661c9ccdc2b1ae
5 years ago
Jakub Libosvar 8b2c40366b ovs-fw: Apply openflow rules immediately during update
Because update operation updates openflow rules three times:
 1) New rules with new cookie
 2) Delete old rules with old cookie
 3) Change new cookie back to old cookie

and the step 2) uses --strict parameter, it's needed to apply rules
before deleting the old rules because --strict parameter cannot be
combined with non-strict. This patch applies openflow rules after
step 1), then --strict rules in step 2 are applied right away and then
rest of delete part from 2) and all new rules from 3) are applied

This patch adds optional interval parameter to Pinger class which sends
more ICMP packets per second in the firewall blink tests to increase a
chance of sending a packet while firewall is in inconsistent state.

Change-Id: I25d9c87225feda1b5ddd442dd01529424186e05b
Closes-bug: #1708731
5 years ago
Sławek Kapłoński 275d406511 Remove usage of TestCase.skip() method
It is deprecated since version 1.9.0 of testtools.
Instead of this deprecated method skipTest() is now used.

Change-Id: I7b6bbdbf43e7f428d5b3873a07fba7e6d629fa82
5 years ago
Boden R ef93f7e7f0 use common agent topics from neutron-lib
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.


Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
5 years ago
Boden R d55e824310 use EGRESS_DIRECTION and INGRESS_DIRECTION from neutron-lib
The EGRESS_DIRECTION and INGRESS_DIRECTION constants live in neutron-lib
now. This patch removes them from neutron and uses lib's version of


Change-Id: I1b81f5c3de9e6f2c0967c2db23ddb716ee7ec6b9
5 years ago
Sławek Kapłoński 6407a71d06 [Fullstack] Unmark test_dscp_marking_packets as unstable
Bug 1733649 is now closed with patch

This commit removes decorator which marked fullstack test
as unstable.

This commit removes also some additional logs of tcpdump's
output added only as temporary "debug" solution.

Change-Id: I0343b1b4fd8d6eed1c26838bf70bce9868e6ee0c
Related-Bug: #1733649
5 years ago
Sławek Kapłoński 386ec26b3d [Fullstack] Change how DSCP marking packets are tested
In fullstack test test_dscp_marking_packets two fake hosts are
created, each of them has got one fake instance (called sender and
Instance called "sender" sends ICMP packets to instance called
"receiver". In receiver's namespace tcpdump process is spawned with
filter to match only packets marked with specified DSCP value.
After sender instance successfully pings receiver, tcpdump process
is killed and stdout from it was examined to search logged ICMP
packets which were send from sender's IP to receiver's IP address.
That check was failing sometimes as is described in bug report.

It was failing when tcpdump doesn't capture any packets so there was
nothing on stdout from it. But even in such case tcpdump reports on
stderr summary of packets and it was like below:

0 packets captured
6 packets received by filter
0 packets dropped by kernel

which means that packets matching our filter was received by tcpdump but
were probably not processed yet. See [1] for more details.

So this patch changes filter used in tcpdump and way how its output is
Now in filter expression there are added src and dst IP addresses also so
it will match only packets send from sender instance to receiver instance.
After that tcpdump's stderr output is examined with regex to check if
line like:

X packets received by filter

is there and if X value is different than 0. If so, it means that
tcpdump received packets with wanted DSCP mark and test should pass.


Change-Id: Ia3522237dc787edb90d162ac4a5535ff5d2a03d5
Closes-Bug: #1733649
5 years ago
Sławek Kapłoński b841a7f5d9 [Fullstack] Shutdown tcpdump process gracefully
In fullstack test for dscp marking packets tcpdump is used to capture
packets send from one namespace (Fake VM) to another one.
When tcpdump is closed quickly with SIGKILL signal it may not displays
all packets and that might cause an issue with failed dscp marking test.

Now tcpdump will be kill with SIGINT signal so it should display
all captured packets to the stdout.
Also it should now return to stderr summary of all captured/dropped
packets which can be helpful in future debugging of issue with those
tests if that will not solve the problem.

Change-Id: Ib2d588d081a48f4f2b6e98a943bca95b9955a149
Related-Bug: #1733649
6 years ago
LIU Yulong d0aaae2800 [L3][QoS] L3 agent side Floating IP rate limit
This patch implements the L3 agent side floating IP rate limit.
For all routers, if floating IP has qos rules, the corresponding
TC filters will be added to:
1. for legacy/HA router, the device is qg-device of qrouter-namespace,
   aka router gateway in network node.
2. for dvr (HA) router in compute node, the device is rfp-device, the
   namespace is qrouter-namespace.
3. for dvr (HA) router in network node, the device is qg-device in

Partially-Implements blueprint: floating-ip-rate-limit

Change-Id: Ie8a5fe4ebaeccfb3998732dd972663c54542b5bf
6 years ago
Sławek Kapłoński 7d5247bc4c [Fullstack] Additional log of tcpdump stderr output
This patch adds additional logging of tcpdump stderr output in
DSCP marking packets tests.
It is required to figure out why those tests are failing sometimes.

Change-Id: I02fe64bf817b47970e4e073f48ea8dea53bdd4f5
Related-Bug: #1733649
6 years ago
Sławek Kapłoński 034934aaf9 [Fullstack] Log tcpdump output lines in DSCP mark tests
This patch adds additional logging of captured tcpdump output during
dscp mark packets tests.

Change-Id: I1ab5727ea70afbe19d2557b9b5c4d77500658cbc
Related-Bug: #1733649
6 years ago
Boden R 502e99bca9 use PROVISIONAL_IPV6_PD_PREFIX from neutron-lib
Commit I9642ed9b513a43c5558f9611f43227299707284a rehomed the
PROVISIONAL_IPV6_PD_PREFIX constant into neutron-lib. This patch
consumes it removing the constant in neutron and using lib's version
of it instead.


Change-Id: I107cb5e0ff2f3e2c5bb9dc501f420d0be08735a0
6 years ago
Jenkins 8df4cf30de Merge "Change ip_lib network namespace code to use pyroute2" 6 years ago
Dongcan Ye 5907b1d924 Fix wrong OS ENV type
If we set environment for Neutron tests in tox.ini, we may get
type error like:
TypeError: %d format: a number is required, not str

os.environ.get method will get string, not integer. This patch
fix it.

Change-Id: Ie71302b8e33586082ae8334e6317e30d382e893a
6 years ago
Brian Haley 4f627b4e8d Change ip_lib network namespace code to use pyroute2
Change network namespace add/delete/list code to use
pyroute2 library instead of calling /sbin/ip.

Also changed all in-tree callers to use the new calls.

Closes-bug: #1717582
Related-bug: #1492714

Change-Id: Id802e77543177fbb95ff15c2c7361172e8824633
6 years ago
Boden R 60f8048c7c use synchronized lock decorator from neutron-lib
neutron-lib contains the synchronized lockutils decorator as well as
the SYNCHRONIZED_PREFIX global. This patch consumes them from
neutron-lib and removes them from neutron.


Change-Id: I729da348e340509f2d09f8a6436716e2398f1583
6 years ago
Sławek Kapłoński 69b8588a19 Remove translation of help messages from tests
There is no need to translate help message for config option
which is added and used only in tests.

Change-Id: I8b8bdd4131e6183245bcd2556b2107d769fd6427
6 years ago
Armando Migliaccio c7ad529983 make net_helpers functions work on OpenSUSE
/sbin may not be in the regular user's PATH or tools like sysctl/ss
may require root privileges to execute correctly on OpenSUSE, and this
makes net_helpers functions fail with OSError. There is no harm in
running ss or sysctl as root user for these functions and that allows
fullstack/functional tests to operate correctly on OpenSUSE.

The change requires a testcase to inherit from BaseSudoTestCase due
to the new run_as_root=True flag.

Change-Id: Ia4f2af1d44faacf5f7ab5471b4f18ecb27f06549
6 years ago
sindhu devale 51ca683797 Refactoring agent linux&ovsdb config
Refactoring neutron agent linux and ovsdb config opts
to be in neutron/conf/agent so that all the config options
reside in a centralized location. This simplifies the
process of looking up the config opts and provides an easy
way to import.


Change-Id: Ib1e0e63dec2985c417412d1ecc68e2a74ef87182
Partial-Bug: #1563069
6 years ago
Jakub Libosvar a41ce8d6bb tests: Log spawned processes by RootHelperProcess
This patch logs the command spawned by RootHelperProcess, it is
handy when debugging failed functional tests on upstream gate.

Change-Id: I743a223c4ff4882fdd760a20823150558d5e5f4a
6 years ago
Ihar Hrachyshka cc69828ff0 Apply network MTU changes to l3 ports
This patch makes L3 agent to update its ports' MTU when it's changed on
core plugin side.

Related-Bug: #1671634
Change-Id: I4444da6358e8b8420a3a365e1107b02f5bb1161d
6 years ago
Jakub Libosvar 6370a04710 ovsfw: Fix overlapping MAC addresses on integration bridge
The patch relies on the fact that traffic not going from instance
(and thus port not managed by firewall) is tagged. Traffic coming from
the instance is not tagged and thus net register is used for marking
such traffic. These two approaches make matching rules unique even if
two ports from different networks share its' mac addressess.

Traffic coming from trusted ports is marked with network in registry
so firewall can decide later to which network traffic belongs.

Closes-bug: #1626010

Change-Id: Ia05d75a01b0469a0eaa82ada67b16a9481c50f1c
6 years ago
Jakub Libosvar d559cd53e8 ovs-fw: Use TRANSIENT table for traffic classification
Commit ce8a0b2b7d introduces a TRANSIENT
table where all traffic local to br-int is sent after it's been
preprocessed by other features using openflow. This patch adopts the

Change-Id: Ic66c186ab73bad6fcd133f2b9d15e07fd0eebb33
Related-bug: #1696983
6 years ago
Jakub Libosvar 330ebbb1b1 functional-tests: Make addresses for tunneling unique
OVS can hold only one tunnel with same endpoints. Some tests had
hardcoded values for both tunnel endpoints which made them unable to run
in parallel manner.

This patch takes always exclusive address using resource allocator.

Change-Id: If81296d54656551b24917d561f235edb96a6c2df
Closes-bug: #1697533
6 years ago
Jenkins 211fb55fab Merge "net_helpers: Set process streams to text mode" 6 years ago
Jenkins fa5babc9a5 Merge "python3: use a list of IPDevice objects in tests" 6 years ago
Jakub Libosvar 5d619e54e2 net_helpers: Set process streams to text mode
RootHelperProcess extends Popen from subprocess and sets all
stdin/stdout/stderr descriptors to PIPE. These descriptors use byte
array by default in Python 3. If universal_newlines [1] is set for Popen
object, then those descriptors work in text mode.


Change-Id: I3fa2192271aed81fb6da658b8196b365a20fa286
6 years ago
Bernard Cafarelli 681f611249
python3: use a list of IPDevice objects in tests
Fixes "TypeError: unhashable type: 'IPDevice'" in

IPDevice class defines an __eq__() method, which in Python 3 disables
the default __hash__() method (and cannot be used in a set). Use a list
instead as it is enough for the test

Change-Id: I09c538908e55df1b8d305265774c57df1ec42f21
6 years ago
Sławek Kapłoński 2d0d1a2d76 Add support for ingress bandwidth limit rules in ovs agent
Add support for QoS ingress bandwidth limiting in
openvswitch agent.
It uses default ovs QoS policies on bandwidth limiting

DocImpact: Ingress bandwidth limit in QoS supported by
           Openvswitch agent

Change-Id: I9d94e27db5d574b61061689dc99f12f095625ca0
Partial-Bug: #1560961
6 years ago
Ihar Hrachyshka fea1b91a88 python3: return str from read_stdout
The result later may be concatenated with another str, and it will then
fail with: TypeError: can't concat bytes to str

It's safer to always return a str into test cases.

Change-Id: I7544322f31b4eda378eb8ee541786ea2574e1cf0
6 years ago
Ihar Hrachyshka d545d19522 Fixed python3 failure in functional tests using net_helpers
.write expects a byte string in python3, while we were passing a
str. It worked in py2 but failed in py3 with:

TypeError: memoryview: a bytes-like object is required, not 'str'

Change-Id: I7f993a06bafa3cad4147f46d5d6dc10efeac3480
6 years ago
Jenkins 0479f0f9d2 Merge "fullstack: Don't let dhcp agents failover" 6 years ago
Jakub Libosvar bc979efdb8 fullstack: Don't let dhcp agents failover
It turned out dhcp tests work only because agents are considered dead
after 10 seconds while they report to server every 60 seconds. This led
to calling network resync after agent revival and hiding the fact dhcp
agent is not capable of receiving any amqp messages.

This patch sets the report interval of agents to the half of
agent_down_time on server side and uses eventlet dhcp agent in order to
trigger eventlet monkey patching code.

Eventlet was behind the failure with messages not getting processed. As
[1] notes: "Note: If the “eventlet” executor is used, the threading and
time library need to be monkeypatched."

Because each port calls dhclient to obtain IP address and each dhclient
instance overwrites /etc/resolv.conf there was added a script that
generates fullstack-dhclient-script from an existing dhclient-script
before starting fulltstack tests. This generated script is passed to
each dhclient process running in fake fullstack machine using -sf


Related-bug: 1453350

Change-Id: I0336176b9c364fe3a95be5cef9e7a3af1ef9d7e9
6 years ago
fpxie 574312165b Replace six.iteritems with dict.items(Part-2)
according to, now we should avoid
using six.iteritems and replace it with dict.items.

Change-Id: I58a399baa2275f280acc0e6d649f81838648ce5c
Closes-Bug: #1680761
6 years ago
Jenkins 5e3abe8040 Merge "Simplify RootHelperProcess._read_stream()" 6 years ago
Jenkins 87cb9d8552 Merge "exclusive_resources: use set remove instead of discard" 6 years ago
Kevin Benton c76164c058 Move conntrack zones to IPTablesFirewall
The regular IPTablesFirewall needs zones to support safely
clearly conntrack entries.

In order to support the single bridge use case, the conntrack
manager had to be refactored slightly to allow zones to be
either unique to ports or unique to networks.

Since all ports in a network share a bridge in the IPTablesDriver
use case, a zone per port cannot be used since there is no way
to distinguish which zone traffic should be checked against when
traffic enters the bridge from outside the system.

A zone per network is adequate for the single bridge per network
solution since it implicitly does not suffer from the double-bridge
cross in a single network that led to per port usage in OVS.[1]

This had to adjust the functional firewall tests to use the correct
bridge name now that it's relevant in the non hybrid IPTables case.

1. Ibe9e49653b2a280ea72cb95c2da64cd94c7739da

Closes-Bug: #1668958
Closes-Bug: #1657260
Change-Id: Ie88237d3fe4807b712a7ec61eb932748c38952cc
6 years ago
John Schwarz 977d254cc6 Throttle SIGHUPs to keepalived
Multiple SIGHUPs in quick succession might cause the master keepalived
to forfeit its mastership (which will cause keepalived to remove IPs of
its external devices, severing connectivity). This can happen when, for
example, associating or disassociating multiple floatingips.

The patch makes the agent throttle SIGHUP sent to keepalived: the very first
SIGHUP is always sent; as for subsequent signals, they are delayed till
agent threshold is reached. (It's 3 seconds by default.)

As an example, when three consequent router updates trigger keepalived
respawn then:
* the very first signal is sent as usual;
* the second signal is deferred and sent in up to 3 seconds since the
  first signal;
* the third signal is ignored, though the change that triggered it will
  be correctly applied by the second signal handler when it is triggered
  after threshold delay.

If the last time a spawn request occurred is older than current-time
minus threshold then there is no delay.

Co-Authored-By: Jakub Libosvar <>
Co-Authored-By: Cedric Brandily <>
Co-Authored-By: Ihar Hrachyshka <>

Closes-Bug: 1647432
Change-Id: I2955e0de835458a2eea4dd088addf33b656f8670
6 years ago
Jenkins 3b119c6471 Merge "Switch RootHelperProcess from select.poll to" 6 years ago
Akihiro Motoki a7898c5495 Simplify RootHelperProcess._read_stream()
select() itself has timeout mechanism, so we do not need to use
wait_until_true wrapper.

Related-Bug: #1674557
Change-Id: I35bc4716f0d1e0d92e7b7a3f6dcb6978e9d725f9
6 years ago