Commit Graph

260 Commits (99f4495c940011293e3cabbb590770dc1e7b6900)

Author SHA1 Message Date
liuchengqian90 58de79a58b Clear residual qos rules after l2-agent restarts.
During the l2-agent stop, if the policy rule is cleared,
after the l2-agent is started, the qos rule that has been applied should be cleared.

Change-Id: Iaaff10dfa8ac6ab8c9dead3124e2bb3caa03a665
Closes-Bug: #1810025
4 years ago
Zuul b832508066 Merge "fullstack: retry on updating port's IP address" 5 years ago
Hongbin Lu ae4b331725 fullstack: retry on updating port's IP address
If the update_port call failed with error IpAddressAlreadyAllocatedClient,
retry a few more times in order to find IP addresses that are available.

Change-Id: I7c5d51b01fa56083b1a689fa629a9a34c8b77012
Closes-Bug: #1808595
5 years ago
Hongbin Lu 02dabd6f10 Add debug information of the dhclient process
Log the standard output/error of the dhclient process which provides
more information of the execution of the DHCP client script.

Change-Id: I6a057d089ee21ea2898078fb7b11dce00a570ec0
5 years ago
Zuul 77a8b020c3 Merge "Change "cmd" folder in fullstack tests" 5 years ago
Rodolfo Alonso Hernandez b617df2b21 Change "cmd" folder in fullstack tests
When fullstack tests are executed manually using a debugger
(e.g.: PyCharm integrated debugger), the "cmd" folder is imported
instead of "cmd" [1] module.

To solve this problem, this folder and the references to this path
must be changed.

[1] https://docs.python.org/3/library/cmd.html

Change-Id: I8e6b6995c10875a882a46ca3a0d779aafda124a3
Closes-Bug: #1805844
5 years ago
Slawek Kaplonski 481211dc3a [Fullstack] Configure policy.json file only if it is set
Path for policy.json file in config of fullstack's neutron-server
is generated automatically. It may happen that this path will have
value ``None`` and in such case it shouldn't be placed in
neutron's config file.

Change-Id: I2416545e7d939f920a1b04bb29bcc1aff86bedd9
5 years ago
Slawek Kaplonski 9b23abbdb6 Add kill_timeout to AsyncProcess
AsyncProcess.stop() method has now additional parameter
kill_timeout. If this is set to some value different than
None, eventlet.green.subprocess.Popen.wait() will be called
with this timeout, so TimeoutExpired exception will be raised
in case if process will not be killed for this "kill_timeout"
time.
In such case process will be killed "again" with SIGKILL signal
to make sure that it is gone.

This should fix problem with failing fullstack tests, when
ovs_agent process is sometimes not killed and test timeout was
reached in this wait() method.

Change-Id: I1e12255e5e142c395adf4e67be9d9da0f7a3d4fd
Closes-Bug: #1798472
5 years ago
LIU Yulong 8c17df7138 Notify router_update after directly gateway IP change
If directly change router gateway port IP address, the gateway IP
does not changed in router related namespace in l3 agent side. This
patch adds a method to catch a 'PORT' IP change event, and notify
the L3 agent.

Closes-Bug: #1795222
Change-Id: If276a7613c156f8c826967c9c8cbd6f2a8d32674
5 years ago
Lajos Katona 9b23e1be07 sriov-agent: fullstack test resource info report
Change-Id: Ibbe656707276d6c3c68b051e6d66d6d388dbfd18
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
5 years ago
Zuul df94340f0f Merge "ovs-agent: fullstack test resource info report" 5 years ago
Zuul 66f0a4d189 Merge "Make port binding attempt after agent is revived" 5 years ago
Slawek Kaplonski f787f12aa3 Make port binding attempt after agent is revived
In some cases it may happen that port is "binding_failed"
because L2 agent running on destination host was down but
this is "temporary" issue.
It is like that for example in case when using L3 HA and when
master and backup network nodes were e.g. rebooted.
L3 agent might start running before L2 agent on host in such case
and if it's new master node, router ports will have "binding_failed"
state.

When agent sends heartbeat and is getting back to live,
ML2 plugin will try to bind all ports with "binding_failed"
from this host.

Change-Id: I3bedb7c22312884cc28aa78aa0f8fbe418f97090
Closes-Bug: #1794809
5 years ago
Lajos Katona 4df02ee93c Enable ingress direction for min_bw rule
Enable QoS minimum-bandwidth rule for ovs and sriov backends for both
ingress and egress directions.
Add qos-bw-limit-direction as supported extension to qos-plugin, and
add create/delete/update_minimum_bandwidth and
delete_minimum_bandwidth_ingress empty methods to sriov/ovs/linuxbridge
extension drivers.

Change-Id: I6eb21ccf0400ea9adae90ff0bf97e08cdb09b8eb
Depends-On: https://review.openstack.org/609368
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
5 years ago
Lajos Katona 277c1e77b2 ovs-agent: fullstack test resource info report
Change-Id: Ia36b05c835339fea8913103dffad1422a9aacaed
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
5 years ago
Brian Haley 2b57f08576 Fix flake8 H404 errors
Fix H404 error and start enforcing it.

Trivialfix

Change-Id: Iaa6fb4f1f07dee32a944259ab65204360d9db7ea
5 years ago
Nguyen Phuong An e4df5d695c Follow-up: add fullstack test for logging
This patch corrects log path and remove redundant code in scenario.

Change-Id: Id59a60b3562e3104b682d42cddc95738b3c3ba14
5 years ago
Nguyen Phuong An 0649112113 Add fullstack test for logging
Since Queens, the security group logging has been merged. But there
is no fullstack test for this feature. So this patch add fullstack
test to avoid regression as https://review.openstack.org/#/c/587681

Co-Authored-By: Yushiro FURUKAWA <y.furukawa_2@jp.fujitsu.com>
Change-Id: Id9bbedc96e399338ea568556bdb17923392512b4
Partially-implements: blueprint security-group-logging
5 years ago
Slawek Kaplonski f9dc83eb95 [Fullstack] Mark securitygroups tests as unstable
It agains fails quite often in iptables scenarios.
We need to debug it more to find out what cause this issue
but for now let's mark this test as unstable to make
life of other people easier.

Change-Id: I7bf6f9b346c6c853193cb045fb364b97375e9d93
Related-bug: #1779328
5 years ago
Zuul a11087ec43 Merge "Trivial: Move platform independent modules to common dir" 5 years ago
Zuul 9798d2bf81 Merge "Add fullstack test to restart agent with active l3-ha router" 5 years ago
Slawek Kaplonski 0bd1cec676 [Fullstack] Add debug_iptables=True in security groups tests
In security groups tests with "iptables" or "iptables_hybrid"
driver it will be useful to have debug_iptables_rules enabled
to check what rules are applied by L2 agent in case of test
failures.

Change-Id: Ib7d12b2e589019bc6043affe371ef5aa5425945b
Related-Bug: #1779328
5 years ago
Lucian Petrut 89915a752e Trivial: Move platform independent modules to common dir
async_process.py and ovsdb_monitor.py are now platform
independent, for which reason we can move them to
neutron/agent/common.

Note that a few subprojects are using async_process. We'll use
debtcollector so that we don't break those projects, while logging
a deprecation warning.

Change-Id: I6a7418cb8680cd71fe16c5d98b9b09ef2d260d37
5 years ago
Bernard Cafarelli 3d06c63e74
[Fullstack] Use string for global_physnet_mtu config option
This was added in Ia838d2a661c5098f90b58b2cb31557f2ebf78868 and breaks
config parser with python3

Closes-Bug: #1783095
Change-Id: I1cd054edb32e7ccf3bd3a356ed535a4a6003a9a1
5 years ago
Zuul c563530af5 Merge "[Fullstack] Remove central_external_bridge" 5 years ago
Zuul f6bcfe1d89 Merge "Fix the tests for filtering with qos_policy_id" 5 years ago
Brian Haley 88a0ebbe7f Add fullstack test to restart agent with active l3-ha router
Re-start of the l3 agent hosting the active l3-ha router
shouldn't cause data plane interruption, assuming there
is no failover.  Create a test explicitly for that.

Change-Id: I5963c21e2b382a09c40b81e2446350696e16d265
Related-Bug: #1776459
5 years ago
Zuul d786643214 Merge "Skip MTU check during deletion of Networks" 5 years ago
Slawek Kaplonski fd126a37c0 [Fullstack] Remove central_external_bridge
It is not necessary to use 2 "central" bridges, one for "data" and
one for "external" network simulation.
Also using 2 bridge will cause problems when "external_network_bridge"
option will be removed from L3 agent and it will use integration bridge
as it should be done.

Change-Id: I68ee51cbc148b2bfce0cba8de7cf9fe08df54c96
5 years ago
Slawek Kaplonski c57a5d9d8a [Fullstack] HA L3 agent restart only standby agents
In fullstack test which is testing if there is no packet lost
during restart of agents there were restarted always all agents
which hosted router.
In case when as first was restarted 'master' agent it might
lead to the case when after restart 'master' node was switched
to second L3 agent and that caused lost of few packets and
failed test.
This test should only check if restart of standby agents will
not cause any packet lost so this patch do it in this way.

Change-Id: I6293169d7d7f35e3a9726071e63003ac569dd01e
Closes-Bug: #1776459
5 years ago
Zuul a388701ddf Merge "[Fullstack] Use string for api_worker config option" 5 years ago
Bernard Cafarelli fad8e4d2cd
[Fullstack] Use string for api_worker config option
Same fix as in Iae6a559451ab03f4a5410626e50ff2c0aa634aea, this causes
tests to fail at configuration load with Python 3

Change-Id: I8f1131bf98e38c716925c562fb95d94bc341cbb0
5 years ago
Reedip 006113e3bf Skip MTU check during deletion of Networks
MTU check can be skipped during deletion of Networks.
The MTU check doesn't provide any additional support during deletion
of the networks.

Also, if a network is created with MTU 'X' and the
global_mtu later on is decreased to 'Y', the created
network cannot be deleted due to the MTU check.

Change-Id: Ia838d2a661c5098f90b58b2cb31557f2ebf78868
Closes-Bug: #1713499
5 years ago
Slawek Kaplonski 0b3a64480b [Fullstack] Ensure connectivity to ext gw before agents restart
In TestHAL3Agent.test_ha_router_restart_agents_no_packet_lost
fullstack test we should first ensure that connection from external_vm
to router's external gateway is possible. If it's fine, we
can restart L3 agents and test if connectivity will not be broken.

Change-Id: I1f153c553cd2dfa846ce80c166e2a35acd9169a3
Related-Bug: #1776459
5 years ago
Hongbin Lu 6f5946d34b Fix the tests for filtering with qos_policy_id
Some QoS tests tried to list ports by attribute qos_policy_id
but this attribute is not a valid filter. In before, the tests
passed because neutron ignored the invalid filter and returned
all the ports which happened to be the correct set. However,
using qos_policy_id as filter is incorrect and this patch fixes it.

Change-Id: Ic3ab5b3ffdc378d570678b9c967cb42b0c7a8a9b
Related-Bug: #1749820
5 years ago
Zuul 2458506344 Merge "Mark test_ha_router_restart_agents_no_packet_lost as unstable" 5 years ago
Slawek Kaplonski ba1e1eb4dd Mark test_ha_router_restart_agents_no_packet_lost as unstable
Fullstack test
test_l3_agent.TestHAL3Agent.test_ha_router_restart_agents_no_packet_lost
is marked as unstable now becuase it is failing quite often recently.
We need to figure out what is the reason of this issue but
to not block gates with many failures, let's mark it as unstable
for now.

Change-Id: I21e590a24390345dfe451b035fd973928445e987
Related-Bug: #1776459
5 years ago
Miguel Lavalle 9237cf374e Improve error message in fullstack test
_assert_ping_during_agents_restart is used in tests of L2 and L3
agents. However, when it raises an exception due to a timeout, the
associated message assumes the agent under test is L2. This patch
fixes that

Change-Id: I3568c97a621e97699fcd93f09897e132d4db402a
5 years ago
Zuul d573e496de Merge "[Fullstack] Wait for SG to be applied by L2 agent" 5 years ago
Slawek Kaplonski 11b41d73b4 [Fullstack] Wait for SG to be applied by L2 agent
In fullstack test_securitygroup there were used simple
net_helpers.assert_ping() and assert_not_ping() functions
which tries to ping IP address 3 times with some short timeout
and test fails if result of ping will not be as expected.

Unfortunately sometimes in fullstack tests it might be not enough
if test is creating new SG or SG rule or apply SG to port and
just after that checks connection because L2 agent don't have
enough time to apply all rules on "host".
This patch changes it to use block_until_ping() and
block_until_no_ping() methods from FakeMachine fixture.
It will also check if ping is possible/not possible but
will try to check it for 1 minute before fails.

Similar change is also done for methods which checks TCP
connectivity using netcat helper class. It now uses
common_utils.wait_until_true() helper function instead of
fail immediatelly.

Change-Id: I9e523d803e3c49d5d090ae5b9d36d43ce7311535
Closes-Bug: #1774006
Closes-Bug: #1767829
5 years ago
Slawek Kaplonski 32f14aa8a0 [Fullstack] Change time waiting for async ping results
During agents restart there is async ping run and there is
called function to wait until all async ping workers will
finish their job.

In TestHAL3Agent.test_ha_router_restart_agents_no_packet_lost
there are 60 pings sent with 1 second timeout so default
wait_until_true timeout which is set to 60 seconds might not
be enough in some cases.
Because of that wait_until_true timeout is now set as
twice higher value than is needed to number of packets to send
with ping_timeout.

This should give enough time to finish all workers.

Change-Id: Ia7c3755c2ba5029bdab3c1dd30b305f3bde19740
Closes-Bug: #1775183
5 years ago
Slawek Kaplonski 3e9e2a5b4b Disable IPv6 forwarding by default on HA routers
In case of HA routers IPv6 forwarding is not disabled by default and
then enabled only on master node.
Before this patch it was done in opposite way, so forwarding was
enabled by default and then disabled on backup nodes.
When forwarding was enabled/disabled for qg- port, MLDv2 packets are
sent and that might lead to temportary packets loss as packets to
FIP were sent to this backup node instead of master one.

Related-Bug: #1771841

Change-Id: Ia6b772e91c1f94612ca29d7082eca999372e60d6
5 years ago
Zuul 9b2d1d53c8 Merge "Fullstack: Add using multiple security groups" 5 years ago
Dongcan Ye 1658e353c3 Fullstack: Add using multiple security groups
Change-Id: I8eadb434be3e7d0849a6e4c3bdf75dbfb5f15a83
Closes-Bug: #1690998
5 years ago
Zuul 06b9603f0c Merge "Monitor phys_bridges to reconfigured it if created again" 5 years ago
Sławek Kapłoński 85b46cd51e Monitor phys_bridges to reconfigured it if created again
In case when external bridge configured in OVS agent's bridge_mappings
will be destroyed and created again (for example by running ifup-ovs
script on Centos) bridge wasn't configured by OVS agent.
That might cause broken connectivity for OpenStack's dataplane if
dataplane network also uses same bridge.

This patch adds additional ovsdb-monitor to monitor if any
of physical bridges configured in bridge_mappings was created.
If so, agent will reconfigure it to restore proper openflow rules
on it.

Change-Id: I9c0dc587e70327e03be5a64522d0c679665f79bd
Closes-Bug: #1768990
5 years ago
Brian Haley 7cfdf4aa81 Fix all pep8 E129 errors
Fixed all pep8 E129 errors and changed tox.ini to no longer
ignore them.

Change-Id: I0b06d99ce1d473b79a4cfdd173baa4f02e653847
5 years ago
Brian Haley c3b83a9ca6 Fix all pep8 E265 errors
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them.  Also removed an N536 comment missed from a
previous change.

Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
5 years ago
Jakub Libosvar 75d28cbc73 fullstack: Migration from iptables_hybrid to openvswitch
Add test validating migration from iptables_hybrid firewall driver to
openvswitch. The test creates simple environment with a single node then
spawns two vms, each has its own security group. Then firewall is
switched and OVS agent is restarted. Connectivity is then validated
again, security groups are removed, tested no traffic is allowed and
then security groups are added back to make sure new firewall driver
works with updates.

Change-Id: Idef80c76c1b82be9f1007f17ea661c9ccdc2b1ae
5 years ago
Jakub Libosvar 98e5f01714 fullstack: Simplify ConfigFixture
ConfigFixture composed ConfigFileFixture instead of inheritance. This
patch uses inheritance over composition in order to make
ConfigFileFixture attributes accesible from the outside.

Change-Id: I8c1c38245867111cb46c6f9493529ff3374593a0
5 years ago