openstack
/
neutron
Code Issues Proposed changes

Re execute a test case if fixtures.TimeoutException is thrown 

Since [1] was introduced, it's very frequent to have
"fixtures._fixtures.timeout.TimeoutException" exceptions during the
execution of UTs and FTs. Because the privsep includes the synchronized
decorator, the synchronization wait is done inside the privsep context.
This is prone to timeouts if the wait is too long.

Until we can reorder the decorators of ip_lib [2] or we can remove the
sync decorators [3], this patch can mitigate the errors in the CI.

[1]https://review.opendev.org/#/c/631275/
[2]https://review.opendev.org/#/c/666853/
[3]https://review.opendev.org/#/c/657608/

Closes-Bug: #1843478

Change-Id: If865c4683645f9bd11f5e1b528bade0547505bfd
This commit is contained in:
Rodolfo Alonso Hernandez 2019-09-11 10:13:19 +00:00
parent f5bcca87d1
commit 63715ea361
1 changed files with 20 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
neutron/tests/base.py
View File

@ -66,6 +66,8 @@ ETCDIR = os.path.join(ROOTDIR, 'etc')
SUDO_CMD = 'sudo -n'
TESTCASE_RETRIES = 3
def etcdir(*p):
return os.path.join(ETCDIR, *p)
@ -173,10 +175,24 @@ class AttributeDict(dict):
def _catch_timeout(f):
@functools.wraps(f)
def func(self, *args, **kwargs):
try:
return f(self, *args, **kwargs)
except eventlet.Timeout as e:
self.fail('Execution of this test timed out: %s' % e)
for idx in range(1, TESTCASE_RETRIES + 1):
try:
return f(self, *args, **kwargs)
except eventlet.Timeout as e:
self.fail('Execution of this test timed out: %s' % e)
# NOTE(ralonsoh): exception catch added due to the constant
# occurrences of this exception during FT and UT execution.
# This is due to [1]. Once the sync decorators are removed or the
# privsep ones are decorated by those ones (swap decorator
# declarations) this catch can be remove.
# [1] https://review.opendev.org/#/c/631275/
except fixtures.TimeoutException:
with excutils.save_and_reraise_exception() as ctxt:
if idx < TESTCASE_RETRIES:
msg = ('"fixtures.TimeoutException" during test case '
'execution no %s; test case re-executed' % idx)
self.addDetail('DietTestCase', msg)
ctxt.reraise = False
return func