Do not retry immediately after server fault

Currently the call_and_ignore_notfound_exc method retries API request
immediately after it receives server failure. However this is not much
robust because retries might be done in a quite short span.

For example in the related bug we've seen that glance-api with cinder
backend needs some time to complete the post tasks to disconnect
the volume when it's run by httpd + mod_wsgi. Currently the 3 delete
calls are executed within 1 second and that is not enough to complete
that task.

This adds 1 second delay between reties, assuming longer delay would
help the server stabilize.

Related-Bug: #2030855
Change-Id: Ic9eed8352c64f183ca4077e141c6b94f81a9b91a
This commit is contained in:
Takashi Kajinami
2023-08-09 21:23:58 +09:00
parent e9b98c6bef
commit be7bed4de2
+1
View File
@@ -93,6 +93,7 @@ def call_and_ignore_notfound_exc(func, *args, **kwargs):
if attempt >= 3:
raise
LOG.warning('Got ServerFault while running %s, retrying...', func)
time.sleep(1)
def call_until_true(func, duration, sleep_for, *args, **kwargs):