octavia/releasenotes/notes/fix-lb-in-PENDING-on-DB-failure-1ffea71a86cd4ea9.yaml at 09b249d0848d315c330c3b5d78c75821ba8c4fe8 - octavia - OpenDev: Free Software Needs Free Tools

openstack/octavia

Files

Gregory Thiemonge 09b249d084 Retry to set loadbalancer prov status on failures

In case of DB outages when a flow is running, an exception is caught and
the flow is reverted. In most of the flows, the revert function of the
first task's (the last to be reverted) unlocks the load balancer by
setting its provisioning status (to ERROR or ACTIVE, depending on the
flow), but it fails if the DB is not reachable, leaving the LB in
a PENDING_* state.
This commit adds tenacity.retry to those functions, Octavia retries to
set the status during ~2h45 (2000 attempts, 1 sec initial delay, 5 sec
max delay).

Note: stable/2023.1 and older, the patch also includes modifications for
v1/tasks/lifecycle_tasks.py

Conflicts:
	octavia/common/config.py
	octavia/tests/unit/controller/worker/test_task_utils.py
	octavia/controller/worker/v1/tasks/lifecycle_tasks.py

Closes-Bug: #2036952
Change-Id: I458dd6d6f5383edc24116ea0fa27e3a593044146
(cherry picked from commit be91493332)
(cherry picked from commit 96782e2c54)
(cherry picked from commit 57833dbdad)
(cherry picked from commit 27060603db)
(cherry picked from commit 5a411a6855)
(cherry picked from commit c8a2cb4fbd)

2023-10-11 10:08:57 +02:00

8 lines

314 B

YAML

Raw Blame History

 ---
 fixes:
   - |
     Fixed an issue with load balancers stuck in a ``PENDING_*`` state during
     database outages. Now when a task fails in Octavia, it retries to update
     the ``provisioning_status`` of the load balancer until the database is back
     (or it gives up after a really long timeout - around 2h45)