Introduce TaskFlowServiceController which uses taskflow
jobboard feature and saves jobs info into persistence backend.
Jobboard could be operated via RedisTaskFlowDriver or
ZookeeperTaskFlowDriver, that could be set via the config.
RedisTaskFlowDriver is intoduced as default backend for jobboard.
Usage of jobboard allows to resume jobs in case of restart/stop
of Octavia controller services.
Persistence backend saves state of flow tasks that required in
case of resuming job. SQLAlchemy backend is used here.
Bump taskflow version to 3.7.1 and add dependency to
SQLAlchemy-Utils (required for taskflow sqlalchemy
backend support).
Story: 2005072
Task: 30806
Task: 30816
Task: 30817
Change-Id: I92ee4e879e98e4718d2e9aba56486341223a9157
Split _get_delete_load_balancer_flow, create
separate methods for store generation.
Change-Id: I71117717a7fdff349473c43975ad639c0448713e
Story: 2005072
Task: 30807
Fixed endpoints logs for listener, pool and member as well.
Rework _get_create_amp_for_lb_subflow due to issue with taskflow
decider and retry subflow.
Retry subflow was not ignored properly for spare amphorae case,
so _get_create_amp_for_lb_subflow has been split to
3 separate subflows each of them linked to graph flow.
This is work around and can be removed when proper mechanism
implemeneted in taskflow library. (added several todos about it).
Change-Id: Ibd114fa14123e6de6c5d6f260e32cf7f2b28805a
Story: 2005072
Task: 30814
This patch converts the member flows to use the provider driver
data model.
It also stops storing sqlalchemy models in the flow storage in
preparation for enabling jobboard.
Change-Id: Ic15e4311ce244e71b65069359c6c08e49f6b8d00
Story: 2005072
Task: 30810
This patch converts the listener flows to use the provider driver
data model.
It also stops storing sqlalchemy models in the flow storage in
preparation for enabling jobboard.
Change-Id: Ic44019b8877f008e6d7a75ceed1b7fd958e051d0
Story: 2005072
Task: 30808
Use taskflow retry for connectivity wait. [1]
This reqired for redis jobboard implementation as each retry expand
claim for job on worker. This means that worker is proccesing job and
it should not be released for other workers to work on it.
Adopted for v2 flows.
[1] - https://docs.openstack.org/taskflow/latest/user/atoms.html#retry
Story: 2005072
Task: 33477
Change-Id: I2cf241ea965ad56ed70ebde83632ab855f5d859e
Fix an issue that prevents graceful shutdown of controller workers.
cotyledon.Service.terminate function is by definition the graceful
termination function and doesn't have any 'graceful' optional boolean
argument (https://cotyledon.readthedocs.io/en/latest/api.html).
Because of this error, message_listener.wait() was never called in the
consumers' termination functions, so flows could be interrupted before
completion and could leave resources such as load balancer in a
PENDING_* provisioning state.
By default cotyledon.Service terminates the server after a timeout if
the worker could not shutdown itself gracefully. The default value
for the timeout is 300 seconds (set in devstack plugin) and can be
overriden using the graceful_shutdown_timeout setting in octavia.conf
The default value will be updated to a lower value when work on
persistant taskflow will be merged.
Story: 2006603
Task: 36770
Change-Id: I3f776bd018246897c9a889699a2d0ecbbfbb7098
This patch creates an Amphora v2 provider driver as well as a
V2 controller worker.
This is in preparation for having the amphora driver use the new
provider driver data models and rely less on native Octavia database
access.
It is also a prepartion step for enabling TaskFlow JobBoard as
this work will move to storing dictionaries in the flows instead
of database models.
Change-Id: Ia65539a8c39560e2276750d8e79a637be4c0f265
Story: 2005072
Task: 30806
Ann Taraday