Joshua Harlow 1de8bbd838 Add a claims listener that connects job claims to engines

To make it easily possible to stop running a engine that was
created from a job, add a claims listener that will be called
on state changes that an engine progresses through. During those
state changes the jobboard will be queried to determine if the
job is still claimed by the respective owner; if not the engine
will be suspended and further work will stop.

Change-Id: I8bbc6a3e03746ba0a7c74139cf9e230631d80d8f

2014-12-01 21:39:23 -08:00

5.3 KiB

Raw Blame History

Notifications and listeners

from taskflow import task from taskflow.patterns import linear_flow from taskflow import engines

Overview

Engines provide a way to receive notification on task and flow state transitions, which is useful for monitoring, logging, metrics, debugging and plenty of other tasks.

To receive these notifications you should register a callback with an instance of the :py~taskflow.utils.misc.Notifier class that is attached to :pyEngine <taskflow.engines.base.EngineBase> attributes task_notifier and notifier.

TaskFlow also comes with a set of predefined listeners <listeners>, and provides means to write your own listeners, which can be more convenient than using raw callbacks.

Receiving notifications with callbacks

Flow notifications

To receive notification on flow state changes use the :py~taskflow.utils.misc.Notifier instance available as the notifier property of an engine.

A basic example is:

>>> class CatTalk(task.Task): ... def execute(self, meow): ... print(meow) ... return "cat" ... >>> class DogTalk(task.Task): ... def execute(self, woof): ... print(woof) ... return 'dog' ... >>> def flow_transition(state, details): ... print("Flow '%s' transition to state %s" % (details['flow_name'], state)) ... >>> >>> flo = linear_flow.Flow("cat-dog").add( ... CatTalk(), DogTalk(provides="dog")) >>> eng = engines.load(flo, store={'meow': 'meow', 'woof': 'woof'}) >>> eng.notifier.register("*", flow_transition) >>> eng.run() Flow 'cat-dog' transition to state RUNNING meow woof Flow 'cat-dog' transition to state SUCCESS

Task notifications

To receive notification on task state changes use the :py~taskflow.utils.misc.Notifier instance available as the task_notifier property of an engine.

A basic example is:

>>> class CatTalk(task.Task): ... def execute(self, meow): ... print(meow) ... return "cat" ... >>> class DogTalk(task.Task): ... def execute(self, woof): ... print(woof) ... return 'dog' ... >>> def task_transition(state, details): ... print("Task '%s' transition to state %s" % (details['task_name'], state)) ... >>> >>> flo = linear_flow.Flow("cat-dog") >>> flo.add(CatTalk(), DogTalk(provides="dog")) <taskflow.patterns.linear_flow.Flow object at 0x...> >>> eng = engines.load(flo, store={'meow': 'meow', 'woof': 'woof'}) >>> eng.task_notifier.register("*", task_transition) >>> eng.run() Task 'CatTalk' transition to state RUNNING meow Task 'CatTalk' transition to state SUCCESS Task 'DogTalk' transition to state RUNNING woof Task 'DogTalk' transition to state SUCCESS

Listeners

TaskFlow comes with a set of predefined listeners -- helper classes that can be used to do various actions on flow and/or tasks transitions. You can also create your own listeners easily, which may be more convenient than using raw callbacks for some use cases.

For example, this is how you can use :py~taskflow.listeners.printing.PrintingListener:

>>> from taskflow.listeners import printing >>> class CatTalk(task.Task): ... def execute(self, meow): ... print(meow) ... return "cat" ... >>> class DogTalk(task.Task): ... def execute(self, woof): ... print(woof) ... return 'dog' ... >>> >>> flo = linear_flow.Flow("cat-dog").add( ... CatTalk(), DogTalk(provides="dog")) >>> eng = engines.load(flo, store={'meow': 'meow', 'woof': 'woof'}) >>> with printing.PrintingListener(eng): ... eng.run() ... taskflow.engines.action_engine.engine.SerialActionEngine: ... has moved flow 'cat-dog' (...) into state 'RUNNING' taskflow.engines.action_engine.engine.SerialActionEngine: ... has moved task 'CatTalk' (...) into state 'RUNNING' meow taskflow.engines.action_engine.engine.SerialActionEngine: ... has moved task 'CatTalk' (...) into state 'SUCCESS' with result 'cat' (failure=False) taskflow.engines.action_engine.engine.SerialActionEngine: ... has moved task 'DogTalk' (...) into state 'RUNNING' woof taskflow.engines.action_engine.engine.SerialActionEngine: ... has moved task 'DogTalk' (...) into state 'SUCCESS' with result 'dog' (failure=False) taskflow.engines.action_engine.engine.SerialActionEngine: ... has moved flow 'cat-dog' (...) into state 'SUCCESS'

Basic listener

taskflow.listeners.base.ListenerBase

Printing and logging listeners

taskflow.listeners.base.LoggingBase

taskflow.listeners.logging.LoggingListener

taskflow.listeners.logging.DynamicLoggingListener

taskflow.listeners.printing.PrintingListener

Timing listener

taskflow.listeners.timing.TimingListener

taskflow.listeners.timing.PrintingTimingListener

Claim listener

taskflow.listeners.claims.CheckingClaimListener

5.3 KiB Raw Blame History