Add a WBE request state diagram + explanation

To make it more clear what the WBE request states are
and what they imply/mean add the appropriate documentation
and diagram that explains it and its states/concepts.

Change-Id: If25b5c6402aff6e294886cc6c5f248413183c4e4

doc/source/workers.rst

@@ -13,7 +13,6 @@ connected via `amqp`_ (or other supported `kombu`_ transports).
     production ready.
 
 .. _blueprint page: https://blueprints.launchpad.net/taskflow?searchtext=wbe
-.. _kombu: http://kombu.readthedocs.org/
 
 Terminology
 -----------
@@ -285,10 +284,53 @@ When **reverting:**
         ]
     }
 
+Request state transitions
+-------------------------
+
+.. image:: img/wbe_request_states.svg
+   :width: 660px
+   :align: left
+   :alt: WBE request state transitions
+
+**WAITING** - Request placed on queue (or other `kombu`_ message bus/transport)
+but not *yet* consumed.
+
+**PENDING** - Worker accepted request and is pending to run using its
+executor (threads, processes, or other).
+
+**FAILURE** - Worker failed after running request (due to task exeception) or
+no worker moved/started executing (by placing the request into ``RUNNING``
+state) with-in specified time span (this defaults to 60 seconds unless
+overriden).
+
+**RUNNING** - Workers executor (using threads, processes...) has started to
+run requested task (once this state is transitioned to any request timeout no
+longer becomes applicable; since at this point it is unknown how long a task
+will run since it can not be determined if a task is just taking a long time
+or has failed).
+
+**SUCCESS** - Worker finished running task without exception.
+
+.. note::
+
+    During the ``WAITING`` and ``PENDING`` stages the engine keeps track
+    of how long the request has been *alive* for and if a timeout is reached
+    the request will automatically transition to ``FAILURE`` and any further
+    transitions from a worker will be disallowed (for example, if a worker
+    accepts the request in the future and sets the task to ``PENDING`` this
+    transition will be logged and ignored). This timeout can be adjusted and/or
+    removed by setting the engine ``transition_timeout`` option to a
+    higher/lower value or by setting it to ``None`` (to remove the timeout
+    completely). In the future this will be improved to be more dynamic
+    by implementing the blueprints associated with `failover`_ and
+    `info/resilence`_.
+
+.. _failover: https://blueprints.launchpad.net/taskflow/+spec/wbe-worker-failover
+.. _info/resilence: https://blueprints.launchpad.net/taskflow/+spec/wbe-worker-info
+
 Usage
 =====
 
-
 Workers
 -------
 
@@ -390,3 +432,5 @@ Interfaces
 .. automodule:: taskflow.engines.worker_based.executor
 .. automodule:: taskflow.engines.worker_based.proxy
 .. automodule:: taskflow.engines.worker_based.worker
+
+.. _kombu: http://kombu.readthedocs.org/

tools/generate_states.sh

@@ -30,3 +30,7 @@ $xsltproc $PWD/.diagram-tools/notugly.xsl /tmp/states.svg > $img_dir/engine_stat
 echo "---- Updating retry state diagram ----"
 python $script_dir/state_graph.py -r -f /tmp/states.svg
 $xsltproc $PWD/.diagram-tools/notugly.xsl /tmp/states.svg > $img_dir/retry_states.svg
+
+echo "---- Updating wbe request state diagram ----"
+python $script_dir/state_graph.py -w -f /tmp/states.svg
+$xsltproc $PWD/.diagram-tools/notugly.xsl /tmp/states.svg > $img_dir/wbe_request_states.svg

tools/state_graph.py

@@ -27,6 +27,7 @@ sys.path.insert(0, top_dir)
 import pydot
 
 from taskflow.engines.action_engine import runner
+from taskflow.engines.worker_based import protocol
 from taskflow import states
 from taskflow.types import fsm
 
@@ -91,6 +92,10 @@ def main():
                       action='store_true',
                       help="use engine state transitions",
                       default=False)
+    parser.add_option("-w", "--wbe-requests", dest="wbe_requests",
+                      action='store_true',
+                      help="use wbe request transitions",
+                      default=False)
     parser.add_option("-T", "--format", dest="format",
                       help="output in given format",
                       default='svg')
@@ -99,9 +104,15 @@ def main():
     if options.filename is None:
         options.filename = 'states.%s' % options.format
 
-    types = [options.engines, options.retries, options.tasks]
+    types = [
+        options.engines,
+        options.retries,
+        options.tasks,
+        options.wbe_requests,
+    ]
     if sum([int(i) for i in types]) > 1:
-        parser.error("Only one of task/retry/engines may be specified.")
+        parser.error("Only one of task/retry/engines/wbe requests"
+                     " may be specified.")
 
     internal_states = list()
     ordering = 'in'
@@ -120,6 +131,10 @@ def main():
         source, memory = r.builder.build()
         internal_states.extend(runner._META_STATES)
         ordering = 'out'
+    elif options.wbe_requests:
+        source_type = "WBE requests"
+        source = make_machine(protocol.WAITING,
+                              list(protocol._ALLOWED_TRANSITIONS), [])
     else:
         source_type = "Flow"
         source = make_machine(states.PENDING,