Include docstrings for parallel engine types/strings supported
When a 'executor' option is passed to a action engine requested to run in parallel mode that option will internally be examined and it will affect the internally used execution model that the engine will use; to make it understandable what the valid options are include a docstring + table(s) that describes the options and there valid values. Change-Id: I9a1852427bae22a01f5993862617e384f10ec005
This commit is contained in:
@@ -161,16 +161,10 @@ Parallel
|
||||
**Engine type**: ``'parallel'``
|
||||
|
||||
A parallel engine schedules tasks onto different threads/processes to allow for
|
||||
running non-dependent tasks simultaneously.
|
||||
|
||||
Additional supported keyword arguments:
|
||||
|
||||
* ``executor``: a object that implements a :pep:`3148` compatible `executor`_
|
||||
interface; it will be used for scheduling tasks. You can use instances of a
|
||||
`thread pool executor`_ or a :py:class:`green executor
|
||||
<taskflow.types.futures.GreenThreadPoolExecutor>` (which internally uses
|
||||
`eventlet <http://eventlet.net/>`_ and greenthread pools) or a
|
||||
`process pool executor`_.
|
||||
running non-dependent tasks simultaneously. See the documentation of
|
||||
:py:class:`~taskflow.engines.action_engine.engine.ParallelActionEngine` for
|
||||
supported arguments that can be used to construct a parallel engine that runs
|
||||
using your desired execution model.
|
||||
|
||||
.. tip::
|
||||
|
||||
@@ -340,6 +334,7 @@ Interfaces
|
||||
.. automodule:: taskflow.engines.action_engine.compiler
|
||||
.. automodule:: taskflow.engines.action_engine.completer
|
||||
.. automodule:: taskflow.engines.action_engine.engine
|
||||
.. automodule:: taskflow.engines.action_engine.executor
|
||||
.. automodule:: taskflow.engines.action_engine.runner
|
||||
.. automodule:: taskflow.engines.action_engine.runtime
|
||||
.. automodule:: taskflow.engines.action_engine.scheduler
|
||||
@@ -360,5 +355,4 @@ Hierarchy
|
||||
.. _executor: https://docs.python.org/dev/library/concurrent.futures.html#concurrent.futures.Executor
|
||||
.. _networkx: https://networkx.github.io/
|
||||
.. _futures backport: https://pypi.python.org/pypi/futures
|
||||
.. _thread pool executor: https://docs.python.org/dev/library/concurrent.futures.html#threadpoolexecutor
|
||||
.. _process pool executor: https://docs.python.org/dev/library/concurrent.futures.html#processpoolexecutor
|
||||
|
||||
@@ -239,7 +239,43 @@ class _ExecutorTextMatch(collections.namedtuple('_ExecutorTextMatch',
|
||||
|
||||
|
||||
class ParallelActionEngine(ActionEngine):
|
||||
"""Engine that runs tasks in parallel manner."""
|
||||
"""Engine that runs tasks in parallel manner.
|
||||
|
||||
Supported keyword arguments:
|
||||
|
||||
* ``executor``: a object that implements a :pep:`3148` compatible executor
|
||||
interface; it will be used for scheduling tasks. The following
|
||||
type are applicable (other unknown types passed will cause a type
|
||||
error to be raised).
|
||||
|
||||
========================= ===============================================
|
||||
Type provided Executor used
|
||||
========================= ===============================================
|
||||
|cft|.ThreadPoolExecutor :class:`~.executor.ParallelThreadTaskExecutor`
|
||||
|cfp|.ProcessPoolExecutor :class:`~.executor.ParallelProcessTaskExecutor`
|
||||
|cf|._base.Executor :class:`~.executor.ParallelThreadTaskExecutor`
|
||||
========================= ===============================================
|
||||
|
||||
* ``executor``: a string that will be used to select a :pep:`3148`
|
||||
compatible executor; it will be used for scheduling tasks. The following
|
||||
string are applicable (other unknown strings passed will cause a value
|
||||
error to be raised).
|
||||
|
||||
=========================== ===============================================
|
||||
String (case insensitive) Executor used
|
||||
=========================== ===============================================
|
||||
``process`` :class:`~.executor.ParallelProcessTaskExecutor`
|
||||
``processes`` :class:`~.executor.ParallelProcessTaskExecutor`
|
||||
``thread`` :class:`~.executor.ParallelThreadTaskExecutor`
|
||||
``threaded`` :class:`~.executor.ParallelThreadTaskExecutor`
|
||||
``threads`` :class:`~.executor.ParallelThreadTaskExecutor`
|
||||
=========================== ===============================================
|
||||
|
||||
.. |cfp| replace:: concurrent.futures.process
|
||||
.. |cft| replace:: concurrent.futures.thread
|
||||
.. |cf| replace:: concurrent.futures
|
||||
"""
|
||||
|
||||
_storage_factory = atom_storage.MultiThreadedStorage
|
||||
|
||||
# One of these types should match when a object (non-string) is provided
|
||||
|
||||
@@ -373,6 +373,7 @@ class ParallelTaskExecutor(TaskExecutor):
|
||||
to concurrent.Futures.Executor.
|
||||
"""
|
||||
|
||||
#: Options this executor supports (passed in from engine options).
|
||||
OPTIONS = frozenset(['max_workers'])
|
||||
|
||||
def __init__(self, executor=None, max_workers=None):
|
||||
@@ -431,6 +432,7 @@ class ParallelProcessTaskExecutor(ParallelTaskExecutor):
|
||||
the parent are executed on events in the child.
|
||||
"""
|
||||
|
||||
#: Options this executor supports (passed in from engine options).
|
||||
OPTIONS = frozenset(['max_workers', 'dispatch_periodicity'])
|
||||
|
||||
def __init__(self, executor=None, max_workers=None,
|
||||
|
||||
Reference in New Issue
Block a user