Merge "Documentation tune-ups"
This commit is contained in:
@@ -23,8 +23,8 @@ parts of :py:class:`linear flow <taskflow.patterns.linear_flow.Flow>` are run
|
||||
one after another, in order, even if engine is *capable* of running tasks in
|
||||
parallel).
|
||||
|
||||
Creating Engines
|
||||
================
|
||||
Creating
|
||||
========
|
||||
|
||||
All engines are mere classes that implement the same interface, and of course
|
||||
it is possible to import them and create instances just like with any classes
|
||||
@@ -43,8 +43,8 @@ might look like::
|
||||
|
||||
.. automodule:: taskflow.engines.helpers
|
||||
|
||||
Engine Configuration
|
||||
====================
|
||||
Usage
|
||||
=====
|
||||
|
||||
To select which engine to use and pass parameters to an engine you should use
|
||||
the ``engine_conf`` parameter any helper factory function accepts. It may be:
|
||||
@@ -53,10 +53,8 @@ the ``engine_conf`` parameter any helper factory function accepts. It may be:
|
||||
* a dictionary, holding engine type with key ``'engine'`` and possibly
|
||||
type-specific engine parameters.
|
||||
|
||||
Known engine types are listed below.
|
||||
|
||||
Single-Threaded Engine
|
||||
----------------------
|
||||
Single-Threaded
|
||||
---------------
|
||||
|
||||
**Engine type**: ``'serial'``
|
||||
|
||||
@@ -70,8 +68,8 @@ on. This engine is used by default.
|
||||
greenthreads and monkey patching). See `eventlet <http://eventlet.net/>`_
|
||||
and `greenlet <http://greenlet.readthedocs.org/>`_ for more details.
|
||||
|
||||
Parallel Engine
|
||||
---------------
|
||||
Parallel
|
||||
--------
|
||||
|
||||
**Engine type**: ``'parallel'``
|
||||
|
||||
@@ -96,8 +94,8 @@ Additional configuration parameters:
|
||||
Running tasks with ``concurrent.futures.ProcessPoolExecutor`` is not
|
||||
supported now.
|
||||
|
||||
Worker-Based Engine
|
||||
-------------------
|
||||
Worker-Based
|
||||
------------
|
||||
|
||||
**Engine type**: ``'worker-based'``
|
||||
|
||||
@@ -118,8 +116,8 @@ operates.
|
||||
.. _wiki page: https://wiki.openstack.org/wiki/TaskFlow/Worker-based_Engine
|
||||
.. _blueprint page: https://blueprints.launchpad.net/taskflow
|
||||
|
||||
Engine Interface
|
||||
================
|
||||
Interfaces
|
||||
==========
|
||||
|
||||
.. automodule:: taskflow.engines.base
|
||||
|
||||
|
||||
@@ -19,8 +19,29 @@ claiming them, and only remove them from the queue when they're done with the
|
||||
work. If the consumer fails, the lock is *automatically* released and the item
|
||||
is back on the queue for further consumption.
|
||||
|
||||
For more information, please see `wiki page`_ for more details.
|
||||
|
||||
Definitions
|
||||
===========
|
||||
|
||||
Jobs
|
||||
A :py:class:`job <taskflow.jobs.job.Job>` consists of a unique identifier, name,
|
||||
and a reference to a :py:class:`logbook <taskflow.persistence.logbook.LogBook>`
|
||||
which contains the details of the work that has been or should be/will be
|
||||
completed to finish the work that has been created for that job.
|
||||
|
||||
Jobboards
|
||||
A :py:class:`jobboard <taskflow.jobs.jobboard.JobBoard>` is responsible for managing
|
||||
the posting, ownership, and delivery of jobs. It acts as the location where jobs
|
||||
can be posted, claimed and searched for; typically by iteration or notification.
|
||||
Jobboards may be backed by different *capable* implementations (each with potentially differing
|
||||
configuration) but all jobboards implement the same interface and semantics so
|
||||
that the backend usage is as transparent as possible. This allows deployers or
|
||||
developers of a service that uses TaskFlow to select a jobboard implementation
|
||||
that fits their setup (and there intended usage) best.
|
||||
|
||||
Features
|
||||
--------
|
||||
========
|
||||
|
||||
- High availability
|
||||
|
||||
@@ -58,29 +79,8 @@ Features
|
||||
user to poll for status (similar in concept to a shipping *tracking*
|
||||
identifier created by fedex or UPS).
|
||||
|
||||
For more information, please see `wiki page`_ for more details.
|
||||
|
||||
Jobs
|
||||
----
|
||||
|
||||
A job consists of a unique identifier, name, and a reference to a logbook
|
||||
which contains the details of the work that has been or should be/will be
|
||||
completed to finish the work that has been created for that job.
|
||||
|
||||
Jobboards
|
||||
---------
|
||||
|
||||
A jobboard is responsible for managing the posting, ownership, and delivery
|
||||
of jobs. It acts as the location where jobs can be posted, claimed and searched
|
||||
for; typically by iteration or notification. Jobboards may be backed by
|
||||
different *capable* implementations (each with potentially differing
|
||||
configuration) but all jobboards implement the same interface and semantics so
|
||||
that the backend usage is as transparent as possible. This allows deployers or
|
||||
developers of a service that uses TaskFlow to select a jobboard implementation
|
||||
that fits their setup (and there intended usage) best.
|
||||
|
||||
Using Jobboards
|
||||
===============
|
||||
Usage
|
||||
=====
|
||||
|
||||
All engines are mere classes that implement same interface, and of course it is
|
||||
possible to import them and create their instances just like with any classes
|
||||
@@ -156,11 +156,6 @@ might look like:
|
||||
time.sleep(coffee_break_time)
|
||||
...
|
||||
|
||||
Jobboard Configuration
|
||||
======================
|
||||
|
||||
Known engine types are listed below.
|
||||
|
||||
Zookeeper
|
||||
---------
|
||||
|
||||
@@ -239,15 +234,11 @@ the claim by then, therefore both would be *working* on a job.
|
||||
.. _idempotent: http://en.wikipedia.org/wiki/Idempotence
|
||||
.. _preemptable: http://en.wikipedia.org/wiki/Preemption_%28computing%29
|
||||
|
||||
Job Interface
|
||||
=============
|
||||
Interfaces
|
||||
==========
|
||||
|
||||
.. automodule:: taskflow.jobs.backends
|
||||
.. automodule:: taskflow.jobs.job
|
||||
|
||||
Jobboard Interface
|
||||
==================
|
||||
|
||||
.. automodule:: taskflow.jobs.jobboard
|
||||
|
||||
.. _wiki page: https://wiki.openstack.org/wiki/TaskFlow/Paradigm_shifts#Workflow_ownership_transfer
|
||||
|
||||
@@ -67,7 +67,7 @@ To receive notification on flow state changes use
|
||||
woof
|
||||
Flow 'cat-dog' transition to state SUCCESS
|
||||
|
||||
Task notifications
|
||||
Task Notifications
|
||||
------------------
|
||||
|
||||
To receive notification on task state changes use
|
||||
|
||||
@@ -105,8 +105,8 @@ A few scenarios come to mind:
|
||||
(when data is saved -- every time it changes or at some particular moments
|
||||
or simply never).
|
||||
|
||||
Persistence Configuration
|
||||
=========================
|
||||
Usage
|
||||
=====
|
||||
|
||||
To select which persistence backend to use you should use the
|
||||
:py:meth:`fetch() <taskflow.persistence.backends.fetch>` function which uses
|
||||
@@ -137,13 +137,17 @@ the following:
|
||||
``'connection'`` and possibly type-specific backend parameters as other
|
||||
keys.
|
||||
|
||||
Known engine types are listed below.
|
||||
Memory
|
||||
------
|
||||
|
||||
**Connection**: ``'memory'``
|
||||
|
||||
Retains all data in local memory (not persisted to reliable storage). Useful
|
||||
for scenarios where persistence is not required (and also in unit tests).
|
||||
|
||||
Files
|
||||
-----
|
||||
|
||||
**Connection**: ``'dir'`` or ``'file'``
|
||||
|
||||
Retains all data in a directory & file based structure on local disk. Will be
|
||||
@@ -152,6 +156,9 @@ from the same local machine only). Useful for cases where a *more* reliable
|
||||
persistence is desired along with the simplicity of files and directories (a
|
||||
concept everyone is familiar with).
|
||||
|
||||
Sqlalchemy
|
||||
----------
|
||||
|
||||
**Connection**: ``'mysql'`` or ``'postgres'`` or ``'sqlite'``
|
||||
|
||||
Retains all data in a `ACID`_ compliant database using the `sqlalchemy`_ library
|
||||
@@ -163,6 +170,9 @@ does not apply when using sqlite).
|
||||
.. _sqlalchemy: http://www.sqlalchemy.org/docs/
|
||||
.. _ACID: https://en.wikipedia.org/wiki/ACID
|
||||
|
||||
Zookeeper
|
||||
---------
|
||||
|
||||
**Connection**: ``'zookeeper'``
|
||||
|
||||
Retains all data in a `zookeeper`_ backend (zookeeper exposes operations on
|
||||
@@ -176,8 +186,8 @@ as the database connection types listed previously).
|
||||
.. _zookeeper: http://zookeeper.apache.org
|
||||
.. _kazoo: http://kazoo.readthedocs.org/
|
||||
|
||||
Persistence Backend Interfaces
|
||||
==============================
|
||||
Interfaces
|
||||
==========
|
||||
|
||||
.. automodule:: taskflow.persistence.backends
|
||||
.. automodule:: taskflow.persistence.backends.base
|
||||
|
||||
Reference in New Issue
Block a user