cea71f2799
This fixes the sporadic of tasks that would happen under certain circumstances. What happened was that a new worker notification would be sent to a callback while at the same time a task submission would come in and there would be a small race period where the task would insert itself into the requests cache while the callback was processing. So to work around this the whole concept of a requests cache was revamped and now the WBE executor just maintains its own local dictionary of ongoing requests and accesses it safely. During the on_wait function that is periodically called by kombu the previous expiry of work happens but now any requests that are pending are matched to any new workers that may have appeared. This avoids the race (and ensures that even if a new worker is found but a submission is in progress that the duration until that submission happens will only be until the next on_wait call happens). Related-Bug: #1431097 Change-Id: I98b0caeedc77ab2f7214847763ae1eb0433d4a78 |
||
|---|---|---|
| doc | ||
| taskflow | ||
| tools | ||
| .coveragerc | ||
| .gitignore | ||
| .gitreview | ||
| .mailmap | ||
| .testr.conf | ||
| CONTRIBUTING.rst | ||
| LICENSE | ||
| README.rst | ||
| pylintrc | ||
| requirements.txt | ||
| run_tests.sh | ||
| setup.cfg | ||
| setup.py | ||
| test-requirements.txt | ||
| tox.ini | ||
README.rst
TaskFlow
A library to do [jobs, tasks, flows] in a highly available, easy to understand and declarative manner (and more!) to be used with OpenStack and other projects.
- Free software: Apache license
- Documentation: http://docs.openstack.org/developer/taskflow
- Source: http://git.openstack.org/cgit/openstack/taskflow
- Bugs: http://bugs.launchpad.net/taskflow/
Join us
Testing and requirements
Requirements
Because this project has many optional (pluggable) parts like
persistence backends and engines, we decided to split our requirements
into two parts: - things that are absolutely required (you can't use the
project without them) are put into requirements.txt. The
requirements that are required by some optional part of this project
(you can use the project without them) are put into our
test-requirements.txt file (so that we can still test the
optional functionality works as expected). If you want to use the
feature in question (eventlet or the
worker based engine that uses kombu or the sqlalchemy persistence backend or
jobboards which have an implementation built using kazoo ...), you should add that
requirement(s) to your project or environment.
Tox.ini
Our tox.ini file describes several test environments
that allow to test TaskFlow with different python versions and sets of
requirements installed. Please refer to the tox documentation to understand how
to make these test environments work for you.
Developer documentation
We also have sphinx documentation in docs/source.
To build it, run:
$ python setup.py build_sphinx