The latest python 2.x release is 2.7.9 and not
2.7.7 so when people are using this script to build
a useful environment (that is lacking py2.7) use this
newer version instead.
Change-Id: I2fdd035e0b4efcacd6ef2da04d6cd8a56dcca512
To make it more clear what the WBE request states are
and what they imply/mean add the appropriate documentation
and diagram that explains it and its states/concepts.
Change-Id: If25b5c6402aff6e294886cc6c5f248413183c4e4
Instead of using strings it is better if we can use constants (that
may be the same/adjusted strings) and use those instead in the state
machine used in the runner.
The names are adjusted (and the state graph diagram and docstring)
to reflect names that fit better with there intended meaning and
usage.
Change-Id: Iaf229d6e37730545ba9f2708d118697cb7145992
The engine state diagram benefits slightly from having
the event labels that cause transitions to other states
so we might as well include it in the generated diagram.
Change-Id: I733eba1d2dc6386c7b7ce8930fbfd41e29cdb602
This just adjusts to install wget and certain other packages
before further work commences to ensure they are ready and
avaiable for further use.
Tested on rhel6.x as a cloud-init userdata script and it appears
to continue to work as expected.
Change-Id: Ia85715e27bc8342fd61d23e2c8a659dbce20aea9
In order to show people that are reading the docs what the current
schema for the database is create a helper tool that upgrades the
schema to the newest version and then uses tabulate to create a
restructured text version of that schema which is then included in
the documentation.
Change-Id: Id215f88430971a4a083f9739fb2ec59d971dc8fa
Instead of just having black text color adjust some of the
states text color depending on there type/name and highlight
some as red, green, orange depending on there underlying
meaning.
Color names are from:
http://www.graphviz.org/doc/info/colors.html
Change-Id: I89f8f90837551a257936d254516ada6130e7b6da
To build the state diagrams switch from using a custom
algorithm that builds a state machine in a table to using
our state machine class to build those same machines before
translation into a dot diagram.
This also allows us to use the state machine that already
is being used in the action engine as the source of transitions
to build the dot diagram with.
Change-Id: I4a0d16a3fb7c620c2774b535ab952b5d5006e9e9
It can be difficult to figure out what to install to get a test
environment that can be used to test all the opportunistic tests,
zookeeper and mysql (with the right user and password and test
database).
This script provides a way to do this on rhel/ubuntu based
environments and can be used to examine how to do this for other
environment types.
I use it with nova boot (on rhel images) and it has worked out
great as a way to rebuild a clean test environment as I need.
Change-Id: I9aef7e5536aa38e951a6411a8e83097339a43955
Switch to using the `state_graph.py` and dot output
as the source of all of the state diagrams (this
makes it easy for anyone to recreate them by just
running the script in the tools directory).
Also update the state diagram creator to have engine
states as well as retry states and replaces all
existing state diagrams with the updated prettified
versions.
Also adjusts some nits around wording and grammar that
were encountered during this updating process.
Change-Id: Ia783aed6c4136763e1e34cbd0b3e57ffb1109abe
Instead of using a helper script that is internal to taskflow
use the doc8 package (which was created based on the check_doc.py
work) instead. This avoids repeating duplicated code when a
stackforge/external pypi package can provide the same
functionality.
Change-Id: Ie5a43f96b20e34f3955657ad2ef8beba05a4300e
Create a tool that can clear any leftover garbage left
by the testing of taskflow unit tests with zookeeper for
when this is needed (for example, a test does not clean up
correctly on its own).
Change-Id: Icfaf28273b76a6ca27683d174f111fba2858f055
What is checked:
- lines should not be longer than 79 characters, also
there are few exceptions;
- no tabulation for indentation;
- no trailing whitespace.
Change-Id: Id5b5dff380460d1fb4ffeeb23f00864e7fc158ed
Most of the utility graph functions we have can
be connected to a directed graph class that itself
derives (and adds on to) the networkx base class.
Doing this allows for functionality that isn't exposed
in networkx to be exposed in our subclass (which is a
useful pattern to have).
It also makes it possible (if ever needed) to replace
the networkx usage in taskflow with something else if
this ever becomes a major request.
Change-Id: I0a825d5637236d7b5dbdbda0d426adb0183d5ba3
Update the docs to explain how to use the toxgen.py
released on pypi and use it to regenerate the tox.ini
file.
Change-Id: I9e0d28f0e40aa0280132a644260d028190b10974
Adjust tests to skip the sqlalchemy test if
sqlalchemy is not installed. Adjust examples
to fallback to a directory based backend if
the sqlalchemy does not load or is not available.
Include a updated tox.ini (generated from the
toxgen.py script) that includes the new venv
variations.
Change-Id: I7686f09901a9b65d7c81b4e037b5bffc24aa7ef7
Instead of requiring all the complicated
venv management logic that tox already provides
just use tox directly instead. This maps more
closely to what jenkins does, and how most other
python development occurs (which does not typically
rely on custom venv scripts).
Change-Id: Iff6cf0a7eb24ddd2ef65ddd723d0e4cb803b9bb9
- Remove versionutils as it is not used and it has
started to pull in oslo.cfg via its direct usage
of the oslo-incubator log module
Change-Id: I364f00d9734b67fe5c7566ccd7a76e6eb18751c2
This commit adds -T aka --format option to state graph tool that
can be used to specify output format. Supported values are 'dot',
'svg', 'svgz' and 'png'.
Change-Id: I27a74b582726da24a1b05d4ffb72bde860fff417
Add a helper tool that can convert the
allowed state transitions into a dot graph
which itself then converts it into a browser
compatible svg file, which can then be looked
at to have a pretty state diagram.
Change-Id: I3cbbc75bd58182700ddc07f00395da5d1093a26c
Removing ThreadGroupExecutor allows to get rid of threading2
dependency. LastFedIter is also not used, so there is no
reason to keep it.
Change-Id: Id396ab59ebf1e2314ef620417d7cac09e1ab8d3b
A executor is a more generic concept than a pool and
also seems to work better with eventlet. This is also
a more supported model of performing concurrent operations
than using the mostly unknown multiprocessing thread
pool implementation.
Fixes: bug 1225275
Change-Id: I09e9a9000bc88cc57d51342b83b31f97790a62e9
Value returned from execute method of task should contain all
results that are mappend with task.rebind. To verify that we
check that the value can be indexed with all necessary indexes.
If that fails, we put warning to log.
Change-Id: I21aac7b36885668155c2d7bd2eedcf4eeba22b11
Remove further reference to oslo-incubator database
layer which itself uses oslo.config and remove its
dependencies and resync with oslo-incubator after
applying these changes.
Implements blueprint eliminate-oslo-cfg
Change-Id: Ie1ee7affae75c60833c012c921e50d7b14026101
Instead of having a pretty restrictive module
based api for saving logbook objects it is much
more friendly and extensible to move toward a more
ceilometer-influenced engine and connection based
storage backend using stevedore to do the backend
loading instead of a custom registration/fetching
mechanism. This allows us to provide a base object
oriented backend api that can be easily inherited
from to allow for customized & pluggable backend
storage modules.
Implements blueprint stevedore-based-backends
Implements blueprint ceilometer-influenced-backends
Change-Id: Ib5868d3d9018b7aa1a3354858dcb90ca1a04055d
Propose a new graph flow that will run every task
in the graph at the same time but will use a count
down latch concept to ensure that a tasks dependents
are provided before the task itself can run. This
allows for tasks to run in parallel (if they have
no dependents or are placed in disjoint parts of
graph).
Note: although this flow uses threads it is typically
expected that the underlying threads would be greenthreads
since python native threading is still suboptimal (for
various reasons).
Implements: blueprint locally-run-many-at-once
Change-Id: If434abd77758aa12fc99da395a2559995305a853
1. Simplify the exposed api to reduce race conditions
which could occur if we allowed flow details, task
details and log books to be deleted at different
times; reduce this down to just being able to save and
delete from logbooks (and only save/update for flow
and task details to) to reduce the problem cases.
2. Introduce a alembic migration with a proper schema so that
the initial database can be created in the first place,
adjust its exposed fields and relations to be defined
by said schema.
3. Use oslo db models instead of our own.
Change-Id: I78bbedf87d506d3f39157198638937c933235b7b
Bring over the sqlalchemy layer + dependents that
oslo provides so that we can use it in a dependent
commit to replace pieces of our own mini
sqlalchemy layer (that was derived from oslos)
Change-Id: Ifda2fd5c5b8eb1d71a4cfce2a2e0046a23010270
In order to rework some of the persistence layer I would
like to move around some of the files first, keeping job
specifics in a jobs folder. Having some of the items which
are root level taskflow items (flow, task) be at the root
of the hiearchy. Also for now until the celery work is
commited move that, since it doesn't make sense in backends
anyway.
Change-Id: If6c149710b40f70d4ec69ee8e523defe8f5e766d
The new way to build packages and perform setuptools
interactions seems to be via the pbr library so to
play well with others, use said library.
Change-Id: I519503fdf59457dc554870b8e6ff61613ab30d11
1. In order to aid future features its useful to split
split apart the linear_flow and have a good base
class that is not connected to a ordered or linear flow.
2. Have the graph flow inherit from the linear flow since
it is performing a linear run using a topological ordering
of the tasks.
Change-Id: I287cd4a666caf09ca3a4cdf1d53814e9c6feb765
The test pip ordering was resulting in the pep8 rules not really
running so this fixes that up and the resulting errors so we can
get a clean run.
Change-Id: Icc27be05cd513c4f04135c7e768007dbf9d07b10
Since python 2.6 is supported in openstack we need to
alter the format messages and the usage of functions
that exist in the 2.7 unittest by including unittest2
and using that instead.
Change-Id: I31c78d1f771603a6ad15ada50507cada0c5e331b
Take the current run_tests.sh from glance and include
that as well as fill in the rest of the pieces needed
for those that use venv functionality. Start adding
memory tests.
