openstack-dev was decomissioned this night in https://review.openstack.org/621258
Update openstack-dev to openstack-discuss
Change-Id: I803a5acabe8ccdf18721707f5637d759d9b3d5b6
Closes-Bug: #1808899
Non-containerized multinode scenario jobs are no longer supported
and should be removed from zuul config.
Change-Id: I9abc7c9db1bd216dcd61925061e8bbc8c7e00b85
* There was a race condition in the test that expected different times
for two workflow runs (the second one is a rerun) although there
was a chance that they could be the same. A delay between two runs
was added to fix that.
* Delays of some workflow tasks in the tests were too long. No need
to do. It only increases test execution times w/o adding any real
value.
* Workflow definitions were moved into the tests where they're used.
It improves readability.
* Fixed style (blank lines, naming, etc.)
Change-Id: Id61bc612014fde600366b1230ff073930a7970ce
Sometimes it is very important to know exact time of task execution, but using fields
created_at and updated_at is incorrect, because in this case duration will consist
of not only Running time.
That new fields solve this problem.
Change-Id: I15be0648a0346f5b3dc9ef4a1b330a6c0e818385
Implements: blueprint mistral-add-started-finished-at
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
* See bug description for the example that didn't work. It was
caused by a simple mistake in a python expression of type
"my_set = my_set or set()" that didn't work as expected, i.e.
it created a new set even if my_set is already an empty set.
So, the proper expression that's needed is
"my_set = set() if my_set is None else my_set"
Change-Id: I2a787921449fecf3301013a770ffe712e9606baf
Closes-Bug: #1803677
* Once in a while we get DetachedInstanceError for action definitions
and it happens when they are fetched from cache. We must always
clone persistent objects before caching them.
Change-Id: I1d0cffea6775eb258dcefc0dbb8a6ee18effe597
Closes-Bug: #1803528
* Previously we used periodic jobs to refresh state of "join" tasks
and there was a guarantee that only one such job could run at a
time, so there wasn't a need in using locking. Now we allow more
than one such jobs run in parallel processes (and threads) so
we have to lock task execution and then check task state again
and update it, if needed.
Change-Id: Icaad486d9c3f830db0314dedb44664940cca0014
Closes-Bug: #1803483
Using "python setup.py check -r -s" method of checking the package has
been deprecated with the new recommendation to build the sdist and
wheel, then running "twine check" against the output.
Luckily, there is already a job that covers this that only runs when the
README, setup.py, or setup.cfg files change, making running this in the
pep8 job redundant. This covered by the test-release-openstack-python3
that is defined in the publish-to-pypi-python3 template.
More details can be found in this mailing list post:
http://lists.openstack.org/pipermail/openstack-dev/2018-October/136136.html
Change-Id: Iab0d2a2086cfecb8cc609c11de67ebbfc9d4d7d5
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
In case of creating/updating multiple workflows from one yaml,
we should not save the whole input to each workflow.
Closes-Bug: #1792975
Change-Id: I724c041ab3441805fcfa2cfc4a50afd774998cc7
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
* Removed using scheduler from action execution heartbeat checker
in favor of regular threads.
* Added the new config options "batch_size" under [action_heartbeat]
group to limit a number of action executions being processed during
one iteration the checker.
* Added a test checking that an action execution is automatically
failed by the heartbeat checker.
Closes-Bug: #1802065
Change-Id: I18c0c2c3159b9294c8af96c93c65a6edfc1de1a1
* Added the new property 'execution_integrity_check_batch_size'
under the [engine] group to limit the number of task executions
that the integrity checker may process during one iteration.
Closes-Bug: #1801876
Change-Id: I3c5074c45c476ebff109617cb15d56c54575dd4f
* action_queue module is replaced with the more generic
post_tx_queue module that allows to register operations that must
run after the main DB transaction associated with processing a
workflow event such as completing action.
* Instead of calling workflow completion check from all places
where task may possibly complete, Mistral now registers a post
transactional operation that runs after the main DB transaction
(to make sure at least one needed consistent DB read) right
inside the task completion logic. It reduces clutter significantly.
* Workflow completion check is now registered only if the just
completed task may lead to workflow completion, i.e. if it's the
last one in a workflow branch.
* Join now checks delayed calls to reduce a number of join
completion checks created with scheduler and also uses post
transactional queue for that.
Closes-Bug: #1801872
Change-Id: I90741d4121c48c42606dfa850cfe824557b095d0
The commands used by constraints need at least tox 2.0. Update to
reflect reality, which should help with local running of constraints
targets.
Change-Id: I0bb160bd02b876ed94a3804c88087289f9c3acc2
* Workflow completion algorithm use periodic scheduled jobs to
poll DB and determine when a workflow is finished. The problem
with this approach is that if Mistral runs another iteration
of such job too soon then running such jobs will create a big
load on the system. If too late, then a workflow may be in
RUNNING state for too long after all its tasks are completed.
The current implementation tries to predict a delay with which
the next job should run, based on a number of incompleted tasks.
This approach was initially taken because we switched to a
non-blocking transactional model (previously we locked the entire
workflow execution graph in order to change a state of anything)
and in this architecture, when we have parallel branches, i.e.
parallel DB transactions, we can't make a consistent read from
DB from neither of these transactions to make a reliable decision
about whether the workflow is completed or not. Using periodic
jobs was a solution. However, this approach has been proven to
work unreliably because such a prediction about delay before the
next job iteration doesn't work well on all variety of use cases
that we have.
This patch removes using periodic jobs in favor of using the
"two transactions" approach when in the first transaction we
handle action completion event (and task completion if it causes
it) and in the second transaction, if a task is completed, we
check if the workflow is completed. This approach guarantees
that at least one of the "second" transactions in parallel
branches will make needed consistent read from DB (i.e. will
see the actuall state of all needed objects) to make the right
decision.
Closes-Bug: #1799382
Change-Id: I2333507503b3b8226c184beb0bd783e1dcfa397f
The new openstacksdk mechanism forces a keystone request to find info
about endpoints. We don't need this for fake client, so skip the
__init__ of the class.
Change-Id: I5b0d89ac57c14f982a6afa638f088d365e0e4ab8
* In the latest version of cachetools lib (3.0.0) the previously
deprecated argument "missing" of cache classes has been removed.
* Disabled test_generator failing due to the changes in the
senlin client until it's fixed by https://review.openstack.org/614211
Change-Id: Iac42f592834734a6fddb743e947860b3bb7e1aba
