heat/heat
Zane Bitter ae2b47d8fd Fix race condition deleting in-progress stack
If we call stack_delete on a stack with an operation in progress, we kill
any existing delete thread that is running. However, we don't wait for that
thread to die before starting a new thread to delete the stack again. If
any part of the cleanup operation in the old thread (i.e. handling of the
GreenthreadExit exception) causes a context switch (which is likely), other
threads can start working while the cleanup is still in progress. This
could create race conditions like the one in bug 1328983.

Avoid this problem by making sure we wait for all threads in a thread group
to die before continuing. (Note that this means the user's API call is
blocking on the cleanup of the old thread. This is sadly unavoidable for
now, but should probably be fixed in the future by stopping the old thread
from the new delete thread.)

This was suggested earlier, but removed without explanation between
patchsets 11 and 12 of I188e43ad88b98da7d1a08269189aaefa57c36df2, which
implemented deletion of in-progress stacks with locks:
https://review.openstack.org/#/c/63002/11..12/heat/engine/service.py

Also remove the call to stack_lock_release(), which was a hack around the
fact that wait() does not wait for link()ed functions - eventlet sends the
exit event (that wait() is waiting on) before resolving links. Instead, add
another link to the end of the list to indicate that links have all been
run. This should eliminate "Lock was already released" messages in the
logs.

Change-Id: I2e4561cbe29ab10554da67859df8c2db0854dd38
2014-06-20 12:37:01 -04:00
..
api Merge "ec2token tolerate fully-qualified ec2authtoken auth_uri" 2014-06-19 19:14:42 +00:00
cloudinit Remove translation from loguserdata 2014-06-01 17:45:46 +02:00
cmd Sync latest oslo.db code 2014-04-11 10:48:46 +03:00
common Merge "Don't set context auth_token in heat_keystoneclient" 2014-06-20 12:47:03 +00:00
db Allow raw templates to be updated in the DB 2014-06-16 11:13:30 -04:00
doc Move resource doc generation to doc/source/ext 2013-11-13 15:21:04 +13:00
engine Fix race condition deleting in-progress stack 2014-06-20 12:37:01 -04:00
locale Remove all mostly untranslated PO files 2014-05-22 18:41:06 +02:00
openstack Sync oslo-incubator.middleware module 2014-06-17 18:37:10 +09:00
rpc Merge "Adding error message for output description" 2014-06-10 12:00:59 +00:00
scaling Use add_resource() to build autoscaling templates 2014-06-05 13:45:14 -04:00
tests Fix race condition deleting in-progress stack 2014-06-20 12:37:01 -04:00
__init__.py Make the first line of every file consistent. 2014-03-04 09:03:04 -06:00
version.py Make the first line of every file consistent. 2014-03-04 09:03:04 -06:00