We currently leak the lock path for LockableZKObjects (which are only
used for zuul-launcher). To correct this, delete the lock path whenever
we delete the associated object.
This approach attempts to minimize additional ZK calls.
The main consideration when deleting lock paths is whether other actors
may be in contetion for the lock at the time of deleting. By deleting
the lock path after deleting the object that we are protecting, that
should reduce contention since other actors should no longer be
interested in locking a path that does not exist. However, due to system
latency and caches, that may still happen.
If we are unable to delete the lock path because another actor has
created a lock contender during our recursive delete, we will retry our
delete for up to 30 seconds. This should give us ample opportunity to
complete the deletion.
If, after we sucessfully delete the lock path, another actor attempts
to acquire the lock, the normal Kazoo Lock recipe would fully re-create
the lock path. In order to avoid this situation, we adjust our subclass
of Lock to indicate that it should not create the path when acquiring the
lock. Instead, we will create the lock path when we originally create
the LockableZKObject itself. Therefore, any failed lock attempt with a
NoNodeError can be interpreted as indicating the object no longer exists
(and is not lockable).
We do not currently create any LockableZKObjects with a corresponding
lock, but that does seem like something we may wish to do in the
future. To accommodate that, we create the lock directory immediately
before creating the object itself. If, in the future, we want to lock
the object on creation, we can insert that instruction between the
two.
Because acquiring and releasing the lock can produce some errors
interesting enough to log, those methods are updated to accept a
ZKContext principally so that we can use both the ZK Client object as
well as a logger. We don't currently check the lock supplied with the
context, but we might wish to do so in the future in the acquisition
method. We will probably never want to check the lock or session in
the release method, but let's still pass a context for consistency.
Change-Id: Ic87e92a5e553e753f6fa43bb0674bbe40e7c2597
50e1338c32