This adds a modified kazoo.DataWatch class which does not set
watches on paths that don't exist. This is mostly so that when
a DataWatched path is deleted, we don't leave a watch in place.
Without that, the executor api will leak watches on the ZK cluster.
We vendor the entire watches.py file from kazoo (ASL2 licensed) to
avoid any issues with API changes. Separately we will see if they
are interested in this class upstream.
This updates the tests to use the wchp 4lw to test that we don't
leak watches.
Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/799334
Change-Id: Ie4491eef03b58d858d95c78ba9454839d169cff1
10966948d7