Allow the watcher to re-register if the session is lost

In cases where the zookeeper cluster loses a member and the
client was connected to that member it will reconnect to another
member automatically. When this happens we should also make sure
that the watcher also correctly is kept active (by switching from
allow_session_lost=False to allow_session_lost=True).

If we don't do this we will not get notified of any new jobs
being added, other jobs being removed by other jobboard entities.

Fixes bug 1315564

Change-Id: Ia87f606f4d0beb7ebe532d1b3a31e06a58c3ac7e

taskflow/jobs/backends/impl_zookeeper.py

@@ -560,11 +560,12 @@ class ZookeeperJobBoard(jobboard.NotifyingJobBoard):
             if self._worker is None and self._emit_notifications:
                 self._worker = futures.ThreadPoolExecutor(max_workers=1)
             self._client.ensure_path(self.path)
-            self._job_watcher = watchers.ChildrenWatch(
-                self._client,
-                self.path,
-                func=self._on_job_posting,
-                allow_session_lost=False)
+            if self._job_watcher is None:
+                self._job_watcher = watchers.ChildrenWatch(
+                    self._client,
+                    self.path,
+                    func=self._on_job_posting,
+                    allow_session_lost=True)
         except excp.IncompatibleVersion:
             with excutils.save_and_reraise_exception():
                 try_clean()