Parallelize config cache loading

Loading config involves significant network operations for each project:

* Loading project keys
* Asking the source for the list of branches for each project
* Retrieving the config file contents from the ZK cache (if present)
* Retrieving the config file contents from git (otherwise)

Only the third item in that list is parallelized currently; the others
are serialized.  To parallelize the remainder, use a thread pool executor.

The value of max_workers=4 is chosen as it appears in practice on OpenDev
to make the most significant reduction in startup time while higher values
make little difference (and could potentially contribute to DoS scenarios
or local thread contention).  Observed config priming times for various
worker counts:

1: 282s
2: 181s
4: 144s
8: 146s

Change-Id: I65472a8af96ed95eb28b88cc623ef103be76a46f

zuul/zk/__init__.py

@@ -17,7 +17,7 @@ from threading import Thread
 from typing import List, Callable
 
 from kazoo.client import KazooClient
-from kazoo.exceptions import NoNodeError
+from kazoo.exceptions import NoNodeError, NodeExistsError
 from kazoo.handlers.threading import KazooTimeoutError
 from kazoo.protocol.states import KazooState
 
@@ -211,8 +211,11 @@ class ZooKeeperClient(object):
         try:
             zstat = self.client.set("/zuul/ltime", b"")
         except NoNodeError:
-            self.client.create("/zuul/ltime", b"", makepath=True)
-            zstat = self.client.set("/zuul/ltime", b"")
+            try:
+                self.client.create("/zuul/ltime", b"", makepath=True)
+                zstat = self.client.set("/zuul/ltime", b"")
+            except NodeExistsError:
+                zstat = self.client.set("/zuul/ltime", b"")
         return zstat.last_modified_transaction_id