nodepool/nodepool/zk
James E. Blair cf5f63bd6f Speed up node listing
This speeds up node listing in the CLI ("nodepool list") and the
webapp ("/node-list") significantly.

The main culprit is that filling in the "locked" field is expensive
because we attempt to lock each node.  To make that faster, we
now just query the lock contenders to determine whether it is locked
(if there are contenders, it's locked).

Further, in the webapp, we can use the cache more aggressively.  First,
we update the cache listener to watch lock contenders and cache those
values on our Node objects.  That means the webapp doesn't even need
to use the optmization above.  Further, we can have the webapp use
cached node ids, at which point it doesn't need to make any ZK queries
at all.

With a local setup of 6000 nodes and a localhost ZK connection (real
world times will be much higher due to network delays), this
takes the web server node list from 3 seconds to 0.009 seconds.

The CLI node list improves from 2.1 seconds to 0.8 seconds (excluding
startup time).

Change-Id: Id857556865b6ad75b9ec404bd7ef0c45e2a527bd
2022-07-12 14:05:48 -07:00
..
__init__.py Use Zuul-style ZooKeeper connections 2022-05-23 07:40:20 -07:00
components.py Add provider/pool priority support 2022-05-23 13:28:21 -07:00
exceptions.py Use Zuul-style ZooKeeper connections 2022-05-23 07:40:20 -07:00
handler.py Use Zuul-style ZooKeeper connections 2022-05-23 07:40:20 -07:00
zookeeper.py Speed up node listing 2022-07-12 14:05:48 -07:00