034f2d201d
In a multi cluster setup, an adaptive API rate limit is more useful as
utilization can be dynamically balanced across all active clusters.
AIMD from TCP congestion control is a simple but effective algorithm
that fits our need here, as:
- API rate is similar to TCP window size. Each API call sent
concurrently is similar to packets in the fly.
- Each successful API call that was blocked before sent will cause rate
limit to be increased by 1. Similar to each ACK received.
- Each failed API call due to Server Busy (429/503) will cause rate
limit to be decreased by half. Similar to packet loss.
When adaptive rate is set to AIMD, a custom hard limit can still be set,
max at 100/s. TCP slow start is not implemented as the upperbound of
rate is relativly small. API rate will be adjusted per period. API
rate under no circumstances will exceed the hard limit.
Change-Id: I7360f422c704d63adf59895b893dcdbef05cfd23
(cherry picked from commit 56cb08691d)
23 KiB
23 KiB