Shawn Wang 56cb08691d Add AIMD for Adaptive API Rate Limit ...

In a multi cluster setup, an adaptive API rate limit is more useful as
utilization can be dynamically balanced across all active clusters.

AIMD from TCP congestion control is a simple but effective algorithm
that fits our need here, as:
- API rate is similar to TCP window size. Each API call sent
  concurrently is similar to packets in the fly.
- Each successful API call that was blocked before sent will cause rate
  limit to be increased by 1. Similar to each ACK received.
- Each failed API call due to Server Busy (429/503) will cause rate
  limit to be decreased by half. Similar to packet loss.

When adaptive rate is set to AIMD, a custom hard limit can still be set,
max at 100/s. TCP slow start is not implemented as the upperbound of
rate is relativly small. API rate will be adjusted per period. API
rate under no circumstances will exceed the hard limit.

Change-Id: I7360f422c704d63adf59895b893dcdbef05cfd23

2020-05-27 16:02:42 -07:00

4.7 KiB

Raw Blame History

View Raw