707a65ab3c
Memcached based cooperative token is a improved version of ghetto lock, see the description of ghetto lock at here: https://github.com/memcached/memcached/wiki/ProgrammingTricks It's used to avoid the thundering herd situation which many caching users face: given a cache item that is popular and difficult to recreate, in the event of cache misses, users could end up with hundreds (or thousands) of processes slamming the backend database at the same time in an attempt to refill the same cache content. This thundering herd problem not only often leads to unresponsive backend; and also those writes into memcached cause premature cache eviction under memory pressure. With cooperative token, when lots of in-flight callers try to get the cached item specified by key from memcache and get cache misses, only the first few query requests (limited by by ``num_tokens``) will be able get the cooperative tokens by creating or incrementing an internal memcache key, and then those callers with tokens can send backend requests to fetch data from backend servers and be able to set data into memcache; all other cache miss requests without a token should wait for cache filling to finish, instead of all querying the backend servers at the same time. Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Yan Xiao <yanxiao@nvidia.com> Co-Authored-By: Alistair Coles <alistairncoles@gmail.com> Signed-off-by: Jianjian Huo <jhuo@nvidia.com> Change-Id: I50ff92441c2f2c49b3034644aba59930e8a99589
11 KiB
11 KiB