openstack/swift
Code Issues Proposed changes

proxy: Add a chance to skip memcache for get_*_info calls

Browse Source 
If you've got thousands of requests per second for objects in a single
container, you basically NEVER want that container's info to ever fall
out of memcache. If it *does*, all those clients are almost certainly
going to overload the container.

Avoid this by allowing some small fraction of requests to bypass and
refresh the cache, pushing out the TTL as long as there continue to be
requests to the container. The likelihood of skipping the cache is
configurable, similar to what we did for shard range sets.

Change-Id: If9249a42b30e2a2e7c4b0b91f947f24bf891b86f
Closes-Bug: #1883324
This commit is contained in:
Tim Burke
2022-01-06 12:09:58 -08:00
committed by Matthew Oliver
parent 24acc6e56b
commit 5c6407bf59
11 changed files with 327 additions and 223 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
etc/proxy-server.conf-sample
View File

@@ -153,8 +153,10 @@ use = egg:swift#proxy
# data is present in memcache, we can periodically refresh the data in memcache
# without causing a thundering herd. Values around 0.0 - 0.1 (i.e., one in
# every thousand requests skips cache, or fewer) are recommended.
# container_existence_skip_cache_pct = 0.0
# container_updating_shard_ranges_skip_cache_pct = 0.0
# container_listing_shard_ranges_skip_cache_pct = 0.0
# account_existence_skip_cache_pct = 0.0
#
# object_chunk_size = 65536
# client_chunk_size = 65536