Prompt
What should I use for low-latency cache under burst traffic?
Latest observation
Nov 18, 2026
OpenAI APIWeb search: off
For low-latency caching under burst traffic, the best choice is usually an in-memory distributed cache, most commonly:
Good default choice
- Redis
- Very low latency
- Handles high request rates well
- Supports eviction policies, TTLs, persistence options, replication, clustering
- Great for session storage, rate limiting, hot data, query/result caching
If you need even faster, single-node local cache
- In-process cache like:
- Caffeine (Java)
- Ristretto (Go)
- Guava Cache (older Java option)
- Fastest possible reads, but only works per instance and is not shared across servers
If burst traffic is large and distributed
Use a two-tier cache:
- Local in-memory cache on each app instance for ultra-low latency
- Redis as the shared distributed cache/fallback
This reduces load on Redis and keeps p99 latency low during bursts.
When to choose what
- Need shared cache across many app instances → Redis
- Need absolute lowest latency per instance → local in-memory cache
- Need both low latency and scalability → local cache + Redis
- Need huge scale and simple key/value access → Redis Cluster or managed Redis
Practical tips for burst traffic
- Use TTL on all cached items
- Add cache warming for known hot keys
- Prevent cache stampede with:
- request coalescing / single-flight
- soft TTL + background refresh
- Size memory carefully and use an eviction policy like LRU/LFU
- Monitor hit rate, eviction rate, and p95/p99 latency
If you tell me your stack (language/framework), whether the cache must be shared, and your expected QPS, I can recommend a specific setup.