why is my api gateway not applying rate limits consistently

Rate limits in API gateways can look “inconsistent” for a few common reasons. Usually it’s not the gateway randomly failing, but one of these patterns:

1) You’re hitting multiple gateway instances

If your gateway is running behind a load balancer and the rate-limit counter is stored in memory on each instance, each instance tracks limits separately.
So if you have 3 gateway pods and a limit of 100/minute, a client may effectively get ~300/minute total unless the limiter uses a shared store like Redis.

2) The limiter uses approximate or sliding windows

Some gateways use:

• fixed windows,
• sliding windows,
• token bucket / leaky bucket algorithms.

These can allow bursts near window boundaries, which can feel inconsistent. Example: you might send requests at the end of one window and again at the start of the next, exceeding the “per minute” rate briefly.

3) Keys are not what you think they are

Rate limits often depend on a key such as:

• API key
• client IP
• user ID
• JWT claim
• route/path

If the key changes unexpectedly, each request may get a different bucket. Common causes:

• missing or inconsistent auth headers
• requests coming through NAT/proxies so IP isn’t stable
• different users/tokens hitting the same endpoint
• gateway configured to rate limit by route + key, but route matching varies

4) Proxy/load balancer headers are misconfigured

If your gateway is behind another proxy, the source IP may always look like the proxy’s IP unless X-Forwarded-For / real IP handling is set correctly.
That can cause:

• all users sharing one limit bucket
• or each request appearing from a different IP depending on chain/order

5) Caches or distributed state are eventually consistent

In some distributed setups, the counter updates may not propagate immediately across nodes. That can create brief overshoots or uneven enforcement.

6) Some traffic bypasses the limiter

Not all requests may be going through the same path:

• internal service-to-service calls bypassing gateway
• health checks excluded
• different hostnames / routes / stages / environments
• fallback or canary routes not configured with the same policy

7) Different policies overlap

You may have:

• a global rate limit,
• a per-route limit,
• a per-user limit,
• a WAF/CDN limit.

These can interact in ways that look inconsistent, especially if one layer blocks and another allows.

8) Clock / window alignment issues

If the gateway and backend store use different clocks or time drift, window resets can occur slightly differently across nodes.

---

What to check first

1. Is the limiter shared across all gateway instances?
   If not, move counters to Redis or another centralized store.

2. What is the exact rate-limit key?
   Confirm whether it’s IP, API key, JWT sub, etc.

3. Are forwarded client IP headers configured correctly?
   Verify X-Forwarded-For, X-Real-IP, or equivalent.

4. Are you using fixed-window limiting?
   If yes, bursts near boundary are expected.

5. Are all routes/stages covered by the same policy?

6. Do logs show the same key/counter for requests that should be limited together?

---

Quick way to diagnose

Log for each request:

• rate-limit key
• gateway instance/pod
• route matched
• current window/bucket id
• counter value before/after
• decision (allowed/blocked)

That usually reveals the issue within minutes.

If you want, tell me which API gateway you’re using (Kong, NGINX, Apigee, AWS API Gateway, Azure API Management, Envoy, etc.) and how it’s deployed, and I can help pinpoint the likely cause.