How do I test third-party API slowdowns under load

To test third-party API slowdowns under load, you want to simulate your system’s real traffic patterns while making the external dependency behave slowly, unreliably, or rate-limited. The goal is to see how your app degrades, not just whether it “works.”

1) Decide what you’re testing

Common goals:

• Timeout handling: does your service fail fast?
• Retry behavior: do retries pile up and make things worse?
• Circuit breaking / fallback: does the system shed load gracefully?
• Queue buildup: do threads, connections, or jobs get stuck?
• User-facing impact: do requests time out, error, or degrade partially?

2) Don’t hit the real third-party API directly in load tests

Instead, use one of these:

A. Mock/stub the API

Create a local fake service that mimics the third-party endpoints and can inject:

• latency
• timeouts
• 429s / 503s
• malformed responses
• intermittent failures

Good for repeatable tests.

B. Proxy/traffic shaping

Put a proxy or gateway in front of the real API and inject delay/failure:

• Envoy / NGINX / HAProxy
• toxiproxy
• service mesh fault injection
• cloud/API gateway latency rules

Good when you want realistic payloads but controlled slowdowns.

C. Record/replay

Capture real responses and replay them locally with added latency.
Good for fidelity without external dependency cost/risk.

3) Use a load tool that supports realistic traffic

Examples:

• k6
• JMeter
• Gatling
• Locust
• custom scripts if needed

Model:

• concurrency
• request rate
• bursts vs steady load
• think time
• distribution of request types

4) Inject slowdown scenarios

Test several cases, not just one.

Slow response scenarios

• +100 ms latency
• +500 ms latency
• +2 s latency
• tail latency spikes (e.g., 5% of requests delayed 10 s)

Failure scenarios

• 1% / 5% / 20% timeouts
• 429 throttling
• 503 intermittent outage
• connection resets
• empty/partial responses

Combined scenarios

• slow + error
• slow + rate limit
• slow + high concurrency

This is often where systems break.

5) Verify your app’s protective controls

Make sure these are configured and tested:

• client timeouts
• max retries
• exponential backoff + jitter
• bulkheads / separate worker pools
• circuit breakers
• rate limiting
• queue limits
• graceful degradation / fallback data
• idempotency for retried writes

A common failure mode is “retry storms” under load.

6) Observe the right metrics

Measure both app and dependency behavior.

Application metrics

• request latency p50/p95/p99
• error rate
• throughput
• thread pool / worker utilization
• connection pool saturation
• queue depth
• retry counts
• circuit breaker open rate
• timeout count

Third-party dependency metrics

• response time
• error rate
• rate-limited responses
• connection errors

Infrastructure metrics

• CPU, memory
• open file descriptors
• network sockets
• GC pauses
• pod/container restarts

7) Run tests incrementally

Start small:

1. Baseline with no slowdown
2. Add mild latency
3. Increase concurrency
4. Add failure rates
5. Push to saturation

This helps identify the exact threshold where behavior changes.

8) If you need a quick setup, use toxiproxy

A common pattern:

• app points to a local proxy
• proxy forwards to mock or real API
• inject latency/faults dynamically

Example ideas:

• latency
• timeout
• bandwidth
• slicer / packet corruption depending on tool

9) Validate user experience, not just server metrics

Check:

• Does the request return partial results?
• Does the UI spin forever?
• Are background jobs delayed?
• Are downstream queues growing?
• Does the system recover automatically after the API recovers?

10) Best practice: test in staging first

If you must use the real third-party API:

• get permission
• use a sandbox/staging account
• respect rate limits
• coordinate with the vendor
• cap concurrency and request volume

Simple test matrix

You can structure your tests like this:

Scenario	Concurrency	Added Latency	Error Rate
Baseline	10	0 ms	0%
Mild slowdown	50	200 ms	0%
Heavy slowdown	100	1 s	0%
Tail latency	100	0 ms avg, 5 s for 5%	0%
Throttling	100	0 ms	10% 429
Outage	100	timeout after 2 s	100%

Recommended approach

If you want the most practical setup:

1. Put a mock API or proxy in front of the third-party dependency
2. Run k6/Locust/JMeter against your app
3. Inject latency and 429/timeout faults
4. Watch timeouts, retries, queues, and saturation
5. Tune client timeouts, retries, and circuit breakers until the system degrades gracefully

If you want, I can give you:

• a k6 example for simulating slow third-party calls,
• a toxiproxy setup,
• or a test plan template for your specific stack.