How do I benchmark proxy performance?

To benchmark proxy performance, measure it under realistic load and compare it against a direct connection baseline. The key is to test latency, throughput, concurrency, and error behavior in a controlled, repeatable way.

1) Define what “performance” means

Common proxy metrics:

• Latency: time to first byte / end-to-end response time
• Throughput: requests/sec, connections/sec, MB/s
• Concurrency: how many simultaneous connections/requests it handles
• Connection setup cost: TCP + TLS + auth overhead
• Reliability under load: error rate, timeouts, retries, dropped connections
• Resource usage: CPU, memory, network, file descriptors
• Tail latency: p95/p99 response times

2) Benchmark against a baseline

Always compare:

1. Direct to origin (no proxy)
2. Through the proxy
3. Optionally, different proxy configurations (auth on/off, TLS on/off, keep-alive on/off, HTTP/1.1 vs HTTP/2)

This tells you the proxy overhead.

3) Use realistic test traffic

Match the workload to real usage:

• HTTP GET/POST mix
• Payload sizes similar to production
• Keep-alive vs short-lived connections
• HTTPS/TLS if your proxy supports it
• Multiple destinations if the proxy routes to different upstreams
• If it’s a SOCKS proxy or transparent proxy, test the protocols you actually use

4) Choose the right tools

Depending on proxy type:

HTTP proxy

• wrk, wrk2, hey, ab, vegeta, curl for simple tests
• fortio is good for latency distributions

SOCKS proxy

• curl --socks5, socat, custom scripts, or app-specific load generators

General network/proxy testing

• iperf3 for raw network throughput
• Custom client scripts if you need to simulate your exact traffic pattern

5) Example benchmark plan

A. Latency test

Run a fixed number of requests and record p50/p95/p99:

curl -o /dev/null -s -w 'time_total=%{time_total}\n' https://example.com

Through proxy:

curl -o /dev/null -s -w 'time_total=%{time_total}\n' -x http://proxy-host:3128 https://example.com

Compare average and tail latencies.

B. Throughput test with wrk

Direct:

wrk -t4 -c100 -d30s https://example.com

Through proxy depends on the tool/proxy support. If wrk doesn’t support your proxy type directly, use a proxy-aware test client or custom script.

C. Connection setup benchmark

Test many short connections to measure handshake overhead:

• Disable keep-alive
• Use small requests
• Measure requests/sec and error rate

6) Test under increasing load

Sweep across concurrency levels:

• 1, 10, 50, 100, 500, 1000 concurrent connections
• Watch when latency spikes or errors start

This shows the proxy’s saturation point.

7) Measure system resources

While running tests, monitor:

• CPU: top, htop, mpstat
• Memory: free, smem
• Network: iftop, nload, sar -n DEV
• Sockets/file descriptors: ss -s, lsof, ulimit -n
• Proxy logs for timeouts or upstream failures

8) Make the results trustworthy

Avoid noisy measurements:

• Run on dedicated or quiet machines
• Warm up caches/JITs
• Repeat each test multiple times
• Use the same origin server and network path
• Pin versions/configs of proxy and client
• Keep DNS, TLS session reuse, and keep-alive behavior consistent

9) Interpret the results

Look for:

• Added latency = proxy overhead
• Reduced throughput = proxy bottleneck
• High p99 = queueing or contention
• Errors/timeouts = capacity or stability issues
• High CPU = protocol parsing, encryption, logging, etc.

10) Simple checklist

• Baseline direct connection
• Same test through proxy
• Vary concurrency
• Measure p50/p95/p99 and throughput
• Test with realistic payloads and connection reuse
• Monitor CPU/memory/network/errors
• Repeat and compare across runs

If you want, I can also give you:

• a benchmark script for HTTP proxies,
• a SOCKS proxy benchmark setup, or
• a Docker-based reproducible test environment.