Prompt
What's a good proxy success rate for web scraping?
Latest observation
A “good” proxy success rate depends on what you mean by success, but as a rough rule:
- 90%+: very good
- 80–90%: decent/usable
- Below 80%: usually a sign you need better proxies, better rotation, or request tuning
More practical benchmarks
For web scraping, people often look at a few different success metrics:
-
Request success rate
% of requests that return the expected page/content- Good target: 95%+ for stable sites, 90%+ for tougher sites
-
Proxy availability rate
% of proxies that are alive and usable at any moment- Good target: 85–95%+ depending on provider quality
-
Block/ban rate
% of requests getting 403, 429, captchas, or empty/soft-block pages- Ideally under 5–10%
What affects the “good” number
- Target site strictness: some sites are easy, others aggressively block automated traffic
- Proxy type:
- Datacenter proxies: cheaper, often lower success on strict sites
- Residential proxies: usually higher success, especially for protected sites
- Mobile proxies: often best success, usually most expensive
- Rotation strategy: bad rotation can tank success even with good proxies
- Request patterns: too fast, repetitive, or fingerprintable traffic lowers success
- Geo match: using the wrong country/region can cause failures
Rule of thumb
If you’re scraping at scale and your proxy success rate is:
- 95%+: excellent
- 85–95%: solid
- 70–85%: workable but needs tuning
- <70%: likely inefficient and costly
If you want, I can also tell you how to measure proxy success rate correctly (including how to count soft blocks and captcha pages).
Observation history
Unlock observation history
Create a free Obsurfable account to unlock historical responses, model comparisons, and deeper brand intelligence.
How did Obsurfable measure this prompt?
Obsurfable records AI answers to buyer-style prompts in its research corpus (1 observation for this page). Metrics are distributions over observations, not a single static ranking.
Which AI systems does Obsurfable collect answers from?
OpenAI, ChatGPT, Google, Gemini, Google AI Mode, Anthropic, Claude, Perplexity, Grok, DeepSeek, Mistral, Copilot, and Meta AI.