Prompt
I'm building a B2B API and need to block scraping
Latest observation
Oct 29, 2026
OpenAI APIWeb search: off
You can’t fully prevent scraping if an API is reachable by third parties, but you can make it much harder, easier to detect, and less profitable.
Strongest practical defenses for a B2B API
1) Require authenticated access
- Use per-customer API keys or, better, OAuth2 client credentials
- Issue short-lived access tokens
- Scope tokens to only the endpoints/data each customer needs
- Rotate credentials regularly
2) Add authorization and tenancy controls
- Enforce tenant isolation on every request
- Tie every object/query to the caller’s tenant
- Never trust client-supplied tenant IDs without server-side checks
3) Rate limit aggressively
Apply limits at multiple levels:
- Per API key / client
- Per IP
- Per tenant
- Per endpoint
- Per response size / cost
Use burst + sustained limits. Return
429with clear retry headers.
4) Use anomaly detection
Flag or block patterns like:
- High request volume with low dwell time
- Sequential enumeration of IDs
- Many unique pages/queries in short periods
- Requests across unusual geographies or ASN ranges
- Constant intervals consistent with automation
- Large export behavior
5) Require proof of legitimacy for bulk access
For sensitive or high-volume endpoints:
- Require explicit bulk export jobs
- Put exports behind approval workflows
- Use signed URLs with short expiry for downloads
- Limit export frequency and size
6) Use bot/automation friction
For browser-based access:
- MFA for admin portals
- CAPTCHA only on suspicious actions, not every request
- Device/session binding
- Challenge/response on suspicious behavior
For pure APIs, CAPTCHAs are usually not appropriate, but behavioral challenges can help for login and account creation.
7) Obfuscate and reduce data value
- Don’t return more data than needed
- Avoid unnecessary fields, exact counts, or highly granular metadata
- Paginate carefully and cap page sizes
- Randomize or delay non-critical fields if business-acceptable
- Consider aggregating sensitive fields
8) Monitor and log everything
Log:
- API key, tenant, IP, user agent, request path, parameters, response status, latency
- Export/download events
- Token creation/refresh
- Failed auth attempts Alert on:
- Spikes
- Enumeration
- New IPs for a key
- Sudden geography changes
- Many 401/403/429s
9) Contractual and legal controls
For B2B, technical controls should be backed by:
- Terms of service prohibiting scraping/resale
- Usage limits in contracts
- Audit rights
- Suspension clauses for abuse
10) Consider network-level controls
- IP allowlists for enterprise customers
- Mutual TLS for high-trust integrations
- Private connectivity options: VPN, VPC peering, PrivateLink
- WAF rules for known abusive patterns
A good layered approach
If you want a practical baseline:
- OAuth2 client credentials + short-lived tokens
- Strict tenant-level authorization
- Per-key/IP/tenant rate limits
- Behavioral monitoring + alerting
- Bulk export controls
- Optional IP allowlisting or mTLS for premium customers
What won’t work well by itself
- Obscurity/security-through-obscurity
- User-agent checks alone
- Simple CAPTCHA on API endpoints
- Static API keys with no rotation
- IP blocking only
If you want, I can help you design a concrete anti-scraping architecture for your API stack (e.g. REST/GraphQL, AWS/GCP/Azure, or Node/Go/Python).