Prompt
What should I use for alerting on API latency and errors?
Latest observation
Nov 10, 2026
OpenAI APIWeb search: off
For API latency and error alerting, a good default is:
- Metrics + alert rules for the main signals
- Tracing to diagnose the cause
- Logging for deep investigation
- Synthetic checks for end-to-end user-facing availability
What to alert on
1) Error rate
Alert when the percentage of failed requests crosses a threshold.
Common examples:
5xx rate > 1% for 5 minutes4xx spikeonly if it indicates a real issue in your API contract/auth flow- Separate alerts for:
- server errors
- dependency errors
- timeout errors
- auth failures, if relevant
2) Latency
Use percentile latency, not just average.
Common examples:
p95 latency > 500ms for 10 minutesp99 latency > 1s for 5 minutes
If your API has user-facing SLOs, alert on the SLO burn rate rather than raw latency thresholds.
3) Availability
Alert if requests are failing or timing out enough that the API is effectively unavailable.
Examples:
success rate < 99.9% over 5 minuteshealth check failing from multiple regions
4) Traffic anomalies
Sometimes high traffic causes latency/errors before outright failures.
Examples:
- sudden request volume spike
- queue depth increase
- saturation of DB connection pool / thread pool / CPU
Best practice: use RED / USE
A simple and effective model:
RED for request services
- Rate: requests per second
- Errors: failed requests
- Duration: latency
USE for infrastructure
- Utilization
- Saturation
- Errors
Recommended tooling
If you want a standard observability stack
- Metrics: Prometheus
- Alerting: Alertmanager
- Dashboards: Grafana
- Tracing: OpenTelemetry + Jaeger/Tempo/Datadog/New Relic
- Logs: Loki/ELK/Cloud logging
Managed alternatives
- Datadog
- New Relic
- Dynatrace
- Grafana Cloud
- AWS CloudWatch + X-Ray
- GCP Cloud Monitoring + Trace
- Azure Monitor
Alerting tips
- Prefer page-worthy alerts only for things users feel immediately.
- Use multi-window burn rate alerts for SLOs to reduce noise.
- Alert on symptoms, not every possible root cause.
- Add labels/tags like:
- service
- endpoint
- region
- status code
- dependency
- Ensure alerts link to a dashboard and logs/traces.
Practical starting point
If you’re starting from scratch, do this:
- Instrument request latency, request count, and request errors
- Create dashboards for:
- p50/p95/p99 latency
- error rate
- request rate
- Add alerts:
- error rate > threshold
- p95 latency > threshold
- availability/SLO burn rate
- Correlate with traces and logs
If you want, I can suggest:
- a specific alert policy
- Prometheus alert rules
- or a Datadog/New Relic setup for your stack.