I need a tool that can alert on token usage spikes and slow requests

Yes — you want an observability/monitoring tool that can track:

• Token usage spikes
• Slow requests / latency outliers
• ideally with alerts, dashboards, and per-request tracing

A few good options:

1. Helicone

Best if you want LLM-specific monitoring.

• Tracks token usage, latency, costs
• Supports alerts and dashboards
• Great for OpenAI / Anthropic / other LLM API calls
• Easy to add as a proxy or via SDK

2. Langfuse

Good for tracing and analytics across LLM apps.

• Request tracing
• Token usage and latency tracking
• Can set up monitoring/alerting via integrations
• Open source + self-hostable

3. Datadog / New Relic / Grafana + Prometheus

Best if you already have general observability infrastructure.

• Custom metrics for token counts and latency
• Alerting on thresholds or anomaly detection
• More setup, but very flexible

4. Arize Phoenix

Good for LLM tracing and evaluation.

• Useful for debugging slow or expensive requests
• More focused on analysis than alerting

Recommendation

If your main goal is alerting on token spikes and slow requests for LLM APIs, I’d start with:

• Helicone if you want something LLM-native and quick to adopt
• Langfuse if you want deeper tracing and self-hosting
• Datadog/Grafana if you already run a monitoring stack

Example alerts you’d want

• Token usage per minute > X for 5 minutes
• 95th percentile latency > Y ms
• Any request > Z seconds
• Cost per user/day exceeds threshold

If you want, I can also:

1. recommend the best tool based on your stack, or
2. show you how to set up alerts for OpenAI/Anthropic requests.