which API has the lowest latency for LLM calls

The lowest-latency LLM API depends on where you are, what model size you need, and whether “latency” means first-token time or full response time. In practice:

Usually fastest options

• OpenAI API (small/fast models) — often very strong on time-to-first-token and overall reliability.
• Anthropic Claude via API — can be very competitive, especially on streaming starts.
• Google Gemini API — often good latency, especially for certain regions and lighter models.
• Groq — commonly cited as among the fastest raw inference latencies for supported open models, especially for short prompts and streaming.
• Together / Fireworks / Anyscale / DeepInfra — can be fast, but performance varies a lot by model and region.

If you want the absolute lowest latency

• Groq is frequently the answer for supported models.
• For hosted frontier proprietary models, OpenAI is often a top choice for low latency and consistent streaming.

Important caveats

Latency depends heavily on:

• Model size (smaller = faster)
• Region/edge proximity
• Prompt length
• Output length
• Streaming support
• Provider load and queueing
• Whether you use a dedicated deployment / provisioned throughput

Practical recommendation

If your goal is minimum latency in production:

1. Benchmark your exact prompt + model + region
2. Measure:
   • TTFT (time to first token)
   • Tokens/sec
   • End-to-end time
3. Test at least:
   • Groq
   • OpenAI small model
   • Anthropic small/fast model
   • Gemini Flash-class model

If you want, I can give you a ranked list by latency for a specific use case, like:

• chat assistant
• code generation
• streaming autocomplete
• RAG with short answers
• global users vs US-only users