Prompt
What should I use to route prompts to the cheapest model that works?
Latest observation
Oct 10, 2026
OpenAI APIWeb search: off
Use a model router rather than sending everything to one LLM.
Good options
- OpenRouter — easiest if you want access to many models and simple fallback/routing.
- LiteLLM — great if you want to build your own routing logic across providers.
- Semantic Router / custom classifier — useful if you want to decide based on prompt type, not just price.
Practical approach
- Start with a cheap model for most requests.
- Escalate only when needed based on:
- prompt length
- task type
- confidence/quality checks
- tool/function-calling needs
- Fallback to a stronger model if the cheap one fails or scores poorly.
Common setup
- Simple Q&A, extraction, summarization → cheap model
- Reasoning, coding, long context, complex tool use → better model
- Sensitive or high-stakes tasks → highest reliability model
If you want a recommendation
- For quickest implementation: OpenRouter
- For custom routing control: LiteLLM
- For logic based on prompt semantics: a small classifier + LiteLLM/OpenRouter
If you want, I can suggest a specific routing architecture or a minimal Python example.
Brands