Prompt
What should I use to avoid rewriting code when switching LLM providers?
Latest observation
Oct 10, 2026
OpenAI APIWeb search: off
Use a provider-agnostic LLM abstraction layer.
Good options:
- LiteLLM – very popular for unifying OpenAI, Anthropic, Azure, Gemini, Mistral, etc. behind one API.
- LangChain / LlamaIndex – useful if you also want orchestration, tools, retrieval, agents.
- Vercel AI SDK – good if you’re building web apps and want a consistent interface across providers.
If your main goal is simply “swap providers without rewriting code”, the best default choice is usually:
LiteLLM + a small internal wrapper/interface
That gives you:
- one consistent code path
- easier provider switching
- centralized handling of retries, logging, rate limits, and model names
A common pattern is to define your own interface like:
interface LLMClient {
chat(messages: Message[]): Promise<string>;
}
Then implement it once using LiteLLM or another adapter.
If you want, I can suggest the best option based on your stack:
- Python or TypeScript
- simple API calls or agents/workflows
- backend, frontend, or both