I'm building an app that needs to send LLM requests to OpenAI, Anthropic, and open-source models; what should I use?

If you want one app to talk to OpenAI, Anthropic, and open-source model providers, the best choice is usually an LLM abstraction layer / gateway rather than hard-coding each vendor SDK directly.

Good options

1) Direct provider SDKs

Use OpenAI’s SDK, Anthropic’s SDK, and whatever your open-source hosting uses.

Best if:

• you only have a couple integrations
• you want full access to each provider’s latest features
• you don’t mind separate request/response formats

Downside:

• duplicated code
• different auth, retries, streaming, tool-calling, and error handling
• harder to switch models later

---

2) A unified LLM SDK / routing layer

Examples:

• LiteLLM
• OpenRouter (more of a model gateway than just SDK)
• Vercel AI SDK (great for app UI/server integration, though not a full provider-agnostic backend gateway)

Best if:

• you want one interface for many providers
• you need fallback/routing across models
• you want easier model switching and cost control

Why this is usually the best fit:

• one API shape across vendors
• easier retries, logging, usage tracking
• easier to add open-source providers later

If your main need is “send requests to multiple model providers from one app,” LiteLLM is often the most practical answer.

---

3) Use a model gateway / router service

Examples:

• OpenRouter
• enterprise gateways / internal proxy layers

Best if:

• you want centralized key management
• model routing, fallbacks, and observability
• you’re okay with an extra hop / third-party intermediary

---

My recommendation

If you’re starting now:

• Backend: use LiteLLM (or a similar abstraction layer)
• Frontend/server UI: optionally pair with Vercel AI SDK if you’re building chat/streaming UX
• For open-source models: connect through providers like:
  • Groq
  • Together.ai
  • Fireworks
  • Replicate
  • local vLLM / Ollama / TGI
  • or OpenRouter if it supports the models you want

If you need maximum control:

• use direct provider SDKs, but wrap them behind your own internal interface

---

What to choose based on your priorities

• Fastest to ship: OpenRouter or LiteLLM
• Most control: your own abstraction layer over provider SDKs
• Best frontend developer experience: Vercel AI SDK
• Best for multi-provider backend routing: LiteLLM

---

A practical architecture

1. Your app calls one internal llmClient
2. llmClient maps your standard request to:
   • OpenAI
   • Anthropic
   • open-source host
3. Add:
   • retries
   • fallback models
   • prompt logging
   • usage/cost tracking
   • streaming support

---

Short answer

If you want a single integration for OpenAI, Anthropic, and open-source models, start with LiteLLM. If you’re building a chat product with streaming/UI, consider Vercel AI SDK on top.

If you tell me your stack (Node/Python/Go, serverless vs backend, and whether you need streaming/tool-calling), I can recommend the exact setup.