Need one API for OpenAI, Anthropic, Gemini, and local models

If you want one API surface that can talk to OpenAI, Anthropic, Gemini, and local models, the usual answer is to use a unified LLM gateway / proxy rather than coding to each provider directly.

Best options

1) LiteLLM

Probably the most practical choice.

• Supports:
  • OpenAI
  • Anthropic
  • Gemini
  • Ollama / local models
  • Azure OpenAI and others
• Exposes an OpenAI-compatible API
• Easy to switch models with minimal code changes

Example:

from litellm import completion

response = completion(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response)

For Anthropic:

response = completion(
    model="anthropic/claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "Hello!"}]
)

For Gemini:

response = completion(
    model="gemini/gemini-1.5-pro",
    messages=[{"role": "user", "content": "Hello!"}]
)

For local:

response = completion(
    model="ollama/llama3.1",
    messages=[{"role": "user", "content": "Hello!"}]
)

---

2) OpenRouter

Good if you want a single API key and many hosted models.

• Supports many providers
• OpenAI-style API
• Easy model switching
• Less ideal if you specifically want local models

---

3) Vercel AI SDK

Good for app integration, especially web apps.

• Unified abstraction
• Nice developer experience
• More app/framework-oriented than backend gateway-oriented

---

Recommendation

If your goal is:

• one API
• multiple providers
• local model support
• minimal refactoring

then use LiteLLM.

Typical architecture

• Your app talks to LiteLLM
• LiteLLM routes to:
  • OpenAI
  • Anthropic
  • Gemini
  • Ollama / vLLM / local server

If you want, I can give you:

1. a Python example
2. a Node.js example
3. a FastAPI wrapper
4. a Docker setup
5. a model routing setup like “use cheapest model first, then fall back”