What should I use to avoid vendor lock-in with LLM APIs?

To avoid vendor lock-in with LLM APIs, use an abstraction layer between your app and the model provider.

Good options:

1. Provider-agnostic SDKs / gateways

   • Examples: LiteLLM, OpenRouter, LangChain/LlamaIndex model wrappers
   • These let you switch between OpenAI, Anthropic, Google, Azure, etc. with minimal code changes.

2. Your own thin wrapper

   • Create an internal interface like:
     • generateText(prompt, options)
     • embedText(text)
     • streamChat(messages)
   • Keep provider-specific code in adapters behind that interface.

3. Standard request/response formats

   • Normalize things like:
     • chat message schema
     • tool/function calling
     • streaming events
     • error handling
   • This makes swapping providers easier.

4. Model gateway / orchestration layer

   • Route requests through a service that can:
     • fail over to another model
     • do A/B testing
     • switch by cost/latency/quality
   • Useful if you expect to manage multiple providers.

5. Avoid provider-specific features in core logic

   • Minimize dependence on:
     • proprietary function-calling formats
     • custom safety settings
     • special prompt templates
     • vendor-only memory/session features

6. Abstract embeddings, rerankers, and vector DBs too

   • Lock-in often happens in the surrounding stack, not just the chat API.

Practical recommendation:

• If you want the fastest path: use LiteLLM or OpenRouter
• If you want the most control: build your own provider abstraction layer
• If you want enterprise routing/failover: use a model gateway

If you want, I can sketch a simple provider-agnostic interface in TypeScript, Python, or Go.