What should I use to deploy models across AWS and Azure?

If you want to deploy models across both AWS and Azure, the best choice is usually a cloud-agnostic model serving layer rather than a cloud-specific managed ML service.

Good options

1. Kubernetes + Kubeflow / KServe

• Best for: portable, production model deployment across clouds
• Why: runs on both AWS EKS and Azure AKS
• Pros: consistent deployment patterns, autoscaling, can standardize CI/CD
• Cons: more infrastructure to manage

2. MLflow + Docker + Kubernetes

• Best for: teams that want lightweight portability
• Why: MLflow handles model packaging/registry; Docker/K8s handles serving
• Pros: simple, cloud-neutral, easy to integrate
• Cons: you’ll need to build more of the serving stack yourself

3. Seldon Core

• Best for: advanced model serving on Kubernetes
• Why: works across clouds via K8s
• Pros: good for inference, A/B testing, canary rollout, observability
• Cons: more complex than basic serving

4. BentoML

• Best for: fast model packaging and deployment portability
• Why: easy to containerize and deploy on AWS/Azure Kubernetes or containers
• Pros: developer-friendly, simple API serving
• Cons: less “platform” than Kubeflow/Seldon

What I’d recommend

• If you want the most portable enterprise setup: Kubernetes + KServe
• If you want the simplest portable setup: BentoML + Docker + Kubernetes
• If you already use MLflow: pair it with Docker + K8s or KServe

What to avoid if you need multi-cloud portability

• AWS SageMaker-only workflows
• Azure ML-only workflows

Those are great within their own ecosystems, but they make cross-cloud portability harder.

If you want, I can also give you:

1. a decision matrix for AWS vs Azure deployment tools, or
2. a reference architecture for multi-cloud model serving.