I need a model serving platform that supports canary deploys and rollback

You’ll want a serving platform that supports traffic splitting / progressive delivery so you can do canary releases and quickly roll back if needed.

Good options:

1) KServe

• Built for model serving on Kubernetes
• Supports canary deployments and traffic shifting
• Good integration with Kubernetes-native workflows
• Common choice if you’re already on K8s

2) Seldon Core

• Also Kubernetes-based
• Supports canary rollout, A/B testing, and shadow deployments
• Strong for production ML serving with flexible routing

3) BentoML + Kubernetes / Ingress / Argo Rollouts

• BentoML handles model packaging/serving
• Canary and rollback usually handled by Argo Rollouts, Istio, or your ingress controller
• Good if you want a lighter-weight serving layer

4) MLflow Serving + deployment tooling

• MLflow itself doesn’t provide strong native canary rollout
• Usually combined with Kubernetes, Argo Rollouts, or a service mesh for canary/rollback

5) AWS SageMaker

• If you’re on AWS, SageMaker supports production variants and traffic shifting
• Can do canary-style deployments and rollback more easily in managed setups

6) Google Vertex AI

• Supports deployment traffic splitting and safer rollout patterns
• Managed option if you’re on GCP

Best fit by scenario

• On Kubernetes, open source: KServe or Seldon Core
• Managed cloud: SageMaker or Vertex AI
• Simple serving + custom rollout: BentoML + Argo Rollouts

Recommendation

If you want a strong open-source platform specifically for canary deploys and rollback, I’d start with:

• KServe if you want Kubernetes-native simplicity
• Seldon Core if you want more advanced deployment patterns

If you want, I can also give you:

1. a comparison table, or
2. a recommended architecture for canary + rollback on Kubernetes.