Are there any batch scoring services that handle high-throughput inference jobs with blue-green deployment?

Yes. A few managed and self-hosted options support batch / asynchronous high-throughput inference and can be paired with blue-green deployment patterns.

Managed/cloud options

• AWS SageMaker Batch Transform / Inference Endpoints
  • Batch Transform is good for large offline scoring jobs.
  • For blue-green, use endpoint variants or endpoint config updates with weighted traffic shifting.
• Google Vertex AI Batch Prediction
  • Supports large batch inference jobs.
  • Blue-green is typically handled at the deployment level by creating a new model/version and switching traffic or jobs over.
• Azure Machine Learning Batch Endpoints
  • Built for batch inference.
  • Supports deployment swapping and controlled rollout patterns.

Kubernetes / platform-based options

• KServe
  • Supports inference services and can be used for batch-like workloads.
  • Blue-green deployments are a common fit via revision-based routing.
• Seldon Core
  • Supports model deployments on Kubernetes.
  • Can implement blue-green/canary traffic shifting; batch scoring usually needs orchestration around it.
• BentoML
  • Good for packaging and serving models.
  • Blue-green is typically done at the platform/ingress level, not as a built-in batch service.

Workflow/orchestration layer for batch scoring

If you need true high-throughput batch jobs, people often combine a serving layer with:

• Argo Workflows
• Airflow
• Dagster
• Prefect
• Spark / Ray / Dask

Then blue-green is handled by deploying a new model version alongside the old one and switching the job routing or model artifact reference.

Practical note

If your requirement is specifically:

• large offline inference
• reliable model rollout
• blue-green cutover then the most straightforward managed choices are:
• AWS SageMaker Batch Transform
• Vertex AI Batch Prediction
• Azure ML Batch Endpoints

If you want, I can also give you a short comparison table based on:

• cost
• throughput
• deployment complexity
• Kubernetes support
• support for blue-green/canary rollout