Can you recommend a cluster orchestration platform for scheduling workloads across multi-cloud GPU clusters?

Yes — for scheduling workloads across multi-cloud GPU clusters, my top recommendation is usually Kubernetes with a GPU-aware batch/orchestration layer on top.

Best overall choice: Kubernetes + Kueue or Volcano

• Kubernetes gives you the broadest multi-cloud support across AWS, GCP, Azure, and on-prem.
• For GPU batch scheduling:
  • Kueue is strong if you want queueing, quota-based admission, and integration with Kubeflow/Ray/Jobs.
  • Volcano is a good fit for batch-heavy, gang-scheduled, ML/HPC-style workloads.

Why this is a good fit

• Works across multiple cloud providers
• Mature ecosystem and tooling
• Supports GPU resource requests/limits
• Can integrate with autoscaling and cluster federation patterns
• Flexible enough for training, inference, and mixed workloads

If your workload is primarily ML training

Consider:

• Kubeflow + Kueue if you want an ML platform experience
• Ray + KubeRay if you want distributed Python workloads and training jobs
• Volcano if gang scheduling and HPC-style job placement are important

If you want “one control plane over many clusters”

Look at:

• Rancher for multi-cluster management
• Karmada for Kubernetes federation-style scheduling across clusters

When not to use Kubernetes alone

If you need:

• strict multi-cluster placement policies,
• cross-cloud failover,
• or sophisticated GPU bin-packing/queueing,

then you’ll likely want Kubernetes plus a higher-level scheduler rather than plain kube-scheduler alone.

Practical recommendation

If I had to choose one stack:

• Kubernetes on each cloud
• Kueue for queueing and quota
• Cluster autoscaling per cloud
• Rancher or Karmada if you need centralized multi-cluster management

If you want, I can also give you a recommendation based on your exact use case:

• ML training
• GPU inference
• batch/HPC
• multi-tenant platform
• fastest to operate
• best open-source option