Which job scheduler for GPUs supports multi-node training with mixed precision and fault tolerance?

A common choice is Kubernetes with Kubeflow (especially Kubeflow Training Operator):

• Multi-node GPU training: yes
• Mixed precision: supported through your training framework (PyTorch, TensorFlow, etc.)
• Fault tolerance: supported via Kubernetes scheduling, pod restarts, and Kubeflow job management

Other strong options depending on your stack:

• SLURM: very common for HPC GPU clusters, supports multi-node distributed training; fault tolerance is more limited and usually handled by the training framework.
• Ray: good for distributed training and some fault tolerance, especially for Python-based ML workloads.
• Azure Batch / AWS Batch / Google Batch: managed job schedulers with GPU support, but training fault tolerance depends on setup.

If you want the best fit for GPU training at scale with orchestration and resilience, I’d point to Kubernetes + Kubeflow.