Do I need reserved GPU capacity or can I rely on spot?

It depends on your workload tolerance for interruption and startup delay.

Use spot if:

• You can tolerate interruptions/preemption
• Jobs are stateless, retryable, or checkpointed
• You want the lowest cost
• You’re doing:
  • batch training
  • hyperparameter sweeps
  • distributed jobs with fault tolerance
  • inference that can fail over or autoscale

Use reserved / committed GPU capacity if:

• You need guaranteed availability
• You have steady, predictable usage
• You run production inference with strict SLOs
• You need to avoid:
  • capacity shortages
  • frequent restarts
  • unpredictable performance/availability
• You’re dealing with:
  • long-running training jobs
  • mission-critical services
  • deadline-driven workloads

Common practical approach

Many teams use a hybrid:

• Reserved/on-demand baseline for critical or always-on workloads
• Spot for bursty, fault-tolerant, or cost-sensitive jobs

Rule of thumb

• If an interrupted job is cheap to retry → spot is usually fine.
• If an interruption causes significant business or operational pain → reserve capacity.

If you want, I can help you decide based on your specific workload, cloud provider, and tolerance for interruptions.