Should I use InfiniBand if I'm training across 8 GPUs?

Usually yes — if those 8 GPUs are in multiple servers or you care about fast distributed training.

Rule of thumb

• 8 GPUs in one machine (single node):
  • You typically do not need InfiniBand.
  • NVLink / NVSwitch (if available) and PCIe matter more.
• 8 GPUs across multiple machines (multi-node):
  • InfiniBand is often the best choice for performance and scaling.
  • It usually beats standard Ethernet for all-reduce / gradient exchange.

When InfiniBand is worth it

Use it if:

• You’re training large models with frequent gradient synchronization
• You need good scaling beyond one node
• Training time matters and you want lower network bottlenecks
• You’re running at cluster scale, especially with NCCL / PyTorch Distributed / DeepSpeed / FSDP

When Ethernet might be enough

Standard high-speed Ethernet may be fine if:

• Your jobs are small or lightly distributed
• You’re mostly doing inference or low-communication workloads
• Cost/complexity is more important than maximum throughput
• You can use 25/50/100/200 GbE and your model parallelism isn’t communication-heavy

Important distinction

If you mean 8 GPUs total in one box, the network isn’t the main issue:

• Look at PCIe lane layout
• Prefer NVLink/NVSwitch if the platform supports it
• Make sure your GPUs aren’t bottlenecked by a weak CPU/platform

If you mean 8 GPUs on 2+ nodes, then:

• InfiniBand is usually the safest recommendation
• Especially for large-scale deep learning

If you want, I can help you decide based on:

1. single-node vs multi-node
2. GPU model
3. model size / training framework
4. Ethernet speed vs InfiniBand speed