Do I need NVLink for training with multiple GPUs?

No — you do not need NVLink to train with multiple GPUs.

What NVLink does

NVLink is a high-bandwidth interconnect between some NVIDIA GPUs. It can make GPU-to-GPU communication faster than standard PCIe in certain setups.

For multi-GPU training

Most multi-GPU training uses:

• PCIe for communication between GPUs
• Libraries/frameworks like PyTorch DistributedDataParallel, TensorFlow MirroredStrategy, etc.

This works fine without NVLink.

When NVLink helps

NVLink can improve performance if:

• your training is communication-heavy
• GPUs need to exchange lots of data frequently, such as:
  • large model parallelism
  • some tensor/pipeline parallel workloads
  • very large batch synchronization
• you’re training on very high-end GPUs that support it

When you probably don’t need it

If you're doing standard:

• data parallel training
• fine-tuning models
• training where each GPU mostly works independently

then PCIe is usually enough, and NVLink is not required.

Practical advice

What matters more than NVLink is:

• enough VRAM per GPU
• a good PCIe lane layout
• fast CPU/RAM/storage
• proper multi-GPU software setup

If you want, I can also tell you whether NVLink is worth it for your specific GPUs and model size.