What GPU setup should I use if I need 80GB memory per card?

If you need 80 GB of GPU memory per card, you’ll want GPUs that are sold with 80 GB VRAM rather than trying to combine smaller cards. The usual options are:

Best-known 80 GB GPUs

• NVIDIA A100 80GB
  Common in servers/datacenters; excellent for training and large inference.
• NVIDIA H100 80GB
  Faster than A100; ideal if budget is less of a concern and you want top performance.
• NVIDIA H200 141GB
  If “at least 80 GB” is fine and you want more memory headroom, this is a newer option.

If you need multiple GPUs

Use a setup with:

• Matching 80 GB cards
• A motherboard / server with enough PCIe lanes
• Proper power and cooling
• Ideally NVLink / NVSwitch support if your workload benefits from fast GPU-to-GPU communication

Good practical setups

• 1× A100 80GB or 1× H100 80GB for a single-GPU system
• 2× or 4× A100 80GB / H100 80GB in a workstation/server if you need more total VRAM or parallelism
• DGX-style server if you want an integrated high-end multi-GPU platform

Important note

If your requirement is 80 GB per GPU, consumer cards like RTX 4090 won’t fit that requirement, even if you use multiple cards. Memory is not pooled automatically for a single model unless your software supports model parallelism.

If you want, I can recommend a setup based on:

1. budget,
2. training vs inference, and
3. how many GPUs you need.