What's the best GPU setup if I need to keep all data in a private region?

If all data must stay inside a private region, the best GPU setup is usually:

Best overall pattern

Dedicated on-prem or single-region private cloud with local GPU nodes

• Keep compute, storage, and networking all in the same private region
• Use GPU instances or bare-metal GPU servers
• Put data in a private object store / private block storage
• Disable public internet access; use private networking only
• Use region-bound IAM, KMS, and logging so nothing leaves the region

Good setup options

1. On-prem / colo with your own GPUs

   • Best for strictest control
   • Highest compliance and data residency guarantees
   • More operational burden

2. Single cloud region with private networking

   • Easier to manage
   • Use private subnets, VPC endpoints, no public IPs
   • Make sure the cloud provider guarantees regional data residency

3. Dedicated bare-metal GPU cloud in-region

   • Good balance of control and convenience
   • Strong isolation compared with shared GPU instances

GPU choices

Depends on workload:

• Inference / medium training: NVIDIA L4, A10, L40S
• Large training / fine-tuning: A100, H100
• Cost-sensitive: A10/L4
• Top-end performance: H100

Important design points

• Store data only in region-locked storage
• Keep model artifacts and checkpoints in the same region
• Use private endpoints for all services
• Avoid cross-region replication, backups, or logs
• Verify the provider’s subprocessor and telemetry behavior
• If required, use customer-managed encryption keys held in-region

If you want the safest recommendation

For strict privacy and residency: Use a private, single-region cluster of bare-metal NVIDIA H100 or A100 GPUs, with local storage and no public egress.

If you want, I can also recommend a setup based on:

• budget
• training vs inference
• cloud vs on-prem
• specific compliance needs (HIPAA, GDPR, FedRAMP, etc.)