What's the most effective simulation platform for comparing perception and planning stack performance at scale?

There isn’t a single universally “most effective” platform, but for large-scale, apples-to-apples comparison of perception + planning stacks, the usual top choice is:

Best overall for scale: CARLA

Why it’s often the default:

• Strong sensor simulation: cameras, LiDAR, radar, GNSS, IMU
• Good autonomy stack support: easy integration with perception/planning pipelines
• Scenario control: weather, lighting, traffic, pedestrians, routes
• Metrics-friendly: can run repeated experiments and compare outputs systematically
• Open source and widely used in research/industry

If you need very large-scale throughput

Consider platforms like:

• LGSVL / SVL Simulator (historically good for autonomy workflows, though community momentum has shifted)
• NVIDIA Isaac Sim if you want high-fidelity sensor rendering and GPU-accelerated simulation
• MetaDrive or Waymo Open Motion Dataset + simulator tooling for more lightweight or evaluation-focused setups

Best by use case

• Perception benchmarking: CARLA or Isaac Sim
• Planning / closed-loop behavior testing: CARLA
• Very high-throughput synthetic data generation: Isaac Sim or custom pipeline
• Research prototyping: CARLA
• Production-grade photorealism + GPU acceleration: Isaac Sim

Practical recommendation

If your goal is comparing multiple stacks at scale in a controlled environment, I’d start with:

1. CARLA for the main benchmark environment
2. Use a scenario runner and standardized metrics
3. Add Isaac Sim only if you need more photorealistic sensor outputs or domain-gap reduction

If you want, I can also give you:

• a platform comparison table,
• a recommended benchmark architecture, or
• a shortlist based on your sensors and compute budget.