How Lightwheel is Building the Simulation Infrastructure of Physical AI with Steve Xie

Lightwheel Builds the Simulation Infrastructure Physical AI Runs On

Lightwheel is building the simulation infrastructure physical AI runs on, betting that calibrated physics and generated data, not graphics, decide whether a robot policy survives the real world. Real teleoperation data covers a fraction of what models need, so the company mass-produces physics-accurate assets and simulation data, then tests models on benchmarks built with factories and hospitals. Xie, who ran simulation at Cruise and then NVIDIA, wants an AI-native agent at the core.

Takeaways

• Simulation is becoming physical AI infrastructure. Xie ran simulation at Cruise from 2018, when it was treated almost as a toy for investors, then led it for autonomous driving at NVIDIA. Seeing Omniverse under Ralph convinced him simulation is next-generation infrastructure for physical AI, not just for self-driving.
• Simulation data supply is scaling tenfold. Real teleoperation data covers 1% or less of what models need. Lightwheel delivered close to 1m hours in the second half of last year, more than 2m already this year, and expects over 10m, with over 1m hours of its EgoSuite egocentric video shipped.
• Physics calibration separates SimReady Assets from game assets. A fridge asset needs its magnetic force and hinges calibrated so the physics solver produces accurate forces, unlike the $9.9 assets sold online. Lightwheel builds SimReady Assets at 99% automation using 3D reconstruction and the USD open standard.
• Industry benchmarks are replacing saturated academic ones. Frontier models pass BEHAVIOR and RoboCasa at 99 to 100%, while industry customers hold tens of thousands of GPUs against academia's 10,000 hours. Real-world evaluation cannot scale, so Lightwheel built RoboFinals with manufacturing, agriculture, hospital and home partners and gives the evaluation infrastructure away free.
• World models sit throughout Lightwheel's pipeline. Marble digitises real scenes into USD for RoboFinals, Cosmos augments auto-generated behaviour data, and Genie 3-style models generate in real time. Stephens says teams make 100 kitchens a day where a 3D artist needs a month in Isaac Sim.
• Robot data collection could become mass participation. Xie predicts 100m to 1bn people could be collecting data for robotics within five years and calls robotics 1,000 times harder than self-driving. He wants an AI-native agent running the loop between real and simulated data.

From Cruise to Lightwheel

1. Xie led the simulation department at Cruise from 2018, measuring how realistic simulation was and improving it with generative AI, at a time when simulation was treated almost as a toy for investors.