Why Data Is the Real Secret to Scaling Physical AI | Varun Nair

Why Robotics Data Is the Real Bottleneck for Physical AI

Physical AI is advancing faster than the data pipelines that feed it. Varun Nair argues the robotics data market is a monopsony, with a few deep-pocketed buyers setting the requirements that data collection and annotation companies must meet. The winning approach is diverse, well-annotated data rather than sheer scale, and simulation and pseudo-labeling could shift where that data comes from. Near term, bimanual arms and teleoperation look more practical than humanoids, and patient capital is better placed to capture the deployment data that becomes post-training data.

Takeaways

• Robotics data is a monopsony. A handful of deep-pocketed buyers set the requirements that data companies and teleoperation providers must meet. Varun Nair says the market is still early, with a lot to build across collection, ingestion, annotation and quality control.
• Diversity beats scale for robotics data. Nair says the biggest metric is not just volume but diversity across tasks, scenes, embodiments and hardware. Researchers are not data scientists and nobody fully understands robotics data yet, so they extrapolate from LLM successes.
• Simulation could eat the data business. Nair predicts calibrated simulation could serve as training data, but good simulation needs both physics engines and world models. Nvidia swapped PhysX for Newton, and Vsim is building its own simulator. World models like World Labs' Atlas handle deformables but understand physics poorly.
• Pseudo-labeling turns small labeled sets into scale. OpenAI's 2022 VPT paper used 2,000 hours of labeled Minecraft data to pseudo-label 70,000 hours of gameplay. Nair recommends a 10,000-hour golden dataset to train an annotation model for modalities such as depth and motion capture, calling tactile the next frontier.
• Astra lacks depth because of its pretraining. Nair found Astra strong at spatial and geometric reasoning but poor at depth because it trained on computer-use data. Robocurve and the RoboDojo paper saw the same, and he predicts the next Astra and OpenAI robotics model will train on depth.
• Bimanual arms beat humanoids near term. Nair is unconvinced humanoids are the right immediate approach and favors bimanual arms. He names robotic welding at Persona and low-tolerance, high-precision tasks as the hardest deployments, and says teleoperation is the right way to start.
• Patient capital will capture robotics value. Nair says impatient capital, often from SaaS software investors, is following robotics, while patient capital will benefit. Deployment data becomes post-training data, and he expects multiple winners rather than one dominant company.

How a robotics dataset gets built