How Close Are We to Robots That Can Do Anything? | Sergey Levine, Physical Intelligence

Physical Intelligence bets robot data will beat better models

Physical Intelligence is building general-purpose robot foundation models, and Sergey Levine argues the hard part is data and prior knowledge rather than model architecture. Robots need common sense from web-scale pretraining, but action data must be bootstrapped from real deployments and human interventions. Levine expects robotics to follow computing toward a horizontal physical intelligence layer that lets many different robots and applications be built.

Takeaways

• Robot data, not model design, is the bottleneck. Levine says the asymptotic picture is excellent because useful robots will be numerous and generate enormous data, but the field must bootstrap from human and intervention data first. The specific model choices matter less than that data trajectory.
• Foundation models supply missing prior knowledge. Grasping robots hit a wall because other tasks need knowledge that cannot be discovered from scratch. Foundation models supply that prior knowledge from web data, though the model is not grounded in the physical world.
• Deployment feedback loops solve the data problem. Human interventions fix mistakes and supply the right data, as when a robot drops an item at a logistics facility. Reinforcement learning then removes clumsiness rather than just teaching recovery.
• Generality beats humanoid-specific vertical stacks. Levine compares robotics to computing, where IBM's vertically integrated machines gave way to factorised layers for hardware and software. He expects an explosion of creativity with different robots and applications enabled by a horizontal physical intelligence layer.
• Hiring favors attention to detail over publications. The robotics stack is deep, and a failure can come from an unattached camera connector or a 16-billion-parameter model that needs 32bn. Physical Intelligence will hire researchers without a great publication record, though signalling is hard.
• Bigger problems are easier than narrow ones. Levine says making a hard intelligence problem as big as possible makes it easier, as machine translation improved by solving all NLP problems at once rather than building a dedicated translation model.

The model roadmap

1. Pi Zero took camera images and language as input and produced continuous actions, a vector of roughly 30 to 50 numbers. It was built on a pretrained LLM plus flow-matching action generation and was meant as an outline, not the ultimate model.
2. Pi 0.5 added an internal reasoning component. Told to clean a kitchen, it breaks the task into thoughts and grounds them into actions. It generalised to unseen kitchens and bedrooms and did longer-horizon, more dexterous work.