Key points
- Luma's Moses, a Prime Video show with Ben Kingsley, is about 90% produced with Luma Agents.
- Jain defines a world model as a single tower jointly modeling language, audio, video, images and physical contact, and Luma's next video model will be unified.
- Amit Jain says foundation labs are the blueprint of future companies, fusing product and research.
Luma AI bets the world model is one unified tower
Luma AI is betting that visual AI does not end at video generation. A real world model would be one network that understands and simulates the physical world, and today's video models only understand video, so they cannot hold a character or follow an instruction however much they scale. Luma funds that research by selling finished creative production to advertisers and studios, where the budgets and the data are.
Takeaways
- A world model must simulate the physical world. Jain says nobody yet knows what a world model is, and being real-time or 3D does not qualify. He defines it as a single tower jointly modelling language, audio, video and physical contact. Today's video models understand only video, and scaling ten times would not fix that.
- Luma sells finished production, not tools. Luma Agents handles end-to-end multimodal creative work, and forward deployed creatives embed it inside customers. Coca-Cola spends about $2.5bn a year on content, and Moses, a Prime Video show with Ben Kingsley, was about 90% produced with Luma Agents.
- Unified models lose on deployment efficiency. Jain expects unified models to replace separate language models in theory but not in practice, since deployment favours efficiency. He cites Qwen 27B outperforming Qwen 289B. Luma fused an image model with a language tower in Uni One and will unify its next video model.
- Businesses, not consumers, buy generated video. After Sora's consumer video effort wound down, Luma chose not to build a generative social network. Jain says what makes a video interesting has little to do with whether it was generated, and businesses produce 99% of the pixels on our screens.
- Product and research are one activity. Starting a new product in a foundation lab costs about 1% of the balance sheet. When a model cannot do something, Luma runs a two or three week data collection job for the next training run rather than building a six to eight month harness.
- Data collection decides which labs win. Jain's rule is that in machine learning, if you do not think about data you are dead, the way distribution kills consumer products. Luma's first product was a free 3D capture app, released to collect 3D data, and it had commercial value too.
How the H100 unlocked video generation
- In late 2018 the K80 was the biggest deal in AI compute. V100s and then A100s followed.
- The H100, built on Hopper, was the first architecture designed for AI, for training and inference.
- It carried 96 GB of VRAM and 800 gigabits per second of interconnect.
- That bandwidth made moving images and video look possible, not just text and small amounts of 3D data.