20VC: Who Wins the Inference Market | Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute | The Real Limits of Post-Training and RL with Erik Bernhardsson, Modal CEO

Why Modal's CEO thinks inference is the next cloud

Erik Bernhardsson argues inference is becoming a new cloud, big enough for many winners but brutal for anyone selling only tokens. Margins across the AI stack sit far below old software, so providers win by bundling many specialised workloads and securing GPUs ahead of demand. Modal, at $1bn in revenue after 15x growth, is betting its platform breadth and developer brand make it that bundle.

Takeaways

• Inference becomes a new cloud, beyond tokens. Selling tokens for dollars carries no switching cost and races to the bottom. Winners will bundle post-training, sandboxes, agentic tools and batch or low-latency inference. Bernhardsson expects a fragmented market of specialised models, with room for a $100bn company.
• AI has ended 90% software margins. Each layer of the AI stack earns roughly 20 to 30 percent. Bernhardsson thinks Modal can reach 40 to 60 percent long term. It is already profitable on a P&L basis, though heavy GPU spending keeps cash flow quite negative.
• GPU access depends on aggregation and forecasting. Modal buys capacity from 25 neocloud providers and lets thousands of customers pay per GPU second. With 30 to 40 percent monthly growth and three to four month lead times, it plans for demand two to three times higher within four months.
• Compute prices rise now, fall later. Buying capacity at Nvidia's GTC feels like an open air drug market, he says. He expects hardware prices to fall eventually and sees no overbuild within five years, though companies burning GPUs without revenue could trigger a pullback.
• Open source inference has not peaked. Jason Lemkin argues cheap frontier lab pricing will plateau open source inference. Bernhardsson rejects a fixed pie and expects both to grow five to twenty times. Modal under-invested until DeepSeek, then built a 30-person team and now ranks among the top providers.
• Post-training works where feedback loops are finite. Reinforcement learning succeeds in areas like coding and maths but may struggle with longer-horizon engineering work. He stays bullish on engineers because efficiency raises demand for software, and Modal spends about $500,000 a month on coding agents.

How Bernhardsson runs Modal

• Raise less money at lower valuations so investors stay hungry. Capital helps now, but he refuses to call it a moat.
• Early startups have no moats, so move faster than everyone. Brand and developer community are where he says Modal beats Baseten and Fireworks.
• Be patient with new products. Sandboxes took four years to succeed and now produce about $300m in run-rate revenue, a third of the total. He hopes the new training product reaches $100m.