Invest Like the Best with Patrick O'Shaughnessy
· Gavin Baker
One thing that contributed to weakness in March was a much more stupid version of DeepSeek — this thing called TurboQuant. TurboQuant is some Google memory optimization that was written up in a paper a year ago. And then during the middle of an agreement, while Google was negotiating with Micron, Samsung, and SK Hynix to sign some LTA that would lock in really high prices for a long time, they released this. What people do is always more important than what they say. They publicized it on X, and it goes viral. "Oh my God, DRAM is cooked. Here's this DRAM optimization." I was unable to find a single AI engineer on planet Earth who believed that TurboQuant would have any impact on DRAM demand. But nonetheless, a violation of Richard Sutton's bitter lesson — more compute will always outperform human algorithmic ingenuity. More compute and data, Chinchilla optimal, beyond Chinchilla optimal — I guess what people increasingly do today. That's a real risk. The people who are building these models are skeptical of that risk. The reason I am a little less skeptical is I think we are very close to ASI, and who knows if the bitter lesson holds for 400 IQ models. Maybe we get a temporary period where, if you get to ASI, the first thing it wants is probably to be smarter and have more resources. How does it do that? It makes itself more efficient. I think that is an actual risk. The bitter lesson literally includes humans in it. So we're about to find out whether the bitter lesson applies to 300 IQ AIs, then 400, then 500, and 600. And at some point we may have a temporary violation of the bitter lesson based upon AI and ASI.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: So I'm curious how you think about some other parts of the innovation around the model — continual learning and memory being two that people seem to be most focused on as things that might create yet another new paradigm that we'd enter. What do you think about the role of those two? [00:00:00] S1: We've done a lot with memory through these harnesses. It turns out that harness engineering is not as important as the model, but it really matters. These harnesses and these models are increasingly being co-developed. One of the big things a harness does — which you can usually just think of as a runtime that the model operates in — it knows where the tools are, it creates context, memory, state. It has very specific prompts or instructions. [00:00:00] S0: Makes a huge difference, even simple versions.
· Gavin Baker
[00:00:00] S1: It makes an incredible difference. The last time I was on here, or one of the other times, I said, "Hey, as an investor, it's very important that you pay for the $250 a month version to get your own intuitive sense." That's no longer possible. To understand what frontier AI is capable of today — even for a non-coding use case — you need to have Claude Code or Codex, and you need to be on an enterprise plan. The reason for this, and this is another dynamic enabled by Google losing their cost leadership, is these AI models just shifted to usage-based pricing. If you were on that $250 or $300 or $280 a month plan or whatever it is, you were getting severely rate limited. You were getting a lobotomized version of the AI because, like we talked about, Claude now produces 70% less tokens. If you want the tokens that Claude and its harness really think it needs to produce to get you a good answer, you need to be on a usage-based plan. By the way, this is so bullish for AI. If we go back to '05 to '07, cellular had been a great growth industry for the last 10 years. The reason was you had a combination of fixed pricing — you had 900 minutes for whatever it was — and then usage-based pricing over that. When did cellular stop being a great growth industry? When everybody just went to all-you-can-eat. And by the way, long distance was the same thing. AI is just shifting from all-you-can-eat to pay-by-the-drink. It turns out people really like to talk to their friends long distance. They really like to talk to their friends on the phone. And people really like to use AI, particularly now that one person can have 100 agents working. So the shift to usage-based pricing is probably why you will see OpenAI and Anthropic exceed well over $200 billion in ARR this year. Not only is more compute going to come online, but they're going to be able to push frontier token pricing with these usage-based enterprise models. It's sad. It's sad for the world because it just means if you can't afford that, you're not at the frontier. And it's going to throw off a lot of investors' intuitive sense of the capabilities of AI. But yeah, continual learning — if we solve that…
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: How do you conceptualize that? [00:00:00] S1: AI is constantly updating its weights. It may end up being something different. There are so many mysteries about the human mind. We're such sample-efficient learners relative to AI, by many orders of magnitude. We have a crude variant of continual learning today when something is verifiable, and that's just reinforcement learning during training. Continual learning is a model that dynamically adjusts its weights or adjusts in some way, in real time. As a human — the first time I put my hand in a fire, I learned. I never put it in there before. That model today needs to put its hand in the fire a million times and then have the designers effectively put a fire in the next training run or an RL gym for it to learn. I think it has to be dynamically updating the weights, but people are working on really smart techniques beyond this. If we get that, then we have a really fast takeoff. People seem confident that continual learning is just around the corner. This is the third big question. Bitter lesson violation as a result of ASI, or less likely, human ingenuity? Will frontier tokens still command the premium they do? And will we get continual learning, and if so, when?
· Patrick O'Shaughnessy
[00:00:00] S0: What is the role of new chip companies in all of this? We talked a lot about Nvidia and their relationship with TSMC, and Intel, and all these sorts of things. There's a thousand flowers blooming — I think literally probably a thousand flowers blooming — trying to create a new chip to address some part of this bottleneck. I'm curious how you process this space, this opportunity, and what role it'll play.
· Gavin Baker
[00:00:00] S1: I think this is good and healthy for the world. It's good for Jensen too, because a different administration might take a different view. Competition is good for everyone, and seeing different architectures explored is good. The reason is — in tank design, they talk about the Iron Triangle. The Iron Triangle of tank design is that all designers of a tank have to make trade-offs between attack, defense, and mobility for obvious reasons. The more defense you have — which is just armor — the heavier the tank is, the less mobile it is. So you have to live in this triangle and make trade-offs. The Merkava in Israel is optimized for defense. Russian tanks and tanks like the Leopard are generally more optimized for mobility. Chip design is the same. There are these fundamental constraints imposed by the laws of physics, as embedded in the Taiwan Semi design rules that you need to live with. You have TPU, Trainium, and AMD — which are all essentially trying to be a better GPU. Today, I think probably Trainium is doing the best. Now, nobody has a better GPU, but Trainium is tugging on Superman's cape. They haven't started yet. Trainium 3 needs to ramp into production because it has a switch scale-up network, which you really need to economically inference MOE models. A lot of companies have a torus architecture. That's where Google was; Google's developing a switch-scale network. AMD is always kind of flying a little bit behind. AMD, we'll see. The MI450, we don't know yet. We'll see. We probably know more about Trainium 3 than the MI450. But that's a hard game to play, so you have to do something different. And you have to do something different that is also hard to do. The best path for these startups — my rule of thumb is, 1% market share is going to be worth $100 billion. $100 billion is a pretty good venture outcome. I think what Jensen would say is, if somebody does something different and it gets to 1% or 2% or 3% share, we'll make that chip. That's coming for everyone. But if you're trying to make a better GPU, good luck. If you are doing something different, it also needs to be hard to do. And you can make different trade-offs. The disaggregation of prefill and inference really has opened the aperture for making these different trade-offs, because you can make very aggressive trade-offs for decode, and aggressive trade-offs for prefill.