Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Yeah. [00:00:00] S1: The companies are all commenting on each other's posts, the research papers come out. There's a list of if on planet Earth there's 500 to 1000 people who really, really understand this and are at the cutting-edge of it, and a good number of them live in China, I just think you have to follow those people closely. And I think there is incredible signal. Everything in AI is just downstream- [00:00:00] S0: Of those people? [00:00:00] S1: Yeah. Everything Andrej Karpathy writes you have to read it three times, minimum. [00:00:00] S0: Yeah. He's incredible. [00:00:00] S1: I would say any time one of those labs, the four labs that matter, OpenAI, Gemini, Anthropic and xAI, which are, clearly, the four leading labs, any time somebody from one of those labs goes on a podcast, I just think it's so important to listen. For me, one of the best use cases of AI is to keep up with all of this. Listen to a podcast, and then if there are parts that I saw were interesting, just talk about it with AI. And I think it's really important to have as little friction as possible. I'll bring it up, "I can either press this button and pull up Grok, or I have this." [00:00:00] S0: Which is so amazing. Can you believe we have this? [00:00:00] S1: I know. [00:00:00] S0: It's freakish. [00:00:00] S1: I think somebody said on X like, "We imbued these rocks with crazy spells, and now we can summon super intelligent genies off our phones over the air." It's crazy.
· Gavin Baker, Patrick O'Shaughnessy
[00:00:00] S1: I think somebody said on X like, "We imbued these rocks with crazy spells, and now we can summon super intelligent genies off our phones over the air." It's crazy. [00:00:00] S0: Crazy. Okay. So, something like Gemini 3 comes out with the public interpretation was, "Oh, this is interesting. It seems to say something about scaling laws and the pre-training stuff." What is your frame on the state of general progress in frontier models in general? What are you watching most closely? [00:00:00] S1: Yeah. Well, I do think Gemini 3 was very important, because it showed us that scaling laws for pre-training are intact. They stated that unequivocally. And that's important, because no one on planet Earth knows how or why scaling laws for pre-training work. It's actually not a law. It's an empirical observation, and it's an empirical observations that we've measured extremely precisely and is held for a long time. But our understanding of scaling laws for pre-training, and maybe this is a little bit controversial with 20% of researchers, but probably not more than that, it's, like, the ancient British people's understanding of the sun. Are the ancient Egyptians understanding the sun? They can measure tit so precisely that the east/west axis of the Great Pyramids are perfectly aligned with the equinoxes, and so, are the east/west axis of Stonehenge. Perfect measurement. They didn't understand orbital mechanics. They had no idea how or why it rose in the east, set in the west, and moved across the horizon. [00:00:00] S0: Because the aliens.
· Gavin Baker
[00:00:00] S1: Yeah. Or God in a chariot. And so, it's really important every time we get a confirmation of that. So, Gemini 3 was very important in that way. But I'd say I think there's been a big misunderstanding of maybe in the public equity investing community, or the broader more generalist community, based on the scaling laws of pre-training, there really should have been no progress in '24 and '25. And the reason for that is after xAI figured out how to get 200,000 Hoppers coherent, you had to wait for the next generation of chips, because you really can't get more than 200,000 Hoppers coherent. And coherent just means you can just think of it as TPU knows what every other GPU is thinking. They are sharing memory. They're connected. They scale up networks, and scale out, and they have to be coherent during the pre-training process. The reason we've had all this progress, and maybe we could show, like, the arc AGI slide where you had zero to eight over four years, zero to 8% intelligence, and then you went from 8% to 95% in three months when the first reasoning model came out from OpenAI. We have these two new scaling laws of post-training, which is just reinforcement learning with verified rewards. And verified is such an important concept in AI. Like, one of Karpathy's great things was with software. Anything you can specify you can automate. With AI, anything you can verify you can automate. It's such an important concept, and I think important distinction. And then test-time compute. And so, all the progress we've had, and we've had immense progress since October '24 through today, was based entirely on these two new scaling laws. And Gemini 3 was arguably the first test since Hopper came out of the scaling law for pre-training, and it held, and that's great, because all these scaling laws are multiplicative. So, now we're going to apply these two new reinforcement learnings, verifiable rewards and test-time compute, to much better base models. There's a lot of misunderstanding about Gemini 3 that I think is really important. So, the most important thing to conceptualize, everything in AI has a struggle between Google and Nvidia. And Google has TPU, Nvidia has their GPUs and Google only has a TPU, and they use a bunch of other chips for networking. Nvidia has the full stack. Blackwell was delayed. Blackwell was Nvidia's next generation chip.
· Gavin Baker, Patrick O'Shaughnessy
The first iteration of that was the Blackwell 200. A lot of different SKUs were canceled. And the reason for that is it was, by far, the most complex product transition we've ever gone through in technology. Going from Hopper to Blackwell, first you go from air-cooled to liquid-cooled. The rack goes from weighing round numbers, 1000 pounds to 3000 pounds, goes from round numbers 30 kilowatts, which was 30 American homes, to 130 kilowatts, which is 130 American homes. I analogize it to imagine if to get a new iPhone, you had to change all the outlets in your house to 220 volt, put in a Tesla power wall, put in a generator, put in solar panels, that's the power, put in a whole home humidification system, and then reinforce the floor, because the floor can't handle this. So, it was a huge product transition. And then just the rack was so dense it was really hard for them to get the heat out. So, Blackwells have only really started to be deployed and really scaled deployments over the last three or four months. [00:00:00] S0: Can you explain why it has been such an important thing that Blackwell was delayed?
· Gavin Baker
[00:00:00] S1: Because Blackwell was so complicated, and it was so hard for everyone to get these exquisitely complex racks working consistently, had reasoning not come along, there would have been no AI progress from mid-2024 through, essentially, Gemini 3. There would have been none. Everything would have stalled. And can you imagine what that would have meant to the markets? For sure, we would have lived in a very different environment. So, Reasoning bridged this 18 month gap, Reasoning saved AI, because it let AI make progress without Blackwell, or the next generation of TPU, which were necessary for the scaling laws for pre-training to continue. Google came out with a TPU v6 in 2024, and the TPU v7 in 2025. In semiconductor time, imagine Hopper. It's, like, a World War II era airplane. And it was, by far, the best World War II era airplane. It's a P-51 Mustang with a Merlin engine. And two years later in semiconductor time, that's, like, you're an F-4 Phantom, okay? Because Blackwell was such a complicated product and so hard to ramp, Google was training Gemini 3 on '24 and '25 era TPUs, which are, like, F-4 Phantoms. Blackwell, it's, like an F-35. It just took a really long time to get it going. So, I think Google for sure has this temporary advantage right now from a pre-training perspective. I think it's also important that they've been the lowest cost producer of tokens. And this is really important, because AI is the first time in my career as a tech investor that being the low-cost producer has ever mattered. Apple is not worth trillions because they're a low-cost producer of phones. Microsoft is not worth trillions, because they're a low-cost producer of software. Nvidia is not worth trillions because they're the low-cost producer of AI accelerators. It's never mattered. And this is really important, because what Google has been doing as the low-cost producer is they have been sucking the economic oxygen out of the AI ecosystem, which is an extremely rational strategy for them. And for anyone who is a low-cost producer. Let's make life really hard for our competitors. So, what happens now? I think this has pretty profound implications. One, we will see the first models trained on Blackwell in early 2026.
· Gavin Baker
I think the first Blackwell model will come from xAI, and the reason for that is just according to Jensen, no one builds data centers faster than Elon. Jensen has said this on the record. And even once you have the Blackwells, it takes six to nine months to get them performing at the level of Hopper, because Hopper is finely tuned. Everybody knows how to use it. The software is perfect for it. The engineers know all its quirks. Everybody knows how to architect a Hopper data center at this point. And, by the way, when Hopper came out, it took six to 12 months for it to really out-perform Ampere, which was the generation before. So, if you're Jensen or Nvidia, you need to get as many GPUs deployed in one data center as fast as possible in a coherent cluster, so, you can work out the bugs. And so, this is what xAI, effectively, does for Nvidia, because they build the data centers the fastest, they can deploy Blackwells at scale the fastest, and they can help work with Nvidia to work out the bugs for everyone else. So, because they're the fastest, they'll have the first Blackwell model. We know that scaling laws for pre-training are intact, and this means the Blackwell models are going to be amazing. Blackwell is ... It's not an F-35 versus an F-4 Phantom, but from my perspective it is a better chip. Maybe it's, like, an F-35 versus a Rafale. And so, now that we know pre-scaling laws are holding we know that these Blackwell models are going to be really good. Based on the raw specs, they should probably be better. Then something even more important happens. So, the GB200 was really hard to get it going. The GB300 is a great chip. It is drop-in compatible in every way with those GB200 racks. Now you're not going to replace the GB200s.