Invest Like the Best with Patrick O'Shaughnessy
· Gavin Baker, Patrick O'Shaughnessy
[00:00:00] S1: I think the first thing is you have to use it yourself. And I would just say I'm amazed at how many famous and august investors are reaching really definitive conclusions about AI based on the free tier. The free tier is, like, you're dealing with a 10 year old. [00:00:00] S0: Right. [00:00:00] S1: And you're making conclusions about the 10 year old's capabilities as an adult, and you could just pay, and I do think actually you do need to pay for the highest tier, whether it's Gemini Ultra, Super Grok, whatever it is, you have to pay the $200 per month tiers, whereas those are, like, a fully-fledged, 30, 35 year old. It's really hard to extrapolate from an eight or a 10 year old to the 35 year old, and yet, a lot of people are doing that. The second thing is there was an insider post about OpenAI, and they said to a large degree, "OpenAI runs on Twitter vibes." And I just think AI happens on X. There have been some really memorable moments. Like, there was a giant fight between the PyTorch team at Meta and the JAX team at Google, on X. And the leaders of each lab had to step in- [00:00:00] S0: Right. And say, "Stop." [00:00:00] S1: ... publicly say, "No one from my lab is allowed to say bad things about the other lab, and I respect them, and that is the end of that." [00:00:00] S0: Yeah.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Yeah. [00:00:00] S1: The companies are all commenting on each other's posts, the research papers come out. There's a list of if on planet Earth there's 500 to 1000 people who really, really understand this and are at the cutting-edge of it, and a good number of them live in China, I just think you have to follow those people closely. And I think there is incredible signal. Everything in AI is just downstream- [00:00:00] S0: Of those people? [00:00:00] S1: Yeah. Everything Andrej Karpathy writes you have to read it three times, minimum. [00:00:00] S0: Yeah. He's incredible. [00:00:00] S1: I would say any time one of those labs, the four labs that matter, OpenAI, Gemini, Anthropic and xAI, which are, clearly, the four leading labs, any time somebody from one of those labs goes on a podcast, I just think it's so important to listen. For me, one of the best use cases of AI is to keep up with all of this. Listen to a podcast, and then if there are parts that I saw were interesting, just talk about it with AI. And I think it's really important to have as little friction as possible. I'll bring it up, "I can either press this button and pull up Grok, or I have this." [00:00:00] S0: Which is so amazing. Can you believe we have this? [00:00:00] S1: I know. [00:00:00] S0: It's freakish. [00:00:00] S1: I think somebody said on X like, "We imbued these rocks with crazy spells, and now we can summon super intelligent genies off our phones over the air." It's crazy.
· Gavin Baker, Patrick O'Shaughnessy
[00:00:00] S1: I think somebody said on X like, "We imbued these rocks with crazy spells, and now we can summon super intelligent genies off our phones over the air." It's crazy. [00:00:00] S0: Crazy. Okay. So, something like Gemini 3 comes out with the public interpretation was, "Oh, this is interesting. It seems to say something about scaling laws and the pre-training stuff." What is your frame on the state of general progress in frontier models in general? What are you watching most closely? [00:00:00] S1: Yeah. Well, I do think Gemini 3 was very important, because it showed us that scaling laws for pre-training are intact. They stated that unequivocally. And that's important, because no one on planet Earth knows how or why scaling laws for pre-training work. It's actually not a law. It's an empirical observation, and it's an empirical observations that we've measured extremely precisely and is held for a long time. But our understanding of scaling laws for pre-training, and maybe this is a little bit controversial with 20% of researchers, but probably not more than that, it's, like, the ancient British people's understanding of the sun. Are the ancient Egyptians understanding the sun? They can measure tit so precisely that the east/west axis of the Great Pyramids are perfectly aligned with the equinoxes, and so, are the east/west axis of Stonehenge. Perfect measurement. They didn't understand orbital mechanics. They had no idea how or why it rose in the east, set in the west, and moved across the horizon. [00:00:00] S0: Because the aliens.
· Gavin Baker
[00:00:00] S1: Yeah. Or God in a chariot. And so, it's really important every time we get a confirmation of that. So, Gemini 3 was very important in that way. But I'd say I think there's been a big misunderstanding of maybe in the public equity investing community, or the broader more generalist community, based on the scaling laws of pre-training, there really should have been no progress in '24 and '25. And the reason for that is after xAI figured out how to get 200,000 Hoppers coherent, you had to wait for the next generation of chips, because you really can't get more than 200,000 Hoppers coherent. And coherent just means you can just think of it as TPU knows what every other GPU is thinking. They are sharing memory. They're connected. They scale up networks, and scale out, and they have to be coherent during the pre-training process. The reason we've had all this progress, and maybe we could show, like, the arc AGI slide where you had zero to eight over four years, zero to 8% intelligence, and then you went from 8% to 95% in three months when the first reasoning model came out from OpenAI. We have these two new scaling laws of post-training, which is just reinforcement learning with verified rewards. And verified is such an important concept in AI. Like, one of Karpathy's great things was with software. Anything you can specify you can automate. With AI, anything you can verify you can automate. It's such an important concept, and I think important distinction. And then test-time compute. And so, all the progress we've had, and we've had immense progress since October '24 through today, was based entirely on these two new scaling laws. And Gemini 3 was arguably the first test since Hopper came out of the scaling law for pre-training, and it held, and that's great, because all these scaling laws are multiplicative. So, now we're going to apply these two new reinforcement learnings, verifiable rewards and test-time compute, to much better base models. There's a lot of misunderstanding about Gemini 3 that I think is really important. So, the most important thing to conceptualize, everything in AI has a struggle between Google and Nvidia. And Google has TPU, Nvidia has their GPUs and Google only has a TPU, and they use a bunch of other chips for networking. Nvidia has the full stack. Blackwell was delayed. Blackwell was Nvidia's next generation chip.
· Gavin Baker, Patrick O'Shaughnessy
The first iteration of that was the Blackwell 200. A lot of different SKUs were canceled. And the reason for that is it was, by far, the most complex product transition we've ever gone through in technology. Going from Hopper to Blackwell, first you go from air-cooled to liquid-cooled. The rack goes from weighing round numbers, 1000 pounds to 3000 pounds, goes from round numbers 30 kilowatts, which was 30 American homes, to 130 kilowatts, which is 130 American homes. I analogize it to imagine if to get a new iPhone, you had to change all the outlets in your house to 220 volt, put in a Tesla power wall, put in a generator, put in solar panels, that's the power, put in a whole home humidification system, and then reinforce the floor, because the floor can't handle this. So, it was a huge product transition. And then just the rack was so dense it was really hard for them to get the heat out. So, Blackwells have only really started to be deployed and really scaled deployments over the last three or four months. [00:00:00] S0: Can you explain why it has been such an important thing that Blackwell was delayed?
· Gavin Baker
[00:00:00] S1: Because Blackwell was so complicated, and it was so hard for everyone to get these exquisitely complex racks working consistently, had reasoning not come along, there would have been no AI progress from mid-2024 through, essentially, Gemini 3. There would have been none. Everything would have stalled. And can you imagine what that would have meant to the markets? For sure, we would have lived in a very different environment. So, Reasoning bridged this 18 month gap, Reasoning saved AI, because it let AI make progress without Blackwell, or the next generation of TPU, which were necessary for the scaling laws for pre-training to continue. Google came out with a TPU v6 in 2024, and the TPU v7 in 2025. In semiconductor time, imagine Hopper. It's, like, a World War II era airplane. And it was, by far, the best World War II era airplane. It's a P-51 Mustang with a Merlin engine. And two years later in semiconductor time, that's, like, you're an F-4 Phantom, okay? Because Blackwell was such a complicated product and so hard to ramp, Google was training Gemini 3 on '24 and '25 era TPUs, which are, like, F-4 Phantoms. Blackwell, it's, like an F-35. It just took a really long time to get it going. So, I think Google for sure has this temporary advantage right now from a pre-training perspective. I think it's also important that they've been the lowest cost producer of tokens. And this is really important, because AI is the first time in my career as a tech investor that being the low-cost producer has ever mattered. Apple is not worth trillions because they're a low-cost producer of phones. Microsoft is not worth trillions, because they're a low-cost producer of software. Nvidia is not worth trillions because they're the low-cost producer of AI accelerators. It's never mattered. And this is really important, because what Google has been doing as the low-cost producer is they have been sucking the economic oxygen out of the AI ecosystem, which is an extremely rational strategy for them. And for anyone who is a low-cost producer. Let's make life really hard for our competitors. So, what happens now? I think this has pretty profound implications. One, we will see the first models trained on Blackwell in early 2026.