Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy
[00:00:00] S0: I will never forget when I first met Gavin Baker in 2017. I find his interest in markets, his curiosity about the world to be as infectious as any investor that I've ever come across. He is encyclopedic on what is going on in the world of technology today, and I've had the good fortune to host him every year or two on this podcast. In this conversation, we talk about everything that interests Gavin. We talk about Nvidia, Google and its TPUs, the changing AI landscape, the math and business models around AI companies, and everything in-between. We even discuss the crazy idea of data centers in space, which he communicates with his usual passion and logic. In closing, at the end of this conversation, because I've asked him my traditional closing question before, I asked him a different question, which led to a discussion of his entire investing origin story that I had never heard before. Because Gavin is one of the most passionate thinkers and investors that I know, these conversations are always amongst my most favorite. I hope you enjoy this latest in the series of discussions with Gavin Baker. I would love to talk about how you, like, in the nitty-gritty process, new things that come out in this AI world, because it's happening so constantly. I'm extremely interested in it, and I find it very hard to keep up, and I have a couple blogs that I go read and friends that I call. But maybe let's take Gemini 3 as a recent example. When that comes out, take me into your office. Like, what are you doing? How do you and your team process an update like that? Given how often these things are happening.
· Gavin Baker, Patrick O'Shaughnessy
[00:00:00] S1: I think the first thing is you have to use it yourself. And I would just say I'm amazed at how many famous and august investors are reaching really definitive conclusions about AI based on the free tier. The free tier is, like, you're dealing with a 10 year old. [00:00:00] S0: Right. [00:00:00] S1: And you're making conclusions about the 10 year old's capabilities as an adult, and you could just pay, and I do think actually you do need to pay for the highest tier, whether it's Gemini Ultra, Super Grok, whatever it is, you have to pay the $200 per month tiers, whereas those are, like, a fully-fledged, 30, 35 year old. It's really hard to extrapolate from an eight or a 10 year old to the 35 year old, and yet, a lot of people are doing that. The second thing is there was an insider post about OpenAI, and they said to a large degree, "OpenAI runs on Twitter vibes." And I just think AI happens on X. There have been some really memorable moments. Like, there was a giant fight between the PyTorch team at Meta and the JAX team at Google, on X. And the leaders of each lab had to step in- [00:00:00] S0: Right. And say, "Stop." [00:00:00] S1: ... publicly say, "No one from my lab is allowed to say bad things about the other lab, and I respect them, and that is the end of that." [00:00:00] S0: Yeah.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Yeah. [00:00:00] S1: The companies are all commenting on each other's posts, the research papers come out. There's a list of if on planet Earth there's 500 to 1000 people who really, really understand this and are at the cutting-edge of it, and a good number of them live in China, I just think you have to follow those people closely. And I think there is incredible signal. Everything in AI is just downstream- [00:00:00] S0: Of those people? [00:00:00] S1: Yeah. Everything Andrej Karpathy writes you have to read it three times, minimum. [00:00:00] S0: Yeah. He's incredible. [00:00:00] S1: I would say any time one of those labs, the four labs that matter, OpenAI, Gemini, Anthropic and xAI, which are, clearly, the four leading labs, any time somebody from one of those labs goes on a podcast, I just think it's so important to listen. For me, one of the best use cases of AI is to keep up with all of this. Listen to a podcast, and then if there are parts that I saw were interesting, just talk about it with AI. And I think it's really important to have as little friction as possible. I'll bring it up, "I can either press this button and pull up Grok, or I have this." [00:00:00] S0: Which is so amazing. Can you believe we have this? [00:00:00] S1: I know. [00:00:00] S0: It's freakish. [00:00:00] S1: I think somebody said on X like, "We imbued these rocks with crazy spells, and now we can summon super intelligent genies off our phones over the air." It's crazy.
· Gavin Baker, Patrick O'Shaughnessy
[00:00:00] S1: I think somebody said on X like, "We imbued these rocks with crazy spells, and now we can summon super intelligent genies off our phones over the air." It's crazy. [00:00:00] S0: Crazy. Okay. So, something like Gemini 3 comes out with the public interpretation was, "Oh, this is interesting. It seems to say something about scaling laws and the pre-training stuff." What is your frame on the state of general progress in frontier models in general? What are you watching most closely? [00:00:00] S1: Yeah. Well, I do think Gemini 3 was very important, because it showed us that scaling laws for pre-training are intact. They stated that unequivocally. And that's important, because no one on planet Earth knows how or why scaling laws for pre-training work. It's actually not a law. It's an empirical observation, and it's an empirical observations that we've measured extremely precisely and is held for a long time. But our understanding of scaling laws for pre-training, and maybe this is a little bit controversial with 20% of researchers, but probably not more than that, it's, like, the ancient British people's understanding of the sun. Are the ancient Egyptians understanding the sun? They can measure tit so precisely that the east/west axis of the Great Pyramids are perfectly aligned with the equinoxes, and so, are the east/west axis of Stonehenge. Perfect measurement. They didn't understand orbital mechanics. They had no idea how or why it rose in the east, set in the west, and moved across the horizon. [00:00:00] S0: Because the aliens.
· Gavin Baker
[00:00:00] S1: Yeah. Or God in a chariot. And so, it's really important every time we get a confirmation of that. So, Gemini 3 was very important in that way. But I'd say I think there's been a big misunderstanding of maybe in the public equity investing community, or the broader more generalist community, based on the scaling laws of pre-training, there really should have been no progress in '24 and '25. And the reason for that is after xAI figured out how to get 200,000 Hoppers coherent, you had to wait for the next generation of chips, because you really can't get more than 200,000 Hoppers coherent. And coherent just means you can just think of it as TPU knows what every other GPU is thinking. They are sharing memory. They're connected. They scale up networks, and scale out, and they have to be coherent during the pre-training process. The reason we've had all this progress, and maybe we could show, like, the arc AGI slide where you had zero to eight over four years, zero to 8% intelligence, and then you went from 8% to 95% in three months when the first reasoning model came out from OpenAI. We have these two new scaling laws of post-training, which is just reinforcement learning with verified rewards. And verified is such an important concept in AI. Like, one of Karpathy's great things was with software. Anything you can specify you can automate. With AI, anything you can verify you can automate. It's such an important concept, and I think important distinction. And then test-time compute. And so, all the progress we've had, and we've had immense progress since October '24 through today, was based entirely on these two new scaling laws. And Gemini 3 was arguably the first test since Hopper came out of the scaling law for pre-training, and it held, and that's great, because all these scaling laws are multiplicative. So, now we're going to apply these two new reinforcement learnings, verifiable rewards and test-time compute, to much better base models. There's a lot of misunderstanding about Gemini 3 that I think is really important. So, the most important thing to conceptualize, everything in AI has a struggle between Google and Nvidia. And Google has TPU, Nvidia has their GPUs and Google only has a TPU, and they use a bunch of other chips for networking. Nvidia has the full stack. Blackwell was delayed. Blackwell was Nvidia's next generation chip.
· Gavin Baker, Patrick O'Shaughnessy
The first iteration of that was the Blackwell 200. A lot of different SKUs were canceled. And the reason for that is it was, by far, the most complex product transition we've ever gone through in technology. Going from Hopper to Blackwell, first you go from air-cooled to liquid-cooled. The rack goes from weighing round numbers, 1000 pounds to 3000 pounds, goes from round numbers 30 kilowatts, which was 30 American homes, to 130 kilowatts, which is 130 American homes. I analogize it to imagine if to get a new iPhone, you had to change all the outlets in your house to 220 volt, put in a Tesla power wall, put in a generator, put in solar panels, that's the power, put in a whole home humidification system, and then reinforce the floor, because the floor can't handle this. So, it was a huge product transition. And then just the rack was so dense it was really hard for them to get the heat out. So, Blackwells have only really started to be deployed and really scaled deployments over the last three or four months. [00:00:00] S0: Can you explain why it has been such an important thing that Blackwell was delayed?