Invest Like the Best with Patrick O'Shaughnessy
00:25:17 · Gavin Baker, Patrick O'Shaughnessy
[00:25:17] S2: Yeah, I am very curious. A lot of people seem to feel Like they are very close to solving continual learning and sample efficient learning, which we've talked about before, and it is possible that if those are solved, that Could that be like a temporary discontinuity in demand if instead of I was trained on effectively twenty billion tokens, and then it's like these models are trained on three hundred trillion tokens? And if, you know, you can train something on ten trillion tokens and then let it out into the world and learn sample efficiently, that doesn't sound good for training demand, but like training has a percentage of semiconductor demand to compute is gonna asymptote to something not approaching Zero, but very small. But I would say that is the most interesting, and, you know, who knows if it's long horizon or short horizon. You know, SSI says that they're gonna come out with their model in August. There's this whole generation of new labs that are focused on this [00:26:20] S0: And this would be good for the world. [00:26:21] S2: This would be amazing for the world. To be clear, yeah, this would be awesome for the [00:26:24] S0: world. We all want, we want this. Yeah, we want this. It
00:26:24 · Patrick O'Shaughnessy, Gavin Baker
[00:26:24] S0: world. We all want, we want this. Yeah, we want this. It [00:26:26] S2: would be amazing for the world, and it's just, it's hard for me to believe that that would actually be negative for AI infrastructure demand. But again, trying to be really, really open-minded I would say that was probably like the biggest scientific or technical takeaway. You know, it's also-- Or you still know. Well, yeah, and also like Nvidia is heavily involved with All of these startups. [00:26:52] S0: If you were just forced to come up with the set of circumstances that would really switch you around and get you really scared, would it just be that this operating cash flow thing doesn't play out and therefore we just need to definance this whole thing? Yeah, if [00:27:03] S2: the operating cash flow does not continue to accelerate, that would be negative, and that to some degree is gonna be a function of how Anthropic, OpenAI, Grok, Cursor, and Open Source do. If there was a pretty dramatic contraction in GPU prices that was kind of sustained, the market would react to that instantly. That would be worrisome if it started to get to be really easy to get GPUs. I mean, have you heard anyone say they have too many GPUs? Like, not, not a single person. No, no, in fact, [00:27:38] S0: it's the opposite. It sounds like a drug market or something. Yeah, it really
00:27:40 · Gavin Baker
[00:27:40] S2: does. It's just wild. But yeah, I mean, I think there's a long list of pretty obvious things. If the sum of these labs plateaus or starts to decline, that's really negative, unless it's just because open source tokens are net growing the pie and taking share. And I do really think the future is multimodal, particularly for the AI natives, they're going to want to take an open source model, it's got all these inference clouds have gotten really good at supervised fine tuning and reinforcement learning, so you can take your data, custom- As an open source model, and then get something that you can put behind a router, and the router routes it to, often first, your model, and then Claude, Frontier model, whatever Claude Grok, checks it, and you can, in a lot of cases, get slightly better outcomes At half the cost. But again, that half the cost, I think a lot of people hear that, they're like, "That's bad for AI demand." It's actually not at all, because the cost the user pays is just a function of the margin on the tokens, and you're literally just shifting tokens from really expensive tokens with like ninety percent gross margins to tokens with maybe, let's call it a thirty percent gross margin. That's where the savings are coming from, but the tokens cost the same amount of compute To produce. And then also all these things are kind of happening on different cycle times. All these big public companies are like, "Oh my God, my AI spend is twenty X, I've burned my budget in three months," so they set up a router, and that actually cuts their AI spend, but it doesn't really impact, it may actually increase the amount of tokens that they are generating just by shifting them to these cheaper open source tokens, and that's just more compute. So a company getting smarter about which model to use for which task, that may lead to a stabilization in their spend or even a decline, but it actually has nothing to do with the amount of GPU compute hours they are effectively consuming behind these model layers of this router. The GPU compute hours probably are going up as you shift to these cheaper tokens you can use more of.
00:29:58 · Gavin Baker, Patrick O'Shaughnessy
That's happening to like a cutting edge of public cloud Companies, and then you have this whole wave of AI natives, they're leading it to this so hard, and they're not hiring humans, they're just putting it mostly into tokens, they're not slowing down. And then you have companies on the east coast of America who have like barely adopted AI, companies broadly speaking, you know, not in the coast, who maybe aren't as cutting, and then Europe, who's just trying to figure out how to regulate AI, you [00:30:30] S0: know, [00:30:31] S2: before [00:30:32] S0: using it. So just
00:30:33 · Gavin Baker
[00:30:33] S2: like there's kind of these differential waves of adoption all happening at the same time, but the thought I can't get out of my mind is like, I think I said it maybe last time, but just y'all access it or not, like five hundred thousand people in the world, two hundred fifty thousand maybe are using agentic AI, and we're in an acute compute shortage. There's seven or eight billion people on the planet. What happens when we go from five hundred thousand to one percent to a hundred million, you know, to five hundred million? It is interesting, you know, a lot of people, I do think it's like helpful to post on X to see the pushback and a lot of people are saying, "We accept your argument that hyperscalers are a dirty, get us compute reprices, their operating cash flows gonna accelerate, and maybe we can fund this, but like, where's that operating cash flow gonna come from? Where is the customer?" And kind of definitionally, it has to either come from faster economic growth through productivity, kind of Satya's comments, like "Either we're gonna start growing ten percent or we're not." Or labor substitution. And for sure, I think in a lot of these AI natives, you're seeing labor substitution, but not because they're firing people, they're just not hiring nearly as many humans. The gross profit dollars per FTE in A16Z, ICANN, a bunch of companies that have done this work, they're vertical, particularly relative to past generations of startups. And then it is interesting, are you doing any surveys of your companies and their token spend relative to labor spend?
00:32:05 · Patrick O'Shaughnessy, Gavin Baker
[00:32:05] S0: Oh yeah, I mean, it's tokens as a percent of total comp spend or something like that. And what, what are the ranges you've seen? I mean, like in the really pilled companies, like it gets really high, twenty percent, twenty-five percent. [00:32:17] S2: Our friend Nile Patel at his, at his company, he's an ASI backseat, but he's at thirty percent. [00:32:23] S0: That's probably the [00:32:24] S2: highest one I've heard. I've actually heard a fifty, and there's twenty-five trillion dollars in knowledge work Let's take your twenty percent number, that's five trillion, and that either comes out of labor substitution or faster economic growth, and we really, really, really want as humans to come from faster economic growth. [00:32:45] S0: One interesting thing I heard this morning from one of the great leading technology CEOs that's founded several companies, if you look at the founder-led and controlled companies and adjust for some of the like COVID-era over-hiring, nobody's really laying people off. These are the people that would probably be most quick to adopt AI to become more efficient or whatever, like they're not really doing jack-assed like huge-scale layoffs, which probably tells you something about where they think there will be lots of opportunity to still have people plus--