Invest Like the Best with Patrick O'Shaughnessy
· Dylan Patel
[00:00:00] S1: So this is the interesting thing. Everyone's like, okay, so Anthropic's just won, right? They had Mythos in February. They never even released it because they didn't feel the need to. They're already sold out. Their revenue is already adding $10 billion a month. And then you've got Opus 4.7 today, all before OpenAI's alleged Spud release, which media such as The Information and others have posted about. So clearly Anthropic is in the lead and OpenAI is cooked. What's interesting is because Anthropic has such bounds on compute, and they can only grow it so fast and to the point of – Dario used to gloat about how OpenAI was being too aggressive on compute and Anthropic was more sensible in their scaling. And now Anthropic is like, fuck, I wish we had a lot more compute. OpenAI is able to pay the bills perfectly fine. In fact, they've raised a ton of money to get incremental compute in addition to the irresponsible levels of compute that they were buying from Oracle and CoreWeave and SoftBank and all these people in Microsoft, such as Trainium. Now they're getting Trainium as well from Amazon. They've done this insane thing on compute. They also know they need more. But what's interesting is if you were to say Opus 4.6, let's ignore models getting better over time. Let's just take diffusion of this technology. You and I may jump on the model immediately day one, but other businesses take time and it takes time for people to learn. And the spark of, oh shit, quad psychosis moment doesn't hit everyone at the same time. And so by the end of the year, let's say a 4.6 Opus tier model, the economy would spend $100 billion on. I don't think that's unreasonable. It's spending $40 billion right now.
· Patrick O'Shaughnessy, Dylan Patel
[00:00:00] S0: That's like a linear extrapolation. [00:00:00] S1: It's a linear extrapolation, not an exponential. To get the exponential, you need the better models. Anthropic won't have enough compute to do that. And presumably OpenAI and Google will hit that tier soon enough. Whoever hits that tier next – sure, Anthropic may get to charge 70 plus percent gross margins. But if OpenAI hits it next, they charge 50% gross margins. They still get all of this incremental demand. And probably they also won't have enough compute to serve all the users. Sure, maybe Mythos is a model where if the world had enough compute, it'd be $500 billion of revenue or something crazy. There is such demand for these tokens and such limitations on compute. We see this with H100 prices skyrocketing and all these other things. The useful life of these GPUs continue to extend and extend. It's pretty clear even the tier 2 lab is going to be sold out of tokens, let alone the tier 1 lab. The tier 1 lab will have better margins, but the tier 2 lab will be sold out. And probably the tier 3 lab will also be close to sold out. The economic value that the best model can deliver is growing faster than our ability to actually serve those tokens to people via the infrastructure. And so this gap will continue to grow and the model labs will continue to have expanding margins. Until people in the hardware supply chain, infrastructure supply chain are like – wait, no, why don't I just jack up my margins?
· Patrick O'Shaughnessy
[00:00:00] S0: So suffice to say, I think the assessment today or your assessment of the demand side is completely explosive in your own particular example here at SemiAnalysis. But just more broadly that – you call it AI psychosis, as people fall into this experience of what they can do, the implementation difficulty going completely away. I've certainly felt that. My own token spend is just through the absolute roof, just in a matter of weeks. So that feels like a pretty good assessment. Anything we're missing on the demand side?
· Dylan Patel, Patrick O'Shaughnessy
[00:00:00] S1: If you don't use more tokens, you'll never escape the permanent underclass. Either you use more tokens and you generate economic value, outsized economic value for the use of those tokens. A lot of people are doing it the boring, lazy way. Oh, I guess I'll just work one hour a day instead of eight hours a day and I'll have AI do most of my job. That's the boring way. The cool way is I'll still work eight hours a day and I'll do 8x the work and maybe I'll make 5x the money. You can't do this with a job, obviously. There's people who have multiple jobs and there's people who start companies and start selling stuff. There's people who are hustling, which is what I view like you and I is doing is we're mostly hustling. Get that economic value on this AI before everyone is using it and it's table stakes, because it's still not table stakes. So if you don't use more tokens and generate the value from them and capture that value, there's three different problems here. Using more tokens, generating value from those tokens, and capturing value from the value that you created from the tokens. If you don't do these three things, you'll never escape the permanent underclass, i.e. as models continue to skyrocket and capability and the concentration of resources potentially happens. [00:00:00] S0: All right, let's talk about supply. What is changing at the frontier of supplying the entire stack that's required to serve all these tokens as the demand curve explodes?
· Patrick O'Shaughnessy, Dylan Patel
[00:00:00] S0: All right, let's talk about supply. What is changing at the frontier of supplying the entire stack that's required to serve all these tokens as the demand curve explodes? [00:00:00] S1: As demand skyrockets, prices are going up for everything on the supply side, whether it be the end GPUs, their prices are going up. In addition, their useful life is extending. [00:00:00] S0: H100 prices look like this.
· Dylan Patel
[00:00:00] S1: Yeah, exactly. There's people who have argued GPU useful lives are less than five years. Complete nonsense. There are clusters now re-signing. Three or four-year-old Hopper clusters re-signing for three or four more years. There's A100 clusters that are re-signing for another couple years. So the useful life is clearly not five years. It's maybe even seven or eight years, arguably. We don't know yet. We'll see when Hopper gets there, but it's clearly not five years. So useful life is extending and the prices are going up on that renewal. In effect, the gross margin was not 35% on a cluster. It's beyond that. So margins are expanding in the cloud layer. Margins are extremely healthy on the hardware layer with NVIDIA still charging 75 or whatever percent gross margin. As we move down the stack memory, obviously margins have skyrocketed there. Places like Optics and Logic, there are large prepayments and margins are growing slowly. More so, the companies that are making chips like NVIDIA are paying huge prepayments. So in effect, the cast of capital or timing of cash flows or return on invested capital is going up even if the gross margin isn't. And you see this across the whole supply chain. You see ASML is completely sold out and they need Carl Zeiss to expand faster. Everyone's either sold out and margins are going up or they're getting prepayments, which increases the return on invested capital because the invested capital is lower. And so this is a consistent trend across any part. It's even like – to make a PCB requires copper foil, and that copper foil is sold out and people are making prepayments for it. Anything and everything that has a pulse and is sold out, people are jumping to get more incremental supply and fighting over the supply for the years after.