Invest Like the Best with Patrick O'Shaughnessy
· Gavin Baker
[00:00:00] S1: I think this is good and healthy for the world. It's good for Jensen too, because a different administration might take a different view. Competition is good for everyone, and seeing different architectures explored is good. The reason is — in tank design, they talk about the Iron Triangle. The Iron Triangle of tank design is that all designers of a tank have to make trade-offs between attack, defense, and mobility for obvious reasons. The more defense you have — which is just armor — the heavier the tank is, the less mobile it is. So you have to live in this triangle and make trade-offs. The Merkava in Israel is optimized for defense. Russian tanks and tanks like the Leopard are generally more optimized for mobility. Chip design is the same. There are these fundamental constraints imposed by the laws of physics, as embedded in the Taiwan Semi design rules that you need to live with. You have TPU, Trainium, and AMD — which are all essentially trying to be a better GPU. Today, I think probably Trainium is doing the best. Now, nobody has a better GPU, but Trainium is tugging on Superman's cape. They haven't started yet. Trainium 3 needs to ramp into production because it has a switch scale-up network, which you really need to economically inference MOE models. A lot of companies have a torus architecture. That's where Google was; Google's developing a switch-scale network. AMD is always kind of flying a little bit behind. AMD, we'll see. The MI450, we don't know yet. We'll see. We probably know more about Trainium 3 than the MI450. But that's a hard game to play, so you have to do something different. And you have to do something different that is also hard to do. The best path for these startups — my rule of thumb is, 1% market share is going to be worth $100 billion. $100 billion is a pretty good venture outcome. I think what Jensen would say is, if somebody does something different and it gets to 1% or 2% or 3% share, we'll make that chip. That's coming for everyone. But if you're trying to make a better GPU, good luck. If you are doing something different, it also needs to be hard to do. And you can make different trade-offs. The disaggregation of prefill and inference really has opened the aperture for making these different trade-offs, because you can make very aggressive trade-offs for decode, and aggressive trade-offs for prefill.
· Patrick O'Shaughnessy
[00:00:00] S0: Prefill being taking in the context, decode being writing the output.
· Gavin Baker
[00:00:00] S1: Yeah. I have a great colleague named Andrew Fox who said, "Picture a British naval ship from the 18th century. Prefill is loading the cannon; decode is firing." What prefill literally is, it's just the model understanding the question, the prompt — and then keeping track of its own answer. That is fundamentally a memory-capacity-bound problem. Decode is the process of generating new tokens, and that is memory bandwidth constrained So if you're a chip designer, this gives you a richer canvas to paint on. But even so, it needs to be hard. If you make different trade-offs in that Iron Triangle to optimize for memory capacity — and they're not hard trade-offs to make — Nvidia is going to make those same trade-offs. They get better prices from Taiwan Semi than you're ever going to get. Good luck. And they have the advantage of working with every model company and optimizing their designs. By the way, another very funny thing — if you're a VC and you are investing in a semiconductor company that is telling you they are going to have an advantage because of a Taiwan Semi process they have special access to — I promise you that Jensen saw that process when it was a twinkle in Taiwan Semi's eyes. They know more about it than this little company with 200 people can imagine. Taiwan Semi, and everybody in the supply chain is showing Jensen everything. The same way they're showing Amazon everything, AMD everything, TPU everything. That's another reason — don't go try to make a better GPU. So you could do something different. You can paint in the prefill canvas, you can paint in the decode canvas. But you also have to do something hard, because if it gets to scale, you're going to have those 4 companies as very fast followers. My firm was a venture investor in Cerebras. What Cerebras has done is something hard and fundamentally different — wafer-scale computing. It comes with a set of trade-offs, but that architectural decision they made was hard and lets them do something that no one else can do. We'll find out how big that is. They're working on really cool things. One of the problems Cerebras has is, once you start needing to glue a lot of chips together and scale up networks or scale out networks, you need a lot of I/O, and I/O is bound by what's called the shoreline — the sides of the chip.
· Gavin Baker, Patrick O'Shaughnessy
Cerebras has an overwhelming ratio of on-chip compute and memory, relative to shoreline I/O. Well, they're really smart people. They did something really hard. They're trying to see if they can put an optical wafer right on top of that, and then that solves that problem. I'm sure they're looking at hybrid bonding of DRAM to get around the — much-discussed on X — alleged limitations that are not true. A Cerebras machine can theoretically run any size model. There are sizes of models where they're much better than other sizes. So Cerebras — what I think is interesting is they did something different that's hard to do, really hard to do — wafer-scale computing. I do think there's a role for these. I would just encourage them all to make a different trade-off, try and do something hard. Everybody's going to get funded after the Cerebras IPO. It's not going to be a problem. But it took Cerebras three generations of chips to get it right. Andrew Feldman, the CEO — you can just see how hard it was, what he did and that whole team did, to get where they are today. They need to have the grit to do that, the resilience. This first chip is a failure. It happens. Can you come back and make a second chip? But one last thing on this topic. This is going to be amazing for the useful lives of GPUs and may single-handedly save private credit. [00:00:00] S0: Say more about that. What do you mean by private credit?
· Gavin Baker
[00:00:00] S1: Well, private credit is in pain from these SaaS loans, and however much they're marked down, they probably need to be marked down more. If the public companies are struggling to adapt, how's a debt-laden company going to adapt and invest in what is a very different margin structure business? There's a lot of private credit in GPUs too, and they were underwriting that to, I think, 3 or 4 years. The disaggregation of inference means that these GPUs are going to have 10 or 15-year lives. The AI skeptics are like, "Oh, these companies are all cooking their books. The useful life of GPUs is only a year or two. The useful life of CPUs is only 4 years because of rapid technological change." No. What rapid technological change has done — with the disaggregation of prefill and inference — is mean that you can put a Cerebras system or Groq LPUs that Nvidia acquired, effectively in front of a Hopper or even an Ampere, use that Hopper and Ampere for prefill, and extend the useful life of that GPU until it melts. They do melt, so they have a time, but maybe you don't have to run them as fast. This is going to be really good for the whole private credit industry. It's going to help finance the AI buildout, because if you can start to finance GPUs at more like 5% or 6% instead of — I think CoreWeave's lowest financing was low 7s — that actually mathematically changes the cost to finance this buildout. We've had this technological innovation that's going to lower the cost of financing, extend the useful life of compute on Earth. One last thing that's interesting about that — my friend Jaimin from Coatue just did a podcast, and Coatue had a deck where they talked about, "Hey, sellers of shortage are doing so much better than the buyers of shortage" — buyers of shortage being the hyperscalers. But if you own a giant installed base of what is currently in shortage, that's also a very good place to be. We're hearing CPUs are way more important than they were in an agentic world. They do all these things around orchestration, tool calls. The biggest CPU fleets in the world sit at the hyperscalers. Some of these hyperscalers may catch up a little bit to the sellers of shortage.
· Patrick O'Shaughnessy
[00:00:00] S0: I want to talk about this idea of different and hard, applied outside of the infrastructure piece of this. Now you're starting to interact with new founders, existing CEOs and founders that have to adjust to this new world. What are you seeing from the most AI-native founders that aren't building chips or infrastructure or models, but just people using this technology to build other stuff? How do they feel the most different to you, if you've observed differences?