Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy
[00:00:00] S0: Prefill being taking in the context, decode being writing the output.
· Gavin Baker
[00:00:00] S1: Yeah. I have a great colleague named Andrew Fox who said, "Picture a British naval ship from the 18th century. Prefill is loading the cannon; decode is firing." What prefill literally is, it's just the model understanding the question, the prompt — and then keeping track of its own answer. That is fundamentally a memory-capacity-bound problem. Decode is the process of generating new tokens, and that is memory bandwidth constrained So if you're a chip designer, this gives you a richer canvas to paint on. But even so, it needs to be hard. If you make different trade-offs in that Iron Triangle to optimize for memory capacity — and they're not hard trade-offs to make — Nvidia is going to make those same trade-offs. They get better prices from Taiwan Semi than you're ever going to get. Good luck. And they have the advantage of working with every model company and optimizing their designs. By the way, another very funny thing — if you're a VC and you are investing in a semiconductor company that is telling you they are going to have an advantage because of a Taiwan Semi process they have special access to — I promise you that Jensen saw that process when it was a twinkle in Taiwan Semi's eyes. They know more about it than this little company with 200 people can imagine. Taiwan Semi, and everybody in the supply chain is showing Jensen everything. The same way they're showing Amazon everything, AMD everything, TPU everything. That's another reason — don't go try to make a better GPU. So you could do something different. You can paint in the prefill canvas, you can paint in the decode canvas. But you also have to do something hard, because if it gets to scale, you're going to have those 4 companies as very fast followers. My firm was a venture investor in Cerebras. What Cerebras has done is something hard and fundamentally different — wafer-scale computing. It comes with a set of trade-offs, but that architectural decision they made was hard and lets them do something that no one else can do. We'll find out how big that is. They're working on really cool things. One of the problems Cerebras has is, once you start needing to glue a lot of chips together and scale up networks or scale out networks, you need a lot of I/O, and I/O is bound by what's called the shoreline — the sides of the chip.
· Gavin Baker, Patrick O'Shaughnessy
Cerebras has an overwhelming ratio of on-chip compute and memory, relative to shoreline I/O. Well, they're really smart people. They did something really hard. They're trying to see if they can put an optical wafer right on top of that, and then that solves that problem. I'm sure they're looking at hybrid bonding of DRAM to get around the — much-discussed on X — alleged limitations that are not true. A Cerebras machine can theoretically run any size model. There are sizes of models where they're much better than other sizes. So Cerebras — what I think is interesting is they did something different that's hard to do, really hard to do — wafer-scale computing. I do think there's a role for these. I would just encourage them all to make a different trade-off, try and do something hard. Everybody's going to get funded after the Cerebras IPO. It's not going to be a problem. But it took Cerebras three generations of chips to get it right. Andrew Feldman, the CEO — you can just see how hard it was, what he did and that whole team did, to get where they are today. They need to have the grit to do that, the resilience. This first chip is a failure. It happens. Can you come back and make a second chip? But one last thing on this topic. This is going to be amazing for the useful lives of GPUs and may single-handedly save private credit. [00:00:00] S0: Say more about that. What do you mean by private credit?
· Gavin Baker
[00:00:00] S1: Well, private credit is in pain from these SaaS loans, and however much they're marked down, they probably need to be marked down more. If the public companies are struggling to adapt, how's a debt-laden company going to adapt and invest in what is a very different margin structure business? There's a lot of private credit in GPUs too, and they were underwriting that to, I think, 3 or 4 years. The disaggregation of inference means that these GPUs are going to have 10 or 15-year lives. The AI skeptics are like, "Oh, these companies are all cooking their books. The useful life of GPUs is only a year or two. The useful life of CPUs is only 4 years because of rapid technological change." No. What rapid technological change has done — with the disaggregation of prefill and inference — is mean that you can put a Cerebras system or Groq LPUs that Nvidia acquired, effectively in front of a Hopper or even an Ampere, use that Hopper and Ampere for prefill, and extend the useful life of that GPU until it melts. They do melt, so they have a time, but maybe you don't have to run them as fast. This is going to be really good for the whole private credit industry. It's going to help finance the AI buildout, because if you can start to finance GPUs at more like 5% or 6% instead of — I think CoreWeave's lowest financing was low 7s — that actually mathematically changes the cost to finance this buildout. We've had this technological innovation that's going to lower the cost of financing, extend the useful life of compute on Earth. One last thing that's interesting about that — my friend Jaimin from Coatue just did a podcast, and Coatue had a deck where they talked about, "Hey, sellers of shortage are doing so much better than the buyers of shortage" — buyers of shortage being the hyperscalers. But if you own a giant installed base of what is currently in shortage, that's also a very good place to be. We're hearing CPUs are way more important than they were in an agentic world. They do all these things around orchestration, tool calls. The biggest CPU fleets in the world sit at the hyperscalers. Some of these hyperscalers may catch up a little bit to the sellers of shortage.
· Patrick O'Shaughnessy
[00:00:00] S0: I want to talk about this idea of different and hard, applied outside of the infrastructure piece of this. Now you're starting to interact with new founders, existing CEOs and founders that have to adjust to this new world. What are you seeing from the most AI-native founders that aren't building chips or infrastructure or models, but just people using this technology to build other stuff? How do they feel the most different to you, if you've observed differences?
· Gavin Baker
[00:00:00] S1: This isn't just for chip design. To me, it's always been a fundamental question for venture. There are different ideas that are obvious to everyone on planet Earth as soon as they hear it. If that's where you are in venture — if it's not hard to do, if it becomes obvious to the world before you have built scale — scale is the ultimate advantage — you're in trouble. The great thing Amazon had was, I think it was obvious to a lot of people, but it wasn't obvious to the retail CEOs. Amazon, they were very smart. Any e-commerce company that VCs invested in, they would destroy. They'd be like, "Oh, that's so cute. We're going to take our margins in that to negative ten thousand percent." That's like the guys at Wayfair, they did something hard. Amazon tried to kill them and they failed. Those were tough, operationally really competent CEOs. For me in venture, I always look — is this going to be obvious to the world before this company could build scale? Or is this both not obvious, different, and really hard to do? A lot of founders are really struggling with this in AI. People are becoming worried today that in Jensen's five-layer cake of AI, the profits are accruing to energy, accruing to data centers, accruing to chips, accruing to models — not really accruing to the applications. Cursor and Cognition got to scale. They focused on coding. 18 months ago, OpenAI was doing everything under the sun. The people focusing on coding were Cursor, Cognition, Anthropic, and it was really right to focus on code. Amjad Masad, the founder of Replit, tweeted something I thought was so smart — something like, "Bitter lesson-adjacent is the fact that coding might be the shortest path to ASI and useful AI. Because if you're really good at coding, you can write yourself code to do anything. " So it was really smart of those companies to focus intensely on coding. They all probably got to a scale where they have a place. I think Cognition is doing something really, really different. But a lot of founders are really struggling. They're trying to get confidence that in "niche-ier" areas…