Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy
[00:00:00] S0: My guest today is Krishna Rao, the CFO of Anthropic. The center of our conversation is how he navigates the decision around procuring and allocating compute, which he describes as the canvas on which everything else gets built. We talk about what he calls the cone of uncertainty, the three chip platforms Anthropic uses fungibly across Trainium, TPUs, and GPUs, and the daily meetings they run to allocate compute between model development, internal use, and serving customer demand. He explains why the returns to frontier intelligence keep getting higher, especially in enterprise, and how Anthropic thinks about the line between platform and application, and why they choose to build their own products like Claude Code. Krishna has such a unique seat watching one of the fastest-growing businesses in history, and he is generous in sharing what he has already learned since joining the company two years ago. Please enjoy my conversation with Krishna Rao. Krishna, I have been so excited for this conversation, because you get to see from the inside one of the most interesting businesses in world history at maybe the most interesting time in world history, at least if you're a technologist or care about technology. One of the things that fascinates me most is this question of compute that you have to deal with all day, every day. It's a key part of what you do. It's a key part of what these companies are doing, and there's this whole revolution happening. I'd love you to start by explaining what it's like to have to deal with that. I understand at one point you were having a daily meeting about how to allocate compute, and to who, and why. Bring us into that part of your life, because I think it's right at the cutting edge of what's going on.
· Krishna Rao, Patrick O'Shaughnessy
[00:00:00] S1: The compute that we procure is the lifeblood of our business. It is the most important thing in the company. It's the canvas on which everything else gets built. The decisions we make on how much compute to buy are some of the most consequential and hardest decisions to make in the entire company. Think of it this way — if you buy too much compute, you go out of business. If you buy too little compute, you can't serve your customers. [00:00:00] S0: You go out of business. [00:00:00] S1: And you're not at the frontier. Same thing. We talk a lot about this cone of uncertainty, but the idea is that these purchases have real-world implications. You can't just go out and buy a gigawatt of compute and have it delivered next week. You have to really think ahead to plan for this. We take a very disciplined approach to how we think about it. We look bottoms up. We model what we think demand will be. Obviously, we sometimes get that wrong. We think about the compute we need to stay at the frontier, and we really look ahead and try to estimate that. As we go out and do these deals to procure compute, flexibility is really important to us. We build that flexibility into the deals themselves. We build that flexibility into how we use the compute as well, because the way we bridge from the position we are today to where we want to go when the business is growing exponentially, is to use that compute as efficiently as possible. I would say I spend 30 or 40% of my time on compute, even today. [00:00:00] S0: What does flexibility mean in that example?
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: What does flexibility mean in that example? [00:00:00] S1: It means a couple of different things. Number one, we use three different chip platforms. We are customers of Amazon's Trainium chip, Google's TPUs, and NVIDIA's GPUs. We use these chips fungibly. If you think about the compute we buy, we're using it for model development, we're using it internally to speed up our own product and model development. We're also using it, obviously, to serve customers. Across those three chip platforms, we're using compute for all of those internal and external uses, and that flexibility — it took us a long time to be able to do that. We've invested in that over multiple years, to be what I believe the most efficient users of compute amongst any of the frontier labs. And that's not something that just happened overnight. When we started using TPUs — I think third-generation TPUs were the first ones we used at scale — people thought, "Oh, well, you're crazy. Everyone's using GPUs. Why aren't you using GPUs?" We've invested very heavily to be able to use that compute incredibly flexibly. We look across the different generations of those chip platforms and use each generation for the best workload internally. We've really built this orchestration layer that gives us that flexibility to use all different types of compute, and in doing so, we're able to get the most value out of it.
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: Am I thinking about this in the right way — that something like CUDA, which has been a part of NVIDIA's story for a long time now, and allows you to do a lot with the underlying actual hardware — that you want to eke your way into being as close to the bare metal as possible? That's part of this flexibility and being able to control as many of the variables as you can. Is that the journey that you've been on? [00:00:00] S1: That's part of the journey, for sure, but it's also been pretty collaborative. We work really closely with the Annapurna Labs team at Amazon to help influence the roadmap of these chips, because we believe what we're doing is really stressing the limits of what these chips are capable of. And that means that a dollar of compute inside our organization goes further than I think it does anywhere else. Importantly, we want to utilize each chip to its best purpose within the company, and that does mean that we're building our own compilers. We're really building things from the chip level up in order to have that customization and that flexibility to use it internally the way we think will generate the most ROI. [00:00:00] S0: Can you explain this cone of uncertainty thing? I want to ask about all the component parts of this, but that feels like a really key starting point or overall frame for how to think about both sourcing and then the uses of compute. Can you explain what that concept is?
· Krishna Rao
[00:00:00] S1: When you're building and growing a business exponentially, really small movements in monthly or weekly growth rates compound into very different outcomes. As we're thinking ahead, even with our revenue growth, it's really hard to predict this business. Humans mostly think linearly. They think incrementally. I've been at the company for two years. That's a paradigm I've had to break for myself, to stop just thinking linearly and think on this exponential. When you're on this exponential, the range of outcomes starts to be really wide. We look at a range of scenarios, and we look at different points in that cone of uncertainty over a one- to two-year period, and then we work backwards from that. What we want is to be at a place where we can still be at the frontier — that's the most important thing. To be able to serve customers, and then to have enough internal compute to accelerate our employees. It's interesting, if we were to say to our employees, "You can't use our models anymore," we could serve billions of dollars of revenue with that compute that we allocate to employees internally. But we want to take a long-term view and a long-term perspective on that cone of uncertainty because we want to range towards the top end of these outcomes. But we have to plan for that. And as we go, that's how we think about buying compute in a disciplined way. The most important thing is what happens if you are at one point in the cone of uncertainty, but you've only bought compute for a different point. That's where this compute efficiency has really helped us out.
· Patrick O'Shaughnessy
[00:00:00] S0: Can you bring us into the room for the conversations around the trade-offs between those? I'm so interested by those three buckets of training, research, internal use, broadly speaking, and then serving customer demand. Naively, you might think it's a third, a third, a third allocation or something. How much does that range around? What are the trade-offs? What is that discussion like on an ongoing basis?