Invest Like the Best with Patrick O'Shaughnessy
· Sam Altman, Patrick O'Shaughnessy
[00:00:00] S1: I would say we got real conviction with GPT-4, not even 3.5. [00:00:00] S0: What was it? [00:00:00] S1: It was seeing the model was smart enough that we knew we'd be able to figure out an approach that worked for reasoning. And then a belief that if we got reasoning to work, that would bring about what is now called agents. We called it different things at the time, but the ability to go do hugely valuable pieces of economic work and make people's lives easier and better in a lot of ways we still haven't seen. [00:00:00] S0: What was the first meeting where you sat down and said, okay, we need to make an outrageous outlay to this? What then happened once you had the realization? What did you do next? [00:00:00] S1: We started calling the clouds. We started calling the chip fabs. We started calling energy providers and everyone's like, "You're totally crazy. This is impossible. No industry has ever moved like this. We've been around, there's these booms and busts. It's not going to go up in a straight line. This is reckless. And we've talked to everybody." It actually reminded me of fundraising for an early stage startup. Most people tell you no, but all you need is one or two yeses. And most people told us no, and we got one or two yeses and we were able to start building. [00:00:00] S0: Who was the first yes? [00:00:00] S1: Microsoft was the first yes. Oracle then became a very big yes on the cloud side. Nvidia has been a tremendous partner.
· Sam Altman, Patrick O'Shaughnessy
[00:00:00] S1: Microsoft was the first yes. Oracle then became a very big yes on the cloud side. Nvidia has been a tremendous partner. [00:00:00] S0: Now there's a thousand flowers blooming of ways to be creative and innovative in how we serve inference and do training in data centers, different kinds of data centers and stuff. I'd love you to reflect on where you see innovation, what you want to do, why people seem to hate these things so much. What's to be done about that?
· Sam Altman
[00:00:00] S1: I have been thinking about how we can organize field trips to a gigawatt data center for people, because it is one thing to say, it is another thing to see a photo or a video of. And then it's a whole other thing to stand there and be like, oh man, this is an unbelievable scale. Building one of these is on the order of 10,000 construction workers going full-time for a year and a half. The energy that flows through one of these things could power a small city. Again, we lost all sense of scale, but each of these would've been among the most expensive infrastructure projects that humanity's ever done. And now we've done a lot of them. First of all, I understand emotionally why people don't want data centers in their backyard. I don't really want a nuclear power plant next to my house, even though I know it's a super safe thing. Unlike power plants — and even power plants have gotten better on this point — we can put a data center anywhere. We should go put it off in the desert, somewhere no one wants to be. This is fine. The AI system is very happy to be there. We have been able to make a lot of progress with innovation on some of the concerns. For example, years ago we were evaporating water to cool these systems. They needed tremendous amounts of water. And now we use these closed-loop systems, and a modern data center uses only as much water as an office building would for the kitchen and the bathrooms. On power, we are moving from energy sources that are burning fossil fuels to systems that are going to be powered by solar or nuclear. And I think that's obviously great. So there may be a deep human thing there to some people, even though they create jobs and are very clean and have all these other positive effects. But in terms of the environmental concerns, we did a great job addressing the water needs and energy is next.
· Patrick O'Shaughnessy, Sam Altman
[00:00:00] S0: What else creative can we do about compute? I'm curious to hear about Jalapeño or other ideas that you've had or thought about for how do we speed up flops and everything available to us? [00:00:00] S1: I think probably the biggest return right now is creative software ideas to squeeze more intelligence out of the units of compute that we have. And my sense is there's orders of magnitude to go there. Jalapeño is a great example of a very efficient chip. So by saying we're going to make a chip that is really good at a specific workflow and gets it some generality, and we want to get some tokens per watt win out of that, I think that's awesome. I think Jalapeño and its successors are going to be a huge competitive advantage for us from that perspective. There are new technologies. I assume at some point we'll figure out optical computing and that'll be a huge win of intelligence per watt. So I think all of those things will happen. [00:00:00] S0: The most interesting thing happening this week is this Kimi release and this idea of the frontier, and all the returns being at the frontier, and distillation, and China versus America. How do you process this? What seems like one of these milestone events? DeepSeek in hindsight looks like it was a quick speed bump. This one — you never know in the moment — how do you process it?
· Patrick O'Shaughnessy, Sam Altman
[00:00:00] S0: The most interesting thing happening this week is this Kimi release and this idea of the frontier, and all the returns being at the frontier, and distillation, and China versus America. How do you process this? What seems like one of these milestone events? DeepSeek in hindsight looks like it was a quick speed bump. This one — you never know in the moment — how do you process it? [00:00:00] S1: Our goal is to offer, at every point along the Pareto optimal frontier, the best option for intelligence and price. And that includes open source. You get a better deal today, at least at a particular latency, using OpenAI's models than Kimi. We distill our own models. That's how we make smaller, cheaper models. I think that's a very good thing to do. And there will be clearly an important place for open-source models in the world and people that will want their own weights for all sorts of reasons, and the ability to modify those. But our goal is the best intelligence price trade-off everywhere on the curve, and we'll continue to do that. [00:00:00] S0: What do you think or hope will happen in the American system and what could block that future? What legislation would worry you? What regulation would worry you? It seems like you've been pretty proactive in showing up in DC.
· Patrick O'Shaughnessy, Sam Altman
[00:00:00] S0: What do you think or hope will happen in the American system and what could block that future? What legislation would worry you? What regulation would worry you? It seems like you've been pretty proactive in showing up in DC. [00:00:00] S1: I haven't thought deeply about the distillation issue. It's clearly a top-of-mind issue now for a lot of people, all of a sudden. But I have always assumed that there are going to be great cheap models in the world, and we better be the greatest and the cheapest. Other people can do what they're going to do, but I think we can really win at our own game here. [00:00:00] S0: Now, the Kimi example is interesting because like you said, you're cheaper on parts of the curve. But the previous story had been, you spend all the money to train the models and then I just distill it and offer it for 1/100th of the cost, how can you make enough money to keep training? [00:00:00] S1: We have so much usage of our models that we do not need to be a gigantically high-margin business to be able to afford model training. So much of our future compute plans will be used to sell inference to customers that even if we can enjoy a modest margin on trillions of dollars of revenue, we can afford to train some giant models. [00:00:00] S0: So the ratio of inference to training is like the thing that…