00:26:18 · John Collison, Sam Altman
[00:26:18] S0: lot about... The real RSI. [00:26:19] S1: Yeah. People talk a lot about RSI as like an AI that does AI research. There's very little... Relatively little talk about self replicating data centers. You know, this idea that you can use AI to control robots and run fabs and build turbines or solar panels and, you know, build data centers. But I think we are... I think we have that, like, within grasp. But... And so we'll try to make a lot of really cheap compute. [00:26:44] S0: But aren't data centers already not labor bottlenecked? They're permitting bottlenecked and components bottlenecked. [00:26:52] S1: Could we make more components to the robots? [00:26:55] S0: Eventually. Yeah. [00:26:56] S1: Yeah. Yeah. And I'm... All of this is, like Yeah. Yeah. How we solve this over a long period of time. [00:27:00] S0: Like, my observation in the computer stuff is they say the solution to high prices is high prices. That's like, okay, step one, we have the high prices for memory and GPUs and everything, and the solutions should be here any moment now. [00:27:13] S1: How are [00:27:13] S0: we doing on the solutions front? Bad. And I'm not saying there's an OpenAI's job. I feel like this is the Invisible Hands job. Right? Like, memory's gotten really expensive, and so new memory makers are gonna flood the market any moment I [00:27:26] S1: think the Invisible Hand is failing right now. I mean, the Invisible Hand, it has, like, a checkered track record at best. But I think it's, like, not quite working here. [00:27:37] S0: Why not? Like, is it just hysteresis or...
00:27:37 · John Collison, Sam Altman
[00:27:37] S0: Why not? Like, is it just hysteresis or... [00:27:40] S1: I think it's... I I think the, like, specifics of each area matter here, but in the... To start with the memory guys. Like, you know, for a very long time, if you were a producer of DRAM, you would invest $20,000,000,000 of CapEx to make $19,000,000,000, and you would go through these like booms and busts. Like, maybe some years you would make $21,000,000,000, some years you would make $18,000,000,000, but, like, until right now, I think making memory has ever been a great business over a long period of time. And then all of a sudden, know, these guys are each going to make like a $100,000,000,000 of free cash flow in a year. Unbelievable. But but I think they all, like, remember their history and how many times they almost went bankrupt by making too many fabs and being promised this time it was different. [00:28:26] S0: Which leads them to be conservative in their CapEx. Clearly. It feels like what you should expect to see is that every part of the supply chain, both new start ups coming along, and maybe more relevantly, people with relevant experience getting into... So, you know, Intel makes chips. They can probably make... They can get back into the memory game, as is tradition. As is tradition. Switch back.
00:28:26 · John Collison, Sam Altman
[00:28:26] S0: Which leads them to be conservative in their CapEx. Clearly. It feels like what you should expect to see is that every part of the supply chain, both new start ups coming along, and maybe more relevantly, people with relevant experience getting into... So, you know, Intel makes chips. They can probably make... They can get back into the memory game, as is tradition. As is tradition. Switch back. [00:28:59] S1: I think there are like all of these ways where the supply chain... It's just like super non obvious. An example of this is people... A lot of people want local inference, and somehow they assume that, like, the problem is solved now because we're going to have open source models, and we're going be able to run them on our computers, and so we're not going to need all the stuff in the data center. But from, like, a global allocation perspective, actually, it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used 5% of the time. Like, you want maximum utilization in the cloud if you're trying to, like, deal with this very supply constrained environment, And and somehow everyone's... Many people's intuition is the opposite of that. [00:29:44] S0: Where else have we seen the market respond to high prices? Not just in memory, but across the the supply chain. [00:29:51] S1: Turbines? Mhmm. Batteries? Concrete?
00:29:51 · Sam Altman, John Collison
[00:29:51] S1: Turbines? Mhmm. Batteries? Concrete? [00:29:55] S0: Sorry. The market is responding in in high prices. Yeah. I'm seeing where... Are you seeing any interesting market responses in in terms of high prices pulling in innovation? Components for... Some components for robots. [00:30:07] S1: And not like the humanoid, like, you know, kind of like fake demos, like the industrial robots that can just make stuff faster. [00:30:12] S0: There's new actuator makers Yeah. And that kind of Honestly, [00:30:17] S1: I mean, maybe this... If OpenAI sells our chip, maybe that would be an example. [00:30:20] S0: Okay. Yeah. Wanna get to your chip because that was gonna be my example. I think the the solution to high prices and short, you know, limited demand... Or sorry, limited supply, excuse me, is you make your own chip. Presumably, you're using a lot of AI to make the chip. I would have thought that's hard because I would have thought that chip design is one of those places limited by the public tokens you can get where there aren't actually that many blog posts on the public Internet on how to build a, you know, leading edge two nanometer chip.
00:30:20 · John Collison, Sam Altman
[00:30:20] S0: Okay. Yeah. Wanna get to your chip because that was gonna be my example. I think the the solution to high prices and short, you know, limited demand... Or sorry, limited supply, excuse me, is you make your own chip. Presumably, you're using a lot of AI to make the chip. I would have thought that's hard because I would have thought that chip design is one of those places limited by the public tokens you can get where there aren't actually that many blog posts on the public Internet on how to build a, you know, leading edge two nanometer chip. [00:30:51] S1: Well, in terms of, like, how to do the lithography... Yeah, I think there's, one company on Earth that seems to know how to do that. In terms of how to do the chip design, like, to build the file that you eventually send the tape out, that is well understood, and that is a great problem for AI. And given that the world is going to be, like, limited in its silicon capacity, we all... We have, a shared interest. One of the things that I'm proud of about OpenAI, and maybe it sounds obvious, but it was it was maybe, like, not as obvious at the time, is we have a culture where people, even if they've done something the old fashioned way for a long time, are are super willing to, like, try using AI to do it better. Mhmm. [00:31:35] S0: What are other examples of that? [00:31:37] S1: AI research, software engineering, but also increasingly, like, procurement or things that people think are very outside the realm of what most people are using AI to do.
00:31:37 · Sam Altman, John Collison
[00:31:37] S1: AI research, software engineering, but also increasingly, like, procurement or things that people think are very outside the realm of what most people are using AI to do. [00:31:48] S0: How does AI driven AI research actually work? Like, the math proofs where you say... Or the find the PHP vulnerability where you say, please find me some AI advances? So [00:31:59] S1: I think we don't know yet how it truly is gonna work. We know today how to say, like, find me a 5% compute efficiency win. [00:32:05] S0: But but it's that high level. Like, I could be an incredibly successful AI researcher now. [00:32:11] S1: I don't know, incredibly. But, yeah, if you were looking for a new job, you can come do it. What I don't think anyone knows how to do yet is to say, like, go discover a new architecture that is as big of a single leap forward as the transformer was. And I think that's where things are heading. Mhmm. And what's missing there? Smarter models... Smarter models that are smarter in this dimension. [00:32:30] S0: The high level planning and conceptual reasoning? [00:32:32] S1: I think if you apply the same pipeline that we applied to get our models to be so good at math to AI research, like, it will probably work. Now, per an early part of this conversation, I kind of think that at least for a while, maybe for a long while, we are going to have to agree to some limits about how smart we let these single models go. But a lack of progress will come from that decision, not from the AI's inability to do good research.