00:28:26 · John Collison, Sam Altman
[00:28:26] S0: Which leads them to be conservative in their CapEx. Clearly. It feels like what you should expect to see is that every part of the supply chain, both new start ups coming along, and maybe more relevantly, people with relevant experience getting into... So, you know, Intel makes chips. They can probably make... They can get back into the memory game, as is tradition. As is tradition. Switch back. [00:28:59] S1: I think there are like all of these ways where the supply chain... It's just like super non obvious. An example of this is people... A lot of people want local inference, and somehow they assume that, like, the problem is solved now because we're going to have open source models, and we're going be able to run them on our computers, and so we're not going to need all the stuff in the data center. But from, like, a global allocation perspective, actually, it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used 5% of the time. Like, you want maximum utilization in the cloud if you're trying to, like, deal with this very supply constrained environment, And and somehow everyone's... Many people's intuition is the opposite of that. [00:29:44] S0: Where else have we seen the market respond to high prices? Not just in memory, but across the the supply chain. [00:29:51] S1: Turbines? Mhmm. Batteries? Concrete?
00:29:51 · Sam Altman, John Collison
[00:29:51] S1: Turbines? Mhmm. Batteries? Concrete? [00:29:55] S0: Sorry. The market is responding in in high prices. Yeah. I'm seeing where... Are you seeing any interesting market responses in in terms of high prices pulling in innovation? Components for... Some components for robots. [00:30:07] S1: And not like the humanoid, like, you know, kind of like fake demos, like the industrial robots that can just make stuff faster. [00:30:12] S0: There's new actuator makers Yeah. And that kind of Honestly, [00:30:17] S1: I mean, maybe this... If OpenAI sells our chip, maybe that would be an example. [00:30:20] S0: Okay. Yeah. Wanna get to your chip because that was gonna be my example. I think the the solution to high prices and short, you know, limited demand... Or sorry, limited supply, excuse me, is you make your own chip. Presumably, you're using a lot of AI to make the chip. I would have thought that's hard because I would have thought that chip design is one of those places limited by the public tokens you can get where there aren't actually that many blog posts on the public Internet on how to build a, you know, leading edge two nanometer chip.
00:30:20 · John Collison, Sam Altman
[00:30:20] S0: Okay. Yeah. Wanna get to your chip because that was gonna be my example. I think the the solution to high prices and short, you know, limited demand... Or sorry, limited supply, excuse me, is you make your own chip. Presumably, you're using a lot of AI to make the chip. I would have thought that's hard because I would have thought that chip design is one of those places limited by the public tokens you can get where there aren't actually that many blog posts on the public Internet on how to build a, you know, leading edge two nanometer chip. [00:30:51] S1: Well, in terms of, like, how to do the lithography... Yeah, I think there's, one company on Earth that seems to know how to do that. In terms of how to do the chip design, like, to build the file that you eventually send the tape out, that is well understood, and that is a great problem for AI. And given that the world is going to be, like, limited in its silicon capacity, we all... We have, a shared interest. One of the things that I'm proud of about OpenAI, and maybe it sounds obvious, but it was it was maybe, like, not as obvious at the time, is we have a culture where people, even if they've done something the old fashioned way for a long time, are are super willing to, like, try using AI to do it better. Mhmm. [00:31:35] S0: What are other examples of that? [00:31:37] S1: AI research, software engineering, but also increasingly, like, procurement or things that people think are very outside the realm of what most people are using AI to do.
00:31:37 · Sam Altman, John Collison
[00:31:37] S1: AI research, software engineering, but also increasingly, like, procurement or things that people think are very outside the realm of what most people are using AI to do. [00:31:48] S0: How does AI driven AI research actually work? Like, the math proofs where you say... Or the find the PHP vulnerability where you say, please find me some AI advances? So [00:31:59] S1: I think we don't know yet how it truly is gonna work. We know today how to say, like, find me a 5% compute efficiency win. [00:32:05] S0: But but it's that high level. Like, I could be an incredibly successful AI researcher now. [00:32:11] S1: I don't know, incredibly. But, yeah, if you were looking for a new job, you can come do it. What I don't think anyone knows how to do yet is to say, like, go discover a new architecture that is as big of a single leap forward as the transformer was. And I think that's where things are heading. Mhmm. And what's missing there? Smarter models... Smarter models that are smarter in this dimension. [00:32:30] S0: The high level planning and conceptual reasoning? [00:32:32] S1: I think if you apply the same pipeline that we applied to get our models to be so good at math to AI research, like, it will probably work. Now, per an early part of this conversation, I kind of think that at least for a while, maybe for a long while, we are going to have to agree to some limits about how smart we let these single models go. But a lack of progress will come from that decision, not from the AI's inability to do good research.
00:32:32 · Sam Altman, John Collison
[00:32:32] S1: I think if you apply the same pipeline that we applied to get our models to be so good at math to AI research, like, it will probably work. Now, per an early part of this conversation, I kind of think that at least for a while, maybe for a long while, we are going to have to agree to some limits about how smart we let these single models go. But a lack of progress will come from that decision, not from the AI's inability to do good research. [00:32:57] S0: Concretely, if I'm using AI to develop improved AI... Because we should expect GPT seven to be really good now because you have GPT six and whatever Yeah. Know, unreleased version of it to develop it. And so my expectations are sky high. But I come into work, you know, with my backpack, and I come into the OpenAI office in San Francisco, and I sit down at my desk. Like, am I seeing find me more intelligence in this model? Am I giving a real direction of research? How directed does it get? What does the iteration loop look like? Just what does RSI... Because you are... You have a front row seat to RSI, and curious what it looks like.
00:32:57 · John Collison, Sam Altman
[00:32:57] S0: Concretely, if I'm using AI to develop improved AI... Because we should expect GPT seven to be really good now because you have GPT six and whatever Yeah. Know, unreleased version of it to develop it. And so my expectations are sky high. But I come into work, you know, with my backpack, and I come into the OpenAI office in San Francisco, and I sit down at my desk. Like, am I seeing find me more intelligence in this model? Am I giving a real direction of research? How directed does it get? What does the iteration loop look like? Just what does RSI... Because you are... You have a front row seat to RSI, and curious what it looks like. [00:33:42] S1: One of the sort of interesting things that people talk a lot about OpenAI, which sounds very obvious, but sounds like a good answer, just like, well, build me build me RSI or whatever is, What what are the questions that you're supposed to ask of GPT seven? Probably, it's not enough to just say, like, make me RSI. And also, like, it's probably not enough to say, like, buy me a 5% compute efficiency win. I think this is, like, one of the most interesting debates going on right now. And the answer is not just, like, what question to ask you, but it's things like agent swarms. Like, what are we... How how are we gonna use these models in a way that, like, actually maximizes what we can get out of them? I think we're gonna learn a lot about that in the next, like, three months.