00:33:42 · Sam Altman, John Collison
[00:33:42] S1: One of the sort of interesting things that people talk a lot about OpenAI, which sounds very obvious, but sounds like a good answer, just like, well, build me build me RSI or whatever is, What what are the questions that you're supposed to ask of GPT seven? Probably, it's not enough to just say, like, make me RSI. And also, like, it's probably not enough to say, like, buy me a 5% compute efficiency win. I think this is, like, one of the most interesting debates going on right now. And the answer is not just, like, what question to ask you, but it's things like agent swarms. Like, what are we... How how are we gonna use these models in a way that, like, actually maximizes what we can get out of them? I think we're gonna learn a lot about that in the next, like, three months. [00:34:26] S0: Okay. But it sounds like you guys do not have a codified OpenAI standard way of doing an AI enabled AI development. It's still lots of experimentation yourself. It's [00:34:37] S1: also experimentation.
00:34:37 · Sam Altman, John Collison
[00:34:37] S1: also experimentation. [00:34:38] S0: Yeah. When we're talking about the token budgets, it just strikes me that this is one of the big unresolved questions for AI use in enterprises generally, is how... You know, we don't have hard budgets for individual engineers. You don't get cut off at a certain point. You're like, sorry. You've spent all your arcade tokens. Maybe we should, but we don't. And again, I think we all believe in the scale up of AI. But if you take that seriously, it means the dollars of compute or AI per person will go up. And so I feel like we're pretty soon to now actually just running into this question of how you actually decide how much spending power to give individual people inside the organization?
00:35:24 · Sam Altman, John Collison
[00:35:24] S1: There was a time when a lot of companies had this, like, thing in their Google Calendar which showed the, like, approximate cost of each meeting in terms of everybody's comp that had to be there. I think that was always still an underestimate of it. And there was this observation that, Paul Graham made it at one point, you know, and many of them made it too, which is that there was like, there are people at companies that have incredible power to save, like, you know, $17 on the snack budget of the office every week. And yet, can have these meetings for, like, unbelievable amounts of money that are adding no value, and that there's kind of like no equivalent of the ability to stop that. I think that most use... Okay. So I'm like really talking about my book here, I'm very biased, but I genuinely believe it. That use of AI is, like, not a company's biggest spend problem relative to other things that happen. And even if it's, like, somewhat inefficient, it is not where I would focus... [00:36:23] S0: Well, hang on. That's not scaling law pilled enough. Like, I'm not... Okay. Two years from now... [00:36:28] S1: You want go send all Stripe's revenue... All of Stripe's profit on OpenAI services and then complain about this? I'd be delighted to have a conversation. [00:36:33] S0: But we're on a path [00:36:34] S1: to that. Like, you know, we... [00:36:35] S0: Great. Yeah. We now spend... You know, 1% ish shorter magnitude of Stripe's OpEx is tokens. That is obviously... [00:36:43] S1: That feels too low.
00:36:43 · Sam Altman, John Collison
[00:36:43] S1: That feels too low. [00:36:44] S0: Yeah. So we're we're kind of fixing it. Okay. And it'll get to 51020%. And and then you have a real question. [00:36:52] S1: Do you have an intuition on what the right number is? Like, what it should be today? [00:36:56] S0: What it should be today is maybe tricky, but steady state, isn't it going to be... Depending on the kind of company, isn't it going to be tens of percentage points of their OpEx? [00:37:16] S1: I think for many companies, the answer to that is clearly yes. And I think there will come a time when the right thing to do is to really optimize that. And also, like, another thing I'm proud of about OpenAI is we've tried to really drive the cost per unit of intelligence down and down and down. I think we've been, like, a good spokesperson [00:37:41] S0: Mhmm. [00:37:41] S1: And good champion of this. But still, I think the problem, even at current costs, I think there are more companies the problem of not using enough AI rather than using too much [00:37:53] S0: Do the dollars per gigawatt of leading edge AI go up or down? [00:37:57] S1: That should go down. Down? That should go down. [00:38:00] S0: Why? Because it's gone up to up to this point. Right? [00:38:05] S1: I mean, the reason it could go up is that if AI just gets increases in value per what it can do per watt so quickly, and there's such a supply shortage that people are willing to spend much more. [00:38:15] S0: But you would view that as, like, compute not keeping up?
00:38:15 · John Collison, Sam Altman
[00:38:15] S0: But you would view that as, like, compute not keeping up? [00:38:17] S1: I would view that as compute not keeping up. And if I look at the, like, fundamental cost of turning the sand into intelligence, that should go down. The cost of energy should go down. The cost of chips should go down. The rest of the cost of the supply chain should go down. If it doesn't go down, I'll be very sad. But for it to go down, given how much value there is in this, supply's got to go away. [00:38:38] S0: Okay. So you would like to see a compute glut where we start producing a lot more memory, a lot more data centers, lot more of all the inputs, and as a result, anyone who owns a computer for whatever reason can have one. [00:38:50] S1: People give different numbers for the kind of the cost per gigawatt measured in, like, the five year total cost of ownership right now. But, like, let's say... Let's pick $70,000,000,000 as a number. I think if that number... [00:39:02] S0: That's a cost to install... [00:39:04] S1: And run it.... Five years. Yep. Okay. I think if that number were, like, 20 instead of 70, we should be much happier. And we should still demand to get, you know, 10 x better performance out of it with smarter models every year. [00:39:17] S0: Presuming you don't think, like, two is a good number. Like, something's gone wrong if we're seeing two because it would be surprising if we were... Or is two better than 20? [00:39:27] S1: Why is it not? Like, I don't think that will happen, but why do we not want that number to be as low as possible?
00:39:27 · Sam Altman, John Collison
[00:39:27] S1: Why is it not? Like, I don't think that will happen, but why do we not want that number to be as low as possible? [00:39:33] S0: I don't know. This is your theory. You have a view that generally deflationary technology is good. [00:39:37] S1: I I do believe in that. Yeah. I'm pro deflation in this sense. Pro deflationary technology. [00:39:41] S0: So how are we gonna get there? What's gonna happen between now? [00:39:43] S1: Well, $2 a gigawatt. I can't imagine. [00:39:45] S0: Okay. What's gonna happen between now and $20 a gigawatt? [00:39:47] S1: $20,000,000,020 a gigawatt would be really [00:39:49] S0: great. Okay. Yeah. 20,000,000,000 a gigawatt. [00:39:50] S1: Even to get down to 20, you're getting into the realm of energy's gotta get cheaper. Mhmm. But certainly, like, all of the gross margin stacking that happens between melting sand... Or the equivalent of melting sand and getting the token back, a lot of that's got to get compressed. [00:40:05] S0: You could just get there with eliminating profits, or do you need to also make the energy cheaper? I don't know the math off the top of my head. [00:40:12] S1: I mean, if you assume, like, all materials were free and no one made a profit, maybe you could get there in the current energy envelope, but only just...