00:32:32 · Sam Altman, John Collison
[00:32:32] S1: I think if you apply the same pipeline that we applied to get our models to be so good at math to AI research, like, it will probably work. Now, per an early part of this conversation, I kind of think that at least for a while, maybe for a long while, we are going to have to agree to some limits about how smart we let these single models go. But a lack of progress will come from that decision, not from the AI's inability to do good research. [00:32:57] S0: Concretely, if I'm using AI to develop improved AI... Because we should expect GPT seven to be really good now because you have GPT six and whatever Yeah. Know, unreleased version of it to develop it. And so my expectations are sky high. But I come into work, you know, with my backpack, and I come into the OpenAI office in San Francisco, and I sit down at my desk. Like, am I seeing find me more intelligence in this model? Am I giving a real direction of research? How directed does it get? What does the iteration loop look like? Just what does RSI... Because you are... You have a front row seat to RSI, and curious what it looks like.
00:32:57 · John Collison, Sam Altman
[00:32:57] S0: Concretely, if I'm using AI to develop improved AI... Because we should expect GPT seven to be really good now because you have GPT six and whatever Yeah. Know, unreleased version of it to develop it. And so my expectations are sky high. But I come into work, you know, with my backpack, and I come into the OpenAI office in San Francisco, and I sit down at my desk. Like, am I seeing find me more intelligence in this model? Am I giving a real direction of research? How directed does it get? What does the iteration loop look like? Just what does RSI... Because you are... You have a front row seat to RSI, and curious what it looks like. [00:33:42] S1: One of the sort of interesting things that people talk a lot about OpenAI, which sounds very obvious, but sounds like a good answer, just like, well, build me build me RSI or whatever is, What what are the questions that you're supposed to ask of GPT seven? Probably, it's not enough to just say, like, make me RSI. And also, like, it's probably not enough to say, like, buy me a 5% compute efficiency win. I think this is, like, one of the most interesting debates going on right now. And the answer is not just, like, what question to ask you, but it's things like agent swarms. Like, what are we... How how are we gonna use these models in a way that, like, actually maximizes what we can get out of them? I think we're gonna learn a lot about that in the next, like, three months.
00:33:42 · Sam Altman, John Collison
[00:33:42] S1: One of the sort of interesting things that people talk a lot about OpenAI, which sounds very obvious, but sounds like a good answer, just like, well, build me build me RSI or whatever is, What what are the questions that you're supposed to ask of GPT seven? Probably, it's not enough to just say, like, make me RSI. And also, like, it's probably not enough to say, like, buy me a 5% compute efficiency win. I think this is, like, one of the most interesting debates going on right now. And the answer is not just, like, what question to ask you, but it's things like agent swarms. Like, what are we... How how are we gonna use these models in a way that, like, actually maximizes what we can get out of them? I think we're gonna learn a lot about that in the next, like, three months. [00:34:26] S0: Okay. But it sounds like you guys do not have a codified OpenAI standard way of doing an AI enabled AI development. It's still lots of experimentation yourself. It's [00:34:37] S1: also experimentation.
00:34:37 · Sam Altman, John Collison
[00:34:37] S1: also experimentation. [00:34:38] S0: Yeah. When we're talking about the token budgets, it just strikes me that this is one of the big unresolved questions for AI use in enterprises generally, is how... You know, we don't have hard budgets for individual engineers. You don't get cut off at a certain point. You're like, sorry. You've spent all your arcade tokens. Maybe we should, but we don't. And again, I think we all believe in the scale up of AI. But if you take that seriously, it means the dollars of compute or AI per person will go up. And so I feel like we're pretty soon to now actually just running into this question of how you actually decide how much spending power to give individual people inside the organization?
00:35:24 · Sam Altman, John Collison
[00:35:24] S1: There was a time when a lot of companies had this, like, thing in their Google Calendar which showed the, like, approximate cost of each meeting in terms of everybody's comp that had to be there. I think that was always still an underestimate of it. And there was this observation that, Paul Graham made it at one point, you know, and many of them made it too, which is that there was like, there are people at companies that have incredible power to save, like, you know, $17 on the snack budget of the office every week. And yet, can have these meetings for, like, unbelievable amounts of money that are adding no value, and that there's kind of like no equivalent of the ability to stop that. I think that most use... Okay. So I'm like really talking about my book here, I'm very biased, but I genuinely believe it. That use of AI is, like, not a company's biggest spend problem relative to other things that happen. And even if it's, like, somewhat inefficient, it is not where I would focus... [00:36:23] S0: Well, hang on. That's not scaling law pilled enough. Like, I'm not... Okay. Two years from now... [00:36:28] S1: You want go send all Stripe's revenue... All of Stripe's profit on OpenAI services and then complain about this? I'd be delighted to have a conversation. [00:36:33] S0: But we're on a path [00:36:34] S1: to that. Like, you know, we... [00:36:35] S0: Great. Yeah. We now spend... You know, 1% ish shorter magnitude of Stripe's OpEx is tokens. That is obviously... [00:36:43] S1: That feels too low.
00:36:43 · Sam Altman, John Collison
[00:36:43] S1: That feels too low. [00:36:44] S0: Yeah. So we're we're kind of fixing it. Okay. And it'll get to 51020%. And and then you have a real question. [00:36:52] S1: Do you have an intuition on what the right number is? Like, what it should be today? [00:36:56] S0: What it should be today is maybe tricky, but steady state, isn't it going to be... Depending on the kind of company, isn't it going to be tens of percentage points of their OpEx? [00:37:16] S1: I think for many companies, the answer to that is clearly yes. And I think there will come a time when the right thing to do is to really optimize that. And also, like, another thing I'm proud of about OpenAI is we've tried to really drive the cost per unit of intelligence down and down and down. I think we've been, like, a good spokesperson [00:37:41] S0: Mhmm. [00:37:41] S1: And good champion of this. But still, I think the problem, even at current costs, I think there are more companies the problem of not using enough AI rather than using too much [00:37:53] S0: Do the dollars per gigawatt of leading edge AI go up or down? [00:37:57] S1: That should go down. Down? That should go down. [00:38:00] S0: Why? Because it's gone up to up to this point. Right? [00:38:05] S1: I mean, the reason it could go up is that if AI just gets increases in value per what it can do per watt so quickly, and there's such a supply shortage that people are willing to spend much more. [00:38:15] S0: But you would view that as, like, compute not keeping up?