00:45:30 · Sam Altman, John Collison
[00:45:30] S1: I super trust that. Yeah. [00:45:32] S0: No. I'm just really struck by this dynamic. Like, my new thinking, in particular with regards to Agenda Commerce, but also anything, is any time there is a web form field that you as a human are filling out, these days, that is below your pay grade. Because you are probably not inventing that information for the first time. Like, we've all filled out the medical forms where it's like, oh, your family history or whatever. Yeah. And that's so frustrating because how many times have you filled out your dad's your mom's medical history and things like that? It just... It feels so wasteful. But there's million versions of that. You are never inventing new information for a web form. [00:46:07] S1: I would be so happy to never do, like, fake work at a computer again. Yeah. [00:46:13] S0: And my argument is that any web form is basically fake work. [00:46:16] S1: I agree. But but, like, the the... I mean, I was, like, one of the very earliest adopters that I knew of, of those, like, plugins that put in your, like, name and address, very promissory, you know, because I was just like, this is... I cannot do this again. [00:46:28] S0: Mhmm. [00:46:29] S1: But I kinda feel like that using, like, most of the web at this point.
00:46:29 · Sam Altman, John Collison
[00:46:29] S1: But I kinda feel like that using, like, most of the web at this point. [00:46:33] S0: Yeah. I just find this phenomenon really interesting where businesses right now are really attached to their HTML and mobile apps as sources of business defensibility. And it just feels very trans... Like, either they're wrong and it's not a source of defensibility, or it may be, but it's transitory and they just need to get over [00:46:54] S1: The last example of this that I think we really witnessed was, like, the app stores. Mhmm. The iPhone app stores specifically. And I remember every start up was like... Or many start ups, like most of the YC batch was like, I'm gonna build an iPhone app. This is like new distribution. This is awesome. I can do a new thing I couldn't do before. And many new startups got created. And all of the big companies felt like they had to make one, but they kind of didn't want to, because they're like, oh, I'm gonna like hand my, you know, relationship with my consumer over to Apple. And it was great for my... [00:47:23] S0: It was just a new UI. Yeah. Yeah. Okay. I'm gonna go more rapid fire. People complain about bench maxing. Is that a real concern? Are OpenAI models bench maxed? Do you... Are you actually Rick Rubining it where you just feel like the vibe of this model is right and you don't care what sweepench gives us?
00:47:23 · John Collison, Sam Altman
[00:47:23] S0: It was just a new UI. Yeah. Yeah. Okay. I'm gonna go more rapid fire. People complain about bench maxing. Is that a real concern? Are OpenAI models bench maxed? Do you... Are you actually Rick Rubining it where you just feel like the vibe of this model is right and you don't care what sweepench gives us? [00:47:39] S1: Maybe there are, like, two things we should really care about models being good at. One is, like, you know, scientific discovery and doing things that people cannot do. And the other is, like, being enjoyable to work with. And I don't think we have great benchmarks for being enjoyable to work with. I'm actually not sure we have great benchmarks for the smartest scientists either, but we endeavor not to bench max. And I I I kind of... I keep thinking that bench maxing is, like, over as a phenomenon, and people keep doing it. [00:48:14] S0: So are you better than other labs at not getting sucked in by the siren call of Benchmax? [00:48:20] S1: I believe we are better at that. [00:48:21] S0: It feels like... I don't think [00:48:24] S1: we've always been better at that in our history. I believe we are better than that right now. [00:48:27] S0: It feels like the industry is still figuring out the being nice to work with, where there was the glazing era. And more recently, there was the stroppy, ornery era of models that were anti glazing, and as a result, tried to disagree with you and nitpick at you on every minor thing. The current ones are pretty good, actually.
00:48:27 · John Collison, Sam Altman
[00:48:27] S0: It feels like the industry is still figuring out the being nice to work with, where there was the glazing era. And more recently, there was the stroppy, ornery era of models that were anti glazing, and as a result, tried to disagree with you and nitpick at you on every minor thing. The current ones are pretty good, actually. [00:48:44] S1: It's gotten... I think the latest models are Yeah. Much better than we've had in a long time. [00:48:49] S0: But it also feels like there's probably higher peaks to Ascend. [00:48:52] S1: Emmett Shear said to me years ago that what he wanted was a really thoughtful and challenging but still respectful senior coworker. And I have still not heard a better, like, brief encapsulation than that. [00:49:06] S0: Yes. After working with him for many years, what's your grand unified theory of Greg Brockman? [00:49:10] S1: Think Greg Brockman is, like, one of a kind in the best sense. [00:49:13] S0: He's singular on the Greg Brockman scale, and you just kinda have to be there to understand what that is. [00:49:16] S1: But I think it's it's, like, a combination of, like, yeah, intensity, like, monster productivity, extreme dedication, and, like, focus on sixteenth of mission. [00:49:26] S0: It's it's [00:49:26] S1: very cool.
00:49:26 · Sam Altman, John Collison
[00:49:26] S1: very cool. [00:49:27] S0: Hate tokens as a metric for anything, really. Pricing or just kind of... It's not a standard unit of AI. And yet we found ourselves there. You know, we did talk... And you might say outcomes, but I don't think it can be outcomes. The outcomes is too Yeah. Different across different use cases. It's... What's the right standardized unit? [00:49:51] S1: We almost made a very... What would have been bad call to say we're not going to price our next model in tokens at all. And you're just going to give it a prompt, and we're going to give you a price back. And we're going to do it. And it's going to be like AWS. You just do something and we send you a bill. But the customer response on that is so negative. They're like, I need to know what I'm paying for somehow. Yes. And I asked a few people, like, would you rather pay more for the same output but have it priced in some number of consistent tokens or something, even if we inflated the tokens. And I go, I'd much rather inflate you to inflate the tokens and pay more, because I know there's a real thing there. So I think you are right that people don't just want us to give them a price because they feel like over the long term they're going to get screwed somehow, In the same way that maybe people would rather pay for CPU hours rather than just getting their thing done, even if it was less expensive.
00:49:51 · Sam Altman, John Collison
[00:49:51] S1: We almost made a very... What would have been bad call to say we're not going to price our next model in tokens at all. And you're just going to give it a prompt, and we're going to give you a price back. And we're going to do it. And it's going to be like AWS. You just do something and we send you a bill. But the customer response on that is so negative. They're like, I need to know what I'm paying for somehow. Yes. And I asked a few people, like, would you rather pay more for the same output but have it priced in some number of consistent tokens or something, even if we inflated the tokens. And I go, I'd much rather inflate you to inflate the tokens and pay more, because I know there's a real thing there. So I think you are right that people don't just want us to give them a price because they feel like over the long term they're going to get screwed somehow, In the same way that maybe people would rather pay for CPU hours rather than just getting their thing done, even if it was less expensive. [00:50:48] S0: You know the way, like, nanometer... The chip It's totally fine. Yeah. Yeah. The nanometers has very little relation to reality these days. Will we get something like that in tokens where you guys should start saying, here is an intelligence equivalent token like... I can't believe it's that butter. [00:51:03] S1: Of, the smallest feature? Yeah. I... That's, like, made up. Hope not. [00:51:08] S0: Think I'll do that. [00:51:08] S1: It... I mean, it offends me that we, like, use the word nanometers for something that's nothing new with nanometers.