00:06:22 · John Collison, Sam Altman
[00:06:22] S0: I'm with you so far. This [00:06:24] S1: is like a terrible sales pitch. Yeah. And one of the things that I am most proud of about OpenAI is even when it has been hugely unpopular in the AI safety community for us to push for iterative deployment and putting these tools in the hands of people. We have really stuck to that principle, and I think this is the only kind of good path. Think people... It's not enough to tell people, like, you're gonna get the model to do these great things for you. It's it's gotta be, we are going to enable you to have a better life that you define for yourself, be... To do useful things. You're not gonna just sit and, you know, get a basic income and have a bunch of people sitting in a few square miles of San Francisco make all the decisions about the future. [00:07:08] S0: Iterative deployment is this idea that you just have to get the advances out there and you can't kind of stopper them up in some... [00:07:13] S1: I mean, not all. Like, you you still have to make them, like, safe before you put them You out can't, like, cause havoc in the world. But but the but the building up a capability overhang is a thing to fight against. That that, directionally speaking, yes, you wanna get this into the world, you wanna get this into people's hands, and you wanna let people use the technology to figure out all these wonderful things. [00:07:32] S0: The exploits get a lot of headlines. The prevented exploits do not. How quickly are people making progress with Cyberblue? I don't want to
00:07:32 · John Collison, Sam Altman
[00:07:32] S0: The exploits get a lot of headlines. The prevented exploits do not. How quickly are people making progress with Cyberblue? I don't want to [00:07:42] S1: declare victory at all on the PHP thing, because I still don't think we know. And I assume some bad things are going to happen. Relative to my worst fears, if I can, like, think back ten days ago, the first day of that went much better than I expected. And we heard a bunch of stories from people that were using our models to, I mean, not only build patches and build cyber defense, but like individual companies who were just like, I need to just do a quick scan appointment. Like, I'm take gonna my thing down for an hour. I'm gonna use this thing, I wanna put it back up because my application's offline. Now I feel like I really jinxed it, I shouldn't have said that. But the first day did go a lot better than I thought, and I think people have been able to use that. The thing that I've heard a lot of cyber experts talk about is they just no longer are confident that we have the right assumptions baked into our threat model. And they're like, look, if everything is what we think and hardware exploits are as hard as we think they are, then yes, we can see how this is going to work. But if AI is going to discover really deep, bugs in hardware that require physics we didn't understand, but all of a sudden means some assumption doesn't hold, then I think people aren't sure what happens.
00:08:53 · John Collison, Sam Altman
[00:08:53] S0: One thing that always strikes me is that when I talk to very knowledgeable people inside OpenAI and similar companies, there is a pessimism that sandboxing is possible, basically. And that would not have been my mental model going in. It would have been this, yeah, you, have to build a really tight sandbox, and we haven't kind of hardened them historically enough compared to the current capabilities. But ultimately, you can have a maximum security prison for the model to to work in and keep it inside there. It seems like you guys have kind of a pessimism that sandboxing models is possible. [00:09:35] S1: My instinct... Well, first of all, think we knew much better than we're doing now, and we should. I think we we clearly... That is the the main thing to focus on right now is a better security model. If you imagine a theoretical AI that is, like, a million times smarter than the smartest security researcher, would you, like, really be willing to trust that sandbox, especially if the model can, like, discover new science that people don't already... [00:10:01] S0: You you can use the models to construct the sandboxes. [00:10:03] S1: I think you can. Well, so this gets... Let me come back to that. [00:10:07] S0: Yeah.
00:10:07 · John Collison, Sam Altman
[00:10:07] S0: Yeah. [00:10:07] S1: I think sandboxing works to a point. Then I think monitoring works to a further point Mhmm. Where we can look at what the models are doing, not only in their chain of thought, we can do activation monitoring and try to really understand, like, are the models... Are we happy that the models are not deceiving us? And then, eventually, I think you need to believe with huge confidence that the models are truly aligned and are going to value and try faithfully to do... To follow their actual... The intent of the people operating them, and that they're going to value sort of people being in control of the future forever. And if you had a model that you could fully trust the alignment, that you were absolutely certain of the alignment of, then yes, I think you can have that to help you build better and better sandboxes. But, like, if you really let yourself project forward into the, like, what does it mean to have something a million times smarter than we are? I I I don't know. Like, I think that just gets [00:11:09] S0: strange. Because the alignment becomes hard? [00:11:12] S1: Because you need to really, really trust the alignment to trust that these sandboxes stay in ahead of capabilities. [00:11:20] S0: Can you not give models competing incentives and give them incentives to defect from each other? [00:11:24] S1: I think this is a promising direction. I suspect that this idea... I hate the word agent swarms. Let's call them agent teams. [00:11:35] S0: But it seems to
00:11:35 · John Collison, Sam Altman
[00:11:35] S0: But it seems to [00:11:36] S1: be I taking suspect that teams of agents have very interesting scaling law properties, but also very interesting alignment properties. And I think that's, like, a direction we should push in more, just from capabilities, from a safety perspective too. [00:11:53] S0: I know it when you talk about agent teams, not swarms. A swarm is such a... I know. It's a locust. Yeah. Yeah. Not good. Not good branding, in keeping with many historical AI industry traditions. [00:12:06] S1: We are still good at this. [00:12:08] S0: We'll get back to branding. But I know that there seems to be a big gap in what the leading AI users are doing and kinda standard corporate use in the use of Swarm is for, like, all the advanced math... Sorry, teams. All the advanced math uses agent teams. Yeah. All the advanced hacking uses agent teams. And then, like, common or garden corporate stuff is not using big swarms. Why has that not done? Because we've had this for many months. [00:12:39] S1: Not to the same degree. Like like, the one way to look at this problem, or this question, is how much compute can we productively throw at a single problem? And, you know, a few years ago... We forget how fast this is all gone. But a few years ago, it was, like, impossible to throw more than a dollar of compute at a problem and have it help you at all. [00:13:00] S0: Why?
00:13:00 · John Collison, Sam Altman
[00:13:00] S0: Why? [00:13:02] S1: You could have GPT-four, like, grind out for you, you know, like a 100,000 tokens or something, and it was going to answer your question. And it certainly was not gonna be able to go right productively, like write you a piece of software. We just... We had not figured out how to use a lot of test time compute to get you a I better see. And then we figured out a chain of thought, and the models could think hard about a problem. And now, And then, like, people really started using a lot of compute. And now you can throw a whole team at something, and you can do that, you know, 10,000 times in parallel. You need chain [00:13:37] S0: of thought and test time compute before you can get the swarms. Because you can't have a useful swarm with the one two two two three. Yes. [00:13:42] S1: But so the effect of this, and I think this is not being well followed, paid attention to in the world, but the incredible exponential of how much compute you can throw at a single problem and get better results is one thing. The delta between, like, what most people are using AI for and what the people are using AI the most productively for has just continued to spread and spread and spread. How do you allocate