00:05:04 · Sam Altman, John Collison
[00:05:04] S1: SNL thing was so funny is because it was so true. Humor like that doesn't stick unless there's an element of, like... It's, like, clearly pointing out a ridiculous but true thing about society or someone. And And the cyber thing, there is a point which is like, okay, you know, sure, the models are really good at cyber, but, like, who asked for this? Why are you doing this? Like, one of my big disappointments, criticisms of our field or whatever, and I think maybe the cyber... And I was getting good at cyber as an example of this is, I don't think we have done a great job of explaining to people why we think all of the potential downside of this technology is worth it. Like, the kind of caricature of this is people feel like we're like, all right, you know, fine. You say the model's going to cure cancer someday. Maybe it will. And if so, I guess that's good. And you say it's going to, like, make stuff cheaper. And, okay, maybe I already see that's good and I get access to medical advice or legal advice. But but then you paint this picture of the future where it's like a new industrial evolution, and maybe the last one wasn't that good, this one sounds worse, and I will have no job, no ability to, like, have control or choice or freedom of my future, and AI is so dangerous that you're gonna have to, like, have a crazy surveillance state so we don't all die. [00:06:22] S0: I'm with you so far. This
00:06:22 · John Collison, Sam Altman
[00:06:22] S0: I'm with you so far. This [00:06:24] S1: is like a terrible sales pitch. Yeah. And one of the things that I am most proud of about OpenAI is even when it has been hugely unpopular in the AI safety community for us to push for iterative deployment and putting these tools in the hands of people. We have really stuck to that principle, and I think this is the only kind of good path. Think people... It's not enough to tell people, like, you're gonna get the model to do these great things for you. It's it's gotta be, we are going to enable you to have a better life that you define for yourself, be... To do useful things. You're not gonna just sit and, you know, get a basic income and have a bunch of people sitting in a few square miles of San Francisco make all the decisions about the future. [00:07:08] S0: Iterative deployment is this idea that you just have to get the advances out there and you can't kind of stopper them up in some... [00:07:13] S1: I mean, not all. Like, you you still have to make them, like, safe before you put them You out can't, like, cause havoc in the world. But but the but the building up a capability overhang is a thing to fight against. That that, directionally speaking, yes, you wanna get this into the world, you wanna get this into people's hands, and you wanna let people use the technology to figure out all these wonderful things. [00:07:32] S0: The exploits get a lot of headlines. The prevented exploits do not. How quickly are people making progress with Cyberblue? I don't want to
00:07:32 · John Collison, Sam Altman
[00:07:32] S0: The exploits get a lot of headlines. The prevented exploits do not. How quickly are people making progress with Cyberblue? I don't want to [00:07:42] S1: declare victory at all on the PHP thing, because I still don't think we know. And I assume some bad things are going to happen. Relative to my worst fears, if I can, like, think back ten days ago, the first day of that went much better than I expected. And we heard a bunch of stories from people that were using our models to, I mean, not only build patches and build cyber defense, but like individual companies who were just like, I need to just do a quick scan appointment. Like, I'm take gonna my thing down for an hour. I'm gonna use this thing, I wanna put it back up because my application's offline. Now I feel like I really jinxed it, I shouldn't have said that. But the first day did go a lot better than I thought, and I think people have been able to use that. The thing that I've heard a lot of cyber experts talk about is they just no longer are confident that we have the right assumptions baked into our threat model. And they're like, look, if everything is what we think and hardware exploits are as hard as we think they are, then yes, we can see how this is going to work. But if AI is going to discover really deep, bugs in hardware that require physics we didn't understand, but all of a sudden means some assumption doesn't hold, then I think people aren't sure what happens.
00:08:53 · John Collison, Sam Altman
[00:08:53] S0: One thing that always strikes me is that when I talk to very knowledgeable people inside OpenAI and similar companies, there is a pessimism that sandboxing is possible, basically. And that would not have been my mental model going in. It would have been this, yeah, you, have to build a really tight sandbox, and we haven't kind of hardened them historically enough compared to the current capabilities. But ultimately, you can have a maximum security prison for the model to to work in and keep it inside there. It seems like you guys have kind of a pessimism that sandboxing models is possible. [00:09:35] S1: My instinct... Well, first of all, think we knew much better than we're doing now, and we should. I think we we clearly... That is the the main thing to focus on right now is a better security model. If you imagine a theoretical AI that is, like, a million times smarter than the smartest security researcher, would you, like, really be willing to trust that sandbox, especially if the model can, like, discover new science that people don't already... [00:10:01] S0: You you can use the models to construct the sandboxes. [00:10:03] S1: I think you can. Well, so this gets... Let me come back to that. [00:10:07] S0: Yeah.
00:10:07 · John Collison, Sam Altman
[00:10:07] S0: Yeah. [00:10:07] S1: I think sandboxing works to a point. Then I think monitoring works to a further point Mhmm. Where we can look at what the models are doing, not only in their chain of thought, we can do activation monitoring and try to really understand, like, are the models... Are we happy that the models are not deceiving us? And then, eventually, I think you need to believe with huge confidence that the models are truly aligned and are going to value and try faithfully to do... To follow their actual... The intent of the people operating them, and that they're going to value sort of people being in control of the future forever. And if you had a model that you could fully trust the alignment, that you were absolutely certain of the alignment of, then yes, I think you can have that to help you build better and better sandboxes. But, like, if you really let yourself project forward into the, like, what does it mean to have something a million times smarter than we are? I I I don't know. Like, I think that just gets [00:11:09] S0: strange. Because the alignment becomes hard? [00:11:12] S1: Because you need to really, really trust the alignment to trust that these sandboxes stay in ahead of capabilities. [00:11:20] S0: Can you not give models competing incentives and give them incentives to defect from each other? [00:11:24] S1: I think this is a promising direction. I suspect that this idea... I hate the word agent swarms. Let's call them agent teams. [00:11:35] S0: But it seems to
00:11:35 · John Collison, Sam Altman
[00:11:35] S0: But it seems to [00:11:36] S1: be I taking suspect that teams of agents have very interesting scaling law properties, but also very interesting alignment properties. And I think that's, like, a direction we should push in more, just from capabilities, from a safety perspective too. [00:11:53] S0: I know it when you talk about agent teams, not swarms. A swarm is such a... I know. It's a locust. Yeah. Yeah. Not good. Not good branding, in keeping with many historical AI industry traditions. [00:12:06] S1: We are still good at this. [00:12:08] S0: We'll get back to branding. But I know that there seems to be a big gap in what the leading AI users are doing and kinda standard corporate use in the use of Swarm is for, like, all the advanced math... Sorry, teams. All the advanced math uses agent teams. Yeah. All the advanced hacking uses agent teams. And then, like, common or garden corporate stuff is not using big swarms. Why has that not done? Because we've had this for many months. [00:12:39] S1: Not to the same degree. Like like, the one way to look at this problem, or this question, is how much compute can we productively throw at a single problem? And, you know, a few years ago... We forget how fast this is all gone. But a few years ago, it was, like, impossible to throw more than a dollar of compute at a problem and have it help you at all. [00:13:00] S0: Why?