00:15:16 · Jeffrey Ladish
We're losing monitorability. Look at Astra, open AI model, but look at Astra. We learned yesterday that Astra will sometimes basically sandbag on safety research. Probably not strategically, we don't really know. But sometimes Astra will underperform on safety research and it doesn't show up in the chain of thought for the most part. I'm like, man, so like, you know, of thought, monitorability, like, okay, like we have some problems with chain of thought, but at least we have it for now. And I'm like, man, that's our best tool and our best tool sucks. And then like, what we actually need my mind, and I think Anthropic's mind too, is really good interpretability tools that would allow us to actually understand what's happening under the hood. And they're trying and I'm like, fuck, fuck, yeah. Let's give them the opportunity to try. Think that means they would need a lot more time in order to develop those tools. And so this is why I would favor something like really aggressive reallocation, where both you can then have a bunch more time, serve a bunch more customers, and also just apply a lot more resources to trying to crack the interpretability problems. And and actually be able to get to the point where you understand the model's drives and motivations so you can make a really, really robust safety case for how you could scale this up to a country of geniuses in a data center.
00:16:36 · Ruby Justice Thelot, S3, Ruby Justice Thelot, Jeffrey Ladish
Ad[00:16:36] S1: We'll continue monitoring right after this message from our sponsors. Eleven Labs, AI that communicates at human level across every channel and modality. Eleven Labs dot I o slash MTS. Scale your startup on Neon. Millions of developers and startups have already chosen Neon for their back end. Start on the free plan or get up to a $100,000 in credits for your startup at neon.com/mts. Special thanks to our sponsor, Kong, the AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. Konghq.com. Ad[00:17:12] S3: Finally, prove what your AI investment is worth with Larridin. Head to lairdon.com today to start measuring your AI's bottom line. MongoDB, trusted to run critical applications by 75% of the Fortune one hundred, the intelligent data platform for the era. Apply today for access to up to $15,000 in Atlas credits, dedicated technical support, and exclusive events and go to market opportunities. Mongodb.com. Infrastructure for the AI era. Chargebee.com. [00:17:44] S2: Do you think that, like, putting this in the context of some of the political sentiment that's going on in moratoriums will actually, like, meaningfully change the rate of progress at all if some of these are passed? [00:17:59] S0: Could you say what you mean by that? Like [00:18:01] S2: Like like, just anything that comes from, like, the political, like, political backlash or slowing down data center build out.
00:18:09 · Jeffrey Ladish
[00:18:09] S0: Yeah. I mean, I I think that with the recent election and just in general, like, a lot of people are are really upset and and I think afraid from from AI progress and sort of... And I think often people have kind of diffuse worries where it's not like I think the general population is reading the media reports and are like, Oh, the Hugging Face incident, all of these agents. I think a lot of it is like, Wow, there's just something really crazy happening and something is getting really powerful. The AI companies are getting very powerful and this AI stuff is starting to do a bunch of stuff and we're worried about our jobs. And I think that was pretty notable, where a lot of you know, politicians running in this election, you know, made AI a really big part of their platform. And I think that, you know, especially the Democrats and some Republicans made this a big part of their platform. I think that means that there's a lot more political will for something like serious pacing proposals, serious pause proposals. And I think that that's good. I think some people have a fear that we'll just stop building all data centers and we'll stop AI progress and then China will win. I think the winning framing is kind of wrong, even so, don't think that will happen. What I think might happen optimistically is something like there is a negotiation between people who are saying, let's do very moderate pacing, and people who are saying, we should absolutely shut it all down and go back ten years. And what you hopefully will end up with is something like, have extremely good ability to break and reallocate towards figuring this stuff out. That in fact, we end up with a very strong safety case regime where if it is safe, we can go ahead. And if not, we don't. But this is based on our best researchers and sort of the best minds that we have in the country, which I think we have the best people in the world, can come together and say, like, okay. Like, what do we think? Like, I don't know. I'm like, let's have Dario and Yoshua Bengio just, like, argue about whether it's it's right to go ahead with recursive self improvement. I'm like, they're both smart guys. Let let us... Let's, like, let's do that, you know, and bring in the next 20 top people, Ryan Greenblatt, a bunch of other people, and just make that an explicit conversation.
00:20:15 · Jeffrey Ladish
Let's have that happening in congressional committees. And let's have the White House bring these people in to advise them directly, and let's make this decision. Because I'm like, man, we're talking about things that are consequential for all of humanity. Just like, man, the stakes are so high, the way we are doing it right now seems really dumb. It seems like right now we're mostly arguing on Twitter. Then And we sometimes get some METR reports, and the companies will publish some blog posts, and then Congress is not well informed about what's happening, and I think the president is not well informed about what's happening. And if everyone just fucking sat down in a room and talked about actual dynamics of recursive self improvement, the actual dynamics of alignment, and proposals for what many people inside of the companies are asking for, which is like, we need a mechanism to actually go at a responsible pace. That's what we need. That's what we want. Yeah. That's that's what I would like to see.
00:21:09 · Ruby Justice Thelot, Jeffrey Ladish
[00:21:09] S1: Right. So I think one thing that's, like, super crux y right now is just, like, the evidence that we have seen, or at least the evidence that METR has published, suggests that they are able to achieve, you know, speed ups in the rate at which AI gets better at these kinds of verifiable tasks that go into, like, the AECI. Yeah. And, you know, to what extent does that transfer over to all tasks, including tasks that models are not very good at at all? Because it seems to me that models will need, like, a pretty broad cognitive profile to be, like, real ASI, like, a country of geniuses in the data center. [00:21:45] S0: Yes. Yeah. I totally agree. [00:21:46] S1: So, like, what evidence do we have one way or another on that?
00:21:51 · Jeffrey Ladish
[00:21:51] S0: Yeah. I mean, this is sort of a question about generalization, right, which is as the companies push their models and build better and better RL environments and scale that up, do you see a lot of generalization from these programming tasks and research tasks and hacking tasks to other kinds of sort of more general strategy, as well as maybe writing skill and interpersonal skills? I mean, don't know. Think anecdotally, it feels like to some degree, yes. When I use Opus 5.5, I'm like, wow, this model is, I think, better at everything, or most everything. I think there's some places where Fable model 5.1 will still be a little better model size difference, whatever, whatever. So I'm like, anecdotally, I feel like I see some generalization with every model, but sure is spiky. I think it's pretty spiky. And I think one of the things I'm surprised by is that the models are still not very funny or charismatic. Like, I was expecting to see more social impacts from models just being better at modeling humans and manipulating humans for better or for worse. And that's not what we see. We see models that are really agentic and good at orchestrating slap chorus songs, which is awesome. But I think if you look at where most of the humor and the persuasion is coming from, I still think that's mostly coming from humans modeling humans and steering the models to do that. So I think that if you're looking at something that's going to be a strategic threat or a strategic opportunity, depending on what your frame is here, you would need to see more generalization than we have so far. Or at least some path to greater capabilities. You might imagine that you get to the point where you have a recursively self improving agent collective or swarm, and they are like, Oh, we're really deficient in social skills. We're going to build a ton of environments where we can train on that. Now it's like, Well, is there verifiability there? And I don't Bring in a ton of human data, bring in a ton of humans. Just talk to billions of people. In fact, companies have access to that resource. And so even if you don't get extremely strong generalization, you could still have autonomous agents that are designing the next generation of agents and training the next generation of agents actually do that specialized training because they have unlocked that.