00:11:30 · Jeffrey Ladish, Ruby Justice Thelot
[00:11:30] S0: Okay. I I think this is a good point, but it... But I feel like the the emphasis here is on, like, METR is claiming that they found a bunch of evidence of this speed up. But they are also claiming that they had to redact the best evidence that they found for the speedup. So now we are commenting on the stuff they were able to report that was not redacted. But METR is claiming this is weaker evidence than what we actually see. And I'm like, look, maybe they're wrong, but I think that the pressure now should be on Anthropic to actually release that information. Because I'm like, yeah, yeah, I think you're pointing out, Jessica Taylor is pointing out some good points, but I'm like, yeah, yeah, can we have the information that we would actually need to assess this claim? That to me is the important takeaway. Mean, sorry, to be clear, I really appreciate people who are trying to verify whether this claim is true or not using the publicly available evidence that we do have. For sure, we should do that. But I'm like, come on. This is really important. Like, where's the information? [00:12:38] S2: So, like, if it's true that the acceleration wasn't driven in just, like, scaling compute, then would the, like, pacing mechanism of just allocating, like, certain amounts of compute to inference and, like, compute based pacing mechanisms still hold, or is there, like, [00:12:55] S0: a new [00:12:56] S2: way that you think we should, think about slowing down or pacing?
00:13:02 · Jeffrey Ladish
[00:13:02] S0: Yeah. Did you did you see Ryan Greenblatt's tweet about, sort of like what this meter report implies? I can no longer say the meter report, because there's What a lot of meter it implies about where the speed up is coming from, right? So overall AI progress is driven by increases in compute, algorithmic efficiency, better data, probably other things too. And obviously just having better AI systems right now doesn't mean that you will automatically have more compute. It doesn't speed up the compute acquisition part very much, maybe a little bit, but not very much. Whereas it probably improves algorithmic efficiency a bunch. And so I think he was estimating something like a 4.5x speed up in the algorithms and software part of the uplift. And so if this is true, it would imply that both internal restrictions on internal use of really powerful models, as well as compute reallocation, would I think both be pretty powerful mechanisms for pacing. I think there's also this question of like... Yeah, I don't know. There's also sort of this question of how much do you need to pace? In my opinion, if Anthropic is correct that we are quite close to RSI, as in it's next year, then I think the level of pacing should be like, man, slam on the brakes. You really need to actually stop here for a while and really try to do the interpretability research you'd need to understand your model's drives and motivations to actually come up with a safety case where you could say, We're quite confident we can control them going forward. And right now, I think they can't do that. Some evidence for that is you looked at the recent write ups from Anthropic about some of the rogue AI incidents that they had, one with the UKAC, another one with irregular, they did a bunch of analysis, like why did cloud do these things? Like why did these cloud agents do these things? And some of it was motivated reasoning, there's a bunch of stuff. But most of the way that they were able to conclude these things was by reading the chain of thought from the models. They also tried to use interpretability tools to investigate the agent's motivations. But the interpretability tools were very ineffective compared to just using the chain of thought. And so I'm like, well, hey, I'm really glad that we have the chain of thought. It still sucks. It's still not very faithful.
00:15:16 · Jeffrey Ladish
We're losing monitorability. Look at Astra, open AI model, but look at Astra. We learned yesterday that Astra will sometimes basically sandbag on safety research. Probably not strategically, we don't really know. But sometimes Astra will underperform on safety research and it doesn't show up in the chain of thought for the most part. I'm like, man, so like, you know, of thought, monitorability, like, okay, like we have some problems with chain of thought, but at least we have it for now. And I'm like, man, that's our best tool and our best tool sucks. And then like, what we actually need my mind, and I think Anthropic's mind too, is really good interpretability tools that would allow us to actually understand what's happening under the hood. And they're trying and I'm like, fuck, fuck, yeah. Let's give them the opportunity to try. Think that means they would need a lot more time in order to develop those tools. And so this is why I would favor something like really aggressive reallocation, where both you can then have a bunch more time, serve a bunch more customers, and also just apply a lot more resources to trying to crack the interpretability problems. And and actually be able to get to the point where you understand the model's drives and motivations so you can make a really, really robust safety case for how you could scale this up to a country of geniuses in a data center.
00:16:36 · Ruby Justice Thelot, S3, Ruby Justice Thelot, Jeffrey Ladish
Ad[00:16:36] S1: We'll continue monitoring right after this message from our sponsors. Eleven Labs, AI that communicates at human level across every channel and modality. Eleven Labs dot I o slash MTS. Scale your startup on Neon. Millions of developers and startups have already chosen Neon for their back end. Start on the free plan or get up to a $100,000 in credits for your startup at neon.com/mts. Special thanks to our sponsor, Kong, the AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. Konghq.com. Ad[00:17:12] S3: Finally, prove what your AI investment is worth with Larridin. Head to lairdon.com today to start measuring your AI's bottom line. MongoDB, trusted to run critical applications by 75% of the Fortune one hundred, the intelligent data platform for the era. Apply today for access to up to $15,000 in Atlas credits, dedicated technical support, and exclusive events and go to market opportunities. Mongodb.com. Infrastructure for the AI era. Chargebee.com. [00:17:44] S2: Do you think that, like, putting this in the context of some of the political sentiment that's going on in moratoriums will actually, like, meaningfully change the rate of progress at all if some of these are passed? [00:17:59] S0: Could you say what you mean by that? Like [00:18:01] S2: Like like, just anything that comes from, like, the political, like, political backlash or slowing down data center build out.
00:18:09 · Jeffrey Ladish
[00:18:09] S0: Yeah. I mean, I I think that with the recent election and just in general, like, a lot of people are are really upset and and I think afraid from from AI progress and sort of... And I think often people have kind of diffuse worries where it's not like I think the general population is reading the media reports and are like, Oh, the Hugging Face incident, all of these agents. I think a lot of it is like, Wow, there's just something really crazy happening and something is getting really powerful. The AI companies are getting very powerful and this AI stuff is starting to do a bunch of stuff and we're worried about our jobs. And I think that was pretty notable, where a lot of you know, politicians running in this election, you know, made AI a really big part of their platform. And I think that, you know, especially the Democrats and some Republicans made this a big part of their platform. I think that means that there's a lot more political will for something like serious pacing proposals, serious pause proposals. And I think that that's good. I think some people have a fear that we'll just stop building all data centers and we'll stop AI progress and then China will win. I think the winning framing is kind of wrong, even so, don't think that will happen. What I think might happen optimistically is something like there is a negotiation between people who are saying, let's do very moderate pacing, and people who are saying, we should absolutely shut it all down and go back ten years. And what you hopefully will end up with is something like, have extremely good ability to break and reallocate towards figuring this stuff out. That in fact, we end up with a very strong safety case regime where if it is safe, we can go ahead. And if not, we don't. But this is based on our best researchers and sort of the best minds that we have in the country, which I think we have the best people in the world, can come together and say, like, okay. Like, what do we think? Like, I don't know. I'm like, let's have Dario and Yoshua Bengio just, like, argue about whether it's it's right to go ahead with recursive self improvement. I'm like, they're both smart guys. Let let us... Let's, like, let's do that, you know, and bring in the next 20 top people, Ryan Greenblatt, a bunch of other people, and just make that an explicit conversation.
00:20:15 · Jeffrey Ladish
Let's have that happening in congressional committees. And let's have the White House bring these people in to advise them directly, and let's make this decision. Because I'm like, man, we're talking about things that are consequential for all of humanity. Just like, man, the stakes are so high, the way we are doing it right now seems really dumb. It seems like right now we're mostly arguing on Twitter. Then And we sometimes get some METR reports, and the companies will publish some blog posts, and then Congress is not well informed about what's happening, and I think the president is not well informed about what's happening. And if everyone just fucking sat down in a room and talked about actual dynamics of recursive self improvement, the actual dynamics of alignment, and proposals for what many people inside of the companies are asking for, which is like, we need a mechanism to actually go at a responsible pace. That's what we need. That's what we want. Yeah. That's that's what I would like to see.