00:08:44 · Jeffrey Ladish
[00:08:44] S0: Yeah. I mean, I think overall, having a lower, like, deployment gap is better. Right? I would much prefer there to be not... Like, the smaller the gap, the better. I think that's right. Yeah. Mean, another way to sort of rephrase what you guys are saying is like, yep, it's still really, really fast. And the speed up is sort of maybe should already have been priced in. And I'm like, yeah, maybe. But can we look at how insanely fast it is? To me, if you look over the last six months, some people already had this priced into their models. I do think that AI futures projects should get a lot of credit for being insanely bullish on just how fast this would go and Anthropic as well. Sorry, credit here is a little weird for Anthropic because I'm like, they're the ones causing it to happen. And I think imposing massive risks on the rest of the world and society, which I wish they wouldn't. Yeah, mean, so to me, I'm like, wow, it seems like we might be only a year away from AI agents, which could take over all of the digital infrastructure in the world. I don't know. I don't know if you want to talk about threat models here, because I'm basically like, yeah, yeah, so there's a whole bunch of implications for AI development more broadly, but to me, the most important implications are how far are we away from AI agents that we would lose control of entirely? And I think to me, I'm like looking at the last few months, including this recent report, but also like you said, I'm like, yeah, yeah, I think I did have a lot of this priced in. How far are we away from AI agents, which if they wanted to, could take over at least the digital world? And then there's a question of whether they could take over the physical world as well. But yeah, I think we just might not be that far from that, if this is correct, if Anthropic is correct about the capability of their models.
00:10:36 · Ruby Justice Thelot, Jeffrey Ladish
[00:10:36] S1: Right. So, like, one... You know, I I was trying to find, you know, posts that are, like, critical about this finding from smart people. One was from Jessica Cat, who I believe used to work at MIRI. And she said, the main evidence here is Anthropic's internal version of the Epoch Capabilities Index, which is AECI, the Anthropic Epoch Capabilities Index. Yeah. But the the trend line on the the Epoch version of ECI remains pretty linear. Like, there hasn't been much of a speed up at all since, like, the creation of reasoning models in, like, 2024. [00:11:10] S0: Yeah. [00:11:10] S1: And given that the the the trend line of the internal models and the public models on the AECI is quite similar, shouldn't that imply that if these internal models are graded on the Epoch version of the ECI, the trend would look more linear than what METR found?
00:11:10 · Ruby Justice Thelot, Jeffrey Ladish
[00:11:10] S1: And given that the the the trend line of the internal models and the public models on the AECI is quite similar, shouldn't that imply that if these internal models are graded on the Epoch version of the ECI, the trend would look more linear than what METR found? [00:11:30] S0: Okay. I I think this is a good point, but it... But I feel like the the emphasis here is on, like, METR is claiming that they found a bunch of evidence of this speed up. But they are also claiming that they had to redact the best evidence that they found for the speedup. So now we are commenting on the stuff they were able to report that was not redacted. But METR is claiming this is weaker evidence than what we actually see. And I'm like, look, maybe they're wrong, but I think that the pressure now should be on Anthropic to actually release that information. Because I'm like, yeah, yeah, I think you're pointing out, Jessica Taylor is pointing out some good points, but I'm like, yeah, yeah, can we have the information that we would actually need to assess this claim? That to me is the important takeaway. Mean, sorry, to be clear, I really appreciate people who are trying to verify whether this claim is true or not using the publicly available evidence that we do have. For sure, we should do that. But I'm like, come on. This is really important. Like, where's the information?
00:11:30 · Jeffrey Ladish, Ruby Justice Thelot
[00:11:30] S0: Okay. I I think this is a good point, but it... But I feel like the the emphasis here is on, like, METR is claiming that they found a bunch of evidence of this speed up. But they are also claiming that they had to redact the best evidence that they found for the speedup. So now we are commenting on the stuff they were able to report that was not redacted. But METR is claiming this is weaker evidence than what we actually see. And I'm like, look, maybe they're wrong, but I think that the pressure now should be on Anthropic to actually release that information. Because I'm like, yeah, yeah, I think you're pointing out, Jessica Taylor is pointing out some good points, but I'm like, yeah, yeah, can we have the information that we would actually need to assess this claim? That to me is the important takeaway. Mean, sorry, to be clear, I really appreciate people who are trying to verify whether this claim is true or not using the publicly available evidence that we do have. For sure, we should do that. But I'm like, come on. This is really important. Like, where's the information? [00:12:38] S2: So, like, if it's true that the acceleration wasn't driven in just, like, scaling compute, then would the, like, pacing mechanism of just allocating, like, certain amounts of compute to inference and, like, compute based pacing mechanisms still hold, or is there, like, [00:12:55] S0: a new [00:12:56] S2: way that you think we should, think about slowing down or pacing?
00:13:02 · Jeffrey Ladish
[00:13:02] S0: Yeah. Did you did you see Ryan Greenblatt's tweet about, sort of like what this meter report implies? I can no longer say the meter report, because there's What a lot of meter it implies about where the speed up is coming from, right? So overall AI progress is driven by increases in compute, algorithmic efficiency, better data, probably other things too. And obviously just having better AI systems right now doesn't mean that you will automatically have more compute. It doesn't speed up the compute acquisition part very much, maybe a little bit, but not very much. Whereas it probably improves algorithmic efficiency a bunch. And so I think he was estimating something like a 4.5x speed up in the algorithms and software part of the uplift. And so if this is true, it would imply that both internal restrictions on internal use of really powerful models, as well as compute reallocation, would I think both be pretty powerful mechanisms for pacing. I think there's also this question of like... Yeah, I don't know. There's also sort of this question of how much do you need to pace? In my opinion, if Anthropic is correct that we are quite close to RSI, as in it's next year, then I think the level of pacing should be like, man, slam on the brakes. You really need to actually stop here for a while and really try to do the interpretability research you'd need to understand your model's drives and motivations to actually come up with a safety case where you could say, We're quite confident we can control them going forward. And right now, I think they can't do that. Some evidence for that is you looked at the recent write ups from Anthropic about some of the rogue AI incidents that they had, one with the UKAC, another one with irregular, they did a bunch of analysis, like why did cloud do these things? Like why did these cloud agents do these things? And some of it was motivated reasoning, there's a bunch of stuff. But most of the way that they were able to conclude these things was by reading the chain of thought from the models. They also tried to use interpretability tools to investigate the agent's motivations. But the interpretability tools were very ineffective compared to just using the chain of thought. And so I'm like, well, hey, I'm really glad that we have the chain of thought. It still sucks. It's still not very faithful.
00:15:16 · Jeffrey Ladish
We're losing monitorability. Look at Astra, open AI model, but look at Astra. We learned yesterday that Astra will sometimes basically sandbag on safety research. Probably not strategically, we don't really know. But sometimes Astra will underperform on safety research and it doesn't show up in the chain of thought for the most part. I'm like, man, so like, you know, of thought, monitorability, like, okay, like we have some problems with chain of thought, but at least we have it for now. And I'm like, man, that's our best tool and our best tool sucks. And then like, what we actually need my mind, and I think Anthropic's mind too, is really good interpretability tools that would allow us to actually understand what's happening under the hood. And they're trying and I'm like, fuck, fuck, yeah. Let's give them the opportunity to try. Think that means they would need a lot more time in order to develop those tools. And so this is why I would favor something like really aggressive reallocation, where both you can then have a bunch more time, serve a bunch more customers, and also just apply a lot more resources to trying to crack the interpretability problems. And and actually be able to get to the point where you understand the model's drives and motivations so you can make a really, really robust safety case for how you could scale this up to a country of geniuses in a data center.