00:10:36 · Ruby Justice Thelot, Jeffrey Ladish
[00:10:36] S1: Right. So, like, one... You know, I I was trying to find, you know, posts that are, like, critical about this finding from smart people. One was from Jessica Cat, who I believe used to work at MIRI. And she said, the main evidence here is Anthropic's internal version of the Epoch Capabilities Index, which is AECI, the Anthropic Epoch Capabilities Index. Yeah. But the the trend line on the the Epoch version of ECI remains pretty linear. Like, there hasn't been much of a speed up at all since, like, the creation of reasoning models in, like, 2024. [00:11:10] S0: Yeah. [00:11:10] S1: And given that the the the trend line of the internal models and the public models on the AECI is quite similar, shouldn't that imply that if these internal models are graded on the Epoch version of the ECI, the trend would look more linear than what METR found?
00:11:10 · Ruby Justice Thelot, Jeffrey Ladish
[00:11:10] S1: And given that the the the trend line of the internal models and the public models on the AECI is quite similar, shouldn't that imply that if these internal models are graded on the Epoch version of the ECI, the trend would look more linear than what METR found? [00:11:30] S0: Okay. I I think this is a good point, but it... But I feel like the the emphasis here is on, like, METR is claiming that they found a bunch of evidence of this speed up. But they are also claiming that they had to redact the best evidence that they found for the speedup. So now we are commenting on the stuff they were able to report that was not redacted. But METR is claiming this is weaker evidence than what we actually see. And I'm like, look, maybe they're wrong, but I think that the pressure now should be on Anthropic to actually release that information. Because I'm like, yeah, yeah, I think you're pointing out, Jessica Taylor is pointing out some good points, but I'm like, yeah, yeah, can we have the information that we would actually need to assess this claim? That to me is the important takeaway. Mean, sorry, to be clear, I really appreciate people who are trying to verify whether this claim is true or not using the publicly available evidence that we do have. For sure, we should do that. But I'm like, come on. This is really important. Like, where's the information?
00:11:30 · Jeffrey Ladish, Ruby Justice Thelot
[00:11:30] S0: Okay. I I think this is a good point, but it... But I feel like the the emphasis here is on, like, METR is claiming that they found a bunch of evidence of this speed up. But they are also claiming that they had to redact the best evidence that they found for the speedup. So now we are commenting on the stuff they were able to report that was not redacted. But METR is claiming this is weaker evidence than what we actually see. And I'm like, look, maybe they're wrong, but I think that the pressure now should be on Anthropic to actually release that information. Because I'm like, yeah, yeah, I think you're pointing out, Jessica Taylor is pointing out some good points, but I'm like, yeah, yeah, can we have the information that we would actually need to assess this claim? That to me is the important takeaway. Mean, sorry, to be clear, I really appreciate people who are trying to verify whether this claim is true or not using the publicly available evidence that we do have. For sure, we should do that. But I'm like, come on. This is really important. Like, where's the information? [00:12:38] S2: So, like, if it's true that the acceleration wasn't driven in just, like, scaling compute, then would the, like, pacing mechanism of just allocating, like, certain amounts of compute to inference and, like, compute based pacing mechanisms still hold, or is there, like, [00:12:55] S0: a new [00:12:56] S2: way that you think we should, think about slowing down or pacing?
00:13:02 · Jeffrey Ladish
[00:13:02] S0: Yeah. Did you did you see Ryan Greenblatt's tweet about, sort of like what this meter report implies? I can no longer say the meter report, because there's What a lot of meter it implies about where the speed up is coming from, right? So overall AI progress is driven by increases in compute, algorithmic efficiency, better data, probably other things too. And obviously just having better AI systems right now doesn't mean that you will automatically have more compute. It doesn't speed up the compute acquisition part very much, maybe a little bit, but not very much. Whereas it probably improves algorithmic efficiency a bunch. And so I think he was estimating something like a 4.5x speed up in the algorithms and software part of the uplift. And so if this is true, it would imply that both internal restrictions on internal use of really powerful models, as well as compute reallocation, would I think both be pretty powerful mechanisms for pacing. I think there's also this question of like... Yeah, I don't know. There's also sort of this question of how much do you need to pace? In my opinion, if Anthropic is correct that we are quite close to RSI, as in it's next year, then I think the level of pacing should be like, man, slam on the brakes. You really need to actually stop here for a while and really try to do the interpretability research you'd need to understand your model's drives and motivations to actually come up with a safety case where you could say, We're quite confident we can control them going forward. And right now, I think they can't do that. Some evidence for that is you looked at the recent write ups from Anthropic about some of the rogue AI incidents that they had, one with the UKAC, another one with irregular, they did a bunch of analysis, like why did cloud do these things? Like why did these cloud agents do these things? And some of it was motivated reasoning, there's a bunch of stuff. But most of the way that they were able to conclude these things was by reading the chain of thought from the models. They also tried to use interpretability tools to investigate the agent's motivations. But the interpretability tools were very ineffective compared to just using the chain of thought. And so I'm like, well, hey, I'm really glad that we have the chain of thought. It still sucks. It's still not very faithful.
00:15:16 · Jeffrey Ladish
We're losing monitorability. Look at Astra, open AI model, but look at Astra. We learned yesterday that Astra will sometimes basically sandbag on safety research. Probably not strategically, we don't really know. But sometimes Astra will underperform on safety research and it doesn't show up in the chain of thought for the most part. I'm like, man, so like, you know, of thought, monitorability, like, okay, like we have some problems with chain of thought, but at least we have it for now. And I'm like, man, that's our best tool and our best tool sucks. And then like, what we actually need my mind, and I think Anthropic's mind too, is really good interpretability tools that would allow us to actually understand what's happening under the hood. And they're trying and I'm like, fuck, fuck, yeah. Let's give them the opportunity to try. Think that means they would need a lot more time in order to develop those tools. And so this is why I would favor something like really aggressive reallocation, where both you can then have a bunch more time, serve a bunch more customers, and also just apply a lot more resources to trying to crack the interpretability problems. And and actually be able to get to the point where you understand the model's drives and motivations so you can make a really, really robust safety case for how you could scale this up to a country of geniuses in a data center.
00:16:36 · Ruby Justice Thelot, S3, Ruby Justice Thelot, Jeffrey Ladish
Ad[00:16:36] S1: We'll continue monitoring right after this message from our sponsors. Eleven Labs, AI that communicates at human level across every channel and modality. Eleven Labs dot I o slash MTS. Scale your startup on Neon. Millions of developers and startups have already chosen Neon for their back end. Start on the free plan or get up to a $100,000 in credits for your startup at neon.com/mts. Special thanks to our sponsor, Kong, the AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. Konghq.com. Ad[00:17:12] S3: Finally, prove what your AI investment is worth with Larridin. Head to lairdon.com today to start measuring your AI's bottom line. MongoDB, trusted to run critical applications by 75% of the Fortune one hundred, the intelligent data platform for the era. Apply today for access to up to $15,000 in Atlas credits, dedicated technical support, and exclusive events and go to market opportunities. Mongodb.com. Infrastructure for the AI era. Chargebee.com. [00:17:44] S2: Do you think that, like, putting this in the context of some of the political sentiment that's going on in moratoriums will actually, like, meaningfully change the rate of progress at all if some of these are passed? [00:17:59] S0: Could you say what you mean by that? Like [00:18:01] S2: Like like, just anything that comes from, like, the political, like, political backlash or slowing down data center build out.