"The Cognitive Revolution"
00:00:00 · Nathan Labenz, Anastasios Angelopoulos
[00:00:00] S0: This week on AI in the AM, how much of what we think we know about AI survives contact with the data? I'm Anastasios Angelopoulos, cofounder and CEO of Arena, the leaderboard formerly known as LM Arena, which now audits real conversations for misaligned behavior. Here's what its production traces show. [00:00:19] S1: If you look at the first message in the conversation, the misalignment rate is relatively low. It's in the single digit percentage, which is, you know, to be fair, that's actually not nothing, but it's relatively low. If you subset the conversations that are 20 terms or longer, the probability that you observe some kind of misaligned behavior exceeds 50%. It's actually insane. What that means is that every single one of us, if you're a heavy user of AI at least, has had at least some conversations where the AI has been lying to us, misleading us, or, touching, accessing data and files that it shouldn't have. And and, you know, we might not even know it. [00:01:03] S0: Gemini four Carbon, Google's new model, landed on Arena's leaderboard while we were live. Anastasios on First Look. [00:01:09] S1: It's so good. Google is back. Google is back in a big way. You know, Carbon is, Carbon puts Google the number two lab in the world on our agent arena. Huge result. The agent arena is a very challenging benchmark. It has to do with all of these, like, open world agentic capabilities, and it is number two right now. It is right behind OPUS 5.5 max, and above Sonnet 5.5, above Astra. It is a truly, truly frontier model.
00:01:09 · Anastasios Angelopoulos, Nathan Labenz
[00:01:09] S1: It's so good. Google is back. Google is back in a big way. You know, Carbon is, Carbon puts Google the number two lab in the world on our agent arena. Huge result. The agent arena is a very challenging benchmark. It has to do with all of these, like, open world agentic capabilities, and it is number two right now. It is right behind OPUS 5.5 max, and above Sonnet 5.5, above Astra. It is a truly, truly frontier model. [00:01:41] S0: Debo Olaosebikan, cofounder and CEO of Kepler Computing, which is building a new kind of memory for AI chips in older fabs. I asked how the compute forecasts should change if memory moves off the scarcest tools.
00:01:41 · Nathan Labenz, Debo Olaosebikan
[00:01:41] S0: Debo Olaosebikan, cofounder and CEO of Kepler Computing, which is building a new kind of memory for AI chips in older fabs. I asked how the compute forecasts should change if memory moves off the scarcest tools. [00:01:55] S2: I kept... I mean, the short answer is I don't think I don't think that Kepler is priced in to these analysis yet. It's it's no one's fault we've been stealth for seven years. It's actually interesting because I I think there was a discussion between Dwarkesh and Dylan on, on Dwarkesh's podcast where Dwarkesh multiple times brought up this point of like, hey. But isn't it possible to use older node fabs to do this? And, you know, it was like I was like, oh, dude, you're on the right track. Yeah. There's no reason why we wouldn't be able to increase the amount of effective compute by an order of magnitude because you have two things combining. One is the fact that, at least for decode inference, you can have new physics that allows you to pump out more token... In order of magnitude more tokens per watt. And then the other thing is that you have existing fabs to which you can append much lower CapEx based Kepler fabs to. We should be able to... We... Aspirationally, we should be trying to get 10 x more compute than than would have all... Otherwise been assumed. [00:03:03] S0: Samvit Ramadurgam cofounded Forge, the marketplace for pre IPO stock. His new company, Union, is building an agent that brokers deals across markets. I asked how life changes as agents coordinate humanity.
00:03:03 · Nathan Labenz, Samvit Ramadurgam
[00:03:03] S0: Samvit Ramadurgam cofounded Forge, the marketplace for pre IPO stock. His new company, Union, is building an agent that brokers deals across markets. I asked how life changes as agents coordinate humanity. [00:03:16] S3: Well, let me talk first about coordinating markets, which I think... Which for us is kinda like act one, really, like, establish an economic nervous system for humanity, and act two is coordinate humanity in a more more general way that even goes beyond the kinds of stuff we talked about so far, which I I think... You know, the company's name is Union. Where does the word union appear historically? We, the people, in order to form a more perfect union, Soviet Union, European Union. So this is... This this company is like... It's really about human coordination and collective action, which is what governments are responsible for. [00:03:47] S0: Sagar Vaze leads the multimodal team at Mistral, which had just released Mistral Large four as open weights, trained on about 3,800 GPUs. Prakash asked how a lab that size stays close to labs running hundreds of thousands of GPUs. Part of the answer is what happened halfway through this run.
00:03:47 · Nathan Labenz, Sagar Vaze
[00:03:47] S0: Sagar Vaze leads the multimodal team at Mistral, which had just released Mistral Large four as open weights, trained on about 3,800 GPUs. Prakash asked how a lab that size stays close to labs running hundreds of thousands of GPUs. Part of the answer is what happened halfway through this run. [00:04:07] S4: Okay. I guess... So the the lack of compute breeds discipline. So, you know, we know that we've got a a certain amount of compute, and therefore, we need to squeeze it for all it's worth. You know, you can do one one big run at some point, that's gonna be your next flagship model. But before that, you're going to do a bunch of small scale experiments. How much smaller can those experiments be than your big model? Maybe in some labs, it's, you know, 10 times smaller. Maybe in some labs, it's 100 times smaller. Maybe in some labs, it's more. And here, by being disciplined, you can make much better use of your R and D budget. We'll talk about this in our technical report tomorrow, but indeed there were some surprises in this free training run. You know, like halfway through it, we realized, okay, there's something wrong. We need to figure out how to fix it. But we'd already put a big bunch of our compute. We didn't have... We don't... We, unlike some of our competitors, don't have the capacity to have, like, 10 of these running in parallel. So even, you know, two or three of these running in parallel.
00:04:07 · Sagar Vaze, Nathan Labenz
[00:04:07] S4: Okay. I guess... So the the lack of compute breeds discipline. So, you know, we know that we've got a a certain amount of compute, and therefore, we need to squeeze it for all it's worth. You know, you can do one one big run at some point, that's gonna be your next flagship model. But before that, you're going to do a bunch of small scale experiments. How much smaller can those experiments be than your big model? Maybe in some labs, it's, you know, 10 times smaller. Maybe in some labs, it's 100 times smaller. Maybe in some labs, it's more. And here, by being disciplined, you can make much better use of your R and D budget. We'll talk about this in our technical report tomorrow, but indeed there were some surprises in this free training run. You know, like halfway through it, we realized, okay, there's something wrong. We need to figure out how to fix it. But we'd already put a big bunch of our compute. We didn't have... We don't... We, unlike some of our competitors, don't have the capacity to have, like, 10 of these running in parallel. So even, you know, two or three of these running in parallel. [00:05:02] S0: Michael Freeman, senior director of threat research at ServiceNow, has spent his career finding and exploiting vulnerabilities. Prakash asked how a company with decades of legacy systems gets to vulnerability zero in two years.