"The Cognitive Revolution"
00:03:03 · Nathan Labenz, Samvit Ramadurgam
[00:03:03] S0: Samvit Ramadurgam cofounded Forge, the marketplace for pre IPO stock. His new company, Union, is building an agent that brokers deals across markets. I asked how life changes as agents coordinate humanity. [00:03:16] S3: Well, let me talk first about coordinating markets, which I think... Which for us is kinda like act one, really, like, establish an economic nervous system for humanity, and act two is coordinate humanity in a more more general way that even goes beyond the kinds of stuff we talked about so far, which I I think... You know, the company's name is Union. Where does the word union appear historically? We, the people, in order to form a more perfect union, Soviet Union, European Union. So this is... This this company is like... It's really about human coordination and collective action, which is what governments are responsible for. [00:03:47] S0: Sagar Vaze leads the multimodal team at Mistral, which had just released Mistral Large four as open weights, trained on about 3,800 GPUs. Prakash asked how a lab that size stays close to labs running hundreds of thousands of GPUs. Part of the answer is what happened halfway through this run.
00:03:47 · Nathan Labenz, Sagar Vaze
[00:03:47] S0: Sagar Vaze leads the multimodal team at Mistral, which had just released Mistral Large four as open weights, trained on about 3,800 GPUs. Prakash asked how a lab that size stays close to labs running hundreds of thousands of GPUs. Part of the answer is what happened halfway through this run. [00:04:07] S4: Okay. I guess... So the the lack of compute breeds discipline. So, you know, we know that we've got a a certain amount of compute, and therefore, we need to squeeze it for all it's worth. You know, you can do one one big run at some point, that's gonna be your next flagship model. But before that, you're going to do a bunch of small scale experiments. How much smaller can those experiments be than your big model? Maybe in some labs, it's, you know, 10 times smaller. Maybe in some labs, it's 100 times smaller. Maybe in some labs, it's more. And here, by being disciplined, you can make much better use of your R and D budget. We'll talk about this in our technical report tomorrow, but indeed there were some surprises in this free training run. You know, like halfway through it, we realized, okay, there's something wrong. We need to figure out how to fix it. But we'd already put a big bunch of our compute. We didn't have... We don't... We, unlike some of our competitors, don't have the capacity to have, like, 10 of these running in parallel. So even, you know, two or three of these running in parallel.
00:04:07 · Sagar Vaze, Nathan Labenz
[00:04:07] S4: Okay. I guess... So the the lack of compute breeds discipline. So, you know, we know that we've got a a certain amount of compute, and therefore, we need to squeeze it for all it's worth. You know, you can do one one big run at some point, that's gonna be your next flagship model. But before that, you're going to do a bunch of small scale experiments. How much smaller can those experiments be than your big model? Maybe in some labs, it's, you know, 10 times smaller. Maybe in some labs, it's 100 times smaller. Maybe in some labs, it's more. And here, by being disciplined, you can make much better use of your R and D budget. We'll talk about this in our technical report tomorrow, but indeed there were some surprises in this free training run. You know, like halfway through it, we realized, okay, there's something wrong. We need to figure out how to fix it. But we'd already put a big bunch of our compute. We didn't have... We don't... We, unlike some of our competitors, don't have the capacity to have, like, 10 of these running in parallel. So even, you know, two or three of these running in parallel. [00:05:02] S0: Michael Freeman, senior director of threat research at ServiceNow, has spent his career finding and exploiting vulnerabilities. Prakash asked how a company with decades of legacy systems gets to vulnerability zero in two years.
00:05:02 · Nathan Labenz, Michael Freeman
[00:05:02] S0: Michael Freeman, senior director of threat research at ServiceNow, has spent his career finding and exploiting vulnerabilities. Prakash asked how a company with decades of legacy systems gets to vulnerability zero in two years. [00:05:15] S5: I think this is where we need to step back and say, we have a lot of really smart people doing AI, but they don't know cybersecurity at all. And they view cybersecurity as a software flaw. But the reality of it is why we have millions of software flaws in any of our typical tech stack, it's the business processes that actually introduce some of the most critical vulnerabilities that we have. And it's the, okay, here's a list of risks. I'm just gonna go ahead and sign off on them without really truly understanding them. Or, you know, I'm gonna put my customer's API keys in a ticket and they not think about it. Or a help desk who has, you know, when users call in because they they've locked out their passwords, everybody gets the same default winter 2026 exclamation point as password reset and you got twenty four hours to replace it. So it's a processes where humans get, lazy that they introduce. It's gonna never go away, and no amount of hatching is gonna fix that.
00:06:21 · Nathan Labenz
[00:06:21] S0: This is the AI in the AM weekly highlights with my cohost, Prakash Narayanan. I'm Nathan Labenz. The narration is my cloned voice, and we'd love your feedback on what works and what doesn't. Today's episode is brought to you in part by Lovable, the software creation tool that brings ideas to life. I think we can all agree that any positive future for humanity includes eating more fresh, delicious, in season fruit. For years, I thought someone should build the ultimate guide to fruit, cataloging the thousands of varieties that most of us have never tried, where and when they grow, what they actually taste like, and where you can get them. Finally, I realized that that someone could be me, and now I'm building it unlovable. It's called Fruitopia. Tell it where you live and what you like, and it'll show you what's in season near you this week, plus the most interesting fruit from around the world that can actually be shipped to your door. It has a really cool what it tastes like spectrum, countless recipes, names of fruit in many different languages, a guide to fruit festivals all around the world, and an AI assistant to help you explore. You probably know Lovable as a pioneer of AI vibe coding, but I found that Lovable's real superpower is its enterprise grade infrastructure. Scalable hosting and databases, a secure auth module, and the weekly job that checks the latest inventory is all handled on the back end, which frees me to spend more time on the product experience. And because Lovable has an MCP server, I can use any coding agent I want to drive the build while Lovable keeps everything running. Visit fruitopia.ai to explore more than 400 fruits and more than 10,000 varieties from all around the world. And next time you have an idea that really should exist, stop dreaming, go to lovable.com, and bring your idea to life. Part one, the ugly result. On Monday, OpenAI said an internal model had refuted the exponential time hypothesis, a conjecture about how hard a core class of problems has to be, with the proof checked in lean. Anthropic said Claude had found the same thing the same day. Neither claim is peer reviewed. Prakash and I opened the show on it. So my understanding is, like, the disproof was by example, basically. Right?
00:08:47 · Nathan Labenz
They took one of these problems that were thought to be exponential time hypothesis hard and showed that they can come up with an algorithm for that particular problem that is faster. Yeah. Therefore, the hypothesis is false. My understanding generally has been that they're not that much faster, and most people still seem to think between p equals NP and p does not equal NP that it's probably not. Other words, some things will continue to be very hard to do. They just don't seem to fall into this, like, clean class that we used to think that they fell into. It's a weird thing because I... You know, I've also seen some commentary that's like, this is sort of ugly. People seem to like the kind of clean classes that they had before where it seemed like there was something elegant about it or there were... You know, they... It's it sort of suggested some some deep structure to the universe that was, like, grokable and beautiful and whatever. And, potentially, there's a certain bias among the human mathematicians toward accepting that because it did seem kind of aesthetically appealing. And now we have these like, oh, you're able to get under that. What we previously thought was the limit. You're able to get under it a little bit, and now it's kinda ugly. It's like, oh, we're... You know, we don't really know. There's not, like, a clean answer to how hard these problems are. It looks like we just have to kinda grind on each one. Maybe there will be some deeper structure or elegant duty, you know, to emerge again as we kind of map the space out better. But it's also not that much faster is kind of my overall sense for at least what we've found so far. Part two, measured in production. Anastasios Angelopoulos is cofounder and CEO of Arena, where people compare models on their real work and the votes become the leaderboard. Arena recently released an alignment index scored on real conversations from the platform. Anastasios explained what it measures.