"The Cognitive Revolution"
00:19:28 · Anastasios Angelopoulos
[00:19:28] S1: saw internally at Arena. At some point, we were looking at all our traces that are being done on our coding agents internally. By the way, we're in the top you know, less than 1% of business in terms of AI spend. We're spending thousands of dollars a month per employee on AI just to help us accelerate our workflow. That's why our sort of revenue to employee ratio is crazy off the charts because we're using a lot of agents internally to do our work. Now what happened is that we were looking through our traces, and we saw these traces, tons and tons of traces that were like, thank you, and then that costs you $40. What? Why is this happening? It's because somebody did a long running task, they let their agent go, it created some artifacts, blah blah blah, very long conversation. They left for a day, they came back, the cash was invalidated, and they're happy with the results. So they say thank you and enter, and that... That's a $40 charge. Terrible. Terrible. Terrible. We had... We're burning money. We're burning VC money. Right? We shouldn't do that. What we should do is we should figure out which tokens are best spent on frontier models, you know, which tokens are best spent on a cheaper model that can do the job, where we should be caching intelligently and, you know, avoiding some of these other types of cache hits, more intelligent caching behavior. And that's why we've been building the Arena router, we call MAX, short for argmax, Is it maximizing capabilities subject to cost constraints?
00:21:06 · Nathan Labenz, Anastasios Angelopoulos
[00:21:06] S0: What can you tell us about the new Gemini four Carbon? I think this has literally, come out and appeared on the Arena leaderboard while we've been live. [00:21:17] S1: And, you know, where it really shines, it shines on steerability. It's able to land user corrections, really effectively. It it shines on task completion. It shines on user preference. Users love working with this model. It's great at three d design. It's great at general agentic work with documents. Congrats to Google. This is just like, what a big comeback. I think that, leadership must be really happy.
00:21:45 · Nathan Labenz
[00:21:45] S0: Hey. We'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Anthropic, makers of Claude. To show off Claude's latest capabilities, I asked Claude to take lead in making this ad. Given a short creative brief, Claude, using Opus 5.5, studied my past Claude ads, wrote this script, directed a clone of my voice, and designed and coded the animation you're watching. My job? Give feedback and decide. Claude is the AI for problem solvers. It's the collaborator that understands your entire workflow and thinks with you, not for you. Whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. And with Claude Code Routines, it keeps working in the cloud even after you close your laptop. For problems worth solving, get started with Claude at claude.aitcr. That's claude.aitcr. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Claude.aitcr. This episode of the Cognitive Revolution is brought to you by OutSystems, the leading agentic systems platform. OutSystems is helping their customers build, modernize, and operate enterprise systems starting from any coding tool in a governed agentic engineering model. OutSystems, you can coordinate and govern your entire agentic workforce within your enterprise ecosystem, accelerate impact with industry proven agentic solutions that are prebuilt, governed, and customizable to your business. It's time to innovate at the speed of AI without compromising quality or control, which is why thousands of enterprises worldwide trust OutSystems for their mission critical apps. Teams any size can use OutSystems to build, deploy, and manage AI apps and agents quickly and cost effectively without compromising reliability and security. With OutSystems, you can accelerate ideas from concept to completion. It's the leading agentic systems platform that is unified, agile, and enterprise proven, allowing you to build your agentic future with AI solutions deeply integrated into your architecture. OutSystems, all your agentic future. Learn more at outsystems.com/tcr. That's outsystems.com/tcr. Part three, an orthogonal axis.
00:24:28 · Nathan Labenz
Debo Olaosebikan is co founder and CEO of Kepler Computing, which came out of stealth in September with a new kind of memory for AI chips, a ferroelectric capacitor stacked on top of each transistor made in older fabs. He started with where the energy goes.
00:24:48 · Debo Olaosebikan
[00:24:48] S2: So then I looked at the existing chips, mainstream ones, you know, the the GPUs or the TPUs, and I say, where's the energy going? And I I I think many people know this, but a lot of people don't know that for inference especially, the bulk of that energy is actually running across the street to fetch the data from the library, you know, before you before you process it and do your fancy matrix multiplications. So if I wanna improve that system for inference by an order of magnitude at the physics level, it's obvious that the first thing I need to go do is address the memory bottleneck. And the one way you can do that is to bring that memory closer to the processor. And the other thing you want to do is increase the amount of memory that you can store within a certain footprint. Both of these things point us to going three d. And then within the chip, how do you stack devices on top of this transistor to pack more bits per millimeter square? Within the chip, use new materials. We use a class of materials called ferroelectric materials. Ferroelectrics have actually been around for hundred plus years. But the specific type of material, the specific, you know, recipe of the cake, if you will, this is this is this is, you know, proprietary to to Kepler. And you basically use that ferroelectric to make a capacitor, a type of... A capacitor like thing. It's really holding a polarization for for people who are, you know, more technical. But roughly speaking, you can say that this capacitor is storing a one or a zero. And if I replaced six transistors with one transistor, one capacitor sitting on top of that transistor, and then eventually put multiple capacitors on top of each other, I can avoid the the rat race of, like, trying to shrink transistors within a chip. So we have technology, roughly speaking, that's saying, how do you go three d inside a chip? How do you go three d between chips? And how do you use those technologies to increase memory capacity, but also bring memory and compute much closer together while avoiding, you know, conventional, like, you know, issues with with thermals and and and and things like this.
00:27:02 · Nathan Labenz
[00:27:02] S0: Prakash asked about manufacturing. Do new materials need a new process? And what does it buy you not to depend on extreme ultraviolet lithography, the EUV tools everyone is short of?