Lenny's Podcast: Product | Career | Growth
00:56:26 · Boris Cherny, Lenny Rachitsky
And this is really the only way to make sure that over the long-term the model is aligned, and it's doing the right things. [00:56:33] S1: It's such a wild space that you work in where there's this insane competition and pace. At the same time, there's this fear that if the God can escape and cause damage, and just finding that balance must be so challenging. What I'm hearing is there's these three layers, and I know there's ... This could be a whole podcast conversation is how you all think about the safety piece, but just what I'm hearing is there's these three layers you work with. There's observing the model thinking and operating. There's tests, evals that tell you this is doing bad things. And then releasing it early. I haven't actually heard a ton about that first piece. That is so cool. So, you guys can ... There's an observability tool that can let you peak inside the model's brain and see how it's thinking and where it's heading.
00:57:16 · Boris Cherny
[00:57:16] S0: Yeah. You should, at some point, have Chris Olah on the podcast, because he's just the industry expert on this. He invented this field of we call it mechanistic interpretability. And the idea is at its core what is your brand? What is it? It's a bunch of neurons that are connected. And so, what you can do is in a human brain or in an animal brain, you can study it at this mechanistic level to understand what the neurons are doing. It turns out surprisingly a lot of this does translate to models also. So, model neurons are not the same as animal neurons, but they behave similarly in a lot of ways. And so, we've been able to learn just a ton about the way these neurons work, about this layer or this neuron maps to this concept, how particular concepts are encoded, how the model does planning, how it thinks ahead. A long time ago we weren't sure if the model was just predicting the next token, or is doing something a little bit deeper. Now I think there's actually quite strong evidence that it is doing something a little bit deeper. And then the structures that let it do this are pretty sophisticated now where as the models get bigger it's not just a single neuron that corresponds to a concept. A single neuron might correspond to a dozen concepts, and if it's activated together with other neurons this is called super position. And together it represents this more sophisticated concept. And it's just something we're learning about all the time. And for Anthropic as we think about the way this space evolves doing this in a way that is safe and good for the world is just this is the reason that we exist, and this is the reason that everyone is at Anthropic. Everyone that is here, this is the reason why they're here. So, a lot of this work we actually open source. We publish it a lot. And we publish very freely to talk about this just so we can inspire other labs that are working on similar things to do it in a way that's safe. And this is something we've been doing for Claude Code also. We call this, "The race to the top" internally. And so, for Claude Code, for example, we released an open source sandbox. And this is a sandbox they can run the agent in, and just make sure that there's certain boundaries and they can't access, like, everything on your system.
00:59:20 · Boris Cherny, Lenny Rachitsky
And we made that open source, and it actually works with any agent, not just Claude Code, because we wanted to make it really easy for others to do the same thing. So, this is just the same principle of race to the top. We want to make sure this thing goes well, and this is the lever that we have. [00:59:37] S1: Incredible. Okay. I definitely want to spend more time on that. I will follow up with this suggestion. Something else that I've been noticing in the field across engineers and product managers, others that work with agents is there's this anxiety people feel when their agents aren't working. There's a sense that [inaudible 00:59:57] has a question [inaudible 00:59:58] answer, or it's, like, blocked on something, or, "There's all this productivity I'm losing. I need to wake up and get it going again." Is that something you feel? Is that something your team feels? Do you feel like this is a problem we need to track and think about?
01:00:11 · Boris Cherny, Lenny Rachitsky
[01:00:11] S0: I always have a bunch of agents running. So, at the moment I have five agents running. And, at any moment, I wake up and I start a bunch of agents. Like, the first thing I did when I woke up was like, "Oh, man. I really want to check this thing." So, I opened up my phone Claude iOS app code tab." Like, agent, do blah, blah, blah. Because I wrote some code yesterday and I was like, "Wait. Did I do this right?" I was double guessing something, and it was correct. But now it's just so easy to do this. So, I don't know. There is this little bit of anxiety maybe. I, personally, haven't really felt it just because I have agents running all the time. And I'm also just not locked into terminal anymore. Maybe a third of my code now is in the terminal, but also a third is using the desktop app. And then a third is the iOS app, which is just so surprising, because I did not think that this would be the way that I could even in 2026. [01:01:03] S1: I love that you describe it as coding still, which is just talking to Claude Code to code for you, essentially. And it's interesting that this is now coding. Coding now is describing what you want, not writing actual code. [01:01:16] S0: I wonder if the people that used to code using punch cards, or whatever, if you showed them software what they would have said. [01:01:22] S1: Isn't that great? Yeah.
01:01:22 · Lenny Rachitsky, Boris Cherny
[01:01:22] S1: Isn't that great? Yeah. [01:01:24] S0: I remember reading something, this was maybe very early versions of ACM magazine, or something, where people were saying, "No. It's not the same thing." Like, "This isn't really coding." And they called it a programming. I think coding is a new word. But I think about this. Back in the ... My family is from the Soviet Union. I was born in Ukraine. And my grandpa was actually one of the first programmers in the Soviet Union, and he programmed using punch cards. My mom growing up told these stories of ... Or she told these stories of when she was growing up, he would bring these punch cards home. And there was these big stacks of punch cards. And for her she would draw all over them with crayons, and that was her childhood memory. But for him that was, like, his experience of programming, and he actually never saw the software transition. But, at some point, it did transition to software. And I think there was probably this older generation of programmers that just didn't take software very seriously. And they would have been like, "Well, it's not really coding." But I think this is a field that just has always been changing in this way. [01:02:27] S1: I don't think you know this, but I was born in Ukraine also. [01:02:30] S0: Oh, I don't [inaudible 01:02:31]. Yeah. [01:02:31] S1: Yes. [01:02:32] S0: Which town? [01:02:32] S1: I'm from Odessa. [01:02:34] S0: Oh, me too. [01:02:35] S1: What? [01:02:36] S0: Yeah. That's crazy. [01:02:39] S1: Wow. Incredible. What a moment. Maybe related in some small way.
01:02:39 · Lenny Rachitsky, Boris Cherny
[01:02:39] S1: Wow. Incredible. What a moment. Maybe related in some small way. [01:02:44] S0: Yeah. [01:02:44] S1: What year did you leave? And your family leave. [01:02:48] S0: We came in '95. [01:02:50] S1: Okay. We left in '88. A little earlier. Yeah. What a different life that would have been to not leave home. [01:02:57] S0: Yeah. I feel so lucky every day that I got to grow up here. [01:03:02] S1: Yeah. My family, any time there's a toast or a meal, they're just like, "To America." [inaudible 01:03:07]. It's, like, "Okay. Enough about that. We get it." Once you start really thinking about what life could have been. [01:03:12] S0: Yeah. Yeah. Exactly. Yeah. We do the same toast, but it's still vodka. [01:03:16] S1: It's still vodka. Absolut. Oh, man. Okay. Let me ask you a couple more things here. You shared some really cool tips for how to get the most out of AI, how to build on AI, how to build great products in AI. One tip you shared is give your team as many tokens as they want. Just let them experiment. You also shared just advice, generally, of just build towards where the model is going, not to where it is today. What other advice do you have for folks that are trying to build AI products?