Lenny's Podcast: Product | Career | Growth
00:22:25 · Boris Cherny
[00:22:25] S0: This is something I have to remind myself once in a while. There's a downside of this, because the model changes so ... Well, there's many downsides that we could talk about, but I think one of them on a personal level is the model changes so often that I sometimes get stuck in this old way of thinking about it. And I even find that new people on the team, or even new grads that join do stuff in a more AGI-forward way than I do. So, sometimes, for example, I had this case a couple of months ago where there was a [inaudible 00:22:57] week. And so, what this is is Claude Code, the memory usage is going up, and, at some point, it crashes. This is a very common engineering problem that every engineer has debugged 1000 times. And, traditionally, the way that you do it is you take a heap snapshot, you put it into a special debugger, and figure out what's going on. You use these special tools to see what's happening. And I was doing this, and I was looking through these traces, and trying to figure out what was going on, and the engineer that was newer on the team just had Claude Code do it. And was like, "Hey, Claude. It sounds like there's a leak. Can you figure it out?" And so, Claude Code did exactly the same thing that I was doing. It took the heap snapshot, it wrote a little tool for itself, so, it can analyze it itself. It was a just in time program. And it found the issue, and put up a pull request faster than I could. So, it's something where for those of us that have been using the model for a long time, you still have to transport yourself to the current moment, and not get stuck back in old model, because it's not Sonnet 3.5 anymore. The new models are just completely, completely different. And just this mindset shift is very different.
00:24:03 · Lenny Rachitsky
[00:24:03] S1: I hear you have these very specific principles that you've codified for your team that when people join you walk them through them. I believe one of them is, "What's better than doing something? Having Claude do it." And it feels like that's exactly what you describe with this memory leak is you almost forgot that principle of like, "Okay. Let me see if Claude can solve this for me."
00:24:21 · Boris Cherny
[00:24:21] S0: There's an interesting thing that happens also when you under-fund everything a little bit, because then people are forced to Claude-ify. And this is something that we see so ... For work, where sometimes we just put one engineer on a project, and the way that they're able to ship really quickly ... Because they want to ship quickly. This is an intrinsic motivation that comes from within. It's just wanting to do a good job. If you have a good idea, you just really want to get it out there. No one has to force you to do that. That comes from you. And so, if you have Claude, you can just use that to automate a lot of work, and that's what we see over and over. So, I think that's one principle is under-funding things a little bit. I think another principle is just encouraging people to go faster. So, if you can do something today, you should just do it today. And this is something we really, really encourage on the team. Early on, it was really important, because it was just me. And so, our only advantage was speed. That's the only way that we could ship a product that would compete in this very crowded coding market. But nowadays, it's still very much a principle we have on the team, and if you want to go faster, a really good way to do that is to just have Claude do more stuff. So, it just very much encourages that.
00:25:32 · Lenny Rachitsky, Boris Cherny
[00:25:32] S1: This idea of under-funding, it's so interesting, because, in general, there's this feeling like AI is going to allow you to not have as many employees, not have as many engineers. And so, it's not only you can be more productive. What you're saying is that you will actually do better, if you under-fund. It's not just that AI can make you faster, it's you will get more out of the AI tooling if you have fewer people working on something. [00:25:54] S0: Yeah. If you hire great engineers, they'll figure out how to do it. And, especially, if you empower them to do it. This is something I actually talk a lot about with CTOs and all sorts of companies. My advice generally is, "Don't try to optimize. Don't try to cost cut at the beginning. Start by just giving engineers as many tokens as possible." And now you're starting to see companies ... Like, at Anthropic, we have ... Everyone can use a lot of tokens. We're starting to see this come up as a perk at some companies where if you join you get unlimited tokens. This is a thing I very much encourage, because it makes people free to try these ideas that would have been too crazy, and then if there's an idea that works, then you can figure out how to scale it, and that's the point to optimize, and to cost-cut, figure out ... Maybe you can do it with a haiku, or with Sonnet instead of Opus, or whatever. But at the beginning, you just want to throw a lot of tokens at it, and see if the idea works, and give engineers the freedom to do that.
00:25:54 · Boris Cherny, Lenny Rachitsky
[00:25:54] S0: Yeah. If you hire great engineers, they'll figure out how to do it. And, especially, if you empower them to do it. This is something I actually talk a lot about with CTOs and all sorts of companies. My advice generally is, "Don't try to optimize. Don't try to cost cut at the beginning. Start by just giving engineers as many tokens as possible." And now you're starting to see companies ... Like, at Anthropic, we have ... Everyone can use a lot of tokens. We're starting to see this come up as a perk at some companies where if you join you get unlimited tokens. This is a thing I very much encourage, because it makes people free to try these ideas that would have been too crazy, and then if there's an idea that works, then you can figure out how to scale it, and that's the point to optimize, and to cost-cut, figure out ... Maybe you can do it with a haiku, or with Sonnet instead of Opus, or whatever. But at the beginning, you just want to throw a lot of tokens at it, and see if the idea works, and give engineers the freedom to do that. [00:26:49] S1: So, the advice here is just be loose with your tokens, with the cost on using these models. People hearing this may be like, "Of course. He works at Anthropic. You want us to use as many tokens as possible." But what you're saying here is the most interesting innovative ideas will come out of someone just taking it to the max and seeing what's possible.
00:26:49 · Lenny Rachitsky, Boris Cherny
[00:26:49] S1: So, the advice here is just be loose with your tokens, with the cost on using these models. People hearing this may be like, "Of course. He works at Anthropic. You want us to use as many tokens as possible." But what you're saying here is the most interesting innovative ideas will come out of someone just taking it to the max and seeing what's possible. [00:27:08] S0: Yeah. And I think the reality is at small-scale, you're not going to get a giant bill for anything like this. If it's an individual engineer experimenting, the token cost is still probably relatively low relative to [inaudible 00:27:21] other costs of running the business. So, it's actually not a huge cost. As the thing scales up, so, let's say they build something awesome, and then it takes a huge amount of tokens, and then the cost becomes pretty big, that's the point at which you want to optimize it. But don't do that too early. [00:27:37] S1: Have you seen companies where their token cost is higher than their salary? Is that a trend you think we're going to find and see? [00:27:44] S0: At Anthropic, we're starting to see some engineers that are spending hundreds of thousands a month in tokens. So, we're starting to see this a little bit. There's some companies that we're starting to see similar things. Yeah. [00:27:58] S1: Going back to coding, do you miss writing code? Is it something you're sad about that this is no longer a thing you'll do as a software engineer?