Lenny's Podcast: Product | Career | Growth
00:17:54 · Boris Cherny
[00:17:54] S0: I think something that's happening right now is Claude is starting to come up with ideas. So, Claude is looking for feedback. It's looking at bug reports. It's looking at telemetry, and things like this, and it's starting to come up with ideas for bug fixes, and things to ship. So, it's just starting to get a little more like a coworker or something like that. I think the second thing is we're starting to branch out of coding a little bit. So, I think, at this point, it's safe to say that coding is virtually solved. At least, for the kinds of programming that I do, it's just a solved problem, because Claude can do it. And so, now we're starting to think about, "Okay. What's next? What's beyond this?" There's a lot of things that are adjacent to coding, and I think this is [inaudible 00:18:35] becoming, but also just general to us. Like, I use Cowork every day now to do all sorts of things that are just not related to coding at all, and just to do it automatically. Like, for example, I had to pay a parking ticket the other day. I just had Cowork do it. All of my project management for the team, Cowork does all of it. It's, like, syncing stuff between spreadsheets, and messaging people on Slack, and email, and all this kind of stuff. So, I think the frontier is something like this. And I don't think it's coding, because I think coding, it's pretty much solved, and over the next few months, I think what we're going to see is just across the industry it's going to become increasingly solved for every kind of code base, every tech stack that people work on.
00:19:14 · Lenny Rachitsky, Boris Cherny
[00:19:14] S1: This idea of helping you come up with what to work on is so interesting. A lot of people listening to this are product managers and they're probably sweating. How do you use Claude for this? Do you just talk to it? Is there anything clever you've come up with to help you use it to come up with what to build? [00:19:30] S0: Honestly, the simplest thing is, like, open Claude or Cowork, and point it at a Slack thread. Like, for us, we have this channel that's all the internal feedback about Claude Code. Since we first released it, even in 2024, internally, it's just been this fire hose of feedback. It is the best. And in the early days, what I would do is any time that someone sends feedback, I would just go in, and I would fix every single thing as fast as I possibly could. So, like, within a minute, within five minutes, or whatever. And this just really fast feedback cycle, it encourages people to give more and more feedback. It's just so important, because it makes them feel heard. Because, usually, when you use a product, you get feedback, it just goes into a black hole somewhere, and then you don't get feedback again. So, if you make people feel heard, then they want to contribute, then they want to help make the thing better. And so, now I do the same thing, but, Claude, honestly, does a lot of the work. So, I pointed at the channel, and it's like, "Okay. Here's a few things that I can do. I just put up a couple PRs. Want to take a look at that one?" I'm like, "Yeah."
00:20:25 · Lenny Rachitsky
[00:20:25] S1: Have you noticed that it is getting much better at this? Because this is the holy grail. Right now it's, cool, building solved. Code review became the next bottleneck with all these PRs. Who is going to review them all? The next big open question is just, like, "Okay. Now humans are necessary for figuring out what to build, what to prioritize," and you're saying that's where Claude Code is starting to help you. Has it gotten a lot better with, like, Opus 4.6, or what's been the trajectory there?
00:20:50 · Boris Cherny
[00:20:50] S0: Yeah. Yeah. It's improved a lot. I think some of it is training that we do specific to coding. So, obviously, the best coding model in the world, and it's getting better and better. Like, 4.6 is just incredible. But also actually a lot of the training that we do outside of coding translates pretty well too. So, there is this transfer where you teach the model to do X, and it gets better at Y. Yeah. And the gains have just been insane. Like, at Anthropic, over the last year, like, since we introduced Claude Code, we probably ... I don't know the exact number. We probably 4X-ed the engineering team, or something like this. But productivity per engineer has increased 200% in terms of pull requests. And this number is just crazy for anyone that actually works in the space and works on dev productivity. Because back in a previous life, I was at Meta, and one of my responsibilities was code quality for the company. So, this is all of our code bases, that was my responsibility. Like, Facebook, Instagram, WhatsApp, all this stuff. And a lot of that was about productivity, because if you make the code higher quality, then engineers are more productive. And things that we saw is in a year with hundreds of engineers working on it, you would see a gain of a few percentage points of productivity, something like this. And so, nowadays, seeing these gains of just hundreds of percentage points is just absolutely insane.
00:22:06 · Lenny Rachitsky
[00:22:06] S1: What's also insane is just how normalized this has all been. Like, we hear these numbers. Like, of course, AI is doing this to us. It's so unprecedented, the amount of change that is happening to software development, to building products, to just the world of tech. It's just so easy to get used to it, but it's important to recognize this is crazy.
00:22:25 · Boris Cherny
[00:22:25] S0: This is something I have to remind myself once in a while. There's a downside of this, because the model changes so ... Well, there's many downsides that we could talk about, but I think one of them on a personal level is the model changes so often that I sometimes get stuck in this old way of thinking about it. And I even find that new people on the team, or even new grads that join do stuff in a more AGI-forward way than I do. So, sometimes, for example, I had this case a couple of months ago where there was a [inaudible 00:22:57] week. And so, what this is is Claude Code, the memory usage is going up, and, at some point, it crashes. This is a very common engineering problem that every engineer has debugged 1000 times. And, traditionally, the way that you do it is you take a heap snapshot, you put it into a special debugger, and figure out what's going on. You use these special tools to see what's happening. And I was doing this, and I was looking through these traces, and trying to figure out what was going on, and the engineer that was newer on the team just had Claude Code do it. And was like, "Hey, Claude. It sounds like there's a leak. Can you figure it out?" And so, Claude Code did exactly the same thing that I was doing. It took the heap snapshot, it wrote a little tool for itself, so, it can analyze it itself. It was a just in time program. And it found the issue, and put up a pull request faster than I could. So, it's something where for those of us that have been using the model for a long time, you still have to transport yourself to the current moment, and not get stuck back in old model, because it's not Sonnet 3.5 anymore. The new models are just completely, completely different. And just this mindset shift is very different.