Lenny's Podcast: Product | Career | Growth
00:16:38 · Lenny Rachitsky, Boris Cherny
[00:16:38] S1: Wow. Even as head of the team? [00:16:41] S0: Yeah. Yeah. Still od a lot of coding. And so, every day, I ship, like, 10, 20, 30 pull requests or something like that. [00:16:47] S1: Every day? [00:16:49] S0: Every day. Yeah. [00:16:50] S1: Good God. [00:16:51] S0: 100% written by Claude Code. I have not edited a single line by hand since November. And, yeah. I do look at the code. So, I don't think we're at the point at where you can be totally hands-off, especially, when there's a lot of people running the program. You have to make sure that it's correct, you have to make sure it's safe, and so on. And then we also have Claude doing automatic code review for everything. So, here at Anthropic, Claude reviews 100% of pull requests. There's still a layer of human review after it, but you still do want some of these checkpoints. Like, you still want a human looking at the code. Unless it's pure prototype code that it's not going to run anywhere. It's just a prototype. [00:17:32] S1: What's the next frontier? So, at this point, 100% of your code is being written by AI. This is, clearly, where everyone is going in software engineering. That felt like a crazy milestone. Now it's just like, "Of course. This is the world now." What's the next big shift to how software is written that either your team is already operating in or you think will head towards?
00:17:54 · Boris Cherny
[00:17:54] S0: I think something that's happening right now is Claude is starting to come up with ideas. So, Claude is looking for feedback. It's looking at bug reports. It's looking at telemetry, and things like this, and it's starting to come up with ideas for bug fixes, and things to ship. So, it's just starting to get a little more like a coworker or something like that. I think the second thing is we're starting to branch out of coding a little bit. So, I think, at this point, it's safe to say that coding is virtually solved. At least, for the kinds of programming that I do, it's just a solved problem, because Claude can do it. And so, now we're starting to think about, "Okay. What's next? What's beyond this?" There's a lot of things that are adjacent to coding, and I think this is [inaudible 00:18:35] becoming, but also just general to us. Like, I use Cowork every day now to do all sorts of things that are just not related to coding at all, and just to do it automatically. Like, for example, I had to pay a parking ticket the other day. I just had Cowork do it. All of my project management for the team, Cowork does all of it. It's, like, syncing stuff between spreadsheets, and messaging people on Slack, and email, and all this kind of stuff. So, I think the frontier is something like this. And I don't think it's coding, because I think coding, it's pretty much solved, and over the next few months, I think what we're going to see is just across the industry it's going to become increasingly solved for every kind of code base, every tech stack that people work on.
00:19:14 · Lenny Rachitsky, Boris Cherny
[00:19:14] S1: This idea of helping you come up with what to work on is so interesting. A lot of people listening to this are product managers and they're probably sweating. How do you use Claude for this? Do you just talk to it? Is there anything clever you've come up with to help you use it to come up with what to build? [00:19:30] S0: Honestly, the simplest thing is, like, open Claude or Cowork, and point it at a Slack thread. Like, for us, we have this channel that's all the internal feedback about Claude Code. Since we first released it, even in 2024, internally, it's just been this fire hose of feedback. It is the best. And in the early days, what I would do is any time that someone sends feedback, I would just go in, and I would fix every single thing as fast as I possibly could. So, like, within a minute, within five minutes, or whatever. And this just really fast feedback cycle, it encourages people to give more and more feedback. It's just so important, because it makes them feel heard. Because, usually, when you use a product, you get feedback, it just goes into a black hole somewhere, and then you don't get feedback again. So, if you make people feel heard, then they want to contribute, then they want to help make the thing better. And so, now I do the same thing, but, Claude, honestly, does a lot of the work. So, I pointed at the channel, and it's like, "Okay. Here's a few things that I can do. I just put up a couple PRs. Want to take a look at that one?" I'm like, "Yeah."
00:20:25 · Lenny Rachitsky
[00:20:25] S1: Have you noticed that it is getting much better at this? Because this is the holy grail. Right now it's, cool, building solved. Code review became the next bottleneck with all these PRs. Who is going to review them all? The next big open question is just, like, "Okay. Now humans are necessary for figuring out what to build, what to prioritize," and you're saying that's where Claude Code is starting to help you. Has it gotten a lot better with, like, Opus 4.6, or what's been the trajectory there?
00:20:50 · Boris Cherny
[00:20:50] S0: Yeah. Yeah. It's improved a lot. I think some of it is training that we do specific to coding. So, obviously, the best coding model in the world, and it's getting better and better. Like, 4.6 is just incredible. But also actually a lot of the training that we do outside of coding translates pretty well too. So, there is this transfer where you teach the model to do X, and it gets better at Y. Yeah. And the gains have just been insane. Like, at Anthropic, over the last year, like, since we introduced Claude Code, we probably ... I don't know the exact number. We probably 4X-ed the engineering team, or something like this. But productivity per engineer has increased 200% in terms of pull requests. And this number is just crazy for anyone that actually works in the space and works on dev productivity. Because back in a previous life, I was at Meta, and one of my responsibilities was code quality for the company. So, this is all of our code bases, that was my responsibility. Like, Facebook, Instagram, WhatsApp, all this stuff. And a lot of that was about productivity, because if you make the code higher quality, then engineers are more productive. And things that we saw is in a year with hundreds of engineers working on it, you would see a gain of a few percentage points of productivity, something like this. And so, nowadays, seeing these gains of just hundreds of percentage points is just absolutely insane.
00:22:06 · Lenny Rachitsky
[00:22:06] S1: What's also insane is just how normalized this has all been. Like, we hear these numbers. Like, of course, AI is doing this to us. It's so unprecedented, the amount of change that is happening to software development, to building products, to just the world of tech. It's just so easy to get used to it, but it's important to recognize this is crazy.