Lenny's Podcast: Product | Career | Growth
00:12:22 · Fiona Fung, Lenny Rachitsky
[00:12:22] S1: That's right. [00:12:22] S0: So, I think what I'm hearing here is this is one of the only ways to stay on top of all the things a teammate is shipping. So this is a really cool thread of just how you found ways to stay on top of this 8X increase in code. What else has worked in helping you and your team stay on top of and maintain quality of all the stuff that y'all are shipping because that's obviously a challenge, a bigger challenge. [00:12:42] S1: Yeah. And so definitely the feedback channels are really important to us, but then we also get a lot of feedback. And for example, even I myself, usually my morning ritual would be I get my morning cup of coffee and then I look at the feedback channels and then I try to pick, oh, if I have some maker time, what's something that maybe I would be able to help out or what I could see some gaps like that. It just used to be something I would do every morning. And yeah, I think maybe a month or two ago we launched Routines and that's also completely changed. Now I just have a routine that automates all this for me. And then there's also, it's almost like before I would be able to generate some prompts, but now with Routines, it's almost like I'm having an agent help me generate the prompts and the PR. So for example, one is, "Hey, keep a look on this feedback channel, what are some of the themes?" And then when I wake up, then I have a really good summary of that and then even some PRs that I'll be able to take a look and review.
00:12:42 · Fiona Fung, Lenny Rachitsky
[00:12:42] S1: Yeah. And so definitely the feedback channels are really important to us, but then we also get a lot of feedback. And for example, even I myself, usually my morning ritual would be I get my morning cup of coffee and then I look at the feedback channels and then I try to pick, oh, if I have some maker time, what's something that maybe I would be able to help out or what I could see some gaps like that. It just used to be something I would do every morning. And yeah, I think maybe a month or two ago we launched Routines and that's also completely changed. Now I just have a routine that automates all this for me. And then there's also, it's almost like before I would be able to generate some prompts, but now with Routines, it's almost like I'm having an agent help me generate the prompts and the PR. So for example, one is, "Hey, keep a look on this feedback channel, what are some of the themes?" And then when I wake up, then I have a really good summary of that and then even some PRs that I'll be able to take a look and review. [00:13:38] S0: And the feedback channel, where's that feedback coming from? Is that like emails, Twitter, or a combo of everything?
00:13:38 · Lenny Rachitsky, Fiona Fung
[00:13:38] S0: And the feedback channel, where's that feedback coming from? Is that like emails, Twitter, or a combo of everything? [00:13:46] S1: Our feedback channels are definitely like we have a lot from internal ends, but also emails, channels. Actually everybody when we all get feedback on either when friend's ping us or on LinkedIn or socials, we'll all actually post all of that in Slack as well. And we also have of course partnerships and so we have different channels for all the various sources, but that's what I mean. I need Claude's help to help me stay on top because there is so much incoming feedback. [00:14:14] S0: Got it. Okay. So this is cool. So this is a way of working that you've built to stay on top of all the stuff shipping, which is this kind of daily ritual/routine where you as a manager look at what people are saying about the current state of Claude Code and Cowork and used to just like, "Okay, someone go fix this, go fix that." Now it's like, "Here's the PR that'll fix this thing. Check it out. We're ready to ship it if you want." Awesome. [00:14:15] S1: That's right. [00:14:41] S0: Okay. Obviously a big challenge for people is also just code review. I imagine Claude is also doing a lot of its own code review. Is there anything there you've recently figured out that allows your teams to ship faster stuff that they are confident is great?
00:14:41 · Lenny Rachitsky, Fiona Fung
[00:14:41] S0: Okay. Obviously a big challenge for people is also just code review. I imagine Claude is also doing a lot of its own code review. Is there anything there you've recently figured out that allows your teams to ship faster stuff that they are confident is great? [00:14:55] S1: Yeah. And honestly, it's crazy when you think we didn't even have Claude Code reviews last year. And so speak of bottlenecks, that was a really, really big bottleneck of the human reviewers. So we definitely, for the important areas that need deep subject matter expertise, we definitely want to make sure we have the proper humans still reviewing. I would say what helps us though is the more that we can automate to almost check in the framework for what good looks like. Claude is very good when you give it a framework to validate against those frameworks. So I had mentioned recently we just updated the content design to have a skill in it. And this is why I think if you have specs or check those into the repo and then make sure the spec also keeps up to date with the code frequently. But that's what I found works really well of anytime you have a statement of what good looks like, get the end to the repo and then Claude Code review can make sure it's still matching what you set up to do. [00:15:53] S0: And basically it's like the evolution of test driven development.
00:15:53 · Lenny Rachitsky, Fiona Fung
[00:15:53] S0: And basically it's like the evolution of test driven development. [00:15:56] S1: Yes. For TDD, because I remember that was a big thing, my gosh, maybe like in the 2000s of the test first and then you can make sure the test fails, then you do actually the code. In principle is really good. But I know I remembered, I myself struggling a bit because it was almost like you have to eat the broccoli first because I was like, "Oh, you have to write this test first." And I just get so much thrill out of shipping and building product. So it's funny, actually the first bug I fixed on Claude Code, I remember asking Claude, "Hey, I want to do test-driven development. Help me write the test first, make sure it's fails and then we'll actually do the fix and then now the test passed." And the fact that used to be that test generation used to just be this tax that I remember having to pay, the fact that that's now automated and you can even revisit all these principles that have been around for a while. But now they actually might be even more efficient just because you have the models that can do more of the work for you. [00:16:51] S0: Yeah, that's so unfair. It just writes the test for you first. So on this point of builders, one of my favorite slides from your talk that I want to ask about is who you are hiring and what you look for in people. And so I'll read what you said there and I want to hear more here. So the two profiles that you now look for when you're hiring are creative builders with product sense and deep systems experts for the hard parts.
00:16:51 · Lenny Rachitsky, Fiona Fung
[00:16:51] S0: Yeah, that's so unfair. It just writes the test for you first. So on this point of builders, one of my favorite slides from your talk that I want to ask about is who you are hiring and what you look for in people. And so I'll read what you said there and I want to hear more here. So the two profiles that you now look for when you're hiring are creative builders with product sense and deep systems experts for the hard parts. [00:17:18] S1: Yeah. The deep subject matter expertise, for example, when I first joined Claude Code, we had really great product generalists and then I realized, oh, we were missing folks with systems background. And so that was definitely an area that we needed more folks with systems and distribute systems expertise. And so I would say it's all about trust but verify. The models are really good, but there are definitely a lot of areas that still need the verification. And so wherever you need the deep subject matter expertise, I would say that's an area to definitely still invest in. And the other one is kind of like the product [inaudible 00:17:57], almost like the dreamers. These are folks that usually will be like, "My gosh, you're really passionate about a product." And they have an idea, they build it, and then it's always like looking at the feedback and then iterating and polishing and making sure that the product is a delightful experience. Owning that product end to end, that's another skillset that served us really well on Claude Code.