Lenny's Podcast: Product | Career | Growth
00:39:08 · Zevi Arnovitz, Lenny Rachitsky
[00:39:08] S1: Yeah, so this is another thing where it's very difficult for me to catch mistakes. So my review process has gone through a bunch of iterations to really be as good as possible and to catch as many things as possible. So I'll always manually QA at first to make sure if I can see any mistakes that Claude made. And then what I'll do is basically /review. And this tells Claude to start reviewing its own code. But what's even cooler and something that I'm really proud of is I will usually do multiple reviews and I'll have Codex, which is ChatGPT's competitor to Claude Code, as well as cursor open, and I will have each of them review the code. And then what I do is I have a /command called peer review, which is really interesting. And basically what it does is it's going to take Claude, which is usually the agent who I'm working with. And just to put this in a mental model, this is basically my dev lead that I'm working with. The /command is basically saying, "You're the dead lead on this project. Other team leads within the company have looked at your code and reviewed it and found these issues." Don't take what they said at face value. The reason is you have more context than them and you led this project. You need to either explain why the stuff they found are not real issues and wrong or fix them yourself. And it's really cool because the way I look at these things is I look at the models, I try to imagine them as people and I can really tell you how each one of these would be as a real human because they have- [00:40:54] S0: Each model.
00:40:55 · Zevi Arnovitz
[00:40:55] S1: Yeah. Each model has such distinct characteristics. So let's say Claude, she would be the perfect CTO. She's very communicative. She's very smart. She doesn't just go with the flow and do whatever you tell her. She's very opinionated, but also super collaborative, which is I think why I'm always drawn to Claude because I need to do so much learning and it's your dream, a very communicative, but very opinionated dev lead, but then there's also Codex. So I use Codex 5.1 Max, whatever. I don't know, they're not the best at naming models, but GPT's model. I always imagine it as like the best coder within the company who comes to the office with a hoodie and sandals and sits in a dark room. And you basically only bother him when you have the worst bugs and you say, "Listen, we have this bug and it will just close the door for two hours and come out and say, I fixed it." And you're like, "Wait, what? Are you going to tell us what happened or whatever?" And he's like, "Don't worry about it. I fixed it." It's like really not communicative, but it solves all the worst problems. And let's say Gemini is like a crazy scientist who's super artsy, super talented at designing, but if you sit next to it and watch it work, it's terrifying. You would fire that person instantly. This might be just my experience, but when I'm using Gemini within antigravity, which is Google's new competitor to Cursor, when it's writing code, you can see the steps it's taking and it's terrifying. You'll say, "I want you to redesign the top of the dashboard." And you're looking at its thought process and it will say, "Oh, first things first, I'll delete the dashboard." And then it'll be like, "Nope, that was a mistake. I'll bring it back." And then it will say, "Oh, can I edit the database?" And you're like, "No, do not edit the database. You're just doing a redesign." And then it will end up designing something beautiful. So the way there is a rollercoaster and very scary, but at the end of the day, Gemini is very good at design. So I think that using all these models and basically playing to their strengths and mitigating their weaknesses by using other models is a game changer for me. So I'll do peer review a bunch of times and I'll have other models review other models code and kind of have them fight it out basically.
00:43:14 · Zevi Arnovitz, Lenny Rachitsky
Sometimes Claude Code will get really sassy and be like, "This has been raised for the third time. And for the third time I'm telling you, this is not an issue. This is by design." So it's just a really cool thing that I've added and I haven't seen many people doing it. [00:43:31] S0: That is such an incredible rant/way to understand what's going on. Okay, awesome. So we just ran the review. So show us what we saw there and let's actually try this peer review. I'm really excited to see what you learned there.
00:43:31 · Lenny Rachitsky, Zevi Arnovitz
[00:43:31] S0: That is such an incredible rant/way to understand what's going on. Okay, awesome. So we just ran the review. So show us what we saw there and let's actually try this peer review. I'm really excited to see what you learned there. [00:43:45] S1: Yeah. So basically Claude has reviewed its code and it's found a bunch of bugs, a critical bug it found in the prompt, some high bugs, some medium bugs, and now what I'll do is I'll do the same thing with the other models. So Codex has a built-in code review that you can do, or I just like to say review all the code in this branch. Of course, branch is referring to the GitHub branch that we're working on. We're not working on the live code base. And then I'll do this with Composer say with, let's do it with Composer One. So I'll do /review here as well. And basically these are both going to run and do a in-depth review similar to what Claude does. But again, because of the differences between the models, they're all going to catch different things and they're all going to look differently, and this is a really cool way to work. It's basically if you had other team leads within the company review the code. Here, you can see how fast Composer is. I think GPT probably will take a bunch of time. Like I said, it's in its own dark room right now reviewing code and we'll come back in a few minutes.
00:43:45 · Zevi Arnovitz, Lenny Rachitsky
[00:43:45] S1: Yeah. So basically Claude has reviewed its code and it's found a bunch of bugs, a critical bug it found in the prompt, some high bugs, some medium bugs, and now what I'll do is I'll do the same thing with the other models. So Codex has a built-in code review that you can do, or I just like to say review all the code in this branch. Of course, branch is referring to the GitHub branch that we're working on. We're not working on the live code base. And then I'll do this with Composer say with, let's do it with Composer One. So I'll do /review here as well. And basically these are both going to run and do a in-depth review similar to what Claude does. But again, because of the differences between the models, they're all going to catch different things and they're all going to look differently, and this is a really cool way to work. It's basically if you had other team leads within the company review the code. Here, you can see how fast Composer is. I think GPT probably will take a bunch of time. Like I said, it's in its own dark room right now reviewing code and we'll come back in a few minutes. [00:44:56] S0: Okay, cool. So we can let these run in. We don't actually have to go through the whole process, but is the idea once you get these results, you run peer review and you copy and paste kind of these results. Is that the idea?
00:44:56 · Lenny Rachitsky, Zevi Arnovitz
[00:44:56] S0: Okay, cool. So we can let these run in. We don't actually have to go through the whole process, but is the idea once you get these results, you run peer review and you copy and paste kind of these results. Is that the idea? [00:45:04] S1: Exactly. I'll copy and paste the results. I'll do peer review and then I'll say dev lead one and then paste from one of the models. And then I'll say dev lead two and paste from the other model and basically have them fight it out until I feel like we have no more issues. [00:45:17] S0: Incredible. [00:45:18] S1: For me, this is super important because I'm not technical and I'm not a developer. And I'll also use /learning opportunity a bunch during this to learn about stuff that I don't understand or don't fully grasp. [00:45:31] S0: Incredible. What a clever solution to solving this code review problem where it's like, I don't know what you ... I don't know how to recode, so what am I going to even ... Yeah. [00:45:39] S1: Yeah. [00:45:40] S0: Okay. Incredible. Let's wrap up this workflow. Is there anything else that's important in this workflow? And again, all this stuff is going to be available where people can just plug this stuff into their Cursor account and use it themselves.