Lenny's Podcast: Product | Career | Growth
00:22:30 · Lenny Rachitsky
[00:22:30] S0: And by the way, as this is loading, I just love the way, the way you described this, especially doing voice mode, it's like exactly how you would talk to an engineer describing a feature, "Here's what I want." And then they ask you questions, here's the clarification.
00:22:43 · Zevi Arnovitz, Lenny Rachitsky
[00:22:43] S1: Yeah. So at first when I was doing this with the CTO, I would do it with ChatGPT voice mode, and that was crazy. That literally felt like ideating with a person. It would push back, ask questions, and maybe one day the coding tools will get there too, but that was exactly ... It really felt like sitting with my CTO. Great. So created STU88. So if we open up Linear now, we should be able to see ... Let's see where STU88. There it is. Fill in the blank questions with drag and drop interface. So it has a TLDR, it has the current state. It did a little bit of research on the code base, I think, expected outcomes, some context. So yeah, so this is basically ready for me to pick up when I'm interested in building. So now let's say a few days go by, I finished the current project I'm working on, I can pick it up. So when I pick it up, I do /exploration phase, which is what we said. And then instead of pressing enter, I'll press tab and I'll show you this. So basically, exploration phase, what it does is it will take an argument. This is basically a placeholder within the prompt, which allows me to enter something that is extra context for the AI. So I can say here, Linear STU88, which is referencing the ticket. And now what it's going to do is it's going to go, it's going to fetch the Linear ticket. [00:24:12] S0: And what's the idea? What's the goal of the exploration phase? This ideate on the idea. Is that the- [00:24:12] S1: Exactly. [00:24:12] S0: Okay.
00:24:12 · Lenny Rachitsky, Zevi Arnovitz
[00:24:12] S0: Okay. [00:24:18] S1: So it's both for the CTO to deeply understand the problem that we're trying to solve and also understand the current state of the code base, what files need to be affected, and how is the best way to implement this technically. And usually what happens is right now, Claude is just basically reading a bunch of files, understanding the basic structure of the code, and then it's going to come back with a bunch of clarifying questions that will decide how we end up implementing this. [00:24:44] S0: So it feels like it's talking to your engineering manager. [00:24:48] S1: Exactly, exactly. 100% this is how I think about it. [00:24:51] S0: And you said that you're a CTO, so you used to use ChatGPT prompt to have a CTO in there. Now the CTO's living inside here in Cursor? [00:25:00] S1: Yeah, because of the way the tools have developed and they've become so good at both exploration and code execution. So now it's just a habit that I call it a CTO, but it's basically all in one. The same agent will both do the exploration and write the plan and end up executing the code. [00:25:19] S0: Got it. So it's basically it's Claude Code. Is there a prompt you gave it to act like that? To act like the right kind of CTO?
00:25:19 · Lenny Rachitsky, Zevi Arnovitz
[00:25:19] S0: Got it. So it's basically it's Claude Code. Is there a prompt you gave it to act like that? To act like the right kind of CTO? [00:25:26] S1: Yeah, so within the Claude.md, which is basically the system prompt that's loaded within Claude's context in every conversation, I have some basic stuff like this is our workflow, this is how we work. Within exploration phase, I want you to challenge my thinking, all kinds of stuff like that that can be loaded within the Claude.md file. [00:25:47] S0: Cool. One last question before we move on here, just because I'm thinking about it as this happens, the Linear issue that you generated, how often is it actually great and ready? How often do you have to edit it? What's the quality of the Linear ticket that it generates? Because a lot of people are probably wondering just like all these terrible linear issues are being created by AI. Are they actually any good? [00:26:08] S1: It's completely different because I'm a company of one. So a lot of the context is within here and there's no need for me to talk to other teams and understand. It's basically very accessible, and also I can easily see when Claude understood something wrong. I don't want to say that I would create Linear issues at work like this, but definitely if you're building your own side project, they're pretty quality. And also, it just kicks off when I want to start working on it. I wouldn't say it's ready to be built. It's ready to start being explored.
00:26:08 · Zevi Arnovitz, Lenny Rachitsky
[00:26:08] S1: It's completely different because I'm a company of one. So a lot of the context is within here and there's no need for me to talk to other teams and understand. It's basically very accessible, and also I can easily see when Claude understood something wrong. I don't want to say that I would create Linear issues at work like this, but definitely if you're building your own side project, they're pretty quality. And also, it just kicks off when I want to start working on it. I wouldn't say it's ready to be built. It's ready to start being explored. [00:26:44] S0: Got it. So it's just the beginning of an idea. Actually, let's come back after we go through this flow of how you would approach this if you were at say Meta or another, maybe a smaller company, how this workflow might work at a larger company that isn't just your own startup. [00:26:56] S1: Yeah. Interesting. [00:26:57] S0: Let's come back to that.
00:26:57 · Lenny Rachitsky, Zevi Arnovitz
[00:26:57] S0: Let's come back to that. [00:26:59] S1: Cool. All right. So this is Claude coming back. I have a comprehensive understanding of the code base. I thoroughly analyzed StudyMate live codebase and understand the current system, feature quest and key areas that it's identified. Usually I'd spend a lot of time going over this because this is super, super important, but just for the sake of development right now, we're going to brush through this. Now Claude basically comes back after it's gone through the code base and understood the way it currently works. It's basically telling me what the current understanding is. So it's talking about how the app is set up at the moment, how the data is structured, what it understood from the feature quest, and what it has identified as key areas, and then it asks me some questions. So it's asking about the scope, it's asking about the data model, the UX/UI of the feature, how it should be validated, how it should be graded, what changes need to be happened to the AI system prompt and all kinds of questions about the app. I've prepared answers to all these questions beforehand because I don't think we all want to sit through this. So I'm just going to paste that in and we'll see what Claude says. [00:28:08] S0: Awesome. I love it. And I love just scanning those questions I was asking. It's like such smart, sophisticated, important questions instead of just, "Cool, here I go, I'm going to build it."