Lenny's Podcast: Product | Career | Growth
00:45:52 · Zevi Arnovitz
[00:45:52] S1: 100%. The one thing I'll say is that I think just like working in general with AI and even just like working on any product, doing constant postmortems is critical. So a lot of times we'll find all these kind of bugs or maybe Claude will fail to execute something correctly. And at the beginning when I started vibe coding, I would basically just keep running at it like running at the wall and until it worked. And once it worked, I was like, "All right, awesome. This works. Let's keep going." But I've learned over time that updating documentation and tooling is one of the biggest hacks for productivity. So when Claude will fail to do something or I'll see this really bad bug that shows that Claude really didn't understand something, I'll ask it, "What in your system prompt or tooling made you make this mistake?" And Claude will kind of like go introspective and think of what made it create that mistake. And then I'll say, "Okay, great. Let's update your tooling and documentation so that this mistake never occurs again." And I do this every time I'm either building an internal tool or anything. And I think this is just like working. If you end up doing a bunch of mistakes and then end up releasing the feature to users, so you're like, "All right, it's a big success." But going back and even when you've succeeded, looking and understanding what you did and what you could have done better is critical. And also using AI, this is probably one of the biggest unlocks. Going back to your prompts, understanding what was not good enough, iterating on them and then seeing how AI's responses get better, I think that's probably one of the most important things and one of the things that divides between people who are okay with using AI and the people who actually know how to use it.
00:47:38 · Lenny Rachitsky, Zevi Arnovitz
[00:47:38] S0: That is such good advice. So what I'm hearing is when the models do something dumb, make a mistake, you ask it to reflect on what the mistake it made was, and then you update the /command prompts with that knowledge so that in the future, it's not making that same mistake, and it just keeps getting better. [00:47:38] S1: Exactly. [00:47:56] S0: These things just keep getting smarter and smarter. So you're building up this really incredible prompt that just gets better and better. [00:48:01] S1: Exactly. Not always the /commands. It will sometimes update different documentation or its tooling, but basically it's understanding what the root cause of the mistake that the AI made and fixing it. [00:48:15] S0: Awesome. So the models are getting smarter and then there's also the other parts of your workflow can get smarter as you find flaws in the way it does stuff. [00:48:25] S1: 100%. Yep. [00:48:26] S0: Amazing. Okay. Is there anything else there before I move in a couple other directions?
00:48:26 · Lenny Rachitsky, Zevi Arnovitz
[00:48:26] S0: Amazing. Okay. Is there anything else there before I move in a couple other directions? [00:48:31] S1: I think that's it. I think we covered pretty much everything. Basically, just to wrap this up, what I do is I do a bunch of code review and then update the documentation so that everything is documented. So the next time I try to build a feature in this area, there won't be any mistakes. And then I'll do a bunch of testing. I'll do some user testing as well before I release this to general availability. Obviously we're not going to release this. This was just a show, but hopefully maybe by the time the podcast comes out, I'll have done this correctly and release the feature. [00:48:59] S0: It's incredible that this was not possible, I don't know, two years ago, maybe a year ago, you are a product manager shipping a product without knowing how to write code, barely knowing how to review code. You said you're afraid of looking at code. As a product manager, you're building a product in Cursor using all of these different AI models. You're making money with this product. We're so used to this now, but it's insane what is now possible.
00:48:59 · Lenny Rachitsky, Zevi Arnovitz
[00:48:59] S0: It's incredible that this was not possible, I don't know, two years ago, maybe a year ago, you are a product manager shipping a product without knowing how to write code, barely knowing how to review code. You said you're afraid of looking at code. As a product manager, you're building a product in Cursor using all of these different AI models. You're making money with this product. We're so used to this now, but it's insane what is now possible. [00:49:29] S1: It's the best time to be live. 100%. I think that I understand the fear, but AI just makes it so much possible. Just a quick side note here, my brother, who I'm building one of the apps with is an entrepreneur. He has a beautiful business that helps old people and seniors understand to use technology and AI better. And he's basically doing the same kind of learning as me, and he's replaced all of the tools he was paying for. I think he was paying for Zapier and Airtable, and he's basically built a full-fledged CRM system and automation system for his business completely alone. So for the people who are curious, optimistic, hardworking, this is the best time to be a builder. [00:50:14] S0: And what I love about this conversation we're having here is it feels like the biggest barrier for a lot of people is like, how do I get started? What exactly do I do? I open up cursor. It looks very intimidating. I don't know how to write code. I don't know how to build stuff. I don't know about databases. And so you're going to be sharing all these /commands and basically this whole workflow with the audience.
00:50:14 · Lenny Rachitsky, Zevi Arnovitz
[00:50:14] S0: And what I love about this conversation we're having here is it feels like the biggest barrier for a lot of people is like, how do I get started? What exactly do I do? I open up cursor. It looks very intimidating. I don't know how to write code. I don't know how to build stuff. I don't know about databases. And so you're going to be sharing all these /commands and basically this whole workflow with the audience. [00:50:34] S1: Yeah. [00:50:36] S0: Okay. [00:50:37] S1: And like I said, just start at GPT. Start on GPT, tell it what your idea is, tell it to explain to you what are even the first steps of thinking, what are the decisions you need to make? And just be inquisitive, learn. Don't rush things. It's very important to just dive in and really spend the time to learn. [00:50:56] S0: And you share this. One of your /commands is learning opportunity, and it's how you learn a lot of these things. Just teach me this thing and how this database issue works. [00:51:04] S1: Exactly. [00:51:05] S0: Okay. There's a couple directions I want to make sure we touch on. One is coming back to a question I asked earlier about how this might work at a larger company. Say it's not like Meta, but just like, I don't know, a thousand person company, 500 people. How much of this can you plug and play into a workflow as a PM at a larger company? What would be your advice for someone that may want to start trying to ship code, at least showing people what's possible?
00:51:05 · Lenny Rachitsky, Zevi Arnovitz
[00:51:05] S0: Okay. There's a couple directions I want to make sure we touch on. One is coming back to a question I asked earlier about how this might work at a larger company. Say it's not like Meta, but just like, I don't know, a thousand person company, 500 people. How much of this can you plug and play into a workflow as a PM at a larger company? What would be your advice for someone that may want to start trying to ship code, at least showing people what's possible? [00:51:28] S1: I think that first making your code base AI native is a really important step, and I think this needs to be done by technical people. So basically my codebase has a ton of just plain text in it. So it will have a bunch of markdown files that explain to agents how to work in certain areas of the code base and high level structure so that the agents navigate through the code base easier. And I think that if this is set up in a really good way, I still don't think PMs should be shipping heavy database chain migrations or any big project, but contained UI projects, especially if you just build it, create the PR and send it to a dev to do the final finishes. I think that's definitely something that's possible. And I think we're going to see that a lot in the next coming years. I think basically everyone's going to become a builder, so it should be really interesting.