Lenny's Podcast: Product | Career | Growth
01:05:37 · Lazar Jovanovic
[01:05:37] S0: Yes. Great question. And absolutely true. No matter how good of a plan you have in place, you're going to run into problems eventually. And I have a small little framework that I call, four by four, just again, analogies. Right? Four by four, if you have it on your car, you're going to get yourself out of the mud much easier than the other way around. So in that sense, four different ways to debug. Attempt one of each only once, and I'll explain why in the end. First one is, again, every tool is different. I'll reference Lovable's workflow, which is when something breaks, Lovable's agent is smart enough to say, "Hey, I made a mistake." It will label that message in orange and have this little button usually, which is called, try to fix. So your agent basically admits it made a mistake. You click on a button and most times, when it's a smaller issue, it corrects the course, fixes it. No problem. Right? Now, there are situations obviously, when the problem is a little bit deeper than that. Right? You click to try to fix, but the problem persists. And sometimes even the problem persists, but Lovable's agent is unaware that it persisted. So there's no more try to fix button. Lovable thinks everything's working, but in reality, it isn't. And the culprit there is usually, you're using a third party integration. You did not give enough context to Lovable what to observe and what to see. So it can't see that the problem exists because Lovable, Cursor, Claude Code, you name it, all of these tools are good enough today to fix any problem they're aware of. Again, awareness is the key here. Right? So when they're unaware of it, there comes the second part, which is, "Okay, I need to bring the awareness layer." And what I do there is I go and very simply open the preview sandbox dev environment of my app, whatever, try to run the function that's broken, right click, read the console log. Right? Every browser allows you to just go and read the console log. And a lot of times, it will record stuff. If it doesn't, you can prompt any tool and say, "Hey, I don't think you're seeing the problem. So instead of me yelling at you, let's find it together." Right? I think it's a problem with X, Y, Z. I want you to write console logs in relevant files so that we can monitor every step along the way. Let's just bring awareness layer into the equation.
01:08:13 · Lazar Jovanovic
It writes the console logs, you rerun it. Guess what? Now you have a full history of everything that was happening. You copy that, you paste it inside your chat. 99% of the time, that's enough, that's already enough. AI is like, "Okay, got it, found it, fixed it." But then, there's situations when even that's not sufficient. So you're like, "Okay, I need to go even deeper." And that's where code reviews and evaluations come into play. My go-to tool today for that is Codex, OpenAI. What I do is any build that I do, I will export it to GitHub. Lovable allows you to own your code cursor as well. All of these tools allow you to have a copy of the code that you can export to GitHub, and then import it into wherever you want to. So I use Codex since beta, like import it in there, and then I'm using an external tool. So I'm like, in the first try, if you remember, I used the tool and I was like, total vibes, I'm relying on the tool. Right? In the second try, I use myself as the awareness facilitator. In the third one, I'm using an external tool as a facilitator, which is like, I'll either connect to Codex and chat with Codex to then fix the problem in Lovable. Right? I don't allow Codex to make code changes for me. A lot of people will say, "Why don't you like, it's a good model?" I just don't know its agent well enough. I don't want to go and use a tool that I don't know how to steer. So I use it only for diagnostic purposes and I'll also do it manually. It's an old workflow that I had before Codex and before Claude Code, which is there's a tool called Repomix, which allows you to compress your entire code base into a single file. You download it, and then I upload it to Claude, just regular Claude or ChatGPT. And I'm like, "This is what I'm building. Read it and this is the problem that I have. These are the console logs." Again, it's almost like having an external consultant at that point. You're hiring help elsewhere because your team just can't handle it. Right? And then the fourth one is usually the best one, because another time when there are problems, it's my fault. Like, no matter how your ego is big, guys, that you're watching this, it's your fault. Trust me. You had a bad prompt. You premised your request in the wrong way. You just don't want to admit it or you can't remember that you did, but it's your fault.
01:10:48 · Lazar Jovanovic
So again, in Lovable and in all these other tools, you can revert back. There's version control built into Lovable, Cursor, Claude Code. You go and say, "Okay, I tried these three things. I'm just going to take three steps back and I'm going to think about my prompt a little bit more." Take a couple breaths, go for a walk, have some coffee, come back with a clear mind and try again. Because guess what? AI is just writing code very fast and sometimes it stumbles on a very small rock and it only happens then and never again. So you just got to make the same request again. And usually, that just fixes the problem. It's just a snag. It's a syntax error. It's something minute. Right? And then I do the final thing, which is this. And this is the key one actually. When the problem gets fixed, I go into the chat mode and I ask Lovable, I say, "Okay, I needed to do four different things to fix this. How can you help me learn how to prompt you better so that next time I have a problem, we do it in one go?" 99% of the time, I get such a great answer that I don't have the problem of not knowing what to do next time. Right? Again, we all need to be aware and realistic. These tools are so good at doing things the right way, if they are used the right way. It's always our fault. I say 90%, but honestly, it's 100% our fault, because they're good enough. It's just that I'm not dynamically shifting token allocation. I didn't reference the right file. I didn't say it the right way. For me, as a non-designer, I don't know any of the terminology, none of the headings and whatnots, and I still don't know it to this day. So when I struggle with prompts, a lot of times, I use chat mode to help me craft a good prompt. Anybody can do this too. If you are just stuck, it's 10:00 PM and you don't know what to ask, switch to chat mode, brain dump and be like, "Help me draft a better prompt. Help me prompt you better." And let the tool effectively prompt itself. A lot of times, you're going to solve your problems by not introducing them at all with bad inputs.
01:13:05 · Lenny Rachitsky
[01:13:05] S1: Oh my God. Everything you share is so interesting. I want to keep digging. So just to reflect back the sequence, and then I want to follow up with another question. The sequence you go through when you get stuck, which is going to happen to everyone. One, is just ask the tool to try to fix it. And oftentimes, it's telling you, "Something is wrong. Can I fix it for you?" And you're like, "Please fix." Sometimes that'll work. Two, is work on adding more debugging messages to the console log. And this advice, I love of just ask it to add more debugging lines to its own console log to help see what's going on. And then you can ask it, "Okay, now that you're looking, look at all the output of your console log, see if you can help find the problem." And then step three is go to Codex, which is so funny. And I hear this a lot, that Codex is the most elite engineer as an AI. Karpathy tweeted this once that... And we had the head of Codex on the podcast too, by the way, that he's like, "Anytime I have the most gnarly bug, I just go to Codex, let it run for half an hour, and it solves it unlike any other tool out there." And so it makes sense that that's where you go. So the idea here is you point Codex to your code, you show all the console output logs, tell it what the problem is and just have it go figure it out. Sweet. And then this final step is so great, and this is where I want to go, which you use this as a learning opportunity so that next time, you solve the problem more quickly or avoid it completely. So what you do there is you ask the agent, "Okay, here's what happened. What can I do? What could I have said? How could I have prompted you better to have gotten this immediately solved?"
01:14:52 · Lazar Jovanovic
[01:14:52] S0: Yeah. And then even deeper than that is like, once you go through this conversation, you're like, "Okay, let me eliminate myself again completely out of the equation because I won't remember to prompt you better two days from now." Put this into rules, put this what we just learned into Rules.md, because I'm making you read the rules every time anyways, so you might as well just record it there. So I'm not going to prompt you better. You're just going to learn that I'm stupid and you're going to prompt yourself better. Right? Again, just eliminate yourself and move the context, you solve 99% of the problems with AI today.
01:15:29 · Lenny Rachitsky
[01:15:29] S1: So idea here is help it build its own brain and rules and way of thinking based on problems you run into. So great. Okay. So I want to come back to this point you've made a couple of times, which is so interesting. This idea that you watch the output of the agent to learn what is going on. There's something I've seen other people. Ben Tossell, who I think is at factory now, shared this recently. He's also basically vibe coding all the time. He was brilliant to no code tools before, and now he's all about vibe coding. Basically, he's learning how coding works and learning how systems work by watching the agent output. And this connects to something Michael Truell shared, the CEO of Cursor. He was on the podcast. He had this vision of Cursor becoming basically what comes after code. What's the layer that we are adding on top of code where people don't need to worry about code anymore? And at that point, it was like a year ago that we chatted and it feels like this is the layer, is the agent conversation of what it's thinking, and then what you tell it back. So essentially, it's English in a conversation, which is like, it's not even pseudocode. It's interesting that that's where it feels like things are heading. The layer over code, just its thinking and your conversation with it.