Lenny's Podcast: Product | Career | Growth
00:21:27 · Simon Willison
[00:21:27] S0: So, one of the most interesting problems we're having with all of this is we've taken the writing code bit and we've massively accelerated that, now the bottlenecks are everywhere else, right? How do we redesign our processes? Now that the bit that used to take the longest, it used to be you'd come up with a spec and you handed it to your engineering team, and three weeks later, if you're lucky, they'd come back with an implementation for you to then start. Now maybe that takes three hours, depending on how well-established the coding agents are for that kind of thing. So, now what, right? Now, where else are the bottlenecks? I mean, there's coming up with the initial ideas. Anyone who's done any product work knows that your initial ideas are always wrong. What matters is proving them, it's testing them. We can test things so much faster now, because we can build workable prototypes so much quicker. So there's an interesting thing I've been doing in my own work where any feature that I want to design, I'll often prototype three different ways it could work, 'cause that takes very little time, and then I can start experimenting them and trying them and seeing which ones I like. That feels to me like the really transformational step here is that when you get AI involved in your ideation phase, it's much more about the prototypes. It's about, okay, we can see a UI prototype is free. Now, ChatGPT and Claude will just build you a very convincing UI for anything that you describe, and that's how you should be working. I think anyone who's doing product design isn't vibe coding little prototypes is missing out on the most powerful boost that we get in that step. But then what do you do, right? Given your three options now that you have instead of one option, how do you prove to yourself which one of those is the best? I don't have a confident answer to that. I expect this is where the good old-fashioned usability testing comes in. Get somebody on Zoom, screen shared using your software, see what happens. You can tell the AI to do it and you can simulate your users with the AI. I don't think that's credible. I don't think you're going to get as good results from ChatGPT pretending to click around on your prototype than you would from an actual human being.
00:23:37 · Lenny Rachitsky
[00:23:37] S1: This is so interesting. A question I've been tackling is just where are human brains going to continue to be valuable? What I'm hearing here is there's the initial idea. You made such a good point here. It's like the initial idea's often not the actual winning idea, it's just the beginning of an idea. So there's the idea for the feature, then there's the try it out, prototype it, help you narrow on the direction, build it, make it awesome, get it out into the world. It feels to me like AI's going to be really good at suggesting ideas and coming up with initial ideas, and I wonder if the human brain ... it's not like maybe someday we don't need human brains at all, and that's a whole other discussion, but maybe the next phase is AI will help us come up with great ideas.
00:24:17 · Simon Willison
[00:24:17] S0: I mean, that's been the case for probably a couple of years now, they've been strong enough to do really good brainstorming. I like to compare it to the thing where when you've got a group brainstorming exercise, you book a meeting room for an hour, you've got a whiteboard, you get a dozen people in, and the first two-thirds of that brainstorming session, honestly it's kind of just everyone going through the most obvious basic ideas, right? You get them all out on the whiteboard, you get them all up, and then things get interesting when you start saying, "Okay, well, let's talk about these, let's start combining them." The AI is so good at that first two-thirds of the ideas. I brainstorm with them all the time where I just get them to spit out all of the obvious stuff, and they'll come up with 20 things and they'll all be kind of dumb, they just won't be very interesting. What gets interesting is when if you ask them for 20 more and now- What gets interesting is when if you ask them for 20 more and now by the end of that list, you're beginning to get things which are not good ideas, but they point you in interesting directions. There are so many other tricks like this. You can tell AI to combine weird fields. You can say, "Okay, I want ideas for marketing my new SaaS platform inspired by marine biology," and you see what happens. And most of it will be complete junk, but there might be a spark that gets you to the good idea. So I love them as brainstorming companions on that front.
00:25:33 · Lenny Rachitsky
[00:25:33] S1: That reminds me of a chat I had with David Placek. He's a expert naming person. He helps companies come up with names for product. And one of the things that he does at his company is he creates three teams to come to brainstorm names. So for example, let's say Windsurf was a product they named. So the first team is, "Okay, this is an AI IDE thing." That's exactly what it is. Second team is, "Okay, this is a boat. You're naming a boat and here's the constraints. And then here, this is a spaceship. So name it from that perspective." And he finds the best names come from those other directions where it's a different metaphor with the same sort of benefits. Okay. So what I'm hearing here is this is good. This is good for humans right now, that there's still opportunity for us to contribute to the process.
00:26:17 · Simon Willison
[00:26:17] S0: And actually, I want to stand in defense of software engineers for a bit because on the one hand, these things can write code. That used to be our thing. I'm finding that using coding agents well is taking every inch of my 25 years of experience as a software engineer, and it is mentally exhausting. This is something which people are talking a lot more about now. I can fire up four agents in parallel and have them work on four different problems, and by 11:00 AM, I am wiped out for the day, because there is a limit on human cognition in how much, even if you're not reviewing everything you're doing, just how much you can hold in your head at one time. And it's very easy to pop that stack at the moment. There's a sort of personal skill that we have to learn, which is finding our new limits. What is a responsible way for us to not burn out and for us to use the time that we have? And I've talked to a lot of people who are losing sleep because they're like, "My agents could be doing work for me. I'm just going to stay up an extra half hour and set off a bunch of extra things and I'm waking up at 4:00 in the morning." That's obviously unsustainable. I hope that that's a novelty thing. The agents only really got good in the past four to five months. We're all learning what that looks like and what that lets us do. But it's concerning. There's an element of gambling and addiction to how we're using some of these tools. But to stand in defense of software engineers, I get great results out of these things because they are amplifiers of existing skills and experience. And I have 25 years of existing pre-AI experience, which I can now amplify because I can talk to the agent at a very high level. I can use sophisticated engineering language that I've mastered over the years, which they appear to know as well, and we can collaborate incredibly effectively. And it means I can look at a problem and I can say, "This problem is a one-sentence prompt and I know it'll find that bug and fix that bug," as opposed to this other problem, which is who knows how big a problem. There is a flip side to this, which is that I've got 25 years of experience in how long it takes to build something, and that's all completely gone. That doesn't work anymore because I can look at a problem and say, "Okay, well, this is going to take two weeks.
00:28:27 · Simon Willison, Lenny Rachitsky
It's not worth it." And now it's like, yeah, but maybe it's going to take 20 minutes, because the reason it would've taken two weeks was all of the crufty coding things that the AI is now covering for us. And that I've been finding really interesting and challenging. I constantly throw tasks at AI that I don't think it'll be able to do, because every now and then it does it. And when it doesn't do it, you learn. You learn, okay, Opus 4.6 still can't do this particular thing, but when it does do something, especially something that the previous models couldn't do, that's actually cutting edge AI research. You can be the first person in the world to spot that AI can now do X just because you were the person you found it couldn't do it and you've been keeping that sort of backlog of interesting tasks for it. [00:29:13] S1: This is such an interesting line of discussion, this idea that let's say 10X engineers, to use that phrase, are going to be more valuable is what you're describing here because you can work with these tools much more effectively. What do you think of junior engineers? Just what's happening there? What's their future?