Lenny's Podcast: Product | Career | Growth
00:16:38 · Simon Willison
[00:16:38] S0: I believe so, but it meant that the software was being very robustly tested in all of these different ways. Yeah, it's kind of similar to having a manual QA team, except one that never sleeps. I thought that was fascinating as a example of thinking outside of the box. Taking this question, how do we tell our software's good if we're not reviewing the code and trying to find creative answers to it. The other thing that was interesting is that the Slack channel itself wasn't actually Slack, because it turns out if you test against real software like Slack and so forth, they all have rate limits and they won't let you just run 10,000 simulated people at a time. So, what they did is they built their own simulation of Slack and Jira and Okta and all of this software they were integrating with. The way they did that is they basically took the API documentation for the public APIs for Slack and the open source client libraries, and they told their coding agents, "Build this. Build me a simulation of this API," and they did. So, this company is ... and this is one of the things ... I went to a demo that they gave back in October. One of the things that really sat with me is that they had their own simulated version of Slack and Jira and all of these different systems that they could then build their software against, which cost them nothing, because once they spun it up it was a little Go binary that sat there, and they even had interfaces. They had a fake version of the Slack interface that they'd vibe coded up that let them see what was going on. Absolutely fascinating.
00:18:08 · Lenny Rachitsky
[00:18:08] S1: That is such a cool story, and I love these stories of just companies at the bleeding edge trying to see what's possible and have an advantage essentially. So, what I'm hearing here is the QA piece is like the new piece in this factory. So we already have Codex, Claude Code, they can go off and build stuff. Is the innovation here, okay, now you've built all this stuff, is it actually any good? Is there a reason why Codex and Claude Code couldn't do this themselves? Why do you need this factory concept?
00:18:32 · Simon Willison
[00:18:32] S0: I think they can. You can tell Claude Code, fire up a subagent that uses Playwright to simulate a browser and all of that kind of thing. You'd have trouble getting it to run 24 hours a day. I mean, maybe it would work. But certainly I think what's interesting to me isn't so much the software you're using. It is these big ideas, these techniques that you're using to try and answer these questions, because even if your QA team, your virtual QA team says this is good, doesn't mean it's secure, right? It doesn't mean that you've got all of those other characteristics that you care about. At the same time, the agents are getting really good at security penetration testing now. This is a new thing ... I think, again, in the past three to six months they've started being credible as security researchers, which is sending shockwaves through the security research industry. They were like, "Wow, we didn't think that they'd get to this point." What's interesting there is both OpenAI and Anthropic have specialist security models that they will not release to the general public, because they can be used to break into websites. So they have invite only registered security researchers can apply for access, and they've been producing vulnerability reports against popular open source software. I think Firefox just a few days ago, maybe last week, said that they'd done a release, which was assisted by Anthropic. Anthropic had discovered 100 potential vulnerabilities in Firefox and responsibly reported them to Mozilla who then fixed them. That's an interesting one as well, because we're seeing a lot of this in the wild and it's just incredibly frustrating for maintainers, because there are these people who don't know what they're doing who are asking ChatGPT to find a security hole and then reporting it to the maintainer. The report looks good. ChatGPT can produce a very well-formatted report of the vulnerability. It's a total waste of time, it's not actually verified as being a real problem. The difference with Anthropic and Firefox is that Anthropic's security team actually did do the work. They didn't report whatever the agent said, they actually verified that it was a good quality report before they handed it over.
00:20:41 · Lenny Rachitsky
[00:20:41] S1: There's going to be a lot to talk about on the security side. You've done a lot of thinking and writing about the dangers there, but I want to follow this thread. So, in terms of what AI's been doing for teams. If you think about it, it's kind of going on the middle and expanding, so it's taking on more and more of the building components. It's doing code reviews now with QA, as you've been describing, constantly building, and it feels like the front of that is the big now gap in opportunity, which is coming up with the idea. What the heck should we build? 'Cause then once you tell the AI, build this thing, as you're describing, it's getting better and better at building something great. Have you had any luck yet with using AI there? And do you think it starts to eat that and just becomes the strategy PM basically?
00:21:27 · Simon Willison
[00:21:27] S0: So, one of the most interesting problems we're having with all of this is we've taken the writing code bit and we've massively accelerated that, now the bottlenecks are everywhere else, right? How do we redesign our processes? Now that the bit that used to take the longest, it used to be you'd come up with a spec and you handed it to your engineering team, and three weeks later, if you're lucky, they'd come back with an implementation for you to then start. Now maybe that takes three hours, depending on how well-established the coding agents are for that kind of thing. So, now what, right? Now, where else are the bottlenecks? I mean, there's coming up with the initial ideas. Anyone who's done any product work knows that your initial ideas are always wrong. What matters is proving them, it's testing them. We can test things so much faster now, because we can build workable prototypes so much quicker. So there's an interesting thing I've been doing in my own work where any feature that I want to design, I'll often prototype three different ways it could work, 'cause that takes very little time, and then I can start experimenting them and trying them and seeing which ones I like. That feels to me like the really transformational step here is that when you get AI involved in your ideation phase, it's much more about the prototypes. It's about, okay, we can see a UI prototype is free. Now, ChatGPT and Claude will just build you a very convincing UI for anything that you describe, and that's how you should be working. I think anyone who's doing product design isn't vibe coding little prototypes is missing out on the most powerful boost that we get in that step. But then what do you do, right? Given your three options now that you have instead of one option, how do you prove to yourself which one of those is the best? I don't have a confident answer to that. I expect this is where the good old-fashioned usability testing comes in. Get somebody on Zoom, screen shared using your software, see what happens. You can tell the AI to do it and you can simulate your users with the AI. I don't think that's credible. I don't think you're going to get as good results from ChatGPT pretending to click around on your prototype than you would from an actual human being.
00:23:37 · Lenny Rachitsky
[00:23:37] S1: This is so interesting. A question I've been tackling is just where are human brains going to continue to be valuable? What I'm hearing here is there's the initial idea. You made such a good point here. It's like the initial idea's often not the actual winning idea, it's just the beginning of an idea. So there's the idea for the feature, then there's the try it out, prototype it, help you narrow on the direction, build it, make it awesome, get it out into the world. It feels to me like AI's going to be really good at suggesting ideas and coming up with initial ideas, and I wonder if the human brain ... it's not like maybe someday we don't need human brains at all, and that's a whole other discussion, but maybe the next phase is AI will help us come up with great ideas.