Lenny's Podcast: Product | Career | Growth
00:14:10 · Simon Willison, Lenny Rachitsky
[00:14:10] S0: So, this is the fascinating thing is ... so there's a policy of nobody writes any code and quite a few companies are beginning to introduce that now, because- [00:14:18] S1: Just to be clear, the policy is you cannot write code, it has to be written by AI. [00:14:21] S0: You can type code into a computer, exactly. [00:14:23] S1: Yeah.
00:14:24 · Simon Willison
[00:14:24] S0: Honestly, six months ago I thought that was crazy, and today probably 95% of the code that I produce I didn't type it myself. So that world is practical already, because the latest models are good enough that you can tell them, "Oh no, rename that variable and refactor that and add this line there," and they'll just do it and it's faster than you typing on the keyboard yourself. The next rule though is nobody reads the code, and this is the thing which StrongDM started doing back in, I think it was August last year. They said, "Okay, we're not going to read the code." So, what does that mean? How do you produce software that works and is good if you're not reading the code? They've come up with a whole bunch of answers. One of the most interesting was the way they did testing where in traditional software some companies will have a QA department. The engineers write about your software and then you throw it over the wall to the QA department and they test it furiously to figure out if it's working or not. That I think went out of fashion a bit over the past five to 10 years from what I've seen in Silicon Valley, because you kind of want your engineers to take responsibility for the code they're writing being good. But what if you can simulate that QA department? So, what StrongDM were doing is they had a swarm of agent testers who were actually simulating end users. So, the software that they were building ... this is crazy. The software is security software for access management, so when you start as a company and somebody needs to assign you access to Jira and then give you access to Slack and all of that kind of thing, they were building software for that. That's very security-adjacent. That's not the kind of thing that you should be vibe coding at all based on most people's understanding of how the world works. They're a legitimate security company who've been doing this stuff without AI for years, so it's not like they didn't understand the risks. So, the way they did their testing is they had this swarm of simulated employees all in a simulated Slack channel saying things like, "Hey, could somebody give me access to Jira?" The Slack channel itself is simulated, we'll talk about that in a moment, and 24 hours a day they're making requests and saying, " Hey, I need access to Jira," and all of those kinds of things at an enormous cost.
00:16:33 · Simon Willison, Lenny Rachitsky
They were spending $10,000 a day on tokens, I think, simulating with these end users. [00:16:38] S1: A day?
00:16:38 · Simon Willison
[00:16:38] S0: I believe so, but it meant that the software was being very robustly tested in all of these different ways. Yeah, it's kind of similar to having a manual QA team, except one that never sleeps. I thought that was fascinating as a example of thinking outside of the box. Taking this question, how do we tell our software's good if we're not reviewing the code and trying to find creative answers to it. The other thing that was interesting is that the Slack channel itself wasn't actually Slack, because it turns out if you test against real software like Slack and so forth, they all have rate limits and they won't let you just run 10,000 simulated people at a time. So, what they did is they built their own simulation of Slack and Jira and Okta and all of this software they were integrating with. The way they did that is they basically took the API documentation for the public APIs for Slack and the open source client libraries, and they told their coding agents, "Build this. Build me a simulation of this API," and they did. So, this company is ... and this is one of the things ... I went to a demo that they gave back in October. One of the things that really sat with me is that they had their own simulated version of Slack and Jira and all of these different systems that they could then build their software against, which cost them nothing, because once they spun it up it was a little Go binary that sat there, and they even had interfaces. They had a fake version of the Slack interface that they'd vibe coded up that let them see what was going on. Absolutely fascinating.
00:18:08 · Lenny Rachitsky
[00:18:08] S1: That is such a cool story, and I love these stories of just companies at the bleeding edge trying to see what's possible and have an advantage essentially. So, what I'm hearing here is the QA piece is like the new piece in this factory. So we already have Codex, Claude Code, they can go off and build stuff. Is the innovation here, okay, now you've built all this stuff, is it actually any good? Is there a reason why Codex and Claude Code couldn't do this themselves? Why do you need this factory concept?
00:18:32 · Simon Willison
[00:18:32] S0: I think they can. You can tell Claude Code, fire up a subagent that uses Playwright to simulate a browser and all of that kind of thing. You'd have trouble getting it to run 24 hours a day. I mean, maybe it would work. But certainly I think what's interesting to me isn't so much the software you're using. It is these big ideas, these techniques that you're using to try and answer these questions, because even if your QA team, your virtual QA team says this is good, doesn't mean it's secure, right? It doesn't mean that you've got all of those other characteristics that you care about. At the same time, the agents are getting really good at security penetration testing now. This is a new thing ... I think, again, in the past three to six months they've started being credible as security researchers, which is sending shockwaves through the security research industry. They were like, "Wow, we didn't think that they'd get to this point." What's interesting there is both OpenAI and Anthropic have specialist security models that they will not release to the general public, because they can be used to break into websites. So they have invite only registered security researchers can apply for access, and they've been producing vulnerability reports against popular open source software. I think Firefox just a few days ago, maybe last week, said that they'd done a release, which was assisted by Anthropic. Anthropic had discovered 100 potential vulnerabilities in Firefox and responsibly reported them to Mozilla who then fixed them. That's an interesting one as well, because we're seeing a lot of this in the wild and it's just incredibly frustrating for maintainers, because there are these people who don't know what they're doing who are asking ChatGPT to find a security hole and then reporting it to the maintainer. The report looks good. ChatGPT can produce a very well-formatted report of the vulnerability. It's a total waste of time, it's not actually verified as being a real problem. The difference with Anthropic and Firefox is that Anthropic's security team actually did do the work. They didn't report whatever the agent said, they actually verified that it was a good quality report before they handed it over.