Lenny's Podcast: Product | Career | Growth
00:06:06 · Cat Wu, Lenny Rachitsky
I think the PMs who do the best on AI-native products are the ones who can figure out, how can I shorten the time from having this idea to actually getting the product in the hands of users and help define what are the most important tasks that need to work out of the box for my product? [00:06:37] S1: So what I love about this is what you're saying is just like people haven't grasped how fast they need to move and how much of the job now is helping the team move fast. What helps do that? What do you do, what does your PM team do to help them move this fast other than have access to the most advanced models?
00:06:57 · Cat Wu
[00:06:57] S0: I think the first thing is to set clear goals. Because LLMs are so general, that actually creates a lot of ambiguity in who we're building for, what problems we're trying to solve, what the top use cases are. And so I think a great PM is able to say, "Okay, our key user is professional developers. The main problem that we want to solve for this feature is maybe there's too many permission prompts and people are feeling fatigue, and the use case is we want professional developers at enterprises to safely get to zero permission prompts." And that actually sets a pretty clear goal because it rules out a lot of potential approaches for reducing permission prompts so that people can get a lot more done with one prompt. And then I think the second thing that's very important is figuring out some repeatable process for getting these features shipped. So for Claude Code, what we do is we actually ship almost all of our features in research preview. We clearly brand this when we ship something so that users know that this is an early product, this is just an idea, this is just something that we're trying to get feedback on and iterating on, and that this might not be supported forever. And what this does is it reduces our commitment for shipping something. We can just get something out in a week or two. And then the third thing that a PM should do is help create the framework for the team so that they know when to pull in cross-functional partners and what those cross-functional partners' expectations are. So for example, we have a really tight process between engineering, marketing, and docs. So when engineers have a feature that they feel is ready and that we've dogfooded internally, they post it in our evergreen launch room, and then Sarah, who leads our docs, and Alex who leads PMM, Antaric and Lydia on DevRel just jump in and can turn around the marketing announcement for it the very next day. And because we have this really tight process, it lowers the friction for any engineer to ship something, and PM is the role that should be setting this up.
00:08:59 · Lenny Rachitsky, Cat Wu
[00:08:59] S1: How do PRDs fit into this and the fact that you said that goals are a really important part, just like being aligned on what does success look like? Who is this for? Who's this now for? Are you writing PRDs, is it just a couple bullet points? How's that evolved in the world of a PM? [00:09:12] S0: So there's two things that we do. One is we have very rigorous metrics and we do metrics readouts with the entire team every week. The goal of this is to make sure that everyone deeply understands all the facets of our business, what our key goals are, how they're trending, and what drives them. The second thing that we do is we have this list of team principles, and this includes who our key users are, why those are our key users, and the reason that we articulate all of this is so that everybody on the team feels like they understand how our business works, they understand what's important to us and what we're willing to trade off, and it lets people make decisions by themselves without feeling like they're blocked on PM or any other stakeholder. [00:09:12] S1: I love how so much of this is like, okay, we still need PMs in the future, and there's so much talk of why do we need PMs? We're just going to ship and build. We need engineers.
00:09:12 · Lenny Rachitsky, Cat Wu
[00:09:12] S1: I love how so much of this is like, okay, we still need PMs in the future, and there's so much talk of why do we need PMs? We're just going to ship and build. We need engineers. [00:10:05] S0: Oh, we actually do PRDs sometimes. So I think for features that are particularly ambiguous, it does help to write out just a one-pager on what the goals are, what the delightful use cases are, what the failure modes currently are that we need to fix. And there are occasionally some projects, especially things that require heavy infrastructure that do take many months, and for those situations, we do write PRDs still. [00:10:29] S1: I want to drill a little bit further into just how you're able to move so fast. I've never seen anything like the pace folks at Anthropic are shipping at. Someone made this calendar of launches across Anthropic, and it was literally every day there was a major feature or product. So one question people had online is you guys just launched this, not launched, but built this incredible model Mythos that is still in preview because it's so powerful, people are a little afraid of what it can do. Have you guys been using this? Is this part of the reason you've been able to move so fast?
00:10:29 · Lenny Rachitsky, Cat Wu
[00:10:29] S1: I want to drill a little bit further into just how you're able to move so fast. I've never seen anything like the pace folks at Anthropic are shipping at. Someone made this calendar of launches across Anthropic, and it was literally every day there was a major feature or product. So one question people had online is you guys just launched this, not launched, but built this incredible model Mythos that is still in preview because it's so powerful, people are a little afraid of what it can do. Have you guys been using this? Is this part of the reason you've been able to move so fast? [00:11:03] S0: We've been moving pretty fast for several quarters now, so I think it's not fully Mythos. Mythos is an incredibly powerful model. We do use the models internally, and I think this has increased our rate of shipping a little bit, but I don't think it explains the bulk of the increase. I think a lot of it is the process and the expectation on the team. So we're very low on process. We want to remove every single barrier to shipping things. We want to make sure every single person on the team feels empowered to take their idea from just an idea to out in the world in less than a week, sometimes even in a day. [00:11:41] S1: Cool. Oh, man. What an advantage to have the best model and also be building product. That's so cool. [00:11:46] S0: We are very lucky to be able to work with the frontier models.
00:11:46 · Cat Wu, Lenny Rachitsky
[00:11:46] S0: We are very lucky to be able to work with the frontier models. [00:11:49] S1: Oh my God, what an awesome advantage, just build a thing and then use it and then accelerate faster. It's so interesting. There's a couple of these other side things, I want to just go on these sidequest on this conversation. There's so much happening with Anthropic, and I just am so curious to get your insight. One is a week ago or so, the whole source code of Claude Code leaked. Somebody got it out there. I think it was a mistake someone made. Is there anything you comment there, just like what happened? What went wrong? What should people know? [00:12:15] S0: So we immediately looked into this when we saw it. We realized that this was the result of human error. There was a human working with Claude to write PR. This was just an update to how we release our packages and it actually went through two layers of human review, and so this was a result of human error and we've hardened our processes to make sure that it doesn't happen in the future. [00:12:40] S1: Is this person still at Anthropic? Are they doing all right? [00:12:42] S0: Yes, yes. It's a process failure, and the most important thing is to just learn from it and to add more safeguards so that doesn't happen again, and so that's what we've been focused on and most of those have shifted.