Lenny's Podcast: Product | Career | Growth
00:47:04 · Cat Wu, Lenny Rachitsky
[00:47:04] S0: Oh, Applied AI is amazing at pushing the boundaries of what Claude Code and Cowork can do. A lot of our Applied AI team spends time with our customers helping them adopt our API, and so sometimes our Applied AI team will, for example, make prototypes on behalf of these customers, which Claude Code makes so much faster than it used to do. They also have the dual goal of needing to manage a lot of customer comms, a lot of customer inbound and historical context, call notes, and so they're both extremely heavy on Cowork and on Claude Code. [00:47:42] S1: And just to understand Applied AI, is that forward to play engineering sort of role? How would most people describe what the Applied AI team is doing? [00:47:52] S0: Yeah, it's helping our customers adopt the latest API and model features across their company, both for powering their company's products and also for internal acceleration. [00:48:05] S1: Got it. So it's like customer success, go to marketing, kind of for deploy engineering sort of thing. [00:48:11] S0: Exactly. It's like a very technical go to-market person. [00:48:13] S1: Got it. Okay, awesome. So you're saying that might be the second org that uses the most tokens?
00:48:13 · Lenny Rachitsky, Cat Wu
[00:48:13] S1: Got it. Okay, awesome. So you're saying that might be the second org that uses the most tokens? [00:48:19] S0: Yeah. And then we also see them pushing the boundaries of what Cowork can do. So for example, if... So a lot of these folks cover multiple customers and in any given day can have five to 10 customer engagements on a high day, and so what they often use Cowork to do is the night before, they'll ask it to summarize, okay, what are all my customer meetings that are coming up the next day? What are all the things that this customer has asked me for? What's top of mind for them? What are the action items from the past meetings? And Cowork will just put together this dossier, this brief of what they should be aware of going into the next meeting. And Cowork can also research answers. So if a customer asks, "Okay, when is feature X going to launch?" Cowork can help the Applied AI person research through Slack to get the latest ETA, add that to the notes so that during the customer call, the Applied AI person has the absolute latest. And these are just workflows that people are building for themselves and sharing with other people on their team. [00:49:25] S1: So cool. Something that kind of this question, this trend, I don't know, question topic comes up a lot recently, which is tokens spend exceeding people's salary where people just use AI and it costs more than how much they're making. Are there any numbers floating around Anthropic of just how much tokens spend, say, engineers spend, I don't know, a month, a day or PMs, anything like that?
00:49:25 · Lenny Rachitsky, Cat Wu
[00:49:25] S1: So cool. Something that kind of this question, this trend, I don't know, question topic comes up a lot recently, which is tokens spend exceeding people's salary where people just use AI and it costs more than how much they're making. Are there any numbers floating around Anthropic of just how much tokens spend, say, engineers spend, I don't know, a month, a day or PMs, anything like that? [00:49:50] S0: It is clear to us that as the models get better, people delegate far more tasks to it and they spend a lot more hours in tools like Claude Code and Cowork, and so we do see the token cost per engineer or per any knowledge worker increase every time that there's a model jump or a substantial product improvement. I think it's still much lower than what the average engineer salary is, but we see the percentage increasing over time. [00:50:22] S1: It's such an interesting... We talked about how you have access to the most cutting-edge models and other advantage of working Anthropic, I believe you guys have basically unlimited tokens. You can use as much as you want. Is that right? [00:50:33] S0: We can use a lot of tokens. Some people do run into limits, so- [00:50:37] S1: Okay, there's a limit. Okay. Boris, shut it down. Okay. It's so interesting how many advantages come from having the most advanced model. It's such an interesting flywheel that starts to kick in.
00:50:37 · Lenny Rachitsky, Cat Wu
[00:50:37] S1: Okay, there's a limit. Okay. Boris, shut it down. Okay. It's so interesting how many advantages come from having the most advanced model. It's such an interesting flywheel that starts to kick in. [00:50:49] S0: I think we also believe a lot in empowering our internal teams to build as fast as possible, and we also trust that everyone understands how much capacity that serving these models truly costs, and we trust our team to use the tokens responsibly. So it's very frowned upon to waste tokens, but we do trust individuals to make that judgment call. [00:51:14] S1: Awesome. Coming back to the PM role, we talked a little bit about this, but I think this will be really interesting for people to hear. Just what I want to understand is what do you think are the emerging skills that PMs need to develop/you most look for, AI companies most look for when they're hiring PMs these days?
00:51:35 · Cat Wu
[00:51:35] S0: I think the hardest skill is being able to define what the product should look like a month from now. I think there's a lot of ambiguity in what models are capable of in that timeline and how user behavior will change, but I think there are patterns that the best PMs can see based on how users are abusing the limits of the existing product and the best PMs can sense that, can set a direction and can steadily execute towards it and change the path if the model capabilities are much better than or worse than what they'd originally expected. I think it is very hard to be the right amount of AGI-pilled because I think everyone can see this future where the models are extremely smart and can do almost everything, in which case you actually don't need that complicated a product, you can actually just have a text box again where you tell the model what you want and it's so smart that it can add any tool or add any integration that it needs to get the job done. It knows when it's uncertain, it can ask clarifying questions. It's kind of very easy to build the product for the super AGI strong model. I think the hard thing is figuring out for the current model, how do you elicit the maximum capability? How do you help users go get onto the golden path? How do you guide users to interact with the model's strengths and patch its weaknesses? This skill is pretty rare.
00:53:19 · Lenny Rachitsky
[00:53:19] S1: And how do you build that skill? Is it just using each... Basically understanding the limits of each model? You talked about taste, understanding, having taste into what the model maybe is capable of, what it's great and not great to at, where it's changed?