Lenny's Podcast: Product | Career | Growth
00:19:16 · Lenny Rachitsky, Dianne Penn
[00:19:16] S1: That's so interesting that you may have developed this AI brain that can do something you're not even aware of. And so part of the job is just uncovering, "Wow, we just got really good at this thing. What can we do with that?" [00:19:27] S0: I think there's product overhang and user overhang, to maybe put it in our PM language, even on today's models. And I think there's a lot that we could be exploring on our current opuses and definitely with Fable, for example. And that discovery is actually another part of what's been in the early days of Anthropic's DNA. And I think it's also continuing to be a big part of how we operate in product, in Labs and across research. [00:20:03] S1: This makes me think about something Garry Tan's been talking about, president of YC, I don't know what his title is. He had this interesting point that if you are willing to spend $100,000 a year right now in tokens, you are living the way somebody in 2028 is going to live. Because by then it'll be really cheap. Everyone can work this way. But there's this alpha opportunity right now to just live in the future, go crazy on token spend. And so there's a big opportunity for people to learn what the future's like and also just build much faster. Thoughts on this idea and the value of token maxing, let's call it.
00:20:37 · Dianne Penn
[00:20:37] S0: Yeah. I think I take more of a almost product lens. It's almost like token spin is more the input. And really the output is what you described of experimentation. And I think if we were orienting goals around experimentation, I feel like that might be the better framing of the outcomes. And therefore there might be different ways of achieving that outcome. I will say internally, some of the most creative thinkers, the best prototypers do spend a lot of time with Claude, with every new version of a research model that we have. And so there is something around you have to be using the models to then come up with good, then great, then better ideas. And there's no substitute for that. It's very hard to come up with a perfect strategy without touching the technology when it's moving this quickly. At the same time, I think there's other things that we could be doing. So one thing that we do a lot is actually working in public internally within Anthropic. And so in the early days when we had less product surfaces, there was a Slack channel where everyone, almost the entire company was testing early versions of Claude and trying different use cases. People were not calling them use cases, but you might be asking it to edit an essay or to come up with the right way to send this email. They were all different use cases, but we all worked in public. And then what you would see magically is different users or different folks on the team coming up with an idea and then other people trying different variations of that idea. And then within maybe 10 or so requests, there was something magical or potentially in a use case that emerges. And I think there's a lot in not just individuals figuring out by themselves how to use this technology. I think we could be doing more to actually bring that communal discovery when we do experimentation. Experimentation is not always necessarily a individual sport.
00:22:55 · Lenny Rachitsky, Dianne Penn
[00:22:55] S1: It's so interesting. Yeah, this idea that we're not sure what this is capable of or what we could do with it. And it takes all this poking around and people trying things, hearing what other people are trying to figure out what's possible. Such an interesting, I don't know, technology. We're just like, okay, here's what... Oh, I figured out it could do this thing. What are you going to do with that? [00:23:14] S0: I think in a broad theme we know. We know that the models could write great essays or could write long form writing, but individual pain points of what can you actually solve with that and bring it to a user level that people can use, I think is something that is more exploration or experimentation based. [00:23:33] S1: So following this thread, you oversee product for the Labs team, which is extremely cool. We've had Ben Mann on the podcast, Mike Krieger, whom both work on Labs now. Talk about Labs. What is Labs? What's come out of Labs? Many people have heard of these things. And how do they work that enables them to create such innovative ideas outside of even the core Anthropic product team? [00:23:58] S0: The thesis of Labs in many ways is identifying and pulling the thread on... ... in many ways is identifying and pulling the thread on the thread of discontinuous large bets that might not be in the core roadmap, and figuring out is there a there there? And also, what is the 10X, 100X, 1000X of the there there? And so for example, things like Claude Code, I think- [00:24:26] S1: I've heard of it.
00:24:26 · Lenny Rachitsky, Dianne Penn
[00:24:26] S1: I've heard of it. [00:24:29] S0: Things like Claude Code, things like skills, and most recently Claude Design, MCP. The thing that we really try to emphasize within the teams is, especially right now, there are so many things that could be built. What does it mean then to have a discontinuous bet? And I think one approach that we're taking this year is you can be very strongly held opinion about the theme or the area, and then more weakly held about the exact prototype. And so there's a culture of experimentation. There's a lot of the bottoms up engineers on the team are very self-enabled, self-driven to test out different ideas. And sometimes, we have a thesis, and it might not work yet, and so we then might revisit it in one to two model generations. And so this idea of these prototypes that actually end up just helping us learn, that's also valuable even if it doesn't lead to something immediately shipping. And so I think that allows the incubation and the charter of Labs to really accelerate and see around corners more broadly for Anthropic. [00:25:44] S1: It's so funny to think about Labs within an Anthropic, which was already so innovative and creative and just shipping like crazy, that there's value to still creating a Labs team within Anthropic. What enables Labs to work as well as it has? Because you listed all these products and it's like, what else has Anthropic shipped? It feels like all the biggest wins almost. I'm sure there are many that I'm not thinking about right now. What's core to creating a successful Labs org within a larger company?
00:25:44 · Lenny Rachitsky, Dianne Penn
[00:25:44] S1: It's so funny to think about Labs within an Anthropic, which was already so innovative and creative and just shipping like crazy, that there's value to still creating a Labs team within Anthropic. What enables Labs to work as well as it has? Because you listed all these products and it's like, what else has Anthropic shipped? It feels like all the biggest wins almost. I'm sure there are many that I'm not thinking about right now. What's core to creating a successful Labs org within a larger company? [00:26:12] S0: I think the team culture, similar to broadly at Anthropic, I think the team culture is very valuable. I think Ben sets an incredible vision and pushes people to think about the 10X, 100X of the idea. And the teams, the pods within Labs is small, sometimes these ideas start with one engineer. And I think sometimes when there's almost really large teams pursuing very ambiguous large ideas, you end up actually being slowed down because of that. So I think it's culture. I think we actually also select for folks who actually want to do that zero to one experimentation. And it's not easy. There's a lot of bets that we end up turning down or turning off. And maybe we revisit them in the future, but that's hard. That's hard when you pour your heart and soul, you're acting as a founder for a bet and it's not working yet. So I think it's like that type, it's selecting for that type of personality, folks who are really passionate and deep about the zero to one.
00:27:30 · Lenny Rachitsky
[00:27:30] S1: So you lead product for the research team, you work with the researchers at Anthropic. A lot of people get a sense of what is research. What do researchers do? I think a lot of people don't totally understand these very valuable people at all the AI labs. The way I think about it, and I want to help people understand, help me understand just what are researchers doing all day. What I imagine is they have a hypothesis for how to improve the model. They find data, they tweak some algorithms, they adjust how it's trained, and they test it, see how it did, keep iterating and keep trying to find ways to improve the model. Is that roughly right? Slash help us understand what researchers are doing all day.