Lenny's Podcast: Product | Career | Growth
00:47:37 · Boris Cherny
[00:47:37] S0: Latent demand is this idea that if you build a product in a way that can be hacked, or can be misused by people in a way it wasn't really designed for to do something that they want to do, then this helps you as the product builder learn where to take the product next. So, an example of this is Facebook Marketplace. So, the manager for the team Fiona, she was actually the founding manager for the Marketplace team and she talks about this a lot. Facebook Marketplace is based on the observation back in ... This must have been, like, 2016 or something like this. That 40% of posts in Facebook groups are buying and selling stuff. So, this is crazy. It's, like, people are abusing the Facebook groups product to buy and sell, and it's not abuse in a security sense. It's abuse in that no one designed the product for this, but they're figuring it out, because it's just so useful for this. And so, it's pretty obvious. If you build a better product to let people buy and sell, they're going to like it. And it was just very obvious that Marketplace would be a hit from this. And so, the first thing was buy and sell groups, so, special purpose groups to let people do that, and the second product was Marketplace. Facebook Dating I think started in a pretty similar place. And I think the observation was if you look at profile views, so, people looking at each other's profiles on Facebook, 60% of profile views were people that are not friends with each other that are opposite gender. And so, this traditional dating setup, people are just creeping on each other. So, maybe if you can build a product for this, it might work. And so, this idea of latent demand I think is just so powerful. And, for example, this is also where Cowork came from. We saw that for the last six months or so, a lot of people using Claude Code were not using it to code. There was someone on Twitter that was using it to grow tomato plants. There was someone else using it to analyze their genome. Someone was using it to recover photos from a corrupted hard drive that was, like, wedding photos. There was someone that was using it for I think ... They were using it to analyze an MRI. So, there's just all these different use cases that are not technical at all, and it was just really obvious. Like, people are jumping through hoops to use a terminal to do this thing.
00:49:43 · Boris Cherny
Maybe we should just build a product for them. And we saw this actually pretty early. Back in maybe May of last year, I remember walking into the office and our data scientist Brendan had a Claude Code on his computer. He just had a terminal up. And I was shocked. I was like, "Brendan, what are you doing?" He figured out how to open the terminal, which is ... It's a very engineer-y product. Even a lot of engineers don't want to use a terminal. It's just the lowest level way to do your work. Just really, really in the weeds of the computer. And so, he figured out how to use the terminal. He downloaded [inaudible 00:50:28]. He downloaded Claude Code. And he was doing SQL analysis in the terminal. It was crazy. And then the next week all of the data scientists were doing the same thing. So, when you see people abusing the product in this way, using it in a way that it wasn't designed in order to do something that is useful for them, it's just such a strong indicator that you should just build a product and people are going to like that. It's something that's special purpose for that. I think now there's also this interesting second dimension to latent demand. This is the traditional framing is look at what people are doing, make that a little bit easier, empower them. The modern framing that I've been seeing in the last six months is a little bit different. And it's look at what the model is trying to do, and make that a little bit easier. And so, when we first started building Claude Code, I think a lot of the way that people approached designing things with LLMs is they put the model in a box, and they were like, "Here's this application that I want to build. Here's the thing that I wanted to do, a model, you're going to do this one component of it. Here's the way that you're going to interact with these tools and APIs," and whatever. And for Claude Code, we inverted that. We said, "The product is the model. We want to expose it. We want to put the minimal scaffolding around it. Give it the minimal set of tools. So, it can do the things, it can decide which tools to run. It can decide in what order to run them in," and so on. And I think a lot of this was just based on latent demand of what the model wanted to do. And so, in research we call this being on distribution. You want to see what the model is trying to do.
00:51:50 · Boris Cherny, Lenny Rachitsky
In product terms, latent demand is just the same exact concept but applied to a model. [00:51:55] S1: You talked about Cowork. Something that I saw you talk about when you launched that initially is your team built that in 10 days. That's insane. [00:52:02] S0: Yeah. [00:52:02] S1: It came out, I think it was used by millions of people pretty quickly, something like that being built in 10 days. Anything there? Any stories there other than it was just, "We used a lot of code to build it and that's it"?
00:52:14 · Boris Cherny
[00:52:14] S0: Yeah. It's funny. Claude Code, like I said, when we released it it was not immediately a hit. It became a hit over time, and there was a few inflection points. So, one was, like, Opus 4, it just really, really inflected, and then in November it inflected, and it just keeps inflecting. The growth just keeps getting steeper and steeper and steeper every day. But for the first few months it wasn't a hit. People used it, but a lot of people couldn't figure out how to use it. They didn't know what it was for. The model still wasn't very good. Cowork when we released it it was just immediately a hit. Much more so than Claude Code was early on. I think a lot of the credit, honestly, just goes to Felix and Sam and Jenny, and the team that built this. It's just an incredibly strong team. And, again, the place Cowork came from is just this latent demand. Like, we saw people using Claude Code for these non-technical things. And we're trying to figure out, "What do we do?" And so, for a few months the team was exploring, they were trying all sorts of different options. And, in the end, someone was just like, "Okay. What if we just take Claude Code and put it in the desktop app?" And that's, essentially, the thing that worked. And so, over 10 days they just completely used Claude Code to build it. And Cowork is actually ... There's this very sophisticated security system that's built in. And, essentially, these guardrails to make sure that the model does the right thing, it doesn't go off the rails. So, for example, we ship an entire virtual machine with it, and Claude Code just wrote all of this code. So, we just had to think about, "All right. How do we make this a little bit safer? A little more self-guided for people that are not engineers." It was fully implemented with Claude Code. It took about 10 days. We launched it early. It was still pretty rough, and it's still pretty rough around the edges. But this is the way that we learn, both on the product side and on the safety side is we have to release things a little bit earlier than we think, so, that we can get the feedback, so, that we can talk to users. We can understand what people want, and that'll shape where the product goes in the future.
00:54:05 · Lenny Rachitsky
[00:54:05] S1: Yeah. I think that point is so interesting, and it's so unique. There's always been this idea, release early, learn from users, get feedback, iterate. The fact that it's hard to even know what the AI is capable of, and how people will try to use it is a unique reason to start releasing things early. So, that'll help you as you exactly describe this idea of what is a latent demand in this thing that we didn't really know? Let's put it out there and see what people do with it.
00:54:30 · Boris Cherny
[00:54:30] S0: Yeah. And for Anthropic as a safety lab, the other dimension of that is safety, because when you think about model safety there's a bunch of different ways to study it. The lowest level is alignment, and mechanistic interpretability. So, this is when we train the model we want to make sure that it's safe. We, at this point, have pretty sophisticated technology to understand what's happening in the neurons to trace it. And so, for example, if there's a neuron related to deception, we're starting to get to the point where we can monitor it and understand that it's activating. And so, this is alignment, this is mechanistic interpretability, it's the lowest layer. The second layer is evals, and this is, essentially, a laboratory setting, the model is in a Petri dish, and you study it. And you put in the synthetic situation and just say, "Okay. Model, what do you do?" And, "Are you doing the right thing? Is it aligned? Is it safe?" And then the third layer is seeing how the model behaves in the wild. And as the model gets more sophisticated, this becomes so important, because it might look very good on these first two layers, but not great on the third one. We released Claude Code really early, because we wanted to study safety. And we actually used it within Anthropic for I think four or five months, or something before we released it, because we weren't really sure. Like, this is the first big agent that I think folks had released at that point. It was definitely the first coding agent that became broadly used. And so, we weren't sure if it was safe. And so, we actually had to study it internally for a long time before we felt good about that. And even since there's a lot that we've learned about alignment. There's a lot that we've learned about safety, that we've been able to put back into the model, back into the product. And for Cowork, it's pretty similar. The model is in this new setting. It's doing these tasks that are not engineering tasks. It's an agent that's acting on your behalf. It looks good on alignment, it looks good on evals, we tried it internally, it looks good. We tried it with a few customers, it looks good. Now we have to make sure it's safe in the real world. And so, that's why we release a little early. That's why we call it a research preview. But, yeah. It's constantly improving.