Lenny's Podcast: Product | Career | Growth
00:24:00 · Lenny Rachitsky, Sean Ellis
[00:24:00] S0: Amazing. Yeah. Sec... I think it's the second most popular episode behind Brian Chesky. Okay. So he had a question of just what are the limitations of the score? When does it break down? When should you not use it if ever? Is there anything of just, like, here's when it's not gonna work for you? [00:24:14] S1: I think one off products would probably... You know, like, how would you feel if you could no longer watch the movie you just watched? Like, I wouldn't care. Even like when I run a workshop, I don't run this as part of my survey after I do a workshop because, like how would you feel that you can no longer attend the workshop you just attended? It doesn't make sense. So I'll ask an NPS question as my filtering question so that I'm focusing in on feedback of people who love it. Also then through a separate lens, looking at people maybe who would be my detractors. So I think one off products are probably not good products to run the question on. There may be other places as well that I'm not thinking of right now, but Okay. [00:24:59] S0: But it sounds like not many. What I'm hearing is it's generally widely applicable. [00:25:04] S1: Yeah. Think it is. Like, at least from from my perspective, like, it's it's been really useful for me anyway.
00:25:04 · Sean Ellis, Lenny Rachitsky
[00:25:04] S1: Yeah. Think it is. Like, at least from from my perspective, like, it's it's been really useful for me anyway. [00:25:11] S0: Awesome. Okay. And then the follow-up question from Shiraz is... And I kind of asked this, but I'm curious if there's anything more here. Just have you seen any instances of startups over relying on the score prematurely declaring product market fit when in reality they haven't reached it yet? And just... Are there any other caveats of, like, cool. I got 40%. Is there anything else you should know? Okay, but maybe check this one thing.
00:25:31 · Sean Ellis
[00:25:31] S1: Yeah. I mean, I think to me, it's kind of like, what really is the definition of product market fit is the definition that people who get through my crappy onboarding and actually experience the product love it. And if I'm able to retain those people, that means I have product market fit. Or is fixing that crappy onboarding part of getting to product market fit as well? I think that's up for debate. To me, the hardest... I wouldn't obsess on onboarding if I know those who kind of get through the challenge of getting started with a product still don't like the product, then feels like it's a core product issue or wrong people using it in the wrong way issue. But once you have that, then ultimately it doesn't mean that you're ready to grow. I focus on growth, then customer acquisition is almost the last step. Once I validate that it's a must have for those early users, then I'm thinking about, Okay, how do I optimize speed to value? How do I make sure that people have the right prompts to come back and use the product at the right time? It's kind of more of that engagement loop. How do I get my existing users to bring in more users if there's something that makes sense on that end? Even how do optimize my revenue model? And once all of those things are working well, then I'll obsess on the customer acquisition side. But customer acquisition is so hard that if you're not really efficient at converting and retaining and monetizing people, you're gonna really struggle on the customer acquisition side.
00:27:08 · Lenny Rachitsky, Sean Ellis
[00:27:08] S0: Yeah. Cool. And we'll talk about customer acquisition slash growth. Sure. Another question I wanted to ask and a couple listeners asked is the 40%. So I had a Jag from Nubank on the podcast. I think you may have worked with them, and they use 50% as their threshold because apparently Brazilians are very nice and they just [00:27:28] S1: Optimistic. He's not gonna use what he said. Yeah. [00:27:30] S0: Yeah. So I guess the question is, you find instances where you should increase that percentage and slash in b two b is anything different? Do you change the percentage in b two b? Any advice there just like when you adjust the threshold?
00:27:42 · Sean Ellis
[00:27:42] S1: Yeah. I hadn't really thought thought too much on that. Again, for for me, generally I'm trying to just figure out, is this a product that can grow? So when I'm using it, I'm trying to just... So if I got a 37%, am I going to be like, Oh, no, this would be impossible? Or if I had a 70%, does that mean I guarantee, say, Oh, yeah, I want to jump in and work with this company. It's more nuanced than that. Obviously, if it's a 70% but I have no idea how I grow the business, I'm going to be stuck there. I do think he brought up a really good point that culturally some people are going to be more optimistic or pessimistic. Interestingly, when I came up with a question, I used to just use kind of a normal satisfaction question. When I was working at Xobni, I'm just like an intensely curious person anyway, so I'm just trying to dig in and understand the customers. And so I've always done lots of surveying, but at Xobni, I was going to ask it as, like use my filter as a satisfaction question. So, you know, how satisfied are you with this? I'm very satisfied. I'm somewhat satisfied. And our main customers were actually senior management. And so I thought, senior management's never satisfied. I'm going to get always this super lukewarm thing. How can I change this question more me a more kind of real answer from these guys? Well, if I flip it and say, How would you feel if you could no longer use this product? I'll probably get a more honest answer back from them. And of course, they're very disappointed if they can't get what they want. And so initially it was just for the case of Xobni, but then I went to Dropbox right after Xobni and I'll try the question again. And the insights I got back were really useful. And so each company I went to, I kept using the question. I'm like, this works way better than your typical satisfaction question. But initially, it was more about thinking just senior management to get a more honest answer out of them.
00:29:54 · Lenny Rachitsky, Sean Ellis
[00:29:54] S0: So that's the origin story right there. [00:29:56] S1: Yeah. [00:29:57] S0: Wow. That senior managers are just very harsh and they... They're not... They don't need anything. Yeah. And you have to flip it. That is so interesting. Just like that question is such a good reminder of how hard it is to build anything people really would be disappointed not to have. Like, that's why this works so well. People are like, I don't need this app. Who cares? Right? Like, that's the core of this. It's just... That is hard.