Lenny's Podcast: Product | Career | Growth
00:04:08 · Lenny Rachitsky
[00:04:08] S0: This episode is brought to you by Gamma, an entirely new way to present your ideas powered by AI. If you hate designing slides and dread that feeling of staring at a blank slide, Gamma is here to help. Just upload your PRD and turn it into a beautiful, ready to present presentation in seconds. Gamma works with all types formats, from Google Docs, PDFs, to PowerPoint. You can even drop in a link to your favorite Lenny's Newsletter post and turn it into a presentation for your team. Gamma has become one of the fastest growing AI web products in the world, adding 20,000,000 new users just this past year and is setting its sights on becoming the modern alternative to PowerPoint. Whether you have design skills or not, Gamma can save you hours of time synthesizing your ideas and shaping your content. Visit gamma.app and use promo code Lenny to get a free month of Gamma Pro. That's gamma. App. Let me tell you about CommandBar. If you're like me and most users I've built product for, you'll probably find those little in product pop ups really annoying. Wanna take a tour? Check out this new feature. And these pop ups are becoming less and less effective since most users don't read what they say. They just wanna close them as soon as possible. But every product builder knows that users need help to learn the ins and outs of your product. We use so many products every day, and we can't possibly know the ins and outs of everyone. CommandBar is an AI powered toolkit for product, growth, marketing, and customer teams to help users get the most out of your product without annoying them. They use AI to get closer to user intent, so they have search and chat products that let users describe what they're trying to do in their own words and then see personalized results, like customer walkthroughs or actions. They do pop ups too, but their nudges are based on in product behaviors, like confusion or intent classification, which makes them much less annoying and much more impactful. This works for web apps, mobile apps, and websites. And they work with industry leading companies like Gusto, Freshworks, HashiCorp, and LaunchDarkly. Over 15,000,000 end users have interacted with CommandBar. To try out CommandBar, you can sign up at commandbar.com/lenny, and you can unlock an extra 1,000 AI responses per month for any plan. That's commandbar.com/lenny.
00:06:30 · Lenny Rachitsky, Sean Ellis
And the idea is that if you... 4040% or more of people say they'd be very disappointed if they can no longer use the product, you essentially have product market fit. [00:06:40] S1: I would say it's a leading indicator of product market fit. The the the lagging indicator is do they actually keep using it? So probably retention cohorts are are more more accurate. But the problem is, like your time at Airbnb, how long do you have to look at a retention cohort before you know that you've actually long term retained someone? And so with this question, can kind of find out day one. You don't actually even need a good analytics system in place to be able to see if product market fit exists. So, yeah, the 40% was not something I originally had in there. Also originally I was trying to have just a filter so that I was not treating all feedback from customers the same, but I was trying to find feedback from customers who actually really cared about the product. And then was over time as I, at the time I was working for a couple of YC backed companies. And so those companies were all pretty connected. And so I would share the question with a lot of other startups in Silicon Valley. So over time, I started to see there was a pattern that once you got a high enough percentage of users saying they'd be very disappointed, most of those very disappointed without the product, most of those products did pretty well. And then if if you felt too low, those products tended to suffer.
00:08:04 · Lenny Rachitsky, Sean Ellis
[00:08:04] S0: Okay, there's two things I want to definitely follow-up on here. The first is such an important point that you made at the beginning when I introduced this test is that you described it as a leading indicator of product market fit and actually retention. People actually using your product. The product actually being used by the market is the actual ultimate test. [00:08:21] S1: Right. [00:08:21] S0: So the idea here is this is a good way to get a sense of before you actually have data, are we headed in a good direction? Can you just speak more about that? Like how to... When to use this and when it's most useful?
00:08:21 · Lenny Rachitsky, Sean Ellis
[00:08:21] S0: So the idea here is this is a good way to get a sense of before you actually have data, are we headed in a good direction? Can you just speak more about that? Like how to... When to use this and when it's most useful? [00:08:33] S1: Mean, for me in particular, when I come into a company, my my my goal is to help them grow. And so I don't want to put myself in a situation where I'm going to fail because no one actually cares about the product. So it can really be asked at a company of any stage. It's helpful to understand who your must have users are. But essentially once you have an even an MVP, very first MVP on the product, you can still get some useful feedback about the product if it's resonating with anyone. I actually had a company where I had committed to work with them. It was right after I left Dropbox and I committed to work with these guys for six months to help them grow. I ran the question and it came back at only 7% of users saying they'd be very disappointed without the product. So I'm like, I have six months to help them grow and they're only at 7% right now. Knows? It might take six months to get the 40%. Am I doing them a disservice by being in a growth role and being on payroll during this period of time? Fortunately, with the signal and the information we got from the initial survey, we were able to get them at 40% in two weeks. [00:09:50] S0: Wow. What did you do there just as a case study?
00:09:53 · Sean Ellis
[00:09:53] S1: Yeah. So the company called Lookout, it's a mobile security company. And now most of the things in Lookout are built into iPhones and Androids. But at that time, the product had everything from backup my data to find my lost phone to protecting your phone a firewall and antivirus. And so when we ran this initial survey, I dug into the 7% who said they'd be very disappointed without the product and found that most of that 7% were focused on the antivirus functionality. So they were like, they know they need to protect their computer from viruses. Smartphones were becoming more like computers. So it just made a lot of sense for them that they'd need to protect their phone. And Interesting, at the time, I think there was only one phone virus that had ever even happened, but it was a pretty easy mental leap for people. So now we knew, okay, it's antivirus that people really valued. And so step one was just reposition the product on antivirus so that that kind of creates a filter. So anyone who now is coming in to sign up for the product who doesn't care about antivirus is not going to convert. And those who are excited about antivirus are going to convert. We already know from the initial survey that people value that after they convert. By setting the right expectations around it upfront, you're going to bring people in with the right expectations. But then the second thing that we did was we streamlined onboarding so that the first thing that they did after signing up for the product was to set up the antivirus and then get a message, you're now protected from viruses. And so it's really the combination of those two things. It's set the right expectations and then speed the value. And so the next cohort of people that we surveyed were at 40% saying they'd be very disappointed without the product. That literally took two weeks to make those changes. Six months later, it was 60% on the score. Then I think they hit the billion dollar valuation four or five years later on ultimately being one of the early unicorns. Interestingly, as all of those things were built into mobile phones now, they've completely changed the business, but they continue to do really well, But they've continued to iterate the business.
00:12:22 · Sean Ellis, Lenny Rachitsky
I think that having that kind of finger on the pulse early in the business was important to build the muscle in the business to be really responsive as the market changed. [00:12:34] S0: Sean, this is already amazing. There's just like a fractal of topics I want to explore from this very short conversation already. So the first is just follow this thread of basically you're sharing kind of a growth strategy that I imagine you execute is look for the percentage of people that would be very disappointed if your product went away. See who they are, see what they're excited about, and lean into that both positioning wise, onboarding wise, and probably also cut out stuff from your product that they don't care about. [00:13:02] S1: Yeah. And I was coming at it from a marketing perspective initially. Over time, I positioned myself more in a growth role with product and marketing as areas I could influence. But as a marketer, probably didn't have a lot of influence on a engineering founded company to say, let's cut out stuff. So it made more sense to say, let's just sequence the onboarding so that we're highlighting this and and onboarding to this. That was a little easier to sell. [00:13:33] S0: And just hearing that you can move this score so quickly without even changing the product substantially, I imagine what surprised a lot of people when you think about, like, removing retention often is really hard, and maybe we talk about that. But I guess it sounds like there's often something you can do that's not very hard that might significantly shift this product market fit test.