Lenny's Podcast: Product | Career | Growth
01:10:28 · Lenny Rachitsky
[01:10:28] S0: I think that last point is really powerful, and I'm just writing it down. You said so. Essentially, one of your tactics is talking to users, asking them how did you find this product, and how do you normally find products like this? Was that the second question? I I think that's that's like such a... It's like similar to your Sean Ellis test. It's such a simple question, but it's so powerful because how else will people find your product? It's... They go to a place to find stuff like this. Yeah. And, like, they... I Google... I search Google for folder sharing. Like there's so much there that I
01:11:01 · Sean Ellis
[01:11:01] S1: I think you need skip think the reason that you don't actually kinda hear people taking the obvious route there a lot of times is because... And I I I used to be in the same thing that people tend to be either over indexed on qualitative or over indexed on quantitative. So it's kind of like, you know, analytics. I'm going to get all my answers from testing and analytics, or I'm going get all my answers from traditional customer research. I was very much in that initial camp for the first five years of my career. I'm just going to measure everything and test the heck out of things and find stuff that works. I had a VC who was our lead VC at LogMeIn who just said, When was the last time you talked to a customer? Just pushed me to survey and talk to customers all the time. At first I was gave like, the smart ass answer. I don't care what they say, I care what they do. That's what he's like, no, you got to talk to them. Then just to appease him, would try to have a conversation every day because he was in our office a lot. And so I could say, Hey, yeah, I talked to a customer today when would ask me. And then I started finding that my experiments were so much better the more I talked to customers. Eventually I became very much like the blend of qualitative and quantitative research leads to much better tests.
01:12:21 · Lenny Rachitsky, Sean Ellis
[01:12:21] S0: That is another amazing story and insight. It's so interesting that people sometimes think of you as growth hacker guy experiments data when most of the advice you've been sharing so far is very qualitative driven, very survey driven, talking to customers driven. [01:12:35] S1: Yeah. And it's it's just really hard to run good experiments when you can't deeply contextualize what's going on. [01:12:42] S0: I love this. By the way, I I don't know if I knew this. So you helped develop the Dropbox referral program?
01:12:47 · Sean Ellis
[01:12:47] S1: I was there at the time. I basically... When I... Even when I first started talking with with Drew before I came in, I was like, I think the way we're gonna grow this business is by leveraging the really passionate customer base, and that's what we need to double down on. And we had tried a similar kind of referral program at Zodney and my friend who actually started Ring, Jamie Siminoff had previously had a company called PhoneTag like way before Ring and he had actually done a lot of the testing on kind of double sided referral programs and having incentive on both sides. He found that that worked the best. And so between what we had tested at Zobny and those conversations with him, I hadn't actually seen PayPal yet at that point what they were doing. But yeah, that was kind of like, yeah, it seems like a referral program where we have incentives on both sides is the best way to go. Interestingly, six months before I was at Dropbox, I was at LogMeIn and I really thought about having incentivized referrals that LogMeIn, but 80% of our new users were coming in through word-of-mouth and I had 100,000,000 devices connected in on our system and I was just so afraid of breaking this growth engine by adding an incentive that I didn't want to risk it. But at Dropbox, was so early that, yeah. I would still say no experiment is one person. It happened to be when I was there, I had some insights that I brought in, ultimately the guy who built it was actually an intern named Albert Ni. He ended up dropping out of MIT to stay with Dropbox for a few years after that. Yeah, he was kind of my right hand guy to collaborating on growth day to day.
01:14:46 · Lenny Rachitsky, Sean Ellis
[01:14:46] S0: Wow. Would say Dropbox as a referral program and the PayPal referral program, as you mentioned, are the two most legendary studied copied referral programs And out [01:14:56] S1: unfortunately, I think that what they don't realize is that before the referral program, Dropbox had amazing referral rate. Companies that are trying to copy it are like, why isn't anyone talking about our product? Let's add a referral program with incentives. Like, to me, I think it's it's a great accelerant when it's already working. But it can't it can't fix it if if people don't want to don't want to talk about your product. [01:15:24] S0: That's an awesome point and something I was just gonna ask about. And just kinda coming back to this topic of growing engagement, growing referrals as a as a growth mechanism, what do you look for to tell you that there's an opportunity there? And I'll just answer it partly as I've seen exactly what you just said, which is you need to already have strong word-of-mouth growth because referrals kinda sits on that and gives you a little more incentive to share. So maybe you agree with that, not agreeing with that. Any other advice on helping figure out is there some kind of loop here that we can build?
01:15:53 · Sean Ellis
[01:15:53] S1: Well, one thing I will say is like, freemium, when we first started with it, as I said, we're like, we we were one of the first with it. The... So it took me a while to figure out exactly how freemium worked, but to me, freemium towards having a free and a premium version of your product to really work in any business, it needs to be that your free product is so good that people naturally have word-of-mouth around that product. Then to be economically viable, you have to have a premium product that's better enough and differentiated enough that people are going to upgrade to the premium product. But I think a lot of times people are so worried about the second part that they make the free version not very good. And then they're surprised when word-of-mouth isn't very strong there. So I think you have to essentially have two distinct products that are, are great on their own. So that would be the one piece. And so, but then obviously companies that have any kind of collaborative layer to them are going to benefit from, are going to be more likely to work well with referral. And then I think on the engagement side, a lot of it comes down to just the nature of the product. Like, Airbnb, you're not going to use it every day, unless you're a vagrant or something maybe, then you won't have money to pay for it. So so there's kind of a natural usage cycle to products and you want to be able to to maximize against that cycle. And I think that's where I was saying, yeah, coming back to the the the hooked model, I think is is a really good way to help to have a framework to think about how do I improve engagement. One good counter example to that though of the natural frequency of using a product is Facebook when they change their North Star metric from monthly active users to daily active users. I think, again, just having what gets measured gets managed. Once Facebook was on a daily active user goal, the team suddenly had a lot more incentive to think about how do I bring people back every day and use this product, where when it was monthly active users, they only got credit for that person for using once in that month. Even if they used 10 times, they didn't get 10 times a credit. It was just like a, Oh, that's cool too, but they weren't measured on that.