Lenny's Podcast: Product | Career | Growth
00:18:16 · Jason Cohen
[00:18:16] S1: Yeah, yeah. Okay. So assuming you agree, yeah, I don't like customers leaving, that sucks, so obviously you got to find out why they're canceling and do something about it. And the kind of root issue here is they don't want to tell you. They're already out the door. They've already stopped investing in you mentally. So the last thing I want to do is spend time with you or really think about it and diagnose it with you. And I have a funny story about this for myself. So at A Smart Bear, people would cancel, we'd put up this form and a dropdown list, too expensive, project ended, this little stuff like we do, so we could gather data. And one of them did have more selection than the rest. And I realized it was the first one on the list. And I thought, huh, I wonder if people are just picking the first one. So then we randomized the list so everyone saw a different order of the list, and now all the items were picked equally, like, oh, right. It's complete noise. And I know other companies have done similar things also with the similar results that this is a global phenomenon. So okay, so what do you do? The point is, it's hard. So the first thing is you want to ask open-ended questions. I know you want to just get a list, but this is the problem. At least with open-ended questions, I mean, most people won't answer, but at least you might be able to get some kind of thing that they generated. And when you do this, the wrong way is to ask, why did you cancel? Because again, this allows them to say something really simple like budget, which may or may not be true. I'll get to that in a second. What you want to do is say, "What made you cancel?" In other words, what about the product or situation or whatever caused the cancellation? Just phrasing it that way, you get much better results. And I stole this from a company called Groove, who has this great case study online about this very thing. They had an email that they sent out, which is a great email, and they started by asking, "Why did you cancel?" They got 10% usable responses. They changed the same email to what made you cancel, and it's 20% usable responses. So there's, I guess, maybe some anecdata that this is a good idea. But the point is you really want them thinking about the product and not just coming up with an excuse.
00:20:25 · Jason Cohen
The next thing is the few times you do talk to them, so you want to go into, delve as far as you can into there, because most people won't talk, the temptation is to hear what they generate at first and say, "That's the answer." So a really common one is it's too expensive. I think anyone who's looked at cancellation data at any company will agree that too expensive is often the number one or at least top three reason in one form or another. And that is never, ever, ever the reason. How do I know? Because they already looked at your homepage, read all the stuff, saw what you promised, looked at the pricing page and decided to buy it. That means it, whatever was in their mind of what it is, is not too expensive. They already decided with their actions. It was not too expensive. Something else happened, but you didn't fulfill the promise that at least they thought you made or something else didn't work or... Now, it is possible they lost budget, but that doesn't mean you're too expensive. That means they lost budget. That's a very different reason. So it's sort of like, this happens in healthcare, for example. So when someone dies, a doctor has to write what's called a proximate cause, which is why did they actually die? But then you try to also write down the real reason. So let's say someone comes in and the proximate cause of death is they stop breathing. Well, you could stop there and that's like listening to it's expensive and going, "That's it." Well, why did they stop breathing? Because they ran their car into a telephone pole and were injured so much that eventually they stopped breathing. Why did they run their car into a telephone pole? Because they passed out at the wheel. Why did they pass out at the wheel? Because they had undiagnosed diabetes. Now we're getting somewhere. It still isn't just one root cause, another as a sidebar. I hate the idea of a root cause. Complex systems do not have one root cause. They often have many interlocking things that could be done to detect earlier or to change it or to reduce and not one root cause. So the root cause analysis to me is, by the way, an incorrect thing. I'm explaining why right now with the healthcare, right? Because, well, what about this undiagnosed?
00:22:32 · Jason Cohen, Lenny Rachitsky
Well, maybe part of the problem is we have a healthcare system that isn't preventative and part of it is that, but they didn't go to the doctor anyway. Okay, so there's all kinds of things that could be useful and interesting to prevent this or make it better. That's the point. That's what an analysis should be, is this array of things, not the root cause. Anyway, something along the lines of undiagnosed diabetes is much more of a cause than stopped breathing. So when we say it's too expensive and that's the reason, you're making this fallacy, you got to go into, well, they wanted this stuff, but it didn't work with Linear, which is what they use. It only works with Jira. And so there's a lack of integration. Now, maybe we should write that integration and maybe we shouldn't. Of course, it depends on how much we hear about it. And of course it's going to depend on other things, but that's the reason, not it is expensive, right? And so this idea of getting into not even the root cause, but let's say root-er causes. [00:23:30] S0: The roots cause.
00:23:31 · Jason Cohen
[00:23:31] S1: Yeah, the more root. I think some people probably say five whys and just paper over what I just said with that. And maybe so, but let's not be so simplistic about that, because again, five why sometimes implies that there's some root cause at the bottom of the whys. Let's be a little more smart about that. So anyway, these things too expensive, this is not it. Maybe project ended really is project ended. Okay. But even there, I see just today, today on an entrepreneur forum I'm on, someone said, "Yeah, we're starting to see more people have project ended as the reason, and so there's nothing we can do about that." Now see, that's incorrect. That's only true if you only look at the proximate thing, which is project ended. You're correct that you can't make that project not end exactly. Yeah. Okay, but wait a minute. If your software was more successful and the project was more successful, would it have ended or is that actually an indicator that your product wasn't that useful or didn't do its job? It's possible, like in this case, who knows, right? But that's possible that it really is your fault. Another example is, but you picked what target segments you are going after. Did you pick a market segment that was easier to sell to, but their projects end like small business and consumers where very often the small business does go out of business where the project ends, et cetera? Because when things are small, they have high variance and lots of things can knock them off the path and so on. And so is it your fault for picking the wrong ideal customer profile or target segment? And so yes, that one case of that one project, that's not your fault, quote-unquote, but by saying that you're just ignoring the fact that there is maybe something to do about it... Now, all this is maybe. None of this proves you should change your market, but when you say there's nothing we can do about it, you are closing the door on these things that might be the right thing. And very often, as I think probably a lot of people here on this listening to this know, the market segment you pick has a lot to do with your retention rate because everyone acts differently.
00:25:36 · Jason Cohen
And so anyway, so I know it's a lot on this topic, but I just feel constantly, people make this particular mistake of just abdicating responsibility or just listening to the first thing they hear and saying that's the reason and that's not right. So that's the big thing about listening. Another thing is you got to ask when people are in trouble, but not yet canceled. You might be able to save them. You certainly can learn more because you can talk to them. They're not shut off yet from you. So this might be, they never uploaded their data, so they're not being successful. They are calling tech support too much. They're in trouble. They're not calling tech support enough. They're not engaged. They didn't log in for a while. There's all kinds of things where... Now, of course, the details are going to depend on the product, obviously, but there are signals that are correlated with cancellation. Now, if you have a lot of data, you can literally correlate signals with cancellation and try to extract that precisely, but even without data, you can guess. And guessing and having a theory, acting accordingly, and as you get more data adjusting your theory, this is a wise way to proceed even without data. So if you can catch them when they seem like they're off the happy path, they're in trouble, that's a better time to do it. And then the last thing I would say about this detection is if you don't know what to do or all else being equal, then focus- ... because if you don't know what to do, or all else being equal, then focus on onboarding. Almost all companies have a whole lot more cancellation in the first day, 30 days, 90 days, depends, but the first period than the whole rest of the customer's life. And also, small changes in the onboarding can have large effects on cancellation, whereas later on, that's not necessarily true. It could be, but it's not necessarily true. So a really dramatic version of this is if you've ever done YouTube videos, which I know you have, but if a listener has ever done a YouTube video and you see the "retention", quote unquote, of the viewer on a YouTube video, it has this thing where it falls just so much, you can't believe, in the first 30 seconds, and then, if it's a decent video, it'll flatten out as people decide to watch the video.
00:27:48 · Jason Cohen
So in that crazy-looking curve, for the people that have watched it for 15 minutes, maybe there's something you could do to keep a few of them staying to the end, but that's not going to change very much how many people get to the end. Whereas for me, I've only done a few, but what I see is about 50% fall off in the first 30 seconds. Well, if I can get that from 50% to 55% stay, that's an additional... And at the end of the line, I only have 20% still there, which is pretty good for a longer video. But if I get it from 50 to 55, I might go from 20 to 25% staying. In other words, if I shifted 10% at the front, which maybe I could do, I can't be dramatic but maybe a little, then in the output, I might be able to increase it by 20, 30%. So that's a huge change, and so the SaaS equivalent is, as we all know, if they leave early, not only is it bad but it's super unprofitable, because you spend all this money to acquire them and then they never stayed around long enough to pay it back, much less to be profitable. So if you can do a little bit in the onboarding or shift the onboarding percentage a little bit, it pays off enormously in revenue and profit over time by making them successful. So again, if you don't know what to do, onboarding is a good bet, and even if you do know what to do, I'll still bet that onboarding is a good bet for where to go.