Lenny's Podcast: Product | Career | Growth
01:29:29 · Lenny Rachitsky, Jason Cohen
[01:29:29] S0: Speaking of the book, let's give you a chance to share what you're working on, and when this is coming out, and where folks can find it. [01:29:36] S1: Sure. So the book is called Hidden Multipliers, and you can pre-order it at hiddenmultipliers.com. Or I guess if this is out long enough, it'll be order it, I guess, depending on when you're listening to this. And it's a lot of stuff like we were talking about today, these questions of, it's called multipliers because the idea is little things that you can do or little decisions you can make that have a huge impact like moving the cancellation rate from five to 4%. It sounds small, has a huge thing, onboarding as opposed to later, huge thing. So those are some examples, but the book is of course full of different kinds of topics, but all of this idea of this stuff that has such a big impact on things like revenue or profit. And either, just as you said earlier, either maybe you have never thought of it that way, so you weren't thinking about it right. Or yeah, you've heard that, you say, "I know." But your actions don't reflect it. And so if we go deep enough with examples and specific things to do, then you can actually act on that supposed knowledge and realize those multipliers. [01:30:40] S0: And just to remind people that you're all hiddenmultipliers.com. [01:30:44] S1: Yeah. [01:30:45] S0: And there's an S at the end, Hidden Multipliers. [01:30:48] S1: Right. There's more than one. [01:30:49] S0: There's many. It's more than one. [01:30:51] S1: Yeah.
01:30:51 · Jason Cohen, Lenny Rachitsky
[01:30:51] S1: Yeah. [01:30:52] S0: Jason, I had other things I wanted to talk about, but I feel like this episode's actually going to be stronger if we just focus on the thing that we've been talking about, which is unstalling growth. So if we do that, is there anything else you want to mention or leave listeners with before we get to a couple corners and then the lightning round? [01:31:13] S1: I think if you tried to find a common thread throughout all this stuff about growth, it comes back to the customer getting value. And I know we already talked about that, but I think if there could be one thing where it would help solve kind of all of it, it would be that they really are getting value. Your product actually promises the right thing, and then it actually delivers on that thing, and the customers can onboard so that they can do the thing, and the customers know, they realize that they're getting the thing, and you're measuring the thing, so you know it's increasing. That is probably, if I was an LLM, I'd probably say, "That's the common thread." There's many ways that manifests, of course, but if that's your North Star is how are we actually creating value in the way the customer values it in their language, in their way, and their way of understanding it, I wouldn't say all the pieces magically fit into place, but certainly isn't that the root thing that is going to make all this stuff work? Then there will be a good way to do pricing. [01:32:15] S0: Yeah.
01:32:15 · Lenny Rachitsky, Jason Cohen
[01:32:15] S0: Yeah. [01:32:16] S1: They will stay as long as possible. And these things will probably be right, if that. So this idea of creating value for the customer and then figuring out how to split with them is probably the root idea. And of course, I hesitate because platitudes like that are actually not actionable, not very actionable. Like, "All right, well, I'll move on with my day." And that's why Twitter's not so useful. But given that we've gone into so much detail, perhaps that's a nice way of summarizing it. [01:32:39] S0: I think that's such an important point. I think what's also interesting is some of your advice is the value, you may be picking the wrong customer, the wrong market, you may be positioning it wrong. So the value may be there, you're just trying to convince the wrong people about it. [01:32:54] S1: Yeah. There's so many ways to get it wrong because like we said, all these things have to be right. And you just said another one, which is, and you have to say in a way that when this person hits the homepage, they know it. It's true, but do they know that? It's just so many things have to go right. [01:33:11] S0: What a tough job we've got over here, just solving people's problems. Come on. Okay. [01:33:15] S1: Well, at least we're growing. [01:33:18] S0: So now we will be after this conversation. Okay. So I'm going to take us to a recurring corner, a recurring segment on the podcast that I call AI Corner. What's one way that you have discovered using AI in your work or in your life that might be helpful for folks to hear?
01:33:18 · Lenny Rachitsky, Jason Cohen
[01:33:18] S0: So now we will be after this conversation. Okay. So I'm going to take us to a recurring corner, a recurring segment on the podcast that I call AI Corner. What's one way that you have discovered using AI in your work or in your life that might be helpful for folks to hear? [01:33:37] S1: There's a lot of data on the internet and it's often in things like images, which makes it hard to do your own analysis, or plug it in, or et cetera, come up with your own models or apply it. But I found that AI is really good actually, especially Gemini, at say, you just give a chart to it and say, "Make this into a table that I can paste," literally say, "that I can paste into Google Sheets." And it will do in a way that literally you can copy, and it will actually paste correctly into Google Sheets, and then you can do stuff. So especially with the book and my articles, I love to use real data whenever I can, of course. And so I do that all the time. So I think that kind of interpretation is very useful. And so all of a sudden you can get 10 examples of something and test a theory, where before it was just too hard and you didn't. [01:34:24] S0: That's an awesome tip because people know you can generate all these infographics, especially with Gemini and all these things. That's really cool to know you can just feed it, "Here's a chart and make a text." [01:34:35] S1: Yeah. [01:34:36] S0: Okay. I'm going to now take us to Contrarian Corner. The question here is, what's something that you believe that most other people don't?
01:34:43 · Jason Cohen
[01:34:43] S1: A/B testing doesn't work very well and it doesn't work on most things. It won't work on strategy, or vision, or insights, nothing actually important to the success of the company. You don't A/B test whether Uber's a good idea. And then, even when you do A/B test the details, where I agree, sometimes that can work. What happens is people will try things like, "Oh, I'll just try this verb, and that verb, and this ad, and that ad." And then like, "Oh, the seventh or eighth one, I got a positive result. That must be good." And what happens is you keep doing that. You pick the best one, and then you go on, and you find another one, and pick the best one. And then a year later you look back and you should be like 50 or a hundred percent better because you've stacked these things, and you look back, and like nothing's different. The conversion rates are the same as they've always been. You say, "What the hell happened? I thought I picked the winner." And the answer is in a combination of the tools not being statistically accurate, which they're not. And the fact that you will get false positives even if the tool is statistically accurate means that most of them are false positives. Even if the tool's 95% accurate, when the thing you're looking for is rare, which it is in the case of A/B testing, the false positives happen more often than the actual thing happens. And so most of the results you get are false positives anyway. So as a result, this isn't true of all A/B testing, but for what most people do when they just do the mundane A/B testing, you can't A/B test the important things and the details are mostly false positives, so it's an enormous waste of time unless you're incredibly sophisticated. I know there's special groups that actually are very sophisticated. Fine. If you're not doing that, it's like the poker table. If you don't know who the patsy is, it's you. If you don't have all of this information and knowledge about A/B testing, then you're the patsy.
01:36:31 · Lenny Rachitsky, Jason Cohen
[01:36:31] S0: Bam. All right. [01:36:32] S1: Yeah. [01:36:33] S0: I will just say that I have found A/B testing useful in my career. I think it's maybe at a certain scale when you're just trying to optimize and continue to grow, you know- [01:36:43] S1: Yeah. [01:36:43] S0: ... where it's like millions of users, like a percentage gain is like millions of dollars. [01:36:47] S1: True. [01:36:47] S0: Most people are not working at that scale, most people. [01:36:49] S1: Right. [01:36:50] S0: Yeah. So just wanted to, for folks that find it valuable. [01:36:54] S1: That's true. [01:36:55] S0: But I love- [01:36:56] S1: However, even so, on your own podcast when you were interviewing the guy from Shopify, and he was saying how maybe a third of the things that they found with their systems just disappear, they just magically disappear. And they have a team of a hundred people, and they're really good at it, and their effects disappear all the time. So they double check later whether the immediate effect goes away because even then. [01:37:19] S0: Right. I think that was the CTO of Shopify conversation. Yeah. [01:37:19] S1: Oh, okay. Yeah. [01:37:22] S0: Yeah. Sweet. Okay. [01:37:24] S1: I just go off of memory because I listen to a lot of episodes, but I don't- [01:37:26] S0: That's such a good one. [01:37:26] S1: ... no, it's- [01:37:27] S0: I love that. Yeah. Where they leave like a holdout group essentially, and then they just look back, "Was the effect something that lasted?" And most times it didn't. [01:37:34] S1: Yeah. [01:37:34] S0: Such a good one.