Lenny's Podcast: Product | Career | Growth
01:03:45 · Lenny Rachitsky
[01:03:45] S1: Along those lines, there's also an advice of just like, don't worry about advisors. There's... Never know if they're gonna be useful. There's always hanger ons that join your startup and wanna help, but they're not helpful. If you give one tip for a founder or team that's interviewing advisers looking for an adviser to vet them to help them find people that are actually great. What would your tip be?
01:04:07 · Elena Verna
[01:04:07] S0: I have actually a good one. Many advisors might not like it though. I think that you should not hire an advisor until you do some sort of workshop with them on the problem that you're experiencing. See them in action. See what kind of information they can provide to you. Because any advisor that is cagey about what they know is not the type of advisor that you're going to want to have on your team. You want to have somebody that has lived through it, that can talk to you, to your face about it, that is able to have hard conversations, that is able to provide you necessary examples, that is able to give you and connect thoughts for you for necessary patterns. So, instead of just saying, Oh, this person looks good. Let me hire them as an advisor. Say, Hey, let's have a workshop first. I have this problem. I think that you can help me fix it. Pay them for it, whatever their rate is for that workshop for sure. But see how they actually interact with your team and then hire them on ongoing retainer bases. Because that creates an interview loop that is very practical, that is very quick to understand whether you can work with them, they can work with you, and whether they have anything to contribute to you. And then every single month, evaluate whether that advisor should stay with you. Some advisors, they only need to be with the company for three months and off they go. That's fine. Some of them might stick with you for four or five years, but every single month you should go and say, Did they add any value? And I'm not saying that advisors should have any monetary expectations, like revenue attribution, so to speak, to them. But did you find value in having conversations with them? Did they offer anything valuable to you?
01:05:48 · Lenny Rachitsky, Elena Verna
[01:05:48] S1: That was an awesome tip. And I could see why people would be like, Goddamn, Elena, don't say that. Makes so much sense. Okay. Here we go. Number nine. Here we go. [01:05:59] S0: My last one is also that I see something way too often nowadays on growth teams specifically. This is very, very growth problem, and that is too much risk averseness on growth, where you're starting to test everything. If every single one of your initiatives that you're doing on growth is an experiment, that's a problem. And that is something that you should take a look at and say, What am I doing? And why do I need a precise scientific measurement for every single thing that I touch in order for it to go to production or like to hit the market. It's almost like a disease, like a paralyzing disease that slows down progress, that slows down velocity, that slows down learnings, that creates very terrible consequences to the output that growth teams produce. And it's a little bit counterintuitive because experimentation is a way to do growth for growth teams, it's their process. Growth teams are meant to experiment. But I also think that experimenting on everything is something that is quite terrible once the team starts to get locked into that state. And it's really hard to get out of because they're then afraid to do any change until or unless that they test.
01:05:59 · Elena Verna, Lenny Rachitsky
[01:05:59] S0: My last one is also that I see something way too often nowadays on growth teams specifically. This is very, very growth problem, and that is too much risk averseness on growth, where you're starting to test everything. If every single one of your initiatives that you're doing on growth is an experiment, that's a problem. And that is something that you should take a look at and say, What am I doing? And why do I need a precise scientific measurement for every single thing that I touch in order for it to go to production or like to hit the market. It's almost like a disease, like a paralyzing disease that slows down progress, that slows down velocity, that slows down learnings, that creates very terrible consequences to the output that growth teams produce. And it's a little bit counterintuitive because experimentation is a way to do growth for growth teams, it's their process. Growth teams are meant to experiment. But I also think that experimenting on everything is something that is quite terrible once the team starts to get locked into that state. And it's really hard to get out of because they're then afraid to do any change until or unless that they test. [01:07:39] S1: So where do you find that balance? I imagine people hearing this are like, oh, yes, this makes sense to me. And then they continue to test basically everything. What's kind of some heuristics you'd recommend for knowing, okay, just don't test that. Don't worry about it. You don't need credit for that win.
01:07:52 · Elena Verna
[01:07:52] S0: First of all, I think that people should trust their intuition a little bit more. Data is good, but data is only good if you have enough of it. So if you have low volume real estate that is gonna take you eight months to reach some sort of answer, do you really wanna test it for eight months? What's the point of it? It's my rule of thumb, if we cannot collect the sample size in a month, we shouldn't test it, period. Because it's then it's not fast enough. We should just go and do pre versus post. Pre versus post is pretty powerful. It's also a way to assess your impact, and you can still roll back if doesn't work, but don't think that everything needs a scientific explanation to whether it needs to be moved forward or not. Also, experimentation cannot be the way that people make decisions in the company. There's still so much about knowing your user, understanding the market, for your brain to connect all of those dots and to know what needs to be an experience that your customers are going to want. That, yeah, if scientific data, like a very tight determination of the probability that this is a success is important, absolutely do it because it might be a really big strategical pivot that you're planning to do. So, it's a data point to validate that all of this extra work will be needed. It might be a very high trafficked real estate that even like 0.1% difference will mean millions of dollars for you. But other than that, it should be just go, go, go. Do pre versus post, by the way. I'm like, I'm not saying just, like, release and release and move on. Release, do seven day, twenty four hour readout, seven day readout, twenty eight day readout, even come back to it a year later and measure some of the retention or expansion data that is associated with it. But to test everything is debilitating to growth teams, and it paralyzes them in its place. So, kind of look at your initiatives and say, Where do I need precision? And it's important, and I can get it fast enough, versus where we should just go for it. And yeah, we will fail there too. And that's okay. And we can roll back and we can figure out how to make it better. But failure is going to happen regardless. And statistics is tricky. Many people take it for face value versus just like a directional data point to say, There is likely a new distribution that has maybe a different mean.
01:10:22 · Elena Verna, Lenny Rachitsky
And 5% of the chances, if you're measuring 95% statistical significance, you might not even be there. And yet, we'll, like, take it for some granted. Oh, it's gonna drive this much lift. So I just think that people stop, like, in this age of data, don't always rely enough on their own intuition. [01:10:42] S1: A lot of contrarian takes here. I love this. Elena, we've reached number 10. [01:10:47] S0: Okay. [01:10:47] S1: And I know number 10 is like a special one where it's more than one quick