00:20:17 · Sandy Diao, Guillaume Moubeche
So that was really, I think, the genesis of how I was able to help a lot of these growth efforts and growth teams move relatively quickly in the early stages. [00:20:26] S1: And have you seen... Or do you have an example where, like, the data you got was actually, like, misleading?
00:20:35 · Sandy Diao
[00:20:35] S0: Yeah. Well, actually, a great example that I'll share is when I was at Descript, one of the decisions that we made was around trying to figure out how we could revamp our pricing. So Descript, for those who aren't familiar, is a audio video editing platform. And its innovation is that it turns content into a text based transcript. So you could edit videos the same way that you edit a Google Doc. And the way that we traditionally priced was by essentially words. Right? Like transcription of that content. So you get a certain number of hours of transcription. And we had this idea of, well, what if people don't understand transcription as a value metric? Why don't we just, you know, essentially give the thing for free to everybody, you know, essentially take those premium features away at some point so that they get a taste of it, and they're gonna really, like, feel the pain of missing out on it? Essentially, what is called a reverse trial. Right? And we thought it was a great idea because we looked at everybody in the space, every prosumer, consumer SaaS company, and everybody was doing reverse trials. So we thought, this thing must be working because everyone's doing this. So a lot of the, I think, the external benchmarking data firstly told us that this was going to work. The second thing that we did was we surveyed our users, and we asked them, like, do you understand what it is that you're paying for? Or, you know, we asked the better version of that, which is what are you paying for? And nobody said transcription. They just said they were paying for editing. So we thought, the data is pointing to basically all the data, external and internal, is pointing to we gotta get rid of this paywall. So we ran an experiment where we shifted our pricing model to a reverse trial. Lo and behold, I wake up the next day after the experiment, and every single user in the exposure group, not every single user, but the conversion rates, you know, basically halved slash worse, you know, over time. And, you know, I was just totally panicking. It's like we spent all this time, you know, optimizing all these different aspects of our pricing and packaging and our free to paid motion, and then all of a sudden, you know, this exposure group is like really, really suffering. What do we do now?
00:22:39 · Sandy Diao
I I think a lot of that, you know, early insight data really pointed us in this direction of saying this is going to work. But I think the lived reality is that when you actually run the experiment, users behave very, very differently. When I kind of reflect on that experience, our biggest learning here is that different types of users have different expectations about the value of these different pricing models. What we found when we segmented the data is if is that businesses or more of that b to b customer had value from that reverse trial because they could essentially kind of leisurely go back to it and test it out over time. And they didn't really have this, like, day to day DAU, WAU distinct need for it. So it felt more aligned with their value model. But for a lot of the consumer, prosumer creator type users using the platform, it didn't make sense to them. Right? Because they just didn't have the two week bandwidth or timeline in their minds for making a decision around a video editing tool. They're like, I need to cut a TikTok short right now. I need to cut a YouTube video right now. It's like, okay. I'll get this thing done, and then I'll disappear, I'll never come back. It doesn't... It didn't really help reinforce the idea of the value of the product. So, you know, we ultimately ended up reverting back to our original pricing model, but that's an example of how that early data really pointed us in the wrong direction.
00:23:58 · Guillaume Moubeche
[00:23:58] S1: Interesting. Yeah. And for me, you know, like, I've seen it multiple times that whenever you want to optimize for certain metrics, you might also, like, compromise a whole other sets, you know, of metrics. So if you want to optimize, let's say, for customer expansion, you're gonna reach out more to existing customer trying to oversell something, and some might actually, like, not like it and churn. So your churn might increase, but your expansion... So you have always, like, side effects. So I'm curious to know, like, from your perspective, whenever you have like a marketing budget or a growth budget, what do you... How exactly do you split it between these are the experiments where I wanna measure, like, a clear outcome versus, you know, like, these are, let's say, belief we have, and kind of like moonshot projects, where we know it's gonna go in the right direction, we know we're gonna not be able to measure exactly, you know, like the impact it has on the growth, but we believe that it's aligned with our brand, who we are, and we have a high conviction that it's beneficial for the company.
00:25:09 · Sandy Diao
[00:25:09] S0: Yeah, great question. Generally, for the growth teams I've led, there are two types of budgets. One is the time budget, and the second is the financial budget. For the time budget, almost all the time, we want to over index or shift most of our time resources on big bets or big swings. Right? We basically wanna find the power distribution channels that will drive the majority of our growth. And if we don't take enough of those swings to find what those channels are, we're really gonna suffer because we're gonna be testing, micro optimizing, you know, focusing on a a local minimum rather than the ultimate global maximum. And then there's the financial budgets. I I think for the financial budgets, we wanna be indexing most of our financial investments in primarily the things that we know are giving us somewhat measurable unit economics. So channels like paid ads, for example, influencer marketing, sometimes SEO if you have sort of this, like, content... The cost behind content creation, although, of course, that's changing a lot with AI these days. But I I think, you know, we generally think about our our budgeting in that way. And then for more of the time investments, we want directional outcomes. Right? We want essentially it to show some results that this is working at a scale or a magnitude that we care about and that's gonna drive a ton of users for us. And then we can get a little bit more detailed and nuanced about how much incrementality or the precise attribution method to use to properly attribute growth there. But the... The, you know, more of the financial budgeting, we probably wanna be pretty precise about that from day one, because the input or the growth lever there is gonna be budget. So that's just gonna be able to scale in the manner that we're able to prove out there's LTV to CAC and that there's actual ROI and return on ad spend. So that's how I've traditionally thought about it.
00:26:58 · Guillaume Moubeche
[00:26:58] S1: And I think you talked about, like, the the big swings at Descript. I think you you did something that generated, you know, like, a lot of the the revenue with... Can you please, like, share a bit the the growth loop that that you put in place there?