00:03:26 · Guillaume Moubeche
[00:03:26] S1: Yeah. That's that's really cool. And you mentioned that you leverage emails to increase activation on specific features. How exactly do you set up everything and track everything? Did you work with a specific data team, or how exactly did that work?
00:03:43 · Sandy Diao
[00:03:43] S0: Yeah. In the early days of using emails for activation, one of the challenges that we had was that we had essentially this benchmark of peer companies. So I think at the time, considered itself to be a social media app, although it is a little bit different if you've, you know, used the product yourself. But I think the challenge is that when you see other companies out there like Twitter, Facebook, and and others set up their emails, you kinda see they have this formula. Right? And it was... At the time, onboarding emails were very education oriented. You know, it was like, you're gonna receive five to 10 emails from us every single day, and here are the things that you should be doing. It was very much treated as early user education, and companies all had that approach. Over time, we realized that, one, the more emails you sent, the more email fatigue there was, and people just stopped opening them. So by the seventh email, no one read what it is that you wanted to sell them. And secondarily, when you actually measured outcomes and engagement based on the emails you're sending, we found that driving specific product actions was far more useful than trying to drive awareness about every possible thing. And instead, the product itself should do the work of educating while the users in the product experience. So we went from this education based approach through our emails to essentially this more data driven action triggered approach. So for example, if a user came in to the platform, they hadn't followed any boards yet, then our our goal would be to get them to follow boards that indicated their interest. If they signed up and they created boards, but they hadn't pinned anything to it yet, then our goal would be to get them to take, you know, a pinning action. So it really was catered and customized based on what a user did or didn't do. So these drop off or fall off states. And a very data driven approach is ultimately what we ended up with. But it did take us some iteration to get there because I think the biggest challenge is coming in. You see what every other peer is doing in this space, and you're like, oh, if Facebook's launching it, it must be working because they have millions of users, and their growth is great. But the big mistake there is you don't know the circumstances of their product.
00:05:44 · Sandy Diao, Guillaume Moubeche
You don't even know if they're measuring the success of these emails. Right? And so I I think it kind of took us several iterative cycles to figure out that more of a data driven transactional approach was gonna be much more beneficial to our growth. [00:05:58] S1: So your focus at the time was how do I connect activation with retention. Is that correct? [00:06:06] S0: Yeah. It was retention at the time. We weren't monetizing quite yet because we were pre revenue. We hadn't built out our ads platform. But I think, you know, it was retention in the in the sort of North Star metric view. In the short term, it was, can we get a user to essentially come back and look at, you know, meaningful day one, week one, month one retention? And then, of course, how does that translate into downstream month six, month 12 retention? [00:06:31] S1: And typically, in the case of Pinterest, I'm making that up, but let's assume that when a user sign up, as you mentioned, they need to, let's say, create their first board, start, like, peeling things, etcetera, etcetera. And once they do that, we believe that they're gonna stay longer. So as as a person working in growth, you have, I'm guessing, like, a list of hypothesis that you wanna test. How exactly do you convince the product team to change things around? So eventually, you know, like, you get to a point where your growth experiments can lead to higher retention?
00:07:09 · Sandy Diao
[00:07:09] S0: So that's a really interesting question because at the time at Pinterest, we didn't really have... At least when I joined because I was so early, we didn't have a product growth team. We didn't really have engineering scope to make some of these changes that would actually drive incremental jumps in, like, conversion rates or activation rates. So I have kind of a a funny story on how I managed to convince an engineer to moonlight this, which is at Pinterest early on in our culture, we, actually had a lot of people who like to play foosball. Are you familiar with foosball, the tabletop soccer name? Yeah. So I noticed that one of the front end engineers who would be able to control some of the onboarding surfaces actually play foosball during lunch. I wasn't necessarily a big foosball fan, but I thought, oh, wow. This is like a good way for me to get in front of this engineer. And so I joined one of the lunch foosball games one day, and I just introduced myself. And then I kind of started to like foosball because it was competitive. It was kind of a nice break from work. And then I ended up staying... This is funny. I haven't really shared this with anyone before, but I stayed after work every day to practice foosball. And I became decent at it, so much so that I ended up playing a lot more of the games. And in one of the games, when I was playing foosball, I mentioned to this engineer that, you know, oh, we have this issue. I don't know who to go to. And he said, oh, you know what? I touched that surface. I have the code there. You know, why don't you draft something up? And even after, you know, that discussion, we had a formal meeting. We found out that it wasn't gonna fit in a proper road map because there were so many other competing priorities and new features to build. But you mentioned, you know, I'll stick behind after work a few days a week, and, you know, we'll moonlight this thing, and we'll test it and see if it works. And so that's actually how I got, you know, the first iteration of the onboarding, emails and email... And onboarding experiences to be coded into the product itself. So I think every company, you know, big or small, has a unique culture like that. But I I think in a lot of growth roles, that's kind of how a lot of big things tend to happen, which is it doesn't get planned for. It's opportunistic.
00:09:09 · Sandy Diao, Guillaume Moubeche
It's very data driven, and it's hard to have this tops down mandate of like, oh, let's drop everything and focus on optimizing our activation rates. So instead, it kind of falls on the person who's responsible for growth and revenue to say, these are important things that we should pause and do, you know, in exchange for, you know, some of the results that we're looking for. So that's how I went about that process. [00:09:30] S1: Nice. So tip number one, play foosball and convince engineer. [00:09:35] S0: Exactly. I hope they bring foosball tables more back to, you know, startups, because I feel like they took them away for quite some time. [00:09:43] S1: They they they switch it with ping pong or [00:09:45] S0: That's right. [00:09:47] S1: Yeah. And you... I think at Pinterest, you know, you also worked on the b two b sides. So what what kind of changed, you know, like, going from... Or transitioning from b two c to b two b?
00:10:00 · Sandy Diao
[00:10:00] S0: Yeah. So going back actually to the tickets that I was responding to, one of my biggest insights there was that the persona or the profile of the user writing in oftentimes wasn't a consumer. Right? We have this, again, massive social media app, and and we started to realize, and I started to see through the questions that people were asking, that these were actually marketers inside of small and medium sized businesses, oftentimes even enterprises. Sometimes you have really big enterprise logos writing in, and they have received the exact same SLAs or service level agreements as the consumers were receiving. Their questions were things like, you know, I want my pin to be seen by more people, or why is this pin only getting one view versus this one getting, you know, hundreds of views? So we started to realize that there were actually businesses with business use cases on the platform, which actually led to one of our biggest growth experiments. This experiment actually still lives on the Pinterest site today. When you create an account, it actually asks you, are you a regular user, or are you a business? And if you are a business, you get to the separate sign up flow where you can create a business profile, forking you and leading you into an entirely separate business experience. It's all about how you can promote your content on the Pinterest platform. So that insight from the ticket basically showed us that we needed to carve out a distinct experience for businesses, a separate onboarding flow that taught businesses how to market their content on the platform. And that actually eventually led into us building an advertising platform, one of the greatest tools that a marketing team or a business can get access to in order to further propagate their content on the Pinterest platform. So, you know, that's actually how that organically came about. And once that became true, we were able to then have distinct product marketing teams that supported the consumer side and the business side.