A Product Market Fit Show | Startup Podcast for Founders
00:09:06 · Pablo Srugo, Scott Stevenson
[00:09:06] S1: You're going after smaller clients. You mentioned running many different experiments. You mentioned hundreds of landing pages. Can you dive in on that? What were some of the things you were testing? Then you mentioned CAC payback period. What were some of the outputs that you looked at to see if those tests worked? [00:09:25] S0: Splits; our director of growth was also a huge part of this also. His name is Kurt Dunphy. We rolled over a hundred landing pages with different value props. Often there was a new feature, a little feature we built to angle the product in a certain way. We monitored these experiments at a variety of levels. Our goal was to run an experiment every two weeks. We would look at a lot of different things. We wouldn't just launch landing pages, we would also launch ads with those landing pages. [00:09:51] S1: What's an example of that? Just take us through one of those experiments that you remember.
00:09:51 · Pablo Srugo, Scott Stevenson
[00:09:51] S1: What's an example of that? Just take us through one of those experiments that you remember. [00:09:55] S0: Yeah, so one of the experiments which actually resonated a lot from a sales standpoint was what we called our menu of services pitch. We actually turned our templating system into a menu of services or a Shopify for law firms that they could put on their website. Now not only did they have templates, they could actually advertise like, "Oh, you need a will or you need an employment agreement, come into my website, click on need an employment agreement, put in all the info." That gets submitted to the lawyer. The lawyer goes into the lawyer's template. The lawyer does their massage on it, and the product goes back to the customer. That was one of our actually hit experiments on the sales side was allowing the lawyer to almost set up a little Shopify shop. [00:10:36] S1: Why do you think that resonated?
00:10:36 · Pablo Srugo, Scott Stevenson
[00:10:36] S1: Why do you think that resonated? [00:11:15] S0: It resonated because lawyers want to make money. They want clients and they want customers. That's one of their biggest pains is where's my next dollar coming from? Where's my next customer coming from? The idea of having a digital storefront was really compelling. How we knew this worked? First we ran ads and people were clicking those ads and we were looking at the cost per click and the cost per conversion into our demo meeting. What's the cost to get a lawyer into a demo meeting? It was way lower than anything else we had run before. We knew that within two weeks of running these ads. We had so much data. We threw up an ad, threw up a landing page. What is the cost to get someone into a demo? We were like, "Wow, this is 90% lower than anything we've run before. They love this." Then we took that and then we were able to go and say, okay, let's take this experiment to the next step. Let's actually roll this feature out to all these customers, implement it and see how it goes. This is where the problems started with that value prop. Again, the time to value issue, getting connected into their website. Lawyers could get this thing up and running, but they had no idea how to drive traffic to it. They're chasing this mirage of – and I'm sure Shopify has this issue too. They probably have tons of people setting up stores and no traffic going to those stores. That was actually a killer problem for that direction for us.
00:11:49 · Pablo Srugo, Scott Stevenson
[00:11:49] S1: It's again this notion that reality doesn't meet the idea? People expect, oh my God, it's going to drive all this traffic, but then the work it takes and all the nuances around it don't let them realize that value? [00:12:01] S0: Exactly, and that passed the CAC payback test. I think our CAC payback was probably six months on that whole motion or less, but it didn't pass the retention and the usage test. We looked at usage retention cohorts a lot. [00:12:15] S1: How much time? That one's tougher because don't you need months of data to see if it flatlines or do you get to see it faster somehow?
00:12:15 · Pablo Srugo, Scott Stevenson
[00:12:15] S1: How much time? That one's tougher because don't you need months of data to see if it flatlines or do you get to see it faster somehow? [00:12:22] S0: In this case actually – I'm probably over-explaining it because in this case, I think you could see what we call the wiggles of death, which is the customer size of the product. You see those first like, oh they logged in, oh they did something and you see a couple wiggles and then nothing. At our company we say actually most things you launch, that's the pattern you see with the wiggles of death. The user is like, "Oh, there's a new feature, I'll play with it." You see a couple little spikes of people using it and then nothing. In my mind that is the default for most companies when they launch anything. I was talking to another company the other day. They built this amazing customer portal and they were amazed that after months of hyping up this portal selling it to customers, they looked at the logs and no customer had ever logged into this portal almost anytime. It had all been their own internal users using it and they thought people were using it. I was like this is normal. This is actually what I see. This is the dirty secret of SaaS is actually getting anyone to use anything. Especially in 2024 when you're so distracted, there's so many other more exciting things to look at, getting anyone to even log into a web app is excruciatingly – [00:13:29] S1: Reminds me of my my usage of threads.
00:13:29 · Pablo Srugo, Scott Stevenson
[00:13:29] S1: Reminds me of my my usage of threads. [00:13:34] S0: Yeah, exactly. Yeah, I logged in once or twice. The wiggles of death, that was my exact usage pattern. I logged in, I went there twice. [00:13:41] S1: That was one experiment. What were some of the others? [00:13:44] S0: It's quite a lot. I think client questionnaires was something we focused on a lot. We had this idea of taking info from a client and feeding it into a template, but actually the questionnaire in its own was useful as a product because when you're onboarding a new legal client, you need to figure out do they conflict with any of your other clients? There's a whole bunch of KYC information you need. We also had this client questionnaire angle of like, watch your lawyers, watch your client questionnaires. That was one that did pretty well. It didn't have that amazing physiological reaction from the customer on it. Some of the others, more advanced types of templating, we even went in terms of – client portal is also something we launched like, hey, lawyers, now not only do you have all these templates on this stuff, you also have a portal that your clients can log into.