A Product Market Fit Show | Startup Podcast for Founders
00:16:29 · Pablo Srugo, Scott Stevenson
[00:16:29] S1: What are some of the arguments because this is where it gets really interesting? I think the rubber meets the road here. On the one hand, you're coming here and you're saying, look, the capital here is not like this, usage and retention, why would I scale it? It seems obvious, but I'm sure there's another argument on the other side that's what is that argument that tells you, no, you're overthinking it, you just got to scale this thing and things will take care of themselves? [00:16:50] S0: I do think there was a period of time where capital was in such excess and there was just this social thing happening where everyone was just watching each other and doing what everyone else was doing. This social mimicry, which I think is – key to business and investing is not mimicking what everyone else is doing, but it's actually in practice very, very hard. The book Blitzscaling was very popular at the time and everyone's like, well, if you don't scale fast enough, a competitor is going to come in or if you don't start scaling up, you're not going to learn fast enough. If you don't scale up, how is your team going to do anything if you don't hire 10 more people? I think there was largely a lack of belief at how how much a small number of people could do for a little while. I think a small number of people can do a lot, sometimes more than a larger number of people. If you're doing a new experiment every two weeks, very hard to get a 50 person team aligned on that. People get burnt out pretty quick, but small team can do that.
00:17:43 · Pablo Srugo, Scott Stevenson
[00:17:43] S1: I think that's one of the most mistaken beliefs by the way is that – it's like there's all these things you should do. There's two worlds in my mind. There's pre-product market fit and the post-product market fit. One of the most classic things I see is, for whatever reason, implementing things that make sense post-product market fit but too early. That goes for like, you should hire a VP of this or a VP of that; you need to scale, you need more developers. You actually don't need more production. You don't need more output until you figure it out in your words what the thing is. Staying lean means there's no overhead. It means you shift quickly. This is my experience is this is one of the things that I think is just not – it's hard to appreciate until you've gone through it which is if you've got a lot of people, the amount of time you need to take to communicate every single time that you decide you're abandoning this and going that way is massive. Then all of a sudden you've got people who worked on this thing, they don't want to let that thing go, they don't necessarily agree with the new direction. You're trying to just go here and then here and then here. It's like you got five people. I don't know how many people you were actually, but I assume it was like five, 10 people or something. [00:18:31] S0: Five to eight people, around there. Yeah. [00:18:31] S1: Everybody's aligned, and so that's a huge difference maker.
00:18:31 · Pablo Srugo, Scott Stevenson
[00:18:31] S1: Everybody's aligned, and so that's a huge difference maker. [00:18:31] S0: Exactly, and I have to give our engineering team so much credit for that as well and our founding engineer Matt Drover rolling with that. Even with a small team of staff, you also have to hire people who are able and willing to do that. Building things over and over and seeing it not stick, it's like not normal. People in big companies just don't – I don't know. You're sheltered from the harsh reality I think a lot of the time that most things actually just don't work and suck and fail. I think when you're up against raw reality with a small team, you see that and feel that, but I think you can live a whole life as a software engineer and not realize just how many things just completely fail and suck. [00:19:40] S1: Did you think everything was going to maybe fail through that period? Did you and or your team think that at times? [00:19:45] S0: Yeah, we did. I don't think that's a good way to communicate it in a large company, but in a small company we're like, we just got to assume everything's going to fail and we have to get to the next experiment. Invalidate this one and move on to the next thing was our approach. [00:19:56] S1: What about, I mean, broader? Did you ever think, you know what? Maybe Rally isn't going to work out, we're just not going to find product market fit? Was that something that – or not really?
00:19:56 · Pablo Srugo, Scott Stevenson
[00:19:56] S1: What about, I mean, broader? Did you ever think, you know what? Maybe Rally isn't going to work out, we're just not going to find product market fit? Was that something that – or not really? [00:20:03] S0: That honestly never really crossed my mind because I had so much faith in the problem and just there must be a way to make legal services 10X more efficient or 100X more efficient. I know that everyone hates getting that really expensive legal bill. Everyone knows how universal that experience is. I think our faith in the problem and our mission was big enough to enable us to do that. Where if it wasn't, if we weren't that motivated by the mission or we started to question now does this problem exist at all, we would've abandoned it for sure. I think that was a really important part, finding a problem that felt – actually had to be a spiky tough problem that hadn't been solved before that we felt motivated that should be solved. It helped our team stick to it and be like, it's almost good that this is a hard problem because otherwise it would've been solved already, so we're just going to keep at it. [00:20:54] S1: When does this – ChatGPT comes out, walk me through how AI starts to play a role and then the transformation into Spellbook.
00:21:04 · Scott Stevenson, Pablo Srugo
[00:21:04] S0: We started experimenting with GPT-2 very early just totally on the side. It's not an OKR. It's on a project. It's on a strategy. It's just playing around and hacking. That's one pattern I had noticed in our successes is that they almost all arose from this evening and weekend hacking. They never arise from the board meeting plan or the quarterly plan. They always arise from like, oh, I'm going to try this thing after hours and see what happens. We started playing with GPT-2 early on. It wasn't really good enough to do anything with. It sucked. Then when GPT-3 and GitHub Copilot came out, that's when we really saw promise. This is when the aha moment really happened and we knew immediately. As soon as I tried GitHub Copilot as a programmer, what I saw is that I could have bespoke code on my screen and the AI copilot would riff off my bespoke code to tell me what I should write next. That was an amazing moment because lawyers had the same problem with our product where they want something that's going to help automate their workflow but it was never bespoke enough. Templates were too rigid. Templating software for lawyers has been around since the 70s. It's called document automation. It was a big thing in the 70s, but it never just caught on. We were like the hundredth company to try this and everyone was like, oh lawyers, they're so bad at adopting technology. I was pissed off at our customers; not really but I was off. I was like, oh, why don't – [00:22:30] S1: Yeah, frustrated, yes.
00:22:30 · Pablo Srugo, Scott Stevenson
[00:22:30] S1: Yeah, frustrated, yes. [00:22:31] S0: Yeah, frustrated like, well, why don't these lawyers use this product. They're foolish. It's actually, no, the lawyers were right. The rigid templating approach to automating contracts just sucked. It took me too long to realize this from our customer feedback, but every deal is unique. Manually massaging text and getting it right for your client really matters. Your client has its own unique history as a corporate entity and its own unique things it's trying to do. Everything actually is really bespoke in law. The amazing aha moment for us was like with GitHub Copilot in software engineering was like seeing that this LLM could riff off of bespoke code and generate new code. We said, oh, this is what our customers want. It wasn't like, oh, we're going to apply AI towards law. It was like we finally have the tool that solves our customer's problem and all the things that they keep saying to us, we finally have the answer to it. Almost immediately we made a V1 of that in about two weeks, actually in evenings and weekends. It wasn't even part of our main plan. We were like, yeah, we're going to hack this together. Really crappy early version, but as soon as we showed it to – this was before ChatGPT actually. [00:23:42] S1: This was what? What was V1?