Invest with AI
00:38:02 · Robert Yang
So that was pretty hard. Now, once we have built these tasks, then it become much easier to kind of iterate on the agent harness, right, which is like everything that goes around the language model. And we did a lot of like um like detailed optimization for the Excel use case, right? Making sure that like essentially um it can really understand the the Excel API, right? Uh, and uh how does the agent manage its context very efficiently so it can it can do the task because like if you think about just naively, how do you take a big big Excel like uh table, right? Um just you have lots of call, you can have lots of columns and lots of rows. How do you put that all in the the context of the language model? And if you're not careful, you're gonna just pollute their context right away, right? So there had to be a lot of engineering to be done. Uh that's very detailed, very specific to spreadsheet. Um and once we started to do these things, then uh it's just started to work, right? We saw all the metrics were going up, like retention is up, like users started to grow, and then we started to get into the enterprise, had figured out how to do like enterprise grade security and things like that, right? We have never done that before. Um, and then it's then it's uh it just started to become better, right? So so um I think turning point was probably November last year, right?
00:39:33 · Brett Caughran
[00:39:33] S1: Um yeah and and uh you walk me through you know, there's there's really been no in sort of the core hedge fund community that that I sort of lived in, there hasn't really been a transcendent product yet. Um I think most hedge funds I I I talked to really mostly use Claude. Um I had a moment sort of independently in September, you know, September 22nd when Opus 5.5 came out, and you know, the the model started to pass my modeling benchmarks in a way that was pretty astounding. The day I tweeted that, David from from Shortcut reached out, he's like, Have you tried shortcut? I'm like, Yeah, man, I tried it last July and it was terrible. And he's like, try it again. Um and um and so from that sort of moment from September to uh September last year to September this year, um you sort of found this ICP in in the hedge fund in the hedge fund level. Can you tell us why that happened?
00:40:37 · Robert Yang
[00:40:37] S0: Yeah, yeah, absolutely. So I think um I I think a lot about this, right? Uh we I don't think we had like great foresight, right? That this is a good um um this is kind of good customer for us, right? Um we were we were still, I would say in the beginning, still didn't have a clear understanding of how to position ourselves, right? We're like we're the best Excel agents, right? But but then mass companies with massive distributions like Microsoft, cloud, how how do you differentiate against them, right? Well, our answer is always like, well, our product is just better, right? Like our agent is better, more accurate, blah, blah, blah. And it turns out for a lot of people, it doesn't matter as much, right? Or like they're not as sensitive to this, right? Because let's say I want to like build a personal budget tracker, right? Um, and I'm not very rich, right? So I'm not going out and buying yachts, right? So it's like it's there's not that much, right? It's like, okay, like try to put your like limit your grocery, right? Um things like that. Um, so so cloud uh for Excel or like um Microsoft Copilot can do these things just fine, right? What the people who are able to most like feel that difference between our product and um and our competitors' product, right? Like cloud for excel are people who are doing really cutting-edge demanding work, right? And it turns out a lot of these are like the hedge fund analysts, right? It's not exclusive to those, but as you know, the hedge fund analysts, the buy-side analysts, these are in some way the pinnacle, right? Of like the work they do is extremely like the sophisticated. It has to be because if it's not good, then you just don't wait, don't make money, right?
00:42:29 · Brett Caughran, Robert Yang
[00:42:29] S1: Um certainly the tolerance for air is uh incredibly low. Yeah, exactly. One error and the one lazy error in the wrong cell of a financial model can be really destructive. [00:42:40] S0: Yeah, exactly, right? You can get fired, right? If you just put give some slop to your uh PM. Uh so so these so these people they really felt the difference. And uh and uh um and I think in retrospect, uh this is a really great match between our company gene and and our customers, right? Our company gene has always been like very research and tech driven, right? That's why I think when a lot of people just are kind of just like a lot of our competitors, maybe not Anthropic, but you know, like there are other competitors, they don't really have benchmarks for a long time, right? It's just kind of vibed, right? Just feel the vibe of it. And we had benchmark which allow our agents to be better, and we can compare it against Claude for Excel and know for a fact it's better, right? Like we can compare it and say, oh, like the token efficiency is higher, the performance is higher, the success rate is higher, right? So that's I think that focus on accuracy, because we have been doing agents, we have been benchmarking agents since 2023, um, just resonated with uh like our investor customers who are also very much about accuracy. And precision and performance.
00:42:40 · Robert Yang, Brett Caughran
[00:42:40] S0: Yeah, exactly, right? You can get fired, right? If you just put give some slop to your uh PM. Uh so so these so these people they really felt the difference. And uh and uh um and I think in retrospect, uh this is a really great match between our company gene and and our customers, right? Our company gene has always been like very research and tech driven, right? That's why I think when a lot of people just are kind of just like a lot of our competitors, maybe not Anthropic, but you know, like there are other competitors, they don't really have benchmarks for a long time, right? It's just kind of vibed, right? Just feel the vibe of it. And we had benchmark which allow our agents to be better, and we can compare it against Claude for Excel and know for a fact it's better, right? Like we can compare it and say, oh, like the token efficiency is higher, the performance is higher, the success rate is higher, right? So that's I think that focus on accuracy, because we have been doing agents, we have been benchmarking agents since 2023, um, just resonated with uh like our investor customers who are also very much about accuracy. And precision and performance. [00:44:02] S1: Yeah, that makes sense. Um and uh you know, building product in um in the world of AI sort of feels like a constant building of sand castles and the better lesson sort of evolving, like as has clock directly has gotten better. How do you think about you know Excel agent as a as a platform?
00:44:23 · Robert Yang
[00:44:23] S0: Yeah, absolutely. So this is something we have been grappling a lot in the company in the last right six months, uh six even twelve uh six months. So first we had like first we were worried about okay, like can this product work at all, right? Let's find uh product market fit, right? So once we found product market fit, um at least I was pretty concerned about the long-term prospect, right? It's you like there were one point early on where like, oh, like maybe this is everyone who uses Excel would use this, right? But at some point it's becoming clear this is not gonna happen, right? The big labs, like most people are gonna use this like widely distributed product, right? So if you're going with uh like um like a customer profile like hedge fund, then you can't you then it's it's like if you're trying to serve them at some point, there's not that much you can do with Excel agent alone that's keep delivering them value, right? Um, so I was thinking a lot about this, and uh and at some point I came to the conclusion that shortcuts or like the generally the product that FRL build have to uh kind of go from Excel agent and go like from this foundation and go beyond it, right? Um and if we look at if we look at analysts, a lot of what they do is is uh well, you build your financial model, you update them, um, but then you have to do research, right? Uh you have to come up with like some thesis, some hypothesis, right? So so there's a lot more than building the financial models. And also these things are are not independent, right? As in the financial model you build can inform your thinking and then inform the research you do, right? Vice versa. So naturally uh we want to expand on this, right? But it's it's not some like arbitrary expansion, right? Because you can say, well, I I have an Excel agent. Now let me have a Word agent and a PowerPoint agent, right? Just like expand along the office suite, right? So we didn't want to just like expand like across more softwares, right? We want to expand, we want to expand along like to go from doing work to really creating value, right? Um, or at least we want to be able to get to the point where uh our users can create value uh better and more easily, right, by using our product.