Invest with AI
00:28:04 · Brett Caughran, Robert Yang
[00:28:04] S1: So he then it's quite heartwarming, actually. [00:28:08] S0: Yeah, yeah, exactly. So that was and I was like, wow, didn't expect it to be so elaborate. Uh and it was we were completely shocked because we didn't we didn't design any of this other than like the fact that there's a priest, there's a mayor, etc. Right? So all of these things came out of just like the agents interacting with one another and behaving like humans. Um, that was for me, that was the coolest, coolest thing that an entire project. [00:28:39] S1: So how did you how did you go from building societies in Minecraft to building Excel agents for Wall Street? What walked me through that journey?
00:28:48 · Robert Yang
[00:28:48] S0: Yeah, yeah, yeah. It's uh um yes, uh absolutely. So so we did project Sid and then um it was very expensive to run, right? So we we're like, how do you turn this into a real product, right? So um we had we had people come in and they were given like one or two agents. It's not the same thing, right? You don't you don't really get to experience the whole like agent society, but even if you did, it probably would be a bad product, right? I think the same reason like moat book right didn't work out, right? Just people don't really want to see a bunch of agents talking to each other and just generating what is mostly slop, right? They're interesting things, but a lot of it is slop, or don't want to see it generated. Um so the product just essentially failed, right? Our like day one retention was 10%. So that was late 2024, and we were kind of in a crisis as a company. Um, and at the time, Anthropic did a computer use demo, and we immediately played with it. And I remember thinking we can do better than this. So we spent a month focused on building better computer use agents, and we actually came up with lots of good ideas. Uh, one of them was essentially we came up with the idea of a skill, right? And if the agent would do some task, and not only a skill, but agent would do some task and would like like introspect their behavior and then write down like a file, right, that will help it do uh do it better next time, right? Which is exactly like how like how right now you can like generate auto-generate skills. So we did that in back in late 2024, and we made the same mistake again, was thinking, oh, we'll turn this into a product right away. And we didn't publish any of this research. We we did use it to raise funding, right? So we raised like a series A of like maybe 30 30 something million dollars. Uh and uh and then now we started to uh just like try to productionize this. Um so we had two paths. One is to build a general computer use agent, another is to do something more focused, right? Like focused on Excel. The reason we focused on Excel is we realized even when we were doing the computer use uh research, we realized the current models are really, really bad at using Excel, but it's not because they don't understand it. It's because they actually don't have the right interface to work with Excel, right?
00:31:23 · Robert Yang
Um, so if you give them a better interface, like again, MCP was not even a concept back then, right? So, but but we realize if you give them a better interface, these models, even in late 2024, are already intelligent enough to operate on Excel, at least do a lot of like the basic work, right? Not not like full-on building of a DCF, but they can use they can build a pivot table, right? This is something basic like that. So that's something we realize essentially people think that agents cannot use Excel, but we realize it's not an intelligence problem, it's like an interface problem. Um, the problem is if we were to focus on Excel, um we had a like a essentially a company gene problem, right? Because we just pivoted from doing games. Uh like the whole team is still kind of thinking a lot about like consumer product, right? You're going from games to um going from games to productivity is already a big change, right? But even within productivity, there are different types of productivity, right? So um so if you're saying with the team that we had, which is a lot of great people, that we're gonna go straight to investment, it would have been impossible, right? Uh so what happened is like we're like um let's do Excel, but no one in the company knows anything about Excel, except for Nico, who's my co-founder, who actually was a financial consultant uh at EY for two years, right? Um, so where he like really like uh did a lot of Excel work. So he was able to see that if we just focus on Excel, then we can build a product that's really valuable. And uh at the same time, we were trying to build this like general computer use product that we just couldn't really find use case. Um in retrospect, there are use cases, but these use cases are probably better for companies that have massive distribution, right? Uh, whereas for like a startup with no distribution, it is better to start with like a concrete, concrete, like vertical, concrete uh customer profile. So we didn't have that understanding back then. Um yeah, so we launched a computer use agent product called Fairies, which I think had a lot of good ideas, and uh, but it just didn't work because there were no clear like use case. And we launched at the same time, uh, around the same time, the Excel agent shortcut, and that was immediately like well received on Twitter, right?
00:33:59 · Robert Yang, Brett Caughran
People are really, really excited about that. [00:34:03] S1: Uh and that was viral viral, viral moment. [00:34:06] S0: Yes, yes, it went 2025, right? Yes. So I think it was the first moment that uh lots of people realized, okay, Excel is within reach by the AI, right? Um, of course, in the last 12 months, what we have seen is okay, um, how do you how do you get it to be good enough so it can really, really deliver like um something that's inspectable, understandable, right? Like um really good looking, doesn't have a lot of errors, right? It's it's one thing to have a demo where it seems to build a financial model that works. It's another thing to put it in the hands of professionals and they are happy enough to give it to their colleagues, right? So I think a lot of the work we did in the last 12 months is to close this gap. And also, thankfully, the models have been getting better and better every release. [00:34:59] S1: Yeah, walk us through that because I remember um yeah, I remember after that moment in summer 25 immediately subscribing and running it through some of my workflows. Yeah, and you could sort of see some interesting effort in the level one workflows, but anything more complicated, it really started to choke on. Uh exactly. Walk us through that experience.
00:35:26 · Robert Yang
[00:35:26] S0: Yeah, yes. So that was a pretty difficult time for us um because it went hyper-viral, right? So we had lots of people come in and test it, um, but then a lot of them churned because uh I think the model is like um not quite there at the point, right? The models was not quite there. Our harness can obviously could have been improved uh more, but I think fundamentally it's just like we were maybe like one generation early, right? Um, so a lot of people come in and try to do very sophisticated things and just realize it's it's not it's not really doing that. Um and uh it was a difficult time because for like two, three months we had on paper, it's very viral. Um, but when we were looking at the metrics, everything was going down, right? Like people were churning and the retention was too low, right? So so the revenue just kept going down. Uh in fact, we attempted a fundraise that didn't work during that time because like people were like, oh, this is very viral, let's talk. And but then when they see the actual numbers, like things were going down. So uh in around October last year, um, we just sat together and we're like, okay, let's let's just focus on making this really, really good. Right. Um, so the team we we um we we essentially cut every other project, just really focused on Excel. And one of the biggest things I think we did is to build a really, really strong internal benchmark, right, for real work. Uh so this is uh Richard, uh who's our chief product officer now, he led that work where he essentially collected uh he essentially built these tasks that are in distribution, which means it's like similar to what our users would do, right? For example, it's like uh leverage buyout, right? This is for private equity, uh discounted cash flow, this is for like like public company investment. Um and and the important thing is is it needs to be hard enough for the agent, but also the you need to be able to trust the score, right? Uh so building a benchmark is very hard because sometimes you you have a task and you have some prompt. Uh, there are many ways to do the task, right? So there is no one correct way, and and then you cannot really like evaluate the output. So the task needs to be designed in a way that's both like natural, as in this is something a user might really do, but also precise enough that there is one correct answer, right?
00:38:02 · Robert Yang
So that was pretty hard. Now, once we have built these tasks, then it become much easier to kind of iterate on the agent harness, right, which is like everything that goes around the language model. And we did a lot of like um like detailed optimization for the Excel use case, right? Making sure that like essentially um it can really understand the the Excel API, right? Uh, and uh how does the agent manage its context very efficiently so it can it can do the task because like if you think about just naively, how do you take a big big Excel like uh table, right? Um just you have lots of call, you can have lots of columns and lots of rows. How do you put that all in the the context of the language model? And if you're not careful, you're gonna just pollute their context right away, right? So there had to be a lot of engineering to be done. Uh that's very detailed, very specific to spreadsheet. Um and once we started to do these things, then uh it's just started to work, right? We saw all the metrics were going up, like retention is up, like users started to grow, and then we started to get into the enterprise, had figured out how to do like enterprise grade security and things like that, right? We have never done that before. Um, and then it's then it's uh it just started to become better, right? So so um I think turning point was probably November last year, right?