00:25:33 · Joe O'Donnell
[00:25:33] S0: Yeah. I mean, I think, you know, there's something against the laws of physics for an investment organization to build, you know, canary inside of their own organization. I'm sure, you know, most investment organizations have a resource set and a team of extremely talented people that could go reconstruct a large percentage or all of what we do. The level of effort required to do that would be extremely high, and I suspect that some people will decide to do that and others will not. I think what we found at least is that in order to, you know, say whatever, our firm's investment process is X, In order to codify that into software, it's nontrivial. I can... Everyone kinda knows what it is when they when they when they think about it or when they look at an individual investment. But but but then when you actually try to turn it into some sort of deterministic logic that sits in a software program, it's not so easy. You know? People plug in a set of things that they think they like, and the output from whatever the... Take your idea generation example is nothing like what they actually think they... What they actually do like. And then there's this iteration process. You know, sort of like automated driving where you can you can, you know, whatever. Maybe this is harder than understanding it, but you get sort of 90% automated in a really easy way. And then the the next 10% end up taking five times as long as the first bet. And, you know, some people will make that investment and maybe they'll build great products and and that... That's great. I think in order to get to that level, it still requires all the infrastructure that I mentioned too. So do you have the right data, which requires building integrations with who knows how many data providers, but that by itself is not sufficient. Well, then what is... What do each of those individual datasets actually mean? And every single one of those needs to be fine tuned. Every single one, at least as far as I can tell. There's nothing that's really just obvious, you know, plug in dataset x, and then a foundation model just intuitively right now understands it at the level of a really good investor. And in order to do that next layer, well, you got to get your really good investors that should be working on investments to go train all those datasets. And that's an ongoing process.
00:27:56 · Joe O'Donnell, Brett Caughran
And then you can build the emergent sort of, you know, what I... Maybe I would call microservices on top of all that stuff. And then you can build that emergent layer of investment analysts. It can be done for sure, but it requires... Well, I can speak for ourselves. It requires years of investment and many thousands of hours of fine tuning and data collection and cleansing and aggregation and stuff like that. And I think every firm has to decide basically what the demarcation line is between what they wanna do themselves and what they wanna outsource to someone else. And there's... Yeah. Every... You all have different answers. [00:28:34] S1: And how's that work today? Because where you may get pushed back is the argument that while correct that most public investors don't have alpha, no public investor believes they don't have alpha. So I think a lot of the mindset with agents is I want to clone my process. I a want to digital analyst that acts and behaves like a senior analyst or a junior analyst on my team. And to that degree, agents have been interesting as a flexible substrate to be able to actually train that analyst. How do you work with funds? Like if I gave you an analyst training manual and said, Joe, make this agent behave like a junior digital analyst. How far away from that reality are we today?
00:29:28 · Joe O'Donnell
[00:29:28] S0: What I would say is for our our products, you know, we have a a product that basically does the job of a junior analyst. Everything post idea generation, get up speed on a company, understand the key investor debates, what are all the key questions, how to do the research, automate all the research, and generate answers. That's called superanalyst. So that was our that was our name first before AlphaSense independently came up with the exact same name for a product a few weeks later. But the... We have a product that does that effectively across all listed companies that refreshes every day and does all the research that one would expect a talented investment analyst to do. It integrates with our own transaction data. For example, we have our proprietary transaction data set that we built from scratch. It integrates with web traffic data, app traffic data, point of sale data, foot traffic data, Google trends. It can build you a scraper. It can do an expert call. And it just does this autonomously across right now, I think close to 4,000 companies and refreshes every day. And our customers who use that product wake up with an email. If there's new research that... To be conducted, wake up with an email with all that research already done. And so, you know, that's... We don't ask questions. We don't say, what kind of research do you want me to do? We just do the research, and we we deliver the final result. And, you know, customers who don't care about it can discard it or they can take some insights or they can disagree with the analysts just like a normal investment firm would would do. And that's something that we already have have built today. I think as you as you move up the stack, it becomes increasingly more difficult. You know? The holy grail is idea generation and maybe after that portfolio construction. And as I said, know, that's sort of a problem we're working on, and and we've started to hopefully solve it at least from the idea generation level. But or... And and also worth mentioning on that same point, our idea generation analysts already do incorporate firm zone preferences into that thing. So the very first thing that happens when someone onboards to one of our idea generation agents is they get a questionnaire. You know? What kind of things do you like? What what don't you like?
00:31:46 · Joe O'Donnell, Khe Hy
If you want, you can send me some research that you've done in the past, and then we'll construct an idea generation funnel around those preferences as part of the solution and then our own models as another part of the solution. And then similarly, there's check-in points throughout the the investment memo generation process where a user can say, follow-up on this point, or I think you might be wrong about this, etcetera, and our models will take that into account, but they don't only do that. They take that into account. They also have their own ideas and combine the two and do further research. And so that's... You know, I I I have no evidence whatsoever to suggest that's the best model, but all I can say is that so far our our customers like it. [00:32:32] S2: Joe, can you talk a bit... You mentioned fine tuning a model. And, again, for for, like, a nontechnical layperson who knows about AI, what does that process actually look like for you guys?
00:32:44 · Joe O'Donnell
[00:32:44] S0: Well, it's terrific. AI sounds really, really, AI honestly is neat. But the actual process of getting an AI model to operate in your image is just horrifically bad. It requires a ton of manual effort. A ton. Know, I mean, to take... You know, just to go back to the disclosure statement example. In order to get a model that can parse through a disclosure statement at the level of a very good analyst, it requires very good analysts going through, you know, whatever, annual reports manually and saying, here's what I would say. And then they look at what the AI says and, you know, through fine tuning, reward functions, etcetera, all that kind of stuff. Over time, you hopefully get, an equivalence. But the only way to do that is by a huge amount. I cannot understate how much manual effort to actually make that work. And one of the reasons for that is somewhat obvious, but another reason which is obvious is the thing I was talking about, the long tail of results. I think there's relatively large Venn diagram overlap between very interesting insights and things that only happen very rarely. And so you... That needs to all be codified. You need to have a sufficiently large sample set that you've actually fine tuned to actually allow the AI models to understand when they see something like that and what it means. And it just requires enormous amount of manual. And that's just disclosures. There's hundreds of those, literally hundreds of those examples that are required to get performant fine tune models, performant defined as what a good investor would think about this particular thing. It needs to be equivalent. My view is that, let's say, I don't wanna pick on anyone in particular, but let's say you could get a result that was 95% as good as a good analyst for task x. Some some tasks, by the way, that's fine. But tasks that are centered around making the right decision, that is not fine. And if you... If it's not possible to get to 100% equivalence, then the default, what I would at least recommend for professional... For public markets investors, the default setting for that would be do it yourself. Do it the old do it the old fashioned way. Because, as I was saying before, AI is a... The the day... One of the dangers of of of overusing AI is higher conviction when you didn't deserve it.
00:35:29 · Joe O'Donnell, Khe Hy
And you never know what the missing percent is or how big the percent is or what what even category it's in, or whether it's a false positive or false negative without doing the AB testing. The only way you can avoid doing the AB testing is if you have AI models that, you rely on either that you buy yourself or that you buy from someone hopefully like us, that where you're convinced it is doing the job at the equivalent of of you or your investment team. [00:36:02] S2: And is it fair to say so someone like an Anthropic is hiring a lot of former bankers, former PE folks, investors, and they're doing a kind of a mass, like a production line version of what you described. And then Canary's alpha in this situation is like the people doing those evaluations and the fine tuning are just more in line with a hedge fund analyst. Like if they're more specific, is that a fair analogy?