Invest with AI
00:33:25 · Brett Caughran
[00:33:25] S1: Can you unpack the knowledge graph a little bit more, some of logistics of that? What does that look like sort of in the specific sort of build structure of a knowledge graph?
00:33:37 · Thomas Li
[00:33:37] S0: So a knowledge graph is nothing about folders and skills, right? If you think about like the Obsidian Wiki or whatever, it's basically sort of folders and markdown files. The magic of a knowledge graph is to have all your markdown files connect to each other and creating a system to generate markdown files automatically, and basically like, quote unquote, delete markdown files that are no longer relevant. And so what you're trying to mimic is basically the human brain, right, you're trying to recreate a set of memory and context and analysis. So what some firms will do is they'll say, Hey, we have 10 MCPs, we have 10 skills, right? If I ran 10 skills on every ticker through every MCP, I can create, I don't know, 10,000 markdown files, right? If I can connect all the markdown files together, and instead of having the MCPs directly connect to my large language model, I connect my endpoint to my folders with my markdown files or the knowledge graph to the large language model. So I think a lot of firms are doing that. There's a tendency for those firms to be more of a one team, one dream type setup. And the key benefit of that is you basically have the analysis and the context pre done, so the AI doesn't have to do analysis. So let's say Amazon drops earnings, instead of a user prompting the system to analyze earnings, it auto analyzes earnings, creates the markdown file, connects to last quarter Amazon earnings, connects to Costco earnings, connects to Nvidia earnings, connects to all these companies as relevant to Amazon's earnings. So when I come into cloud and I say, Hey, Amazon just reported earnings, what are the macro trends I need to take away from Amazon and their peers? And how does that influence how I should think about next week's earnings? A lot of that work has already been pre done. So Cloud's job is really just to walk through your knowledge graph and figure out where all the pieces lie and assemble it together, as opposed to generating that output on the fly, which significantly improves the quality of the output and drops your cost.
00:35:44 · Brett Caughran
[00:35:44] S1: Stepping away from data and Dilupa for a minute, we thought we'd fire a few just broader AI questions at you. And one of the things that it feels like the debate is picking up again, And in a few of my evals, some of the models have gotten surprisingly good on the open source, like Chinese open source models. I'm curious, sort of your perspective on the sort of frontier models versus open source models, how you see that debate evolving?
00:36:14 · Thomas Li
[00:36:14] S0: So this is a personal view, this not a Dilipa view. I'm a big believer in open source. I mean, if you look at Dilipa, GitHub, we've opened... Every time we create a skill or an agent, we open source it. So I'm a big believer in that. My view is, it's very, very hard to build something that is technical faster than an open source creator can, especially when the literal contribution world today is everybody, because everybody can now code, so everybody can contribute to open source. So all it takes is will and passion, I guess. And there are just more people who are not working in Anthropic than is working in Anthropic, right? So it becomes really hard to combat open source, I think. And in this case, we're specifically talking like open weight models, right? Like the K3s of the world. I think it's really hard. I think at the bleeding edge, when you think about how you actually make a model better, there are a couple of different elements to it, right? One element, and this is like way oversimplifying it, but one element is increased parameter size, right? If you can build a 20,000,000,000,000 parameter model, chances are it's going to be better than a 1,000,000,000,000 parameter model, right? Because you just have more weights to play with. But where you get a lot of efficiency actually isn't in the base model, but what you do in post training, right? So if I took a small parameter model, and I really post train it to a very specific task, Let's say this task is building a DCF, right? If I had $10,000,000 to have a model build a DCF, instead of spending it increasing the parameter account of the base model, I just take a small base model, and I just post train the heck out of the DCF piece. Hire a bunch of people, build 1,000 DCFs, feed it in, do it again and again and again, you end up with a model that can build a DCF better than the increased parameter size. Obviously, it taps out, like if you don't have a model of big enough parameter size, like you end up going to these like really, really deep targeted, you know, use cases. But if you think about the use cases of all the different things you do within AI in any industry, it's actually somewhat quantifiable. Of course, there's a long tail.
00:38:40 · Thomas Li, Brett Caughran
But, you know, when you use AI in Excel, right, there's really only like five to 10 different things you're trying to do with, when you use it in a hedge fund, in a chatbot, there's really only five to 10 different things. So there is a case to be made where the labs can work closely because they're enterprises, can work closely with their customers on the post training piece to get the output really precise to what they need to do in a way that the open source guys can't. Because it will be very hard for a hedge fund to speak to an open source model and say, I'm going to give you how I think about the world so that you can train your open source better. Right, so that's a disadvantage of open source, which is industry seekers tend to not get in, so post training becomes difficult. But their advantage is on the pre training side, they have a much lower lift. [00:39:39] S1: I've heard the opposite argued a bit with some of the like local host locally hosted post train Quinn models where that sort of that post training actually lives within the walls of the hedge fund, where there's actually a little bit of growing skepticism about working with the foundation labs as a large asset manager because you're sort of leaking your proprietary knowledge back into the foundation lab. How would you sort of rebut that you sort of seem to have been observing the opposite?
00:40:08 · Thomas Li
[00:40:08] S0: Yeah, I think post training is more art than science. I think yeah, in theory, you can go into GitHub and just download like a post training module and just put all your data in there. And, you know, localhost:quen or k3 or whatever and run post training. I think the reality is like, when you think about what post training really is, it comes down to a very, very simple problem. You have to know your objective function. Right? So, when we say train a model for a specific problem, what we are assuming is that we can in a very quantifiable way, write down this problem. Right? And a lot of business problems cannot do that. So if you go to an analyst, and you say, I'm going to train a model that can do what you do, I need you to write down in a few sentences for me, a measuring stick on how I know the output is good or worse. And the analyst will be like, I don't know, like, yes, the final outcome is P and L, right? We all know that. But when we start to break down what creates P and L, like what makes a pitch better or worse? Like is a 10 page pitch better than a one page pitch? Like probably not. So that's not a good measure. Is detail of analysis better? Maybe. But like how do you know it's more detailed? More numbers? Right? So it becomes very taste driven and very art driven. So anyone can post train, but what is your objective function? That's like the key problem. If we could take a particular problem and say definitively there is a quantifiable objective function, then that problem can be post trained on. So a very good example is math, right? If you're trying to solve math proofs, there is a very quantifiable objective function, because you either prove it or you don't, and approve is either right or wrong, right? So in that world, post training becomes, I wouldn't say easy, but quantifiable, and therefore you can just run cycles on. But if let's say we are writing poetry, what makes one poetry better than the other? Like, I don't know, it's not quantifiable. So if it's not quantifiable, I cannot create a feedback loop of like, this wasn't good, this is good, look like this next time. So that's what makes post training so good.