00:12:45 · Brett Caughran
[00:12:45] S0: Yeah. No, I'm curious yours. I, I think about the investor toolbox as like workflows, kind of end to end workflows. Hey, I want to go do an earnings preview process or this company has a new CEO, I want to go and explore the background and the potential of this CEO. And within that, there can be 20 steps along the process. I want to go understand or her stock grant, past performance, do a background analysis on if this person has nefarious historical affiliations, etc. In the past, I had to create each of those as individual prompts because as you know, a prompt past five or six pages started to lead to a garbled output with chatbots. So I couldn't build a workflow to do all of these things. And many of the tool providers tried to do that, chaining together prompts and building all of this complex engineering around it. Now I can one shot that an agent, So with a skills it really just makes the tools much more user friendly. It gives me an end to end workflow that still isn't perfect, but is shocking in terms of the quality and caliber. So my three zero six prompts have gone down to 30, really like 18 orchestrated pipelines and then individual skills that I can pull off the shelf to dig in more deeply. So you're seeing this abstraction in the complexity of working with these tools. And there's been innovations around MCP and MCP and data as well, too, that sort of the one example you'd see that if you went into ChatGPT last year and asked a question, often the quantitative metric will be pulled from a blog somewhere, right? So if I went in and I said, what is the ten year history of Nvidia's earnings per share? These were exercises that a chatbot couldn't answer accurately. So it's like, if I can't answer a basic question like that, like get out of my face with this AGI thing. But now I can plug Claude Cowork into an MCP from Dilipa or Faxit or FinShat, and that data is piped in through the MCP process with, in my experimentation, much better accuracy. So it's not a solved problem, but there's been huge innovation. Do you think about agentic workflows and what agents unlock?
00:15:29 · Khe Hy
[00:15:29] S1: Think similar to what you said, so I hear kind of like end to end workflows being one versus kind of constantly steering the model. Okay, now tell me this. Now tell me this. The analogy I use is it's like the difference between Waymo and Google Maps, right? Waymo, you give it the destination and then you just kick back. You don't really care how it gets there as long as it gets you there within a standard deviation of your expectations for time and safety, right? And so with the AgenTic, it's like, hey, I would like to, you know, write this deck. But I don't really care if you do slide seven first or this, but I just have a general intent of what I'm looking for. And then the model kind of figures it out. Right. So that that would be kind of like the end to end. And obviously, you mentioned the connections as well. The other thing that I would add with agents is that they tend to, like, self correct along the way. So you might give it a PDF that's too big. And then without telling it, it will run like a PDF extraction tool, right? Like, I don't even know that the PDF extraction tool exists as a thing. I don't even know that PDF's too big. But the agent is making these decisions and what, you know, in AI language, they'll say tool calling, right? And not just tool calling like Slack or Dulupa, tool calling like these weird kind of utility help error apps to, like, chunk a PDF or to use what we would call Command F, you know, is called grep in, you know, in computer science land. And so the agent not only can it get you answers, but it can find and solve its own problems by accessing new tools, which again gets to that point that you mentioned. It makes the end to end workflow more accurate and longer running. So in many ways, that kind of full agreement with what you said, I would add kind of the tooling element that is kind of abstracted away that, like, I don't have to tell it to use the PDF extractor and then the ability for it to reason and act and self correct in the process.
00:17:42 · Brett Caughran
[00:17:42] S0: Yeah. And I'd add probably one of the it's not the flashiest AI pilled moment, but one of the big AI pill moments I had this year was the usability of agents for validation. And so we work in an industry where accuracy is critical. We have a fiduciary duty to our investors to conduct a rigorous investment process. And if you follow any of the finance benchmarks of LLM accuracy, they tend to sort of shake out in the 60 to 70% range. So I'm always like, all right, well, yes, for qualitative use cases where I'm putting in six transcripts that I can go and validate in the transcript, sure, Right? But for quantitative use cases, that was a very difficult mental hurdle for me to get over. Like I don't want to unleash LLMs on financial models where it's 65% accurate. Exercise of Like finding 35% error is like next word prediction, next token prediction, next word prediction, it's fine. Next number prediction is not fine. We don't want a stochastic exercise to sort of make up a plausible sounding number, right? And in a few of these use cases, like LLMs are very good at making something look plausible, which means that where I've seen quantitative errors, sometimes a quantitative error looks more plausible than the actual number, right? And so this was something where I was like pump the brakes on quantitative use cases in 25. So what's changed is MCP, the data's gotten more accurate. But the ability to build agentic, I was using is like, okay, let's check these numbers, use these four LM to check this LLM and go back and forth. And it was okay, but the ability to create agents to validate work now, it's exactly you pointed out to sort of go in tool call, check all these numbers, right? Even give me a checklist by hand to go in and spot check numbers that could really be problematic for my thesis. That to me is a big unlock with agents that didn't exist in chatbots. Curious, Kay, to understand a little bit more about on your consulting side, are there three or four common themes you see come up from clients in terms of the problem space that the firms you work with have in general that you help them walk through?
00:20:20 · Khe Hy
[00:20:20] S1: I think one of the big ones, believe it or not, is just, I would say, kind of a failure of imagination. And what I mean by that is that you're kind of flash with this, you know, you're presented this blank flashing prompt. And it's like, this is, you know, 200 Nobel Prize winners at your disposal. Go nuts. And what happens is people kind of anchor to the existing mental frames that they know. And that's like Google and chat. So immediately, you start off on chat. So that would be kind of the first challenge, which we kind of help people shift into that agentic mindset. And then you treat it like texting, where it's, you know, one sentence prompts instead of a two hour prompt, which, again, prompting has shifted. But it's not like an iMessage. It's much more like a brief in many of these cases. So I think that's the one of the biggest ones is to kind of like unlock what's possible, right? And so, you know, I joke with people. It's like, you now have a data scientist in your pocket, right? But what does that mean? That means you need to have the data that a data scientist works with. You need to understand the language to communicate with a data scientist. But more importantly, you need to look at your problem space and understand what a data scientist could be useful for. And if you never worked with a data scientist and you might have at some of the firms you have, that is a completely different world. Right. And so I really kind of think of it as the expansion of kind of what's possible. That would be one thing. The second thing is usually just like, how do all the pieces fit together? The data, the prompting, the custom instructions, the memory, the statelessness of an LLM, projects. You just kind of like... So I would kind of put that in kind of like marrying the workflow with the tooling and the features. And by the way, there's like, you know, Anthropic's been on a tear, but they've launched a new two new features a week since January of this year, and we're in April. So that'll be another. And the third is and this is actually a harder one is, I think I saw this quote the other day is, so much knowledge work is just vibes in spreadsheets. And people don't actually know what they do for work. It's the vibes. They're like, okay. You know, it could be anchored around thing like quarterly earnings. Like, there's an... There's a huge...
00:23:02 · Khe Hy, Brett Caughran
They know what they do there. But if you ask them, like, what's your screening process? Like, oh, like, sometimes I look at 13 Fs. Like, we'd go to these idea dinners. There's not, like, an actual... It's Vibes, right? Sourcing happens on Vibes. And this actually isn't an LLM question. This is much more of a process... I don't even... I'm not a... I'm not, like, a formally trained consultant, but it's like process excavation exercise, for lack of a better term. [00:23:34] S0: Yeah. [00:23:34] S1: Or maybe workflow mapping, right? And I think that, you know, it's okay to run with Vibes when, you know, without LLMs because you can kind of, like, flow back and forth and you can kind of work in that kind of osmosis, a little bit reactionary way of doing things. But with LLMs, you need to be clear on your intent. You need to be clear on your steps. You really need to, like, deconstruct things. And that is that's the hardest thing with my clients because every analyst is every firm is different and every analyst is different. And mostly, you know, that is kind of it's a very kind of human exercise of, like, extracting out knowledge and then remapping it towards an LLM.
00:24:22 · Brett Caughran
[00:24:22] S0: That's such a salient point. And I think that's where, you know, I'm spending a lot of my time, and I completely agree. It's almost investing after you've done it. It's why it's hard to learn, right? Because really great PMs will have a process, but the process leans more towards an intuitive sort of tacit understanding than anything that's overly robotic, right? Almost no PMs have a 24 page checklist of, If I look on an idea, I go check off every single step. They've done it hundreds of times. And like a master playing a piano, they look at an idea, hit 72 Bloomberg functions, update the model, call the man. And they just sort of do it out of muscle memory such that the workflow is tacit, not explicit. That's really hard to build an agent around. You kind of have to slow down and document that process. And that's really hard to do because investing is a non linear process. That's what we've done for four years at Fundamental Edge. We've tried to create a linearity out of it to bridge that knowledge of the junior analyst to the PM. It's not perfect. But I completely agree that that's a big part of the journey of enabling agents in your process, documenting your process. Also documenting LMs or wonderful pattern recognition engines, like the mental models in your mind that you come back to over and over. How do you codify those such that an agent can think the way that you