Business Breakdowns
· David Plon
[00:00:00] S1: I think the interesting thing about memory is – today, as it's implemented, it's far worse than a human's memory. It is super limited. It's expensive in the sense of having deep memories takes up a lot of context. It lacks a lot of the nuance of – our memories are really useful, both for specific facts, but more so for the abstract concepts that they create and that we learn. When you think of those memories like, a lot of times that connection is where they become really powerful, and that's become pattern recognition for investments. So in the near term, I think memory is a useful tool for all the obvious reasons that you can think about, in terms of just adding more context that persists in the context window. Longer term, everyone knows all the fallibility of human memory. And this is a little bit above my pay grade in terms of the actual ML research and how memory is being incorporated in different ways. But you could imagine any one of the architectures that are being experimented today with memory, and having short-term memory, and longer-term memory, and the ability to look up memories. If implemented correctly, both in terms of how the models are trained as well as the harnesses around them, imagine having a model that has lived every single one of a firm's investments and maybe things they even passed on. There's a world where it becomes so much more powerful than any given human. I look at some of the investors I've worked for, and they just have this insanely, eerily intuitive judgment about things. A lot of that is not just memory of specific facts, but memory that comes from experience and scar tissue. There's a world where that certainly could exist in models. And I think in some sense, people talk about AGI being the ability to be generally intelligent. I think a really interesting test for a model, if it’s AGI is – can it predict the future? You need a really good world model and understanding of lots of variables to predict the future, and that's what an investor does. And memory is clearly a really important ingredient to that. So I'd say near term, I'm a little bearish on just how big of an impact it's going to be relative to just copy-pasting that context into a window.
· David Plon, Matt Reustle
But longer term, I think that's the unlock where this goes from being the useful tool to the real core driver of a lot of the research a firm is doing. [00:00:00] S0: Well, I'll separate it. Near term, I'm almost taking a way that it might be more valuable to limit the memory, therefore reducing the context window, and investing more into the prompt that I'm using. I know they're not like-for-like, but ultimately memory is customizing to you, and if you could just put that in the prompt, that might be more beneficial than sucking up memory.
· David Plon
[00:00:00] S1: It's a near-term convenience. So in other words, if there's knowledge you want to impart in every single time you interact with the model, yeah, that should live in memory. Or if there are aspects of working on a name and you've maybe done 20 queries on a particular name, you maybe want some sense of the memory living in the model. So if you reference something back, or it already knows it did XYZ analysis, it can do that. I think that's where it's useful. It's a bit of a shortcut for just loading in context that you don't have to repeat. I definitely would recommend spending a lot of time being thoughtful around, to the extent you can manage it, the system prompt in an off-the-shelf tool or anything like that. I think we're just very early innings from a technical perspective tapping what it means to really utilize memory in a thoughtful way. And just to illustrate that – and these models have such incredible world knowledge by essentially reading the entire Internet in their pre-training. So they clearly have the capability to store in their weight a lot of knowledge, and then learn important concepts from that knowledge that leads to things like reasoning. So, it's just a matter of time before that same paradigm applies to continual learning on the job. And that to me is going to be really, really fun.
· Matt Reustle
[00:00:00] S0: Yeah, I do think about if Bridgewater has really been recording all those meetings for all of this time, what all of that input could do eventually, and what could happen with that technology. I don't even have an idea of what it could be, but it just feels pretty unique. And seeing how people are testing this, even with their own personal recordings and creating their own things. The last thing I wanted to talk about because it gets brought up – it's a buzzword, it has very practical applications – but agentic AI. The definition gets a little bit fuzzy sometimes, but just thinking about agentic AI as a topic and as it applies to investors. If you could just talk about where that's going, how you would even categorize that within the various tools, and anything else you would mention on that.
· David Plon
[00:00:00] S1: So, there's lots of definitions of agentic AI. I think of it essentially as the model has the capability to reason, reflect, take action, reflect on those actions, ultimately in the pursuit of a goal. And there's obviously different levels of scaffolding and constriction around what the model can and can't do. What's really exciting, especially now, is agents have started to work. And interestingly, when we first started Portrait, we tried, and we did actually have an agent that was running using GPT-4, and it was so hard to get to work. And the way you got it to work was you had to be so prescriptive about what the model could and couldn't do. I think our system prompt was 30,000 tokens or something crazy. Really had to limit it because at any given time, the model, if it started going down the wrong direction, it didn't have the ability to self-correct and be thoughtful about what it's learned and how it should adjust its plan. And that required just a ton of prompting. What's cool today – and I think what's happening in software engineering is the leading edge of what's going to eventually happen in investment research and really any other knowledge field – is the models have now become smart enough to do longer-running tasks through using tools and updating its thinking process as it goes. And you're seeing this with things like Claude Code and Codex, where people are leaving these models alone for multiple hours and it is dynamically operating like a software engineer to understand a code base, make changes to it, test things out, see an error, go look at the trace and figure out why there's a problem, make a fix. That type of thinking, it started out really challenging, and that now works. The nice thing about code is it's the perfect environment for these models to figure this type of capability out. Because one, it's all just text files at the end of the day. It's all contained usually within a repo, so all the context is locally available. And it's super verifiable – the code either runs or it doesn't. Investment research is obviously much different. None of those things are true. The context lives in many different forms and very broad, and some is more accessible than others. It's also much more qualitative in terms of output.
· David Plon, Matt Reustle
But the form factor of, "can these models reason iteratively and arrive at complex answers that require on-the-go changes of plans," the answer is yes. And it's just a matter of the engineering work to get those models into the right context, so that they can operate like a junior analyst and, ultimately like a senior analyst doing long-running research. It's definitely still a buzzword, but there's a lot more meat behind that buzzword than there was maybe a year ago. [00:00:00] S0: The point on code – and I often bring up chess – these constrained environments, that are still very complex, but they're constrained by certain things. You tend to see more of a clean iteration cycle and more productivity that comes out of it. And the more complex and adaptive the system, the harder it is to control the evolution. So, it's one that I think will be interesting to see how it all evolves. But this has been fascinating. Thank you for going on some more philosophical tangents in addition to the very tangible use cases. It's been a pleasure, so thank you for sharing the knowledge with us. [00:00:00] S1: Thank you. It's been really fun.