Business Breakdowns
· David Plon
[00:00:00] S1: It's super dependent on what tool you're using, and in what context you're using it, and the task. To go a little deeper on that, the context windows obviously have been expanding, although really in the past year they've been flat, with Gemini at around a million, GPT I think around 400,000, and Opus or Claude around 200. Where the models have gotten more capable is using that context well. I think historically models struggled if you loaded in, I don't know, the last five 10-Ks. And let's say a 10-K is like 100,000 tokens, so 500,000. The models historically would be really good at say if you say, "What was SG&A in 2023?" They call that a needle-in-a-haystack problem. They would struggle if you said, "Build a three-statement model using these." The model needs to apply its attention across lots of different pieces of the context window simultaneously. I think that still is a roughly helpful rule of thumb, which is the models are generally, if you're using a ton of context, you can do better if you're loading that in because you're looking for very specific data points. That being said, I think a change versus if we had this conversation three months ago is – models have gotten very good at using the context intelligently. If you're thinking about building a project in Portrait or NotebookLM or something like that, I wouldn't really hold back. I would give it pretty much the entire corpus. And you're seeing this a lot in software engineering, where Claude Code and other agentic coding tools are very capably exploring super long context in a way that is efficient and focused on the task at hand. So in the same way that I wouldn't give a junior analyst a task and say, "These are the only four documents you can use." You'd want them to ultimately have the agency to choose what content they're going to use. And I think that holds true. All that being said, if you're going to use the raw model itself, you do not want to overload the context window and use 70% of the context. You will see a degradation in any complex task. In that case, you should break it down. But hopefully the much easier way to do this is use a tool that manages that complexity already, so you don't need to worry about, "Okay, what percent of the context window am I utilizing?" and so forth, is how I think about that today.
· Matt Reustle
[00:00:00] S0: On memory, I think it's obvious where, as memory has now been implemented in a lot of these tools, it customizes to you in that iteration. Again, I'm going to overuse the lens of the employee overseas, that you're creating a better, more aligned working environment. Are there any non-obvious impacts that you think memory will have?
· David Plon
[00:00:00] S1: I think the interesting thing about memory is – today, as it's implemented, it's far worse than a human's memory. It is super limited. It's expensive in the sense of having deep memories takes up a lot of context. It lacks a lot of the nuance of – our memories are really useful, both for specific facts, but more so for the abstract concepts that they create and that we learn. When you think of those memories like, a lot of times that connection is where they become really powerful, and that's become pattern recognition for investments. So in the near term, I think memory is a useful tool for all the obvious reasons that you can think about, in terms of just adding more context that persists in the context window. Longer term, everyone knows all the fallibility of human memory. And this is a little bit above my pay grade in terms of the actual ML research and how memory is being incorporated in different ways. But you could imagine any one of the architectures that are being experimented today with memory, and having short-term memory, and longer-term memory, and the ability to look up memories. If implemented correctly, both in terms of how the models are trained as well as the harnesses around them, imagine having a model that has lived every single one of a firm's investments and maybe things they even passed on. There's a world where it becomes so much more powerful than any given human. I look at some of the investors I've worked for, and they just have this insanely, eerily intuitive judgment about things. A lot of that is not just memory of specific facts, but memory that comes from experience and scar tissue. There's a world where that certainly could exist in models. And I think in some sense, people talk about AGI being the ability to be generally intelligent. I think a really interesting test for a model, if it’s AGI is – can it predict the future? You need a really good world model and understanding of lots of variables to predict the future, and that's what an investor does. And memory is clearly a really important ingredient to that. So I'd say near term, I'm a little bearish on just how big of an impact it's going to be relative to just copy-pasting that context into a window.
· David Plon, Matt Reustle
But longer term, I think that's the unlock where this goes from being the useful tool to the real core driver of a lot of the research a firm is doing. [00:00:00] S0: Well, I'll separate it. Near term, I'm almost taking a way that it might be more valuable to limit the memory, therefore reducing the context window, and investing more into the prompt that I'm using. I know they're not like-for-like, but ultimately memory is customizing to you, and if you could just put that in the prompt, that might be more beneficial than sucking up memory.
· David Plon
[00:00:00] S1: It's a near-term convenience. So in other words, if there's knowledge you want to impart in every single time you interact with the model, yeah, that should live in memory. Or if there are aspects of working on a name and you've maybe done 20 queries on a particular name, you maybe want some sense of the memory living in the model. So if you reference something back, or it already knows it did XYZ analysis, it can do that. I think that's where it's useful. It's a bit of a shortcut for just loading in context that you don't have to repeat. I definitely would recommend spending a lot of time being thoughtful around, to the extent you can manage it, the system prompt in an off-the-shelf tool or anything like that. I think we're just very early innings from a technical perspective tapping what it means to really utilize memory in a thoughtful way. And just to illustrate that – and these models have such incredible world knowledge by essentially reading the entire Internet in their pre-training. So they clearly have the capability to store in their weight a lot of knowledge, and then learn important concepts from that knowledge that leads to things like reasoning. So, it's just a matter of time before that same paradigm applies to continual learning on the job. And that to me is going to be really, really fun.
· Matt Reustle
[00:00:00] S0: Yeah, I do think about if Bridgewater has really been recording all those meetings for all of this time, what all of that input could do eventually, and what could happen with that technology. I don't even have an idea of what it could be, but it just feels pretty unique. And seeing how people are testing this, even with their own personal recordings and creating their own things. The last thing I wanted to talk about because it gets brought up – it's a buzzword, it has very practical applications – but agentic AI. The definition gets a little bit fuzzy sometimes, but just thinking about agentic AI as a topic and as it applies to investors. If you could just talk about where that's going, how you would even categorize that within the various tools, and anything else you would mention on that.