Latent Space: The AI Engineer Podcast
00:18:40 · S1, Joseph Kim, S2
[00:18:40] S1: Yeah. We can shout out Frontier Code from Cognition. So it's not only like, SWE-bench is very, you know, is the answer correct? Right. This is also would you merge the code? So you can have multiple answers that are proper PRs that pass. Right. [00:18:53] S0: But which [00:18:53] S1: one would you merge? And that's why you sometimes see the charts of lower reasoning is better, but it's a taste benchmark as well, right? [00:18:59] S2: Yeah. And one of your other 2027 predictions I thought was interesting was that, you know, the models that you're training will become much better at, you know, investment thesis and sort of generating ideas for investment and trading. But obviously, you know, that's something that humans are are experienced kind of financial professionals are very good at, and they kinda have an intuition for that kind of thing, but it's often counter to the prevailing wisdom out there. Otherwise, know, there's less of a chance of it making a great return. So how are you tackling that problem in terms of training the agents to do that kind of work?
00:19:33 · Joseph Kim
[00:19:33] S0: Yeah. I would say that like up until date, vast the majority of applications of AI and finance have been really focused on, I would say like the junior analysts work. And I think that's a... And I think that's for two reasons. One is you have to build the product based on the like capabilities of the model of today. And so if you look at the capabilities of the model today, they're really quite good at taking pretty concrete instructions of what to do and kind of executing. Right? So build me a slide, I want it to look this way, you know, make it look good, use our template, these types of things. Model's The quite capable of doing those things and so therefore we've built product experiences around that. And the other thing I would call out is there's a strong preference to make the model good at things that humans are not good at or don't like to do. Right? And so human... You know, most humans I talk to, there's obviously exceptions, but a lot of the humans I talk to like humans I talk... People I talk to, you know, they really don't like building Excel models. It's it's a tedious kind of work and it's very detail oriented. It's easy to get wrong. And similarly with slides, you know, you're you're copying logos, you're placing them very precisely on a specific part of the page. That's very tedious work that lot of people don't like to do. And so that's been part of the impetus for us to go and focus on some of these tasks. Right? It's a combination of two of those things. There's a... Like the existing molecule is pretty good at certain types of tasks and also human preferences of what they like to do and not like to do helps drive the direction of the product as well. Right? It's a combination of those two things. I think that like we've solved a lot of the low hanging from those two things. And so going into 2027, we're gonna start to move up the stack. Right? We're gonna go from the task that a GG analyst can do to some of the tasks that an associate can do to some of the tasks that an MD would do and move up that chain. And as you move up the chain, what I would say is the objectives become much fuzzier. It goes from this is clearly a good or bad slide or something that would put in front of a client or not to, well, I don't know if this investment idea is good. And it's...
00:21:19 · Joseph Kim
And it would take me, you know, three years of waiting to really be able to give you a clear concrete answer of like what the market has done. And so the objectives start to become fuzzier. And so the the way you go about training the model and the way you think about how to make the model better at those things becomes more difficult. It becomes more difficult from a research perspective. And so... But it's not impossible is what I would say. And so what it takes is a lot of research focus. It takes a lot of wonderful data. It takes a lot of feedback from experts in the field. We really wanna get the the, you know, the p 99.9 type of feedback to make the model, you know, as good as humans. But I don't think it's an insurmountable problem. And I'd... And what it takes is continued scaling the base model and focus. Focus on the research labs to think that this is an important problem to go solve. And one of the things that I'm viewing on my team is making this an area of focus and priority for OpenAI. And and I think because of the combination of those two things, I feel confident we'll make a lot of strides in 2027.
00:22:11 · S1
[00:22:11] S1: Do you have any comments on any of the current financial benchmarks? So I think we're still in the state of, you know, does this check out numbers wise? How do you think about making your own ad, egg, and taste? I think this is also an open call for people working in this space to work on their own benchmarks. Yeah. And a sort of follow along to that is you said build for... Build systems for models today, but you also give a lot of predictions for the future. Yes. How do you put guardrails, limitations, checks on financial services that you build for people today just so they're confident in, you know, this is not like going out off the rails.
00:22:48 · Joseph Kim
[00:22:48] S0: Yeah. Makes sense. Let's... Yeah. Let's talk a little about evals. Evals is... Yeah. Something near and dear to my heart. Spent a lot of time working on evals. I would say that the, like, the prior that I would share with people is building a good eval is incredibly hard. People... You know, that's not obvious to people who haven't worked on an eval before. To build an eval that actually like correlates with the thing that you're actually trying to measure, like the skill that you're trying to measure, capability you're trying to measure, very difficult. Because whatever you put together is just a proxy for whatever that thing is. Right? And it's just one dimension. So that's one. Two is it's very tedious to get the answer key or the way that you grade this correctly. And if you don't get the way that you get... You grade the answers correctly, it's eval that's very easy to hack, a very easy to reward hack. And therefore, you know, you could score very highly on the eval but actually do really poor on the on the actual underlying skill or objective that you're trying to measure. Right? And so the the the like prior that I was sure is evals are really hard, not just in financial work, but just across the board. And what I would say is we don't think that there's a really good eval today for financial work. There's a bunch of existing prior work. I'll I'll reference some the work that we did which was a GDPval which is like a little bit more broadly in knowledge work. If you guys look at the Astra announcement, don't talk about GDPval. We we know that the eval is completely saturated and that it's... There's a... And the pieces that are not saturated are essentially broken. Right? And so we actually don't report on that. We don't track it anymore. And so one of the tricky things for GPT six Astra was we didn't talk a whole lot about financial evals. We have some internally that we didn't talk too much about and didn't wanna share at the time yet. They're coming. But it was... There was no good way for us to show progress on financial tasks on these evals. There's a funny, like, funny things. Like, I think there's this bias in evals where if the numbers look good, the researchers assume that the model's gotten better and they don't investigate it. If the model... If the...
00:24:31 · Joseph Kim, S1
You've made a much more capable model and the eval numbers look bad or haven't increased in the the scale that you want, then you go investigate. Right? So this is kind of like bias baked into investigation of these evals. Like this happened with GPT-6 where we we ran GPT-6 on a set of financial evals and the numbers didn't really change much. We're like, oh, like what's going on here? Let's take a look at this eval. We we did a deep dive into these different evals and lots of things were broken. We... Like, the... Basically, we found that we had saturated the evals and the questions that we were getting wrong were driven by incorrect answer keys, poorly specified prompts, or just some other flaw with the eval. And so all that to say, like, one of the things that my team is working on currently is to try to build a much better eval that actually predicts the ability to automate the work of that specific profession. And so... Yeah. And so the idea is, like, we'll have something like investment banking eval that predicts... You know, if you score one these eval, you could probably automate a lot of the work of a junior analyst in investment banking. [00:25:22] S1: We gotta wrap here soon. I wanna ask any callouts for people in the finance space, the Latent Space audience? Your team is probably hiring, but last thing to put out there, how how can people be be helpful to you?