Latent Space: The AI Engineer Podcast
00:22:11 · S1
[00:22:11] S1: Do you have any comments on any of the current financial benchmarks? So I think we're still in the state of, you know, does this check out numbers wise? How do you think about making your own ad, egg, and taste? I think this is also an open call for people working in this space to work on their own benchmarks. Yeah. And a sort of follow along to that is you said build for... Build systems for models today, but you also give a lot of predictions for the future. Yes. How do you put guardrails, limitations, checks on financial services that you build for people today just so they're confident in, you know, this is not like going out off the rails.
00:22:48 · Joseph Kim
[00:22:48] S0: Yeah. Makes sense. Let's... Yeah. Let's talk a little about evals. Evals is... Yeah. Something near and dear to my heart. Spent a lot of time working on evals. I would say that the, like, the prior that I would share with people is building a good eval is incredibly hard. People... You know, that's not obvious to people who haven't worked on an eval before. To build an eval that actually like correlates with the thing that you're actually trying to measure, like the skill that you're trying to measure, capability you're trying to measure, very difficult. Because whatever you put together is just a proxy for whatever that thing is. Right? And it's just one dimension. So that's one. Two is it's very tedious to get the answer key or the way that you grade this correctly. And if you don't get the way that you get... You grade the answers correctly, it's eval that's very easy to hack, a very easy to reward hack. And therefore, you know, you could score very highly on the eval but actually do really poor on the on the actual underlying skill or objective that you're trying to measure. Right? And so the the the like prior that I was sure is evals are really hard, not just in financial work, but just across the board. And what I would say is we don't think that there's a really good eval today for financial work. There's a bunch of existing prior work. I'll I'll reference some the work that we did which was a GDPval which is like a little bit more broadly in knowledge work. If you guys look at the Astra announcement, don't talk about GDPval. We we know that the eval is completely saturated and that it's... There's a... And the pieces that are not saturated are essentially broken. Right? And so we actually don't report on that. We don't track it anymore. And so one of the tricky things for GPT six Astra was we didn't talk a whole lot about financial evals. We have some internally that we didn't talk too much about and didn't wanna share at the time yet. They're coming. But it was... There was no good way for us to show progress on financial tasks on these evals. There's a funny, like, funny things. Like, I think there's this bias in evals where if the numbers look good, the researchers assume that the model's gotten better and they don't investigate it. If the model... If the...
00:24:31 · Joseph Kim, S1
You've made a much more capable model and the eval numbers look bad or haven't increased in the the scale that you want, then you go investigate. Right? So this is kind of like bias baked into investigation of these evals. Like this happened with GPT-6 where we we ran GPT-6 on a set of financial evals and the numbers didn't really change much. We're like, oh, like what's going on here? Let's take a look at this eval. We we did a deep dive into these different evals and lots of things were broken. We... Like, the... Basically, we found that we had saturated the evals and the questions that we were getting wrong were driven by incorrect answer keys, poorly specified prompts, or just some other flaw with the eval. And so all that to say, like, one of the things that my team is working on currently is to try to build a much better eval that actually predicts the ability to automate the work of that specific profession. And so... Yeah. And so the idea is, like, we'll have something like investment banking eval that predicts... You know, if you score one these eval, you could probably automate a lot of the work of a junior analyst in investment banking. [00:25:22] S1: We gotta wrap here soon. I wanna ask any callouts for people in the finance space, the Latent Space audience? Your team is probably hiring, but last thing to put out there, how how can people be be helpful to you?
00:25:33 · Joseph Kim
[00:25:33] S0: Yeah. Wonderful. Well, I would say, like, first and foremost, like, please check out ChatGPT financial services. It is an enterprise product, but we're trying very hard to make it available to more prosumers on the pro plan. And so hopefully, we'll have some cool announcements there. Two, yes, my team is hiring. We're seeing incredible customer traction with the product and because of that, we're hiring and scaling the team rapidly. It's a very diverse team with lots of different roles and so I just wanna call out a few of them very quickly. There's obviously the the standard front end, back end engineering, so kind of traditional product building. So that's one. Two, we're very much looking for wonderful product managers. I think in some ways, great product management is the the main bottleneck in trying to go win in this space. And so we're trying to hire wonderful product managers in investment banking, equity research, asset management, you know, across the board. We're we're trying to hire product managers. And then the other role that I wanna give a special shout out to is we're hiring something called a subject matter expert. And, you know, it's really important. One of the things that we did well in in coding is, you know, the the people that were training the models and making them better understood what good code looks like. Right? And that was a huge tailwind for us as we're making the models better at these things. We don't have that advantage when it comes to making the models better at finance. And so we're we're building out and trying to hire a lot of folks that are able to bring that financial expertise and work with the engineers, work with the researchers, work with the product managers to make the model and the product better in your specific fields. And so if any of those resonates with you, like, I highly, you know, recommend that you reach out to me. I'm available on LinkedIn. We also have these job listings posted on our careers page. It's pretty easy to find.
00:27:02 · S2, Joseph Kim, S1
[00:27:02] S2: Correct. Well, you answered my question. I was gonna ask when I when can I have a play with this, but it is coming to Pro? [00:27:08] S0: Yeah. We're working very hard to make it available to more people. [00:27:10] S1: That's very common OpenAI stuff. Know, it makes sense. You make stuff for enterprise and then this trickles down the the regular models. It comes to Pro Plan. So Expect for your general usage down the line. It'll get better at. [00:27:20] S0: Yes. Absolutely. [00:27:21] S1: Awesome. Well, very excited to have you check out the keynote. You gave a lot. You gave good predictions. Yeah. You talked a little bit about some juicy. We fixed pretraining issues. I'll leave breadcrumbs for you guys to go watch, but thanks for joining. [00:27:32] S0: Yeah.