Latent Space: The AI Engineer Podcast
00:06:07 · Joseph Kim, S1
Some some of the MCPs that we connected, some big names, 40% plus error rate on their MCPs. Right? And so MCPs is one of those things where it it looks great on paper, but you have to go see... Look at the data and see what's actually going on. And so for data, we're like, hey. Look. This is really bad. The only way we can go about fixing this at scale is to bring the data in house, like just control that experience end to end. And the extra money that you pay on top to bundle that data is worth it. Right? So that was a product decision that we had to make that was very industry specific, and it's been resonating with customers. Customers love it. You know, the number one thing they call out is the data, and so it worked. [00:06:40] S1: So you had good success in this. Tell us about the technical side of things for people building a harness adaptation for a specific service. Right? So you guys went out, you changed codex, you did post training, you did whatever you did to make ChatGPT for financial services. If someone was to the... To do the same for another domain, say legal sales, whatever, what are the best practices there? What did you learn? What goes into just taking off the shelf and actually, you know, making a certain product around it?
00:07:07 · Joseph Kim
[00:07:07] S0: Yeah. It's a... Yeah. I like the question a lot. It it... You know, I try to allude to this in my keynote as well, but the, like, interesting thing is, you know, it's fun to talk about the post training stuff because it's unique to what we do and gets people excited. But 95% of what we built was something that absolutely a startup could do, another company could do. And we didn't really have to go leverage our ability to go and chain... Make changes in the model to go about building fantastic product. Right? And so I think it's a great call to kind of mention that What I would say is like few fold. One is most of the product that we built was built on top of the codex or I should say ChatGPT now plug in architecture and plug in primitives. And that is something that we've been investing a lot of time and energy in. You know, OpenAI is like an interesting company where, you know, we have... We kind of face the same problems internally that a startup would face if they were to build on top of OpenAI. So in very... In a lot of ways, like, team is kind of like a startup that's building on top of OpenAI technology. Right? And so even internally, we need to find scalable ways for teams like myself to stand on the shoulders of giants and build on top of Codex. Right? Codex is fantastic. It's got great memory. It's got good skills. Like, there's a... You know, it's persistent. There's a lot of things that are wonderful about Codex. But the thesis of my team was Codex was... Codex will get you about 80% of the way to a product that would be successful in the market. But, you know, the 20% is missing. And so we need to find some way for my team to go build that 20% to actually go because the other half of the thesis was if you just took Codex and launched it as a finance product, it would fail. And so that 20% is actually pivotal. So what I would say is we we wouldn't be built on top of the plug in architecture. It's using the same primitives that are available to any of the builders that wanna go build on top of ChatGPT. And, you know, I had a line in my presentation, but I I really do think that there's gonna be a billion dollar business built on top of a a ChatGPT plugin. Right? Like, it's... Like, you're gonna be able to leverage all of that horizontal functionality and just add your secret sauce on top.
00:08:57 · Joseph Kim, S1, S2
And that secret sauce might be data, it might be UI changes, it could be an integration, but, you know, that secret sauce is really valuable. And I think there's there's a billion dollars worth of value to be built [00:09:07] S1: there. So, [00:09:09] S2: yeah, it's an interesting time for the industry because we're at a a time now where orchestration is becoming a lot more automated, you know, teams of agents going out and doing things. Yep. But for finance, obviously, there are a lot of risks involved in that. So can you talk a bit about about the safety sort of that you've baked into the product?
00:09:26 · Joseph Kim
[00:09:26] S0: Yeah. Safety is super, super important. I would say that like... And it's only going to get more important because the the, you know, the agents are already getting more autonomous and more independent. And I think that trend we expect it to continue into '27 into '28. I would say for the for the product that we built and launched last month, we... There was a bunch of paradigms that already existed as it pertains to safety and, you know, being thoughtful around auditing the work, making sure that the model is not making a mistake that's gonna impact business outcomes. And we're able to lean heavily into those. And and what I would say is the existing product that we launched is a productivity tool for knowledge workers working in the space. And so the idea behind it is there's a deep assumption that there's always a human in the loop. Right? Like, so the idea there is, you know, you're a financial analyst and you have a a laundry list of things you wanna get done that day. And and just like how a software engineers do with Codex, for every one of those items, the first thing you do is try to build it with ChatGPT. Right? And it might get you 30% of the way there. It might get you 80% of way there depending on the but there's always this presumption of humans in the loop that are presiding over that work. And that made it very easy for us to to build a product where we could kind of pass off the responsibility of auditing review into people's hands and into people's responsibility. And that worked for us well and it was easy to communicate. So that was wonderful. I will say we're thinking very deeply about what comes next and and what comes next looks different than than a productivity tool that sits and and is always being monitored by a human. And I think as we think about what comes next, there's kind of two things we're thinking about. One is how do we kind of shape the security safety profile of the product into something that people understand intuitively? And some... I would say something that people understand very intuitively is how to work with other people. And so our thesis is if we make the product very similar to how you work with your your coworkers, your interns, people around you, that is something that people can pick up and learn very quickly, very easily. And we...
00:11:16 · Joseph Kim
And therefore, we need to shape the security profile of that to look like a person. And so there's a few things I'll call out there. One is there's this notion of read, write, you know, even for people. Like, you're a new software engineer at a company. You oftentimes... You know, when I was at Google, you need you need something called readability. And so there's these kind of, like, checks and balances and systems in place before you can start mucking around and touching the system. And so there's already systems in place to monitor the work of people. So if mod... If models behave more like people, then you can leverage those systems, And that makes it very easy to kind of talk about and work through this with these financial institutions. So that's something we're leading into pretty heavily coming up. And the very last thing I would say is, at least in the context of banking, is where we kind of started, there's already this system in place for junior analysts doing a lot of the groundwork. Their work gets passed off to associates. Associates are reviewing the work. Then the associates are kind of adding their secret sauce, their magic, their reasoning. That gets passed off to the VP. VP goes up to the director, managing director. So there's this kind of like a hierarchical system where there's this implicit work checking that's happening that's already been built out to stop... To try to prevent against mistakes that humans make. Right? Like, humans make just as many mistakes as models, and so those systems are built there. And so we're also leaning heavily in terms of those systems as it as it pertains to building financial products.
00:12:36 · S1, Joseph Kim, S2
[00:12:36] S1: Okay. I'd say there's this, like, sort of misconception where in the finance world, one mistake really compounds like your your whole career is at risk if you submit something wrong. How should people think about getting started and using it? So if you're an analyst, how should you use these tools? And do have like any analogies towards where we're at right now? So there's like levels of autonomy and self driving. One of your slides, think said like 2027, you see no more PowerPoint. A lot of people will be using AI tools in the loop. So how should people think about where we are now and where things are going and how should they adopt these tools if they have that misconception of like one mistake my career is over. Obviously there's human in the loop, but how should they also think about it going forward as they need to offload more and more? [00:13:18] S0: Or Yeah. No. That's that's wonderful. I I have two perspectives on it, and they come... They start with different assumptions. I think I I think the notion that, like, my career is over if you make one mistake is an incorrect notion, And I have a line of thought that, like, follows that. But let me... Let's first assume that it's true. Right? And and in the case that it's true, what I would say is, like, not all mistakes are created equally [00:13:38] S2: Yeah.