00:15:21 · Ronak Malde, Pat Grady, Lauren Reeder
So what we've realized is, let's say I have 10,000 traces. If I can smartly select a 100 of them that are basically representative of all of these broader 10,000, and then have the customer or the user basically densely say, okay, here's what good looks like, here's a rubric or verifiable criteria for just those a 100 traces. And then what we can do is basically train a meta grader that learns from those behaviors of the 100 to the broader 10,000. And with that, you get a really clean interface that allows you to just propagate this reward across the board. So I think obviously it's not the perfect solution in that direction. The ideal goal actually is that you have some sort of observability interface where you're like, here's the five things my agent's good at, here's the five things it's bad at, and I just circle those five things and hit train. That's the world I want to get to, and that's where we're pushing that research direction in. [00:16:16] S2: Very cool. [00:16:16] S1: So you have a product that you just released to do some of this. Tell us about it. What have you, what are you really seeing in the product and who is it for?
00:16:16 · Lauren Reeder, Ronak Malde
[00:16:16] S1: So you have a product that you just released to do some of this. Tell us about it. What have you, what are you really seeing in the product and who is it for? [00:16:22] S0: Yeah, up until now, it's taken a massive research team, a ton of time and a ton of compute in order to train these models. What we wanted to do is just make the easiest and most intuitive interface to train models and customize them for what you care about. And so Trajectory, the platform that we released, is basically a very simple system where you can take your agent traces and what good looks like. It's a one click train interface that just takes a couple agent prompts, and then you can eval the model and then deploy it all in one place. And I think the really magical part about this, a lot of our philosophy in product actually comes from Apple, and when, in like the early 2000s, you had a computer that had access to the internet, and you had a phone that you could make phone calls, and I think no one really thought to put the two together until Apple said, okay, what if you actually put the internet on a handheld device and combine the two? And I think by doing this, of basically putting training, eval, and deployment in one place, making that just dead simple for every single person to do, and then you can close the loop and basically now upload that model back to production, gather new traces, it suddenly is this magical paradigm that I think every single company is going to adopt very soon.
00:16:22 · Ronak Malde, Pat Grady
[00:16:22] S0: Yeah, up until now, it's taken a massive research team, a ton of time and a ton of compute in order to train these models. What we wanted to do is just make the easiest and most intuitive interface to train models and customize them for what you care about. And so Trajectory, the platform that we released, is basically a very simple system where you can take your agent traces and what good looks like. It's a one click train interface that just takes a couple agent prompts, and then you can eval the model and then deploy it all in one place. And I think the really magical part about this, a lot of our philosophy in product actually comes from Apple, and when, in like the early 2000s, you had a computer that had access to the internet, and you had a phone that you could make phone calls, and I think no one really thought to put the two together until Apple said, okay, what if you actually put the internet on a handheld device and combine the two? And I think by doing this, of basically putting training, eval, and deployment in one place, making that just dead simple for every single person to do, and then you can close the loop and basically now upload that model back to production, gather new traces, it suddenly is this magical paradigm that I think every single company is going to adopt very soon. [00:17:37] S2: Very cool. Who is the product targeted at? Like, who is the ideal customer for you guys today?
00:17:37 · Pat Grady, Ronak Malde, Lauren Reeder
[00:17:37] S2: Very cool. Who is the product targeted at? Like, who is the ideal customer for you guys today? [00:17:41] S0: Yeah. So what we want to do is make this very simple for companies that have already their agent harness in order, they've spent a lot of time there, but that next step of basically modifying the most valuable part, which is the model, making that super, super easy. So right now, what the product is meant for is mainly AI native companies like Harvey and Rogo, and some larger companies as well, like NVIDIA, are already using the platform. But the next step after this is I want to also move beyond just the model. So another important piece is the harness as well, because I think the model gets you efficiency. It gets you a model that has seen a task a 100 times or a thousand times over and making that faster, cheaper, and more reliable. What a harness gets you also is new capabilities. It's just something that was not possible before. And I think a lot of larger companies that are adopting AI, they also haven't done their work to optimize the harness for sub agents or for basically squeezing out that last mile, even out of the frontier tokens. And so the eventual world is maybe you use a frontier model like Fable for 10% of tasks, and then for 90% of the time, you can smartly route to a model that you've trained that is far more efficient and also gets you better margins. [00:18:57] S1: Tell us some stories about some of the customers you've worked with so far. You gave us a little hint at Harvey, but maybe a couple more examples.
00:18:57 · Lauren Reeder, Ronak Malde, Pat Grady
[00:18:57] S1: Tell us some stories about some of the customers you've worked with so far. You gave us a little hint at Harvey, but maybe a couple more examples. [00:19:03] S0: Yeah, so one really interesting one here is Rogo, and I think Rogo is taking a very interesting paradigm to agents that I think will become the future. So right now the way people use agents are very much like synchronous, right? You prompt the agent, it goes and does its thing, and then it comes back. What Rogo is very interested in is not just like this kind of verbose model that will go and basically like spit out a bunch of garbage, right? What they want is just like a very concise and clear finance agent that basically like does the work that you care about, It can autonomously do things on the side, but then it comes back and is very efficient. And so what the goal of Trajectory is, is to basically say, okay, Rogo, what good looks like for them is they also want the idea of intelligence density. They want the idea that a model can do the same task, but do that far more efficiently than the frontier models. They have the data to do that. And so with Trajectory, what they're able to do is train a model that is several times more efficient in terms of its tokens, and also happens to be cheaper and faster, but then is also able to gain and climb on the things that they care about. Very cool. [00:20:15] S2: Okay. So we talked a little bit about, like, group size one with GRPO. What's next? Like, where is continual learning headed?
00:20:15 · Pat Grady, Ronak Malde
[00:20:15] S2: Okay. So we talked a little bit about, like, group size one with GRPO. What's next? Like, where is continual learning headed? [00:20:22] S0: Yeah. So a lot of what we've been talking about right now is mainly just helping these companies, like, translate over what good looks like for them into improvements for the model. I think to get to this true world where humans, we are extremely efficient about learning from the real world, and we're able to have, I think, inbuilt curiosity as well about the world, and I would say that's kind of one of the missing pieces to get us to agents actually learning on a kind of autonomously on their own via continual learning. The models right now, they do a task, but they don't actually accumulate any experience or learn anything about the real world. Whereas if you think about like, let's say I'm throwing a football, right? I feel like as a human, maybe if I've never seen a football before, I actually have no idea what's gonna happen, and, but I do that a 100 times over, and now I suddenly build this heuristic, right? [00:21:17] S2: Yeah.