00:17:37 · Pat Grady, Ronak Malde, Lauren Reeder
[00:17:37] S2: Very cool. Who is the product targeted at? Like, who is the ideal customer for you guys today? [00:17:41] S0: Yeah. So what we want to do is make this very simple for companies that have already their agent harness in order, they've spent a lot of time there, but that next step of basically modifying the most valuable part, which is the model, making that super, super easy. So right now, what the product is meant for is mainly AI native companies like Harvey and Rogo, and some larger companies as well, like NVIDIA, are already using the platform. But the next step after this is I want to also move beyond just the model. So another important piece is the harness as well, because I think the model gets you efficiency. It gets you a model that has seen a task a 100 times or a thousand times over and making that faster, cheaper, and more reliable. What a harness gets you also is new capabilities. It's just something that was not possible before. And I think a lot of larger companies that are adopting AI, they also haven't done their work to optimize the harness for sub agents or for basically squeezing out that last mile, even out of the frontier tokens. And so the eventual world is maybe you use a frontier model like Fable for 10% of tasks, and then for 90% of the time, you can smartly route to a model that you've trained that is far more efficient and also gets you better margins. [00:18:57] S1: Tell us some stories about some of the customers you've worked with so far. You gave us a little hint at Harvey, but maybe a couple more examples.
00:18:57 · Lauren Reeder, Ronak Malde, Pat Grady
[00:18:57] S1: Tell us some stories about some of the customers you've worked with so far. You gave us a little hint at Harvey, but maybe a couple more examples. [00:19:03] S0: Yeah, so one really interesting one here is Rogo, and I think Rogo is taking a very interesting paradigm to agents that I think will become the future. So right now the way people use agents are very much like synchronous, right? You prompt the agent, it goes and does its thing, and then it comes back. What Rogo is very interested in is not just like this kind of verbose model that will go and basically like spit out a bunch of garbage, right? What they want is just like a very concise and clear finance agent that basically like does the work that you care about, It can autonomously do things on the side, but then it comes back and is very efficient. And so what the goal of Trajectory is, is to basically say, okay, Rogo, what good looks like for them is they also want the idea of intelligence density. They want the idea that a model can do the same task, but do that far more efficiently than the frontier models. They have the data to do that. And so with Trajectory, what they're able to do is train a model that is several times more efficient in terms of its tokens, and also happens to be cheaper and faster, but then is also able to gain and climb on the things that they care about. Very cool. [00:20:15] S2: Okay. So we talked a little bit about, like, group size one with GRPO. What's next? Like, where is continual learning headed?
00:20:15 · Pat Grady, Ronak Malde
[00:20:15] S2: Okay. So we talked a little bit about, like, group size one with GRPO. What's next? Like, where is continual learning headed? [00:20:22] S0: Yeah. So a lot of what we've been talking about right now is mainly just helping these companies, like, translate over what good looks like for them into improvements for the model. I think to get to this true world where humans, we are extremely efficient about learning from the real world, and we're able to have, I think, inbuilt curiosity as well about the world, and I would say that's kind of one of the missing pieces to get us to agents actually learning on a kind of autonomously on their own via continual learning. The models right now, they do a task, but they don't actually accumulate any experience or learn anything about the real world. Whereas if you think about like, let's say I'm throwing a football, right? I feel like as a human, maybe if I've never seen a football before, I actually have no idea what's gonna happen, and, but I do that a 100 times over, and now I suddenly build this heuristic, right? [00:21:17] S2: Yeah.
00:21:18 · Ronak Malde
[00:21:18] S0: And the important part is, I think with humans, like, as kids, we are constantly exploring the world. We see a soccer ball, we see a football, and we kind of just try out things to build this mental model, and that makes us incredibly intelligent, whereas with agents that doesn't exist right now. And so we've done a lot of thinking in how do you actually quantify curiosity? And I think one interesting way to think about this is, so there's this idea in RL called a Markov decision process, And it's basically, can characterize the world as a bunch of state action pairs. So a state is like, I'm holding the ball in my hand, and then an action is I throw it, and then the following state is where does the ball end up? What LLMs right now are really good at is basically going straight from state zero to state one, their next token predictors, right? But what's missing is that intermediate part of what is that transition function? When you actually like go from or S0 to S1, what is that transition function that happens? And LLMs don't really have a way of encoding that right now. So I think the interesting thing here is with humans, we are very aware of when we don't know what a transition function is in the world, and we say, I'm basically going to try to figure out what this transition function is, and I think this is where curiosity comes in. And so if there's a way to basically quantify, here's all of the world that I don't know about right now. And my goal as an agent is to figure out if I do this task, what happens? I build this mental heuristic. Then I think we are starting to solve this curiosity problem and get to a world where you can autonomously kind of learn and continually learn from the world.
00:23:01 · Pat Grady, Ronak Malde
[00:23:01] S2: How do you make sure that an agent's curiosity is aligned with a user's objectives? [00:23:07] S0: So I think that's actually one of the reasons why I'm very passionate about working on continual learning. So there's two kind of directions that it seems like research is going right now. There's RSI, Recursive Self Improvement, and there's continual learning. And both of these are very tightly tied together, right, or very similar concepts. They're essentially just how does an agent actually learn from experience. The thing is RSI is all about taking a model, putting it in a black box, and basically making it the best AI researcher itself so that it can improve itself, right? But this is kind of a scary world, right? Because it exactly begs the question of, are these models and agents actually going to be aligned and have the best interests of humans in mind? Where I think continual learning takes a very different approach, is it says, I want to learn from experience, but learn from the experience of actually interacting with humans, and then getting better at the tasks that they set out and that they want to do. And I think that's where the key difference is, is you need to allow these models and build the algorithm so that it keeps humans in the loop. I don't want to end up in a world where these agents are going rogue, I think the way to do that is to keep on this alignment path.
00:23:07 · Ronak Malde, Lauren Reeder
[00:23:07] S0: So I think that's actually one of the reasons why I'm very passionate about working on continual learning. So there's two kind of directions that it seems like research is going right now. There's RSI, Recursive Self Improvement, and there's continual learning. And both of these are very tightly tied together, right, or very similar concepts. They're essentially just how does an agent actually learn from experience. The thing is RSI is all about taking a model, putting it in a black box, and basically making it the best AI researcher itself so that it can improve itself, right? But this is kind of a scary world, right? Because it exactly begs the question of, are these models and agents actually going to be aligned and have the best interests of humans in mind? Where I think continual learning takes a very different approach, is it says, I want to learn from experience, but learn from the experience of actually interacting with humans, and then getting better at the tasks that they set out and that they want to do. And I think that's where the key difference is, is you need to allow these models and build the algorithm so that it keeps humans in the loop. I don't want to end up in a world where these agents are going rogue, I think the way to do that is to keep on this alignment path. [00:24:19] S1: Guys have done a bunch of unique things across product and research at Trajectory to bring these two halves of the whole together. Tell us more about what makes building at Trajectory unique, and who are some of the people you bring together to make it happen?