00:23:07 · Ronak Malde, Lauren Reeder
[00:23:07] S0: So I think that's actually one of the reasons why I'm very passionate about working on continual learning. So there's two kind of directions that it seems like research is going right now. There's RSI, Recursive Self Improvement, and there's continual learning. And both of these are very tightly tied together, right, or very similar concepts. They're essentially just how does an agent actually learn from experience. The thing is RSI is all about taking a model, putting it in a black box, and basically making it the best AI researcher itself so that it can improve itself, right? But this is kind of a scary world, right? Because it exactly begs the question of, are these models and agents actually going to be aligned and have the best interests of humans in mind? Where I think continual learning takes a very different approach, is it says, I want to learn from experience, but learn from the experience of actually interacting with humans, and then getting better at the tasks that they set out and that they want to do. And I think that's where the key difference is, is you need to allow these models and build the algorithm so that it keeps humans in the loop. I don't want to end up in a world where these agents are going rogue, I think the way to do that is to keep on this alignment path. [00:24:19] S1: Guys have done a bunch of unique things across product and research at Trajectory to bring these two halves of the whole together. Tell us more about what makes building at Trajectory unique, and who are some of the people you bring together to make it happen?
00:24:31 · Ronak Malde
[00:24:31] S0: Yeah, so the cool part that we are sitting in is the intersection of frontier research and product, where we have to modify the core RL algorithms in order to get to a place where you can actually learn from the world, but also package that into a really beautiful and clean interface for people to train these models. And so some of the research team that we've built that had contributed to, like for example, the co founders to Gemini three and Gemini Robotics, other folks to GPT four point zero at OpenAI, and building Muse infrastructure at MSL. But then on the product side as well, having people that have built scalable infrastructure at Stripe for payment SDKs. Also, have the youngest design engineer at Netflix. These sort of people, and I think the intersection between them is where the magic starts to happen. So the way we've actually structured the team, normally you have like a research stack that's this spaghetti code base in one corner of the company, and then you have the product that's kind of well built out. What we've done is actually productionized the code base for training as well. So this means microservices, Kubernetes scaling, auto scaling of every single part of the trainer, the rollout engine. And then on the design side as well, our design engineers are kicking off training runs all of the time and understanding deeply what their research experience is. I think this is a really interesting and cool place to be in. A lot of this is also only possible because of Frontier coding agents. Actually, most of the time now, we are spending our time whiteboarding, and we'll spend a lot of time just arguing about system design. And then the actual implementation is just a software factory of agents that will go in and actually implement it.
00:26:08 · Lauren Reeder, Ronak Malde
[00:26:08] S1: So building a product here is not easy. What is the role of a product builder in a world of continual learning? [00:26:13] S0: I think as a product builder, you should be thinking about how you actually take these traces and experience in the real world and start to shape and improve the product autonomously. So we're starting with the model, right, for Trajectory, and I think there's some interesting work to do there. Eventually that will be the model and the harness and basically self improving both of those. And I think once you have the harness and products are kind of becoming more AI native, and so the harness is kind of becoming most of the product for a lot of these companies, right? Well, and if you have the right to kind of self improve the harness and then the product, then you have the right to kind of improve and self, kind of self improve the entire company. And I think there then products and product builders start to become almost like a gardener, like you have your plant and it's going to grow autonomously on its own, And your goal as a product builder is to basically just set these trellises in the ground of here's where I want to shape my product. And I think this is going to be a very exciting world where you can just put a product out to a couple 100 users, it's going to grow on its own. And now we're gonna have basically you just imbue your taste, into where the company grows. So I think that's gonna be a very exciting world and companies should kind of start building towards that.
00:27:26 · Pat Grady
[00:27:26] S2: Yeah. For somebody listening who's thinking, boy, Trajectory sounds like a cool place to work, what's the pitch? Like, why why might somebody want to work at Trajectory?
00:27:35 · Ronak Malde
[00:27:35] S0: So I think there's a couple of things. The first thing is continual learning as a paradigm is is the next learning paradigm. Any massive company that you can think of of the previous age, so let's take Google or Amazon or Facebook, they were all doing continual learning, basically. If you think about like recommender systems or ads, the idea is taking all of these preferences from the real world and then creating online systems that can learn and get smarter the more they're used. And I think for agents that hasn't existed yet, and that's exactly the place, that Trajectory I want to be in. We're starting with just V1 of the product, which is a very easy way to train models, but in aggregate, I want it to be a massive recommender system of every single agent should be attuned to what companies want and what every single user eventually wants. I And think that's going to be a fundamentally exciting world to get to. There's a lot of work that needs to happen, and I also think Trajectory is not the only company in the ecosystem that needs to work really well to make this happen as well. You need, like two blocks down from us is Modal at our office that works on sandboxes, and that powers a lot of the infrastructure that we have, and two blocks down from that is Thinking Machines also in SF, and I think there's like all these sort of companies that need to be built to basically power all the infrastructure to make continual learning work, but I think the the product interface is where a lot of the exciting work can happen.
00:29:02 · Pat Grady
[00:29:02] S2: Yeah. One of the things you said when we were first getting to know you many, many months ago was this idea that the world right now has plentiful intelligence thanks to the Foundation Model Labs and plentiful experience thanks to all the people who do hard work in lots of different domains every day, but there's this big gap between the two And in some ways, continual learning is kind of the thing that bridges that gap, and it's kind of the thing that honors all of this earned experience that people have and, like, makes it legible to the intelligence coming out of the foundation models. And I thought that was, like, a very kinda elegant and beautiful way to think about what it is you guys are doing. And then the question that that leads to is what what is your most optimistic view of the future in terms of where continual learning can take us?