00:07:22 · Sergey Levine
[00:07:22] S0: Yeah. So a robot foundation model needs to be able to take as input the things that the robot is observing. So those are typically going to be camera images. You could not use depth. You could use LIDAR, but in the end, it's some sort of, like, image structured data. And it needs to produce actions. And, again, you could have a few choices for actions, but they're all kind of the same. Something continuous that represents either positions of the robot's end effectors or joint angles or joint motor commands, torques, whatever. The... Basically, a vector of numbers, roughly, you know, 30 to 50 numbers. So that's the basics. Now if you want it to be a foundation model, you need one more thing, which is you need to tell it what you want it to do. And, you know, the most obvious way to do that is to tell it in language, So therefore, it also gets language as input. There are other ways to tell it what to do as well that are a little bit more involved, but language is a fairly basic one. So that gives us, like, the starting outlines of this thing. It needs to produce continuous actions. It needs to take in lots of images, and it needs to take in language. So now you have a few choices. One very obvious choice is train the model from scratch on all of your robot data. So that's basic imitation learning. The trouble with this is it's very difficult from just the robot data you have to get everything that you need for the system to be successful. So that's that prior knowledge piece that I mentioned before. So then the next logical step is to say, well, if you need prior knowledge, let's start it off from a model that kind of provides the complement to your robot data. So you don't wanna start from a model that's as close as possible to a robot, because you need to give it the stuff that the robot data doesn't have, and that's the prior knowledge. So that's why the LLM is a really good starting point, because the LLM kinda has like exactly what you lack. It has lots of knowledge, but it is not grounded. So the LLM is a good starting point. Now I don't think that's the only way to do it. I think that once you have that, that... That's kinda like the outlines of a thing, but then you can fill in... You can color in the outlines and put in more stuff.
00:09:11 · Sergey Levine, Aditya Agarwal
So you could, for example, and I think we've already seen some of this, and we'll see more of it in the future. You can include other data and training. You can include videos. You can include, you know, data from humans, all this other stuff. But the Pi Zero model really was not intended to be the ultimate thing that does everything. It was intended to be the outline. And once we draw the outline, then we can color in the stuff inside. We can add in reasoning, chain of thought, other data sources, and so on. That's lot of what we've been doing since then. [00:09:38] S1: Okay. So maybe to move on from Pi Zero to Pi 0.5, I think one of the cool things about the second model... It was the second model. Right? Yep. Was that it started to demonstrate some amount of generalization. It started to kind of like work in environments where it had not been explicitly trained. It cleaned kitchens and bedrooms that it had never seen before. Also, it just did more long horizon kind of like work that was more dexterous. So what is the key ingredient that kind of unlocked the capability kind of, like, step function change or, I guess, generalization from Pi zero to Pi 0.5?
00:10:14 · Sergey Levine
[00:10:14] S0: So there are quite a few things, but, you know, a a few of the really important principles for robot foundation models. These principles are not actually all that surprising because they look just like the principles for other foundation models, but they have to be mapped onto the robotics problem in the right way. So the the first and most obvious one is that you need the right data, and getting the right data is not is not just an industrial problem, it is also a research problem. In fact, that's why it's such a difficult problem is that it combines, you know, economics, industrial scale operations, and research. You have to figure out the right thing, scale it up in the right way. You usually can't answer what the right thing is until you scale it up to some level. So that's the data piece. You also need the model to be able to understand complex instructions and break them down into manageable pieces. So this is where the LLM part actually becomes really important. So the Pi0.5 model actually had an internal reasoning component. So if you tell it to clean a kitchen, it doesn't just try to map clean the kitchen to robot actions. It breaks it down into thoughts, and the thoughts might be like, pick up the dish, pick up the the towel, wipe the spill, and then from there grounds that into actions. And this is tremendously powerful. Like, this is why... This is this is the same reasons why coding agents work. Right? Coding agents today, you don't just tell them what you want and they spit out lots of code. They actually make a plan. They break it down into steps. And Pi0.5 was our first model that did something like that. And since then, we've extended it. We can do reasoning in other modalities. We can do reasoning of variable length and so on. And these are very important things for doing really complex tasks.
00:11:45 · Aditya Agarwal
[00:11:45] S1: How much of the capabilities increase, whether it be Pi 0.5 or 0.7, is coming in some ways of the afterglow of what we are seeing of, like, you know, kind of frontier, more standard LLM kind of research or, I guess, capabilities versus, hey. It's great that we figured out reasoning models. Maybe we'll figure out how to get, you know, swarms or tens of thousands of essentially agents collaborating to do things and then figuring out how to apply it to robotics. Guess, do you think of that as a kind of like a, you know, concrete strategy at Pi, which is that, like, it's great. We have, you know, OpenAI and Anthropic with, like, hundreds of billions of dollars worth of kind of research capabilities going in, and we'll be able to kind of draft off of that kind of like, you know, for more specific domains.
00:12:34 · Sergey Levine
[00:12:34] S0: So the trouble is that the the ideas are not actually the same. Like, there's analogies to be drawn, but the analogies from my experience often end up being more obvious in hindsight than they are in foresight. So for example, one of the things that turned out to be really important to get great, zero shot generalization in Pi0.7, you know, Pi0.7 is a very powerful model. It can actually generalize to entirely new tasks. You can give it like a kitchen appliance that the robot has never used, and it can kinda muddle through and figure out how to use that kitchen appliance. One of the important things for that is to have the right annotations for your robot examples. Now in hindsight, this is blindingly obvious. Like, the biggest innovation for video models has been prompt expansion. Prompt expansion is what enables video models to actually... Not just to take complex instructions, but to fit complex data. But the trouble is that for a robot task, obviously, you can't do the same thing as you do for video models. Because for video models, a prompt expansion, you tell it, you know, make a video of a dog running through the grass, and it'll say, you know, there's beautiful green grass with, like, dandelions growing on it, and the dog has, like, flowy golden fur. Like, the robot doesn't care about any of that. The robot needs something else. It needs something that is much more tied to the behavior that needs to exhibit. So the kind of... The principle is kind of shared. Why does prompt expansion work? Because more detailed prompts have higher mutual information with the output that you want so that the same principle... You still have to figure out how to instantiate that principle in a robotics context so that actually works.
00:13:57 · Aditya Agarwal
[00:13:57] S1: Talk a little bit about Pi 0.7, your latest model. Right? Like, what is the big delta between zero point five and zero point seven? And insofar as you're able to share, is the