Chelsea Finn: This is the State of the Art in Robotics

00:30:00 · Chelsea FinnPlay from here

And on the bottom right, the robot is replacing a trash bag in a trash can. And so we had two goals at the start of this. The first was to move towards an out of the box model. And even those videos showed that out of the box, the model is able to do quite a bit. But really the key question here is how does this pre trained model compare to the specialists that were trained specifically for coffee making, specifically for box building that I talked about previously? And if we measure the throughput and the success rate of this single pi0.7 model versus the fine tuned pi0.6 model, we see that the... Across the board, the single PIO... Like, pre trained pi0.7 model matches or outperforms the fine tuned specialists that were developed with reinforcement learning post training for those downstream tasks. And so we see that it's able to match the performance of specialists, it also holds for SFT specialists, just RL post trained models as well, suggesting that we do indeed have a single model that can do a lot of different tasks with a really high degree of performance out of the box. Okay, so that was the first goal of out of the box models. And the second goal is compositional generalization. And there's a few different ways to measure this. There's many different ways you might try to combine concepts in robotics. The first test that we wanted to do was to see if a robot could interact with an appliance that's quite rare, like an air fryer. And this is kind of an example. We basically wanted to see if it could open an air fryer, put a sweet potato in the air fryer and close the air fryer. And we picked this because we thought that the data set didn't have any air fryers in it. We didn't intentionally collect any training data with air fryers. After we did some analysis on the data set, we actually found that our data set was so diverse that it did actually have three episodes with air fryers in it. We expect that they likely weren't having an impact and that even if you didn't include those exact three episodes, it likely would still work. But what we found generally is that the robot was able to interact with the appliance that was basically hardly represented at all in the training dataset and combine the skill of interacting with it, like opening it, closing it, and so forth with this object that it hasn't seen before.

00:32:21 · Chelsea FinnPlay from here

And then after instructing it, like Lucy did, we can train a high level policy to do this task fully autonomously. And you can see, the robot doing that in this video. So that's the first form of compositional generalization. The second compositional generalization test that we wanted to look at is we wanted to see if we could compositionally generalize between tasks and robot platforms. And so we wanted to take a robot platform called the Biarm UR5E robot. It's actually a very large, kind of industrial robot platform. And we want to see if it could fold clothes, despite the fact that we didn't collect any folding data on, this robot platform. So specifically, we had data of, folding clothes, like folding a shirt on the robot platform that's pictured here on the left. And then we wanted to see like out of the box without collecting any folding data on this very different robot platform on the right, could the robot successfully do the task? And what we see in this video is, the... We indeed did see that, it kind of compositionally generalized in this manner. The first time we saw the robot do this, we were like floored, because we... Like, there was no training data for this, for this task. And the robot here is quite different from the other robot, not just in size, but also in the lengths of the linkages of the robot, in the configuration of the joints of the robot, and so forth. Okay, we can, and this is a 1x speed video, so it's not the fastest thing. And obviously, if you haven't seen any training data on something, might not, it's literally the robot's first time folding a shirt, it might take a few attempts. But eventually, it will get to the folded shirt. You can also see the generated subgoal images on the top left. So those are basically the models trying to generate images that will make progress on the folding task. And then those are passes input to the model. And we see the folded shirt here. I think it's going make a couple small corrections at the end to try to make it a little bit smoother. Cool. So the takeaway here is that both in terms of language object interactions and in terms of task robot interactions, we see strong signs of compositional generalization in this model. Okay.

00:34:35 · Chelsea FinnPlay from here

And then quantitatively, we also see that, as we kind of get to these more advanced models like the pyo seven model, the performance of folding, towels and folding shirts on this platform that hasn't seen before, increases dramatically. And it even approaches the performance of human teleop, despite the fact that we didn't have any robot specific training data, for folding clothes. And then for the last experiment that we did here, I think this is maybe perhaps the most interesting experiment is we wanted to test how important are the two ingredients that I mentioned? How important is diverse data and how important is this sort of like capacity or detailed prompting for the kinds of results that I showed? And so if we remove the most diverse data from the model training shown in like the grayish color, we find that the performance on held out tasks decreases dramatically. Whereas if we, just take out a random 20% of the data that's less diverse than the most diverse subset, the performance, only decreases a little bit. So this suggests that actually having really diverse data, plays an important role in enabling it to generalize to new tasks. And then we tried to also ablate the fact that we are prompting the model with metadata. And for this experiment, we looked at, with and without prompting with metadata. So with prompting is shown in yellow and without prompting is shown in the gray color. And, with prompting, helps, kind of significantly. But the most interesting thing is if you look at, when you add... So this plot is showing as you add more and more data, and specifically as you add more and more low quality data, what is the performance? And without metadata prompting, when you add lower quality data from 80% data to a 100% data, the performance actually decreases, which is perhaps not too surprising because you're adding low quality data to your data mixture. Whereas with the metadata prompting, the performance actually increases when you add that low quality data. So Justin is actually able to make... Get get a lot more juice out of even low quality data, when you include this kind of prompting. Cool.

00:36:43 · Chelsea FinnPlay from here

So the takeaways here are that, we found that we're able to train a single model to control the robots that matches or exceeds the performance of specialized post trained models, kind of like going from a BERT like pretrained model to a model that really works out of the box like GPT. We also saw strong signs of compositional generalization in a dolly like way. For example, in compositionally generalizing skills applied to appliances and skills applied to new robots in ways that weren't seen in the training data. Cool. And then all the, like, videos and experiments that I showed were just evaluating the model out of the box without any post training. And the paper and the technical report online have a lot more experiments and a lot more details. Okay. So we talked about long term autonomy. We then showed how we can kind of develop that in a single general purpose model. Where are we at now? The first thing that I'll mention is if we go back to the timeline of generalist AI, I think that we now kind of firmly have physical intelligence in the right side of this timeline. We're kind of firmly more in like a GPT and DALL-E like era for robotics and physical intelligence, which is really exciting. And I think that... And we've kind of went there in just a few years. And lastly, we also have these models that are actually deployed in real world circumstances. So the two videos on the top are actually two YC companies, Ultra and Weave, that have taken find, PI models and post train them to do, in deployment to do tasks like folding laundry and packaging in a warehouse. The kind of video on the bottom, bottom left is, the video that I showed previously. And this kind of model works for a really diverse set of robot embodiments. The ones on kind of the top and the left are kind of a more standard bimanual platform, but it also can be adapted to things like drones, quadcopters, surgical robots, and on the bottom right, for things like tractors. So this is really kind of truly showing how physical intelligence can make an impact, not just in demos, in research and so forth, but actually in real world deployment. And I think that we'll, as we go, we'll start to see robots that are, more and more robots actually deployed for real in the physical world with all the advances that we've been seeing over the past few years. Awesome.

00:39:05 · Chelsea Finn, S1Play from here

So the last thing that I'll mention shamelessly is that, we are hiring at Physical Intelligence. So if you're excited about some of the stuff that I talked about, we encourage you to take a look at some of the open roles and apply. And yeah, definitely have time for questions and happy to get all your thoughts. Thanks. [00:39:24] S1: Okay, so the first question is, how far away are we from a ChatGPT moment for robotics and what will that look like? [00:39:31] S0: So I'll start with the second part actually, which is that I'm not sure it will really look like the ChatGPT moment that we saw in language models, which is that with something like ChatGPT, we saw like, it passed like what, a million new users in five days. I think that the distribution channel for physical models is gonna be slower, unfortunately, because you actually need a physical robot there. Mean, I think that we've seen for something like Waymo, it's actually been incredible to see the rollout that they've had, but it still takes time, to actually deploy things on physical devices. So I don't know if we'll have a single moment that has the distribution that ChatGPT had. At the same time, in terms of the capabilities of these models, I think that we are really starting to get to the point where these models are actually useful in the real world. And I think that getting to the kind of the capabilities of ChatGPT, I I think is, yeah, very much on the horizon in the next few years. Cool.

00:39:31 · Chelsea Finn, S1Play from here

[00:39:31] S0: So I'll start with the second part actually, which is that I'm not sure it will really look like the ChatGPT moment that we saw in language models, which is that with something like ChatGPT, we saw like, it passed like what, a million new users in five days. I think that the distribution channel for physical models is gonna be slower, unfortunately, because you actually need a physical robot there. Mean, I think that we've seen for something like Waymo, it's actually been incredible to see the rollout that they've had, but it still takes time, to actually deploy things on physical devices. So I don't know if we'll have a single moment that has the distribution that ChatGPT had. At the same time, in terms of the capabilities of these models, I think that we are really starting to get to the point where these models are actually useful in the real world. And I think that getting to the kind of the capabilities of ChatGPT, I I think is, yeah, very much on the horizon in the next few years. Cool. [00:40:31] S1: The second question is when should a small team switch from scaling per site models to a generalist policy and what does that transition actually look like? What signals tell you it's time?