The Robot Report Podcast
00:45:51 · Eugene Demetri, Scott Kuindersma, Steve Crowe
[00:45:51] S3: be able to get there. [00:45:52] S4: Yeah, I mean, really is. It's also like a very powerful thing, right? Because it might not even be obvious in some cases. Like if you ask me, can the robot do X? And it was some manipulation tasks we just hadn't tried before. Now the question, the sort of corollary question, you know, in a behavior cloning type setting like this large behavior model setting is, can I teleoperate that? If I can teleoperate what you're asking, then the answer is almost certainly yes, we can get the robot to do that. We basically know the process for doing it. That's really powerful. Before we had this type of machine learning based approach to programming the robot, we have to think carefully about, okay, well, you know, what are the, you know, the physical properties of the objects that may come into play and do I... Will those be captured by the models that are embedded in my planning and control stacks and all these various things? Now it's like, okay, well, we can grab a tablecloth and flip it out and cover a table. And no one had to write a model for that. No one had to write a tablecloth detector. No one had to do anything other than strap on a VR system and pick up a tablecloth and flip it over, you know, a few 100 times. [00:47:08] S0: What? This might be a stupid question. It might not be. Like, why does this work so well?
00:47:13 · Scott Kuindersma
[00:47:13] S4: It's not a stupid question. I think it's a really deep question, and I'm not sure we have the full answer to it, to be honest. So, I think there's also like a different answer depending on the scale of what you're observing. So, it's people, you know, in the original diffusion policy paper, for example, that is not what what anyone would call a large behavior model. That is taking some number of demonstrations and then training a diffusion style neural network model in order to reproduce those demonstrations. And essentially, that's a very high capacity model. And it's able to... I wouldn't say it it'd be oversimplifying to say it's memorizing those demonstrations, but it knows them very well and it's able to use feedback from the robot sensors in order to basically do something that feels like a nearest neighbor search and like playback the relevant pieces of the demonstration data in order to reproduce the behavior that the demonstrators performed. And so I think that's just a more of a benefit of the diffusion style architecture that allows you to represent very complex distributions. I'm trying to avoid going into too much technical detail here, but there's this property called multimodality where essentially, like a simple example would be, imagine there's two ways to go from point A to point B, one that goes around the right hand side of an obstacle and one that goes around the left hand side of an obstacle. And demonstrators are giving you both answers with some, you know, different probability. If you didn't have a model that could capture multimodality, it might try to memorize something that looks like the average correct answer, which would be to go straight through the obstacle. And that's just not the right answer. And so you need an expressive enough model to be able to represent both answers, even though they're kind of different, they're both correct. And a lot of the manipulation tasks that we were wanting robots to do now have that kind of multimodality. And so having identified an architecture that's able to handle that appropriately is part of the reason why this works well.
00:49:39 · Eugene Demetri
[00:49:39] S3: I hear a lot about emergent behavior, but so much, like you said, comes down to that probabilistic modeling and and weighing of of outcomes. So let's go down a couple of levels. We read a little bit about how both Atlas' arms and legs have been controlled by the same model, but with feet as additional hands. New approach and how does that work? There must be instances where behaviors for one might not work for the other.
00:50:08 · Scott Kuindersma
[00:50:08] S4: Yeah. Basically, what we did was so we have this teleoperation interface that we built that's based on a Vive style VR system. So there's a a head mounted display, and so people are looking into a VR headset. And so they don't have line of sight to the robot. They can only see through the robot's camera sensors. They have trackers on their wrists and torso and feet, and then they have some kind of controller for the hands. And we have two different things. One is a VR style controller where we can use button presses to switch between different hand shapes and then use triggers in order to open and close the hands. Or we have motion capture gloves that allow people to do finger level control of the hands for more dexterous things. And so when we say that the hands are... That the feet are essentially like additional hands, what we mean is that the trackers that are attaching to the feet and hands are essentially the same, and those get turned into just desired reference streams into an underlying controller that's coordinating the entire body. So part of the teleoperation system is resting on the years of work that we did in whole body control. So we're able to basically just take these raw streams of information from the VR system and throw it into this whole body controller. And it's able to do all the coordination, maintaining balance, avoiding self collisions. All that stuff is just happening underneath the teleoperation so that the human being doesn't have to worry about all that. A person's not literally trying to balance the robot through moving their own body, or they're not really worried about, well, if I move my foot from here to here too quickly or whatever, it sort of doesn't matter because the underlying controller is gonna sort of do it in a way that doesn't knock the robot over. And then once we've done that and we collect the data, the policy uses the exact same interface as the teleop system. So from the underlying control system, it doesn't really know if it's a teleoperator or if it's a policy. And so that's also been a really flexible way to architect the thing.
00:52:22 · Steve Crowe
[00:52:22] S0: Are there things holding back how quickly you guys could scale these behaviors on Atlas to more tasks or even broader tasks? Like, I think you guys use the new Jetson Thor platform, but is it computing power? Is it training data and diversity of that data? Like are those sort of, I don't mean this in a negative way, but what are those bottlenecks that are holding you guys back from taking this even further more quickly?
00:52:50 · Scott Kuindersma
[00:52:50] S4: Yeah, for sure it's data. That's just because in order to collect, like the highest quality data that you can get is data for the tasks you care about on the robot you care about. And there's just a limited amount of that. So you only have so many humanoids. Humanoids are among the most complex robots that anyone's trying to do this kind of work on. So, I think as you look at other types of robot platforms like table mounted arms and things like that, I think your ability to scale it up to just massive data collect facilities is a little better than it is with humanoids. So, I think that's one thing. But one of the really exciting things in this space is that there's other scaling bets that we can leverage here. So I think if the only answer we believe was true is that you needed on hardware experience in order to train these models and nothing else would work. That would be pretty problematic because I think in order to get to even like GPT, I don't know, two level scale would be a tremendous undertaking, right? And so what people tend to talk about is something like a data pyramid. And so you could think about the tip of that pyramid as being the highest quality data that if you just had more and more of that, things are only gonna get better. And so that's sort of like the teleoperation on Atlas. And then as you go down that pyramid, the scalability proposition gets a lot better, but the embodiment gap maybe gets bigger as well. So maybe if I go down a little bit, I can talk about simulation data, right? So if I go down somewhere in the pyramid, having simulations of my robot doing manipulation tasks could be used as co training data. So this is data that we throw into the mix with the real robot data. We've actually found in our experience that, and other groups have too, that incorporating simulation data as part of the training mixture actually improves performance on the physical robot, which is really exciting. One of the other opportunities about humanoids specifically is maybe there's more of an opportunity to use human demonstration data that isn't teleop. So if I have recordings of people, people wearing instrumentation, egocentric videos of people doing manipulation tasks, if their bodies are not too dissimilar from my robot's body, that may be an easier thing to translate into and inform robot behavior.