The Robot Report Podcast
00:42:39 · Scott Kuindersma
[00:42:39] S4: Yeah. I mean, I think it's... We've been really impressed with the... I'll just say I've been very impressed with the progress that the team has made, you know, over the past year largely pivoting the approach that we're taking to programming the robot. So, many, like I've had many years of experience in what I would call the kind of bread and butter approaches to programming robots, where we build modular software stacks that are combining different elements. Yes, some machine learning based elements that maybe were mostly in charge of doing computer vision tasks and model based elements like model predictive controllers and planners and things like that. And what we showed last year with e Atlas doing Hyundai part sequencing tasks was a demonstration that you could take a system like that and you could make it do really impressive and approaching real world work. The problem with those types of approaches is that in order to take that robot and put it in front of an entirely new task that maybe we hadn't anticipated yet, the process for onboarding that new task would almost certainly involve a tremendous level of robotics expertise. And this could be from PhD roboticist to, or even just the field application engineer type folks to get the robot to do that thing. That's okay, but there's a scalability problem there, right? So if we're talking about what's the opportunity of humanoids, I think it's that these are robots that are physically suited to doing many, many different labor tasks, and that their ability to be re tasked is sufficiently low overhead that this becomes like a very flexible thing that you can just scale out to tens of thousands of robots. Now, in order for that to be true, I think it can't be the case that you need a team of robotics experts involved every time you wanted to get the robot to do something new. And so what's exciting about these approaches is that it's a repeatable process of just providing demonstration style inputs to the robot. And if you can demonstrate it, then you can train a neural network to make it autonomous. And that's really, really powerful. And so one of the things that's impressed me is seeing that comparison back to back, right?
00:45:01 · Scott Kuindersma, Steve Crowe, Eugene Demetri
Just having a really deep understanding of both approaches now and just seeing how different it feels to program the robot this way and the opportunity for scalability. So that's the one thing that I'm just really excited about right now. [00:45:17] S0: Yeah, let's dive into that more because that's something that we wrote down that we wanted to ask you. I wrote down a couple of the quotes from the blog. We don't have to read them verbatim, but you guys basically wrote with TRI in the blog that the training process is the same. This is an exact quote. So the training process is the same, whether it's stacking rigid blocks, folding a t shirt. If you can demonstrate it, the robot can learn it. I mean, again, an outsider, those are pretty lofty statements, but if that's true, like it just must open up a whole new world for what you guys and what other robotics developers think is possible and how quickly you're going to [00:45:51] S3: be able to get there.
00:45:51 · Eugene Demetri, Scott Kuindersma, Steve Crowe
[00:45:51] S3: be able to get there. [00:45:52] S4: Yeah, I mean, really is. It's also like a very powerful thing, right? Because it might not even be obvious in some cases. Like if you ask me, can the robot do X? And it was some manipulation tasks we just hadn't tried before. Now the question, the sort of corollary question, you know, in a behavior cloning type setting like this large behavior model setting is, can I teleoperate that? If I can teleoperate what you're asking, then the answer is almost certainly yes, we can get the robot to do that. We basically know the process for doing it. That's really powerful. Before we had this type of machine learning based approach to programming the robot, we have to think carefully about, okay, well, you know, what are the, you know, the physical properties of the objects that may come into play and do I... Will those be captured by the models that are embedded in my planning and control stacks and all these various things? Now it's like, okay, well, we can grab a tablecloth and flip it out and cover a table. And no one had to write a model for that. No one had to write a tablecloth detector. No one had to do anything other than strap on a VR system and pick up a tablecloth and flip it over, you know, a few 100 times. [00:47:08] S0: What? This might be a stupid question. It might not be. Like, why does this work so well?
00:47:13 · Scott Kuindersma
[00:47:13] S4: It's not a stupid question. I think it's a really deep question, and I'm not sure we have the full answer to it, to be honest. So, I think there's also like a different answer depending on the scale of what you're observing. So, it's people, you know, in the original diffusion policy paper, for example, that is not what what anyone would call a large behavior model. That is taking some number of demonstrations and then training a diffusion style neural network model in order to reproduce those demonstrations. And essentially, that's a very high capacity model. And it's able to... I wouldn't say it it'd be oversimplifying to say it's memorizing those demonstrations, but it knows them very well and it's able to use feedback from the robot sensors in order to basically do something that feels like a nearest neighbor search and like playback the relevant pieces of the demonstration data in order to reproduce the behavior that the demonstrators performed. And so I think that's just a more of a benefit of the diffusion style architecture that allows you to represent very complex distributions. I'm trying to avoid going into too much technical detail here, but there's this property called multimodality where essentially, like a simple example would be, imagine there's two ways to go from point A to point B, one that goes around the right hand side of an obstacle and one that goes around the left hand side of an obstacle. And demonstrators are giving you both answers with some, you know, different probability. If you didn't have a model that could capture multimodality, it might try to memorize something that looks like the average correct answer, which would be to go straight through the obstacle. And that's just not the right answer. And so you need an expressive enough model to be able to represent both answers, even though they're kind of different, they're both correct. And a lot of the manipulation tasks that we were wanting robots to do now have that kind of multimodality. And so having identified an architecture that's able to handle that appropriately is part of the reason why this works well.
00:49:39 · Eugene Demetri
[00:49:39] S3: I hear a lot about emergent behavior, but so much, like you said, comes down to that probabilistic modeling and and weighing of of outcomes. So let's go down a couple of levels. We read a little bit about how both Atlas' arms and legs have been controlled by the same model, but with feet as additional hands. New approach and how does that work? There must be instances where behaviors for one might not work for the other.
00:50:08 · Scott Kuindersma
[00:50:08] S4: Yeah. Basically, what we did was so we have this teleoperation interface that we built that's based on a Vive style VR system. So there's a a head mounted display, and so people are looking into a VR headset. And so they don't have line of sight to the robot. They can only see through the robot's camera sensors. They have trackers on their wrists and torso and feet, and then they have some kind of controller for the hands. And we have two different things. One is a VR style controller where we can use button presses to switch between different hand shapes and then use triggers in order to open and close the hands. Or we have motion capture gloves that allow people to do finger level control of the hands for more dexterous things. And so when we say that the hands are... That the feet are essentially like additional hands, what we mean is that the trackers that are attaching to the feet and hands are essentially the same, and those get turned into just desired reference streams into an underlying controller that's coordinating the entire body. So part of the teleoperation system is resting on the years of work that we did in whole body control. So we're able to basically just take these raw streams of information from the VR system and throw it into this whole body controller. And it's able to do all the coordination, maintaining balance, avoiding self collisions. All that stuff is just happening underneath the teleoperation so that the human being doesn't have to worry about all that. A person's not literally trying to balance the robot through moving their own body, or they're not really worried about, well, if I move my foot from here to here too quickly or whatever, it sort of doesn't matter because the underlying controller is gonna sort of do it in a way that doesn't knock the robot over. And then once we've done that and we collect the data, the policy uses the exact same interface as the teleop system. So from the underlying control system, it doesn't really know if it's a teleoperator or if it's a policy. And so that's also been a really flexible way to architect the thing.