The Robot Report Podcast
00:40:27 · Scott Kuindersma, Steve Crowe
[00:40:27] S4: Yes, it is. So I think it's a really great observation. So one of the challenges with evaluation, not only the fact that just doing things on robots is time consuming and challenging, but just knowing to what extent you're actually covering the relevant distribution of things that might happen when you actually want to run this policy is a really hard thing to do. And I think the answer is we basically will never be able to anticipate all of the things that the world will present to the robot ahead of time. And so I think longer term where I see this work going is having learning feedback mechanisms that are available to the robot in deployment so that when the arbitrary complexity of the world creates scenarios that none of us could have possibly anticipated ahead of time, that there's some way to at least observe what's happening and then maybe not instantly, but take those experiences and then be able to iterate and create a policy that's able to deal with those. I think that's a fundamentally iterative process that we're going have to engage in because we just have to be okay with the fact that we're not going to be able to anticipate everything and pre build everything such that when we say, deploy a robot into a factory, it never messes up. It's just not gonna work. So we just have to identify those robust processes that allow us to create... Sorry, to climb the robustness curve for the behaviors. Who's... Who has [00:41:54] S0: the hockey stick? That's not you behind the camera with the hockey stick.
00:41:54 · Steve Crowe, Scott Kuindersma
[00:41:54] S0: the hockey stick? That's not you behind the camera with the hockey stick. [00:41:56] S4: It wasn't... Definitely wasn't me this time. Yeah. So there's a... I think probably it was different people in different parts of that video, but a couple members of our team, I think Pat and Russell were among the hockey stick wielders this time. [00:42:10] S0: If you ever... I never played hockey, but if you need a volunteer, like an outsider, I'm your guy. I'll I'll be happy to help. Are there... I mean, there's some... Neither of Gene or I are robotics engineers, so so much of this stuff fascinates us. And What surprises you? When you guys are watching this demo, I don't know how many times you did it. It doesn't matter to me how many times you guys did it to get this video that you put out. You guys watching this in real time? And are you surprised by certain things that happen?
00:42:39 · Scott Kuindersma
[00:42:39] S4: Yeah. I mean, I think it's... We've been really impressed with the... I'll just say I've been very impressed with the progress that the team has made, you know, over the past year largely pivoting the approach that we're taking to programming the robot. So, many, like I've had many years of experience in what I would call the kind of bread and butter approaches to programming robots, where we build modular software stacks that are combining different elements. Yes, some machine learning based elements that maybe were mostly in charge of doing computer vision tasks and model based elements like model predictive controllers and planners and things like that. And what we showed last year with e Atlas doing Hyundai part sequencing tasks was a demonstration that you could take a system like that and you could make it do really impressive and approaching real world work. The problem with those types of approaches is that in order to take that robot and put it in front of an entirely new task that maybe we hadn't anticipated yet, the process for onboarding that new task would almost certainly involve a tremendous level of robotics expertise. And this could be from PhD roboticist to, or even just the field application engineer type folks to get the robot to do that thing. That's okay, but there's a scalability problem there, right? So if we're talking about what's the opportunity of humanoids, I think it's that these are robots that are physically suited to doing many, many different labor tasks, and that their ability to be re tasked is sufficiently low overhead that this becomes like a very flexible thing that you can just scale out to tens of thousands of robots. Now, in order for that to be true, I think it can't be the case that you need a team of robotics experts involved every time you wanted to get the robot to do something new. And so what's exciting about these approaches is that it's a repeatable process of just providing demonstration style inputs to the robot. And if you can demonstrate it, then you can train a neural network to make it autonomous. And that's really, really powerful. And so one of the things that's impressed me is seeing that comparison back to back, right?
00:45:01 · Scott Kuindersma, Steve Crowe, Eugene Demetri
Just having a really deep understanding of both approaches now and just seeing how different it feels to program the robot this way and the opportunity for scalability. So that's the one thing that I'm just really excited about right now. [00:45:17] S0: Yeah, let's dive into that more because that's something that we wrote down that we wanted to ask you. I wrote down a couple of the quotes from the blog. We don't have to read them verbatim, but you guys basically wrote with TRI in the blog that the training process is the same. This is an exact quote. So the training process is the same, whether it's stacking rigid blocks, folding a t shirt. If you can demonstrate it, the robot can learn it. I mean, again, an outsider, those are pretty lofty statements, but if that's true, like it just must open up a whole new world for what you guys and what other robotics developers think is possible and how quickly you're going to [00:45:51] S3: be able to get there.
00:45:51 · Eugene Demetri, Scott Kuindersma, Steve Crowe
[00:45:51] S3: be able to get there. [00:45:52] S4: Yeah, I mean, really is. It's also like a very powerful thing, right? Because it might not even be obvious in some cases. Like if you ask me, can the robot do X? And it was some manipulation tasks we just hadn't tried before. Now the question, the sort of corollary question, you know, in a behavior cloning type setting like this large behavior model setting is, can I teleoperate that? If I can teleoperate what you're asking, then the answer is almost certainly yes, we can get the robot to do that. We basically know the process for doing it. That's really powerful. Before we had this type of machine learning based approach to programming the robot, we have to think carefully about, okay, well, you know, what are the, you know, the physical properties of the objects that may come into play and do I... Will those be captured by the models that are embedded in my planning and control stacks and all these various things? Now it's like, okay, well, we can grab a tablecloth and flip it out and cover a table. And no one had to write a model for that. No one had to write a tablecloth detector. No one had to do anything other than strap on a VR system and pick up a tablecloth and flip it over, you know, a few 100 times. [00:47:08] S0: What? This might be a stupid question. It might not be. Like, why does this work so well?
00:47:13 · Scott Kuindersma
[00:47:13] S4: It's not a stupid question. I think it's a really deep question, and I'm not sure we have the full answer to it, to be honest. So, I think there's also like a different answer depending on the scale of what you're observing. So, it's people, you know, in the original diffusion policy paper, for example, that is not what what anyone would call a large behavior model. That is taking some number of demonstrations and then training a diffusion style neural network model in order to reproduce those demonstrations. And essentially, that's a very high capacity model. And it's able to... I wouldn't say it it'd be oversimplifying to say it's memorizing those demonstrations, but it knows them very well and it's able to use feedback from the robot sensors in order to basically do something that feels like a nearest neighbor search and like playback the relevant pieces of the demonstration data in order to reproduce the behavior that the demonstrators performed. And so I think that's just a more of a benefit of the diffusion style architecture that allows you to represent very complex distributions. I'm trying to avoid going into too much technical detail here, but there's this property called multimodality where essentially, like a simple example would be, imagine there's two ways to go from point A to point B, one that goes around the right hand side of an obstacle and one that goes around the left hand side of an obstacle. And demonstrators are giving you both answers with some, you know, different probability. If you didn't have a model that could capture multimodality, it might try to memorize something that looks like the average correct answer, which would be to go straight through the obstacle. And that's just not the right answer. And so you need an expressive enough model to be able to represent both answers, even though they're kind of different, they're both correct. And a lot of the manipulation tasks that we were wanting robots to do now have that kind of multimodality. And so having identified an architecture that's able to handle that appropriately is part of the reason why this works well.