The Robot Report Podcast
00:36:29 · Steve Crowe
[00:36:29] S0: So we'll link to this blog that you guys wrote. It's fascinating. It gets pretty in-depth to all this stuff as well. And in it, you mentioned there's a four stage pipeline you use to, I guess, train these LBMS, Teleop, to collect the data and there's data processing and annotation model training and evaluation. Can you quickly take us through that four stage effort for this specific task? I mean, how intensive is it? How long does it take to collect and fine tune all the data that was necessary to put this together?
00:37:03 · Scott Kuindersma
[00:37:03] S4: Yep, so for the Spot Workshop task, I think probably had on the order of twenty hours of data. So this is the... When we say data here, this is humans tele operating the robot to do the task and variations of the task, recoveries, like things get dropped while they're being teleoperated, they have to recover for it. All that stuff is in the data. We take that, and then when we prepare for training, we also do various augmentations to it. So one of those standard tricks is to mess with the visual observations in the data. So you don't just take the perfect camera streams from the robot. You're doing all this different types of cropping and other types of visual ablations, and that's to increase the visual robustness of the resulting policy. We also annotate the demonstrations with language. So we're using just natural language descriptions of what is happening in those demonstrations, and then that becomes also an input to the policy so that later, when you have a policy that's seen many different example tasks and they've all been labeled with natural language, then you can steer the policy using natural language in real time. So, take that data, it's annotated now, we run training. This is typically done a cloud computing cluster, lots of GPUs, and then we get a policy back. And then the question is, okay, how do you know if this policy is good or not? And that's actually one of the really hard things about doing this work is doing careful evaluation. Evaluation. So being able to know when some change you've made to the data mixture or your augmentation strategies or to the neural network architecture, any of these design decisions, how to know when you've done something good or whether it's made a difference or not, it can actually be pretty subtle. And so being able to run many of these evaluations on the hardware is quite important. It's also really useful to leverage simulation as a way to get many, many evaluation rollouts. So our collaborators TRI had a paper that they released a month ago, I think did a great job of this and just leveraging lots and lots of simulation rollouts so they can get statistically significant statements about the difference in performance, for example, a policy that was trained using a large collection of data versus a more narrow task focused policy.
00:39:38 · Eugene Demetri
[00:39:38] S3: And you had mentioned obviously that in some ways LVMs are no different than LLMs if you haven't taken enough corpus of data. But the chat GPs of the world have been trained on the entire Internet. Right? They have a lot of data available to them, whereas you're having to create, whether it's your simulation or a real world tele op, you're having to to create these datasets yourselves. How hard is that when we're talking about the exception handler? Right? You want to build a robust model, as you said, when we're messing with the totes or a part gets dropped on the floor or you're trying to get the robot to open and close a lid. Like, those are... You know, it seems easy to us as humans. Right? But to get... How do you know when you're doing the evaluation that you're at that desired level of of robustness? It because sounds like a moving target to me.
00:40:27 · Scott Kuindersma, Steve Crowe
[00:40:27] S4: Yes, it is. So I think it's a really great observation. So one of the challenges with evaluation, not only the fact that just doing things on robots is time consuming and challenging, but just knowing to what extent you're actually covering the relevant distribution of things that might happen when you actually want to run this policy is a really hard thing to do. And I think the answer is we basically will never be able to anticipate all of the things that the world will present to the robot ahead of time. And so I think longer term where I see this work going is having learning feedback mechanisms that are available to the robot in deployment so that when the arbitrary complexity of the world creates scenarios that none of us could have possibly anticipated ahead of time, that there's some way to at least observe what's happening and then maybe not instantly, but take those experiences and then be able to iterate and create a policy that's able to deal with those. I think that's a fundamentally iterative process that we're going have to engage in because we just have to be okay with the fact that we're not going to be able to anticipate everything and pre build everything such that when we say, deploy a robot into a factory, it never messes up. It's just not gonna work. So we just have to identify those robust processes that allow us to create... Sorry, to climb the robustness curve for the behaviors. Who's... Who has [00:41:54] S0: the hockey stick? That's not you behind the camera with the hockey stick.
00:41:54 · Steve Crowe, Scott Kuindersma
[00:41:54] S0: the hockey stick? That's not you behind the camera with the hockey stick. [00:41:56] S4: It wasn't... Definitely wasn't me this time. Yeah. So there's a... I think probably it was different people in different parts of that video, but a couple members of our team, I think Pat and Russell were among the hockey stick wielders this time. [00:42:10] S0: If you ever... I never played hockey, but if you need a volunteer, like an outsider, I'm your guy. I'll I'll be happy to help. Are there... I mean, there's some... Neither of Gene or I are robotics engineers, so so much of this stuff fascinates us. And What surprises you? When you guys are watching this demo, I don't know how many times you did it. It doesn't matter to me how many times you guys did it to get this video that you put out. You guys watching this in real time? And are you surprised by certain things that happen?
00:42:39 · Scott Kuindersma
[00:42:39] S4: Yeah. I mean, I think it's... We've been really impressed with the... I'll just say I've been very impressed with the progress that the team has made, you know, over the past year largely pivoting the approach that we're taking to programming the robot. So, many, like I've had many years of experience in what I would call the kind of bread and butter approaches to programming robots, where we build modular software stacks that are combining different elements. Yes, some machine learning based elements that maybe were mostly in charge of doing computer vision tasks and model based elements like model predictive controllers and planners and things like that. And what we showed last year with e Atlas doing Hyundai part sequencing tasks was a demonstration that you could take a system like that and you could make it do really impressive and approaching real world work. The problem with those types of approaches is that in order to take that robot and put it in front of an entirely new task that maybe we hadn't anticipated yet, the process for onboarding that new task would almost certainly involve a tremendous level of robotics expertise. And this could be from PhD roboticist to, or even just the field application engineer type folks to get the robot to do that thing. That's okay, but there's a scalability problem there, right? So if we're talking about what's the opportunity of humanoids, I think it's that these are robots that are physically suited to doing many, many different labor tasks, and that their ability to be re tasked is sufficiently low overhead that this becomes like a very flexible thing that you can just scale out to tens of thousands of robots. Now, in order for that to be true, I think it can't be the case that you need a team of robotics experts involved every time you wanted to get the robot to do something new. And so what's exciting about these approaches is that it's a repeatable process of just providing demonstration style inputs to the robot. And if you can demonstrate it, then you can train a neural network to make it autonomous. And that's really, really powerful. And so one of the things that's impressed me is seeing that comparison back to back, right?