The Robot Report Podcast
00:32:49 · Steve Crowe, Scott Kuindersma
[00:32:49] S0: one of [00:32:49] S4: the things we're excited about with humanoids, of course, is the full breadth of manipulation skills we know they can perform. And so what we wanted to do was show that we could first build an interface to the robot that would allow us to do a really expressive set of data collection. So I could teleoperate the robot to do almost anything that it could physically do, right? Stepping, crouching, picking up things with one or both hands, hand level manipulation, things like this. And then once we were able to do that and collect that data, show that the same sort of techniques that have been gaining traction in the larger behavior models literature also translate to this task space where it's now not just moving hands around and moving things on a tabletop, but actually coordinating an entire humanoid robot body to do a manipulation task. So once we decided, you know, this is the thing we wanted to do, it was really just a matter of walking around our building and trying to figure out what are some tasks that kind of tick these boxes. And we literally walked into the Spot Repair Shop and just saw their shelves of all these Spot Parts. And we saw what the people in that shop were doing. And we thought, okay, we could just pull all these elements together and create a task in our lab based on it. And that's what we did.
00:34:05 · Eugene Demetri
[00:34:05] S3: And that's the cool thing about Boston Dynamics is doing with humanoid robots, right? That we're taking full advantage of real world surroundings and applications and not just having a humanoid do what you have some other robot can already do, but to take advantage of the humanoid form factor. But you mentioned a large behavior models, and we've heard a lot about large language models, but maybe not everyone's familiar with LBMs. Can you explain what they are and how are they helping Atlas learn new skills?
00:34:35 · Scott Kuindersma
[00:34:35] S4: Yeah. So the naming of it, which I think was coined by our collaborator CRI, is meant to draw an analogy to large language models. And so basically the story that we all know from large language models and vision language models and all these multimodal variants is that by pulling together large amounts of diverse data and training a single model, basically using supervised learning, sometimes other types of learning to do additional fine tuning after. But by compiling this all into one model, you can actually use that same model to do many different downstream tasks. We can build large language models that are really good at just being chatbots, but also good at solving math problems or coding problems and things like that. Observation is that, well, the same sort of generalization So, should be available to physical there's nothing inherently special, we think, about language or about images or anything like that, that if we also had a similar sized corpus of embodied data that involved robot state observations and actions that the robot took to change the state of itself and of the world, that if you fed these models that same sort of, that kind of data, that they should be able to generalize in similar ways over behavior. And so that's the sort of hypothesis. And so what we were doing here is using primarily teleoperation data on a couple of different robot platforms over many, many tasks, and then compiling that all together to train a model that is able to take information from the robot sensors, information from the robot cameras, and language commands, and turn that into closed loop manipulation policies like we showed in the video.
00:36:29 · Steve Crowe
[00:36:29] S0: So we'll link to this blog that you guys wrote. It's fascinating. It gets pretty in-depth to all this stuff as well. And in it, you mentioned there's a four stage pipeline you use to, I guess, train these LBMS, Teleop, to collect the data and there's data processing and annotation model training and evaluation. Can you quickly take us through that four stage effort for this specific task? I mean, how intensive is it? How long does it take to collect and fine tune all the data that was necessary to put this together?
00:37:03 · Scott Kuindersma
[00:37:03] S4: Yep, so for the Spot Workshop task, I think probably had on the order of twenty hours of data. So this is the... When we say data here, this is humans tele operating the robot to do the task and variations of the task, recoveries, like things get dropped while they're being teleoperated, they have to recover for it. All that stuff is in the data. We take that, and then when we prepare for training, we also do various augmentations to it. So one of those standard tricks is to mess with the visual observations in the data. So you don't just take the perfect camera streams from the robot. You're doing all this different types of cropping and other types of visual ablations, and that's to increase the visual robustness of the resulting policy. We also annotate the demonstrations with language. So we're using just natural language descriptions of what is happening in those demonstrations, and then that becomes also an input to the policy so that later, when you have a policy that's seen many different example tasks and they've all been labeled with natural language, then you can steer the policy using natural language in real time. So, take that data, it's annotated now, we run training. This is typically done a cloud computing cluster, lots of GPUs, and then we get a policy back. And then the question is, okay, how do you know if this policy is good or not? And that's actually one of the really hard things about doing this work is doing careful evaluation. Evaluation. So being able to know when some change you've made to the data mixture or your augmentation strategies or to the neural network architecture, any of these design decisions, how to know when you've done something good or whether it's made a difference or not, it can actually be pretty subtle. And so being able to run many of these evaluations on the hardware is quite important. It's also really useful to leverage simulation as a way to get many, many evaluation rollouts. So our collaborators TRI had a paper that they released a month ago, I think did a great job of this and just leveraging lots and lots of simulation rollouts so they can get statistically significant statements about the difference in performance, for example, a policy that was trained using a large collection of data versus a more narrow task focused policy.
00:39:38 · Eugene Demetri
[00:39:38] S3: And you had mentioned obviously that in some ways LVMs are no different than LLMs if you haven't taken enough corpus of data. But the chat GPs of the world have been trained on the entire Internet. Right? They have a lot of data available to them, whereas you're having to create, whether it's your simulation or a real world tele op, you're having to to create these datasets yourselves. How hard is that when we're talking about the exception handler? Right? You want to build a robust model, as you said, when we're messing with the totes or a part gets dropped on the floor or you're trying to get the robot to open and close a lid. Like, those are... You know, it seems easy to us as humans. Right? But to get... How do you know when you're doing the evaluation that you're at that desired level of of robustness? It because sounds like a moving target to me.