Going Direct Conversations
00:16:07 · Chris Paxton, Jordan Wolfe
[00:16:07] S0: Is that okay? Yeah. The thing is that you can't just get a good feel for how a robot works from a video because you can always get a good video. Honestly, the other thing is that I think people have seen lots of really good robot videos for a long time because they've seen Star Wars or whatever. You've seen robots that can do things that they can't even do now. I think you have to be able to show robots in the real world and see it for yourself to be able to see what it can do. I think a lot of videos can be misleading. I wanted to drill into the different types of training, methods for training. [00:16:44] S2: I wanted to get your perspective on it. Because that's also another aspect that's kind of a little bit unclear. What is the most effective? Is it a mix of all of these? Let's start with just teleoperation. Michael, what's your perspective on what does teleoperation truly allow you to do to solve some of the training gaps that currently exist?
00:16:44 · Jordan Wolfe, Michael Cho
[00:16:44] S2: I wanted to get your perspective on it. Because that's also another aspect that's kind of a little bit unclear. What is the most effective? Is it a mix of all of these? Let's start with just teleoperation. Michael, what's your perspective on what does teleoperation truly allow you to do to solve some of the training gaps that currently exist? [00:17:08] S1: Yeah, I would say teleoperation is the gold standard amongst the three types of data I mentioned just now. But obviously, it's extremely costly to collect. I say it's a gold standard because you are doing in the real physical world with a real physical robot. So you don't suffer from, let's say, embodiment gap. And you collect exactly, typically, I mean, this is humans controlling the robot, right? So you also, in a way, collect human intention, like what would a human do, like in this instance, right? So there's some social compliance stuff that also naturally get collected. So I think it's great for all those reasons. But of course, these robots break all the time, you know, and you can't respawn them, you know, like in a simulation. So that's a huge thing. Yeah. So I, I think my thing is that if you can gamify it, then, then hopefully you get more people to constantly do it, but you still cannot get away from the fact that these things will break all the time and makes it very, very hard to scale. [00:18:08] S2: Yeah. Yeah. What's your thought, Chris, on the teleoperation side?
00:18:08 · Jordan Wolfe, Chris Paxton
[00:18:08] S2: Yeah. Yeah. What's your thought, Chris, on the teleoperation side? [00:18:13] S0: I mean, I think that, yeah, like you're saying, that's the gold standard, getting real robot data. And I think the gold standard is actually probably, the best data is probably the data the robot collects itself. It's probably like reinforcement learning kind of data. But that's pretty rare still. But the robots are... Teleoperation data is great as long as you can actually get it, right? But while scaling, it's hard. And so that was actually the next question was kind of reinforcement learning. [00:18:45] S2: So Chris, can you kind of explain that for folks that might not be so understanding of what that actually means in the term of robotics space?
00:18:53 · Chris Paxton
[00:18:53] S0: Yeah, sure. Yeah, so I apparently even wrote a book chapter on this for what it's worth. So reinforcement learning is... What's the best way to describe this? I really should have a better answer for you off the top of my head. But it's basically the idea that the robot needs to be able to teach itself from its own experiences. So the way that you or I learn to perform a task is not by watching. I don't watch... If I wanted to learn how to... hit a tennis ball. I think that's like to serve a tennis ball or something like this. Like I wouldn't do that by just watching a bunch of YouTube videos. I would do that by tossing a tennis ball and trying to hit it, right? And then do practicing for days or weeks or months or years depending on how good I wanted to get, right? And that's what reinforcement learning is in general. It is saying that the robot is going to just like sit there and take in information about the world and it's going to And it's going to make a decision and try something, and if it fails, it's going to take that and think about it and do better next time, hopefully. Or if it succeeds, it's going to be like, okay, that was good. Let's try again a little bit. Let's try to do more like that. And the reason that I'm saying it's the best is because it's the same reason that it's best for you or me. If you want to learn to do something, You'll have your own embodiment and your own version, your own thoughts about how to do that, and you're never going to really get that from watching someone. But you'll do pretty good from watching someone. You'll learn a lot from watching someone, but you still at some point always need to do it yourself.
00:20:27 · Jordan Wolfe, Chris Paxton
[00:20:27] S2: Right? And so that's a good segue into kind of another aspect, which is kind of one I really wanted to better understand, which is learning from kind of human video. So the concept of can you collect, you know, you collect data, you label it, you provide contacts on and on and on. You have your own data schema. So Mike, what's kind of your perspective on, because of just what Chris said around like you actually have to do the thing to learn the thing. At some point, at some point. [00:20:55] S0: You can get pretty far without it, right? Let's dig into that because this is like a very abstract thing for folks that [00:21:01] S2: aren't in the robotic space. So how useful is kind of human video for training robots, Mike? I mean, do you think, is that a critical piece right now? And will it be a critical piece going forward? Or is it going to be the robot physically doing like physically watching someone and not video.
00:21:19 · Michael Cho
[00:21:19] S1: Yeah, I think it's a critical piece, but you, it can't be the only piece. I think, like I said just now, there are three types of data. You don't need all three types is my view. You need to co-train on all three types. Video being one of them. Video is great because it's so easy to collect video. We already have a lot of it on YouTube, let's say. You can also quite easily just collect ecosystem video, that sort of stuff. The issue, well, I mean, the great thing about video is because it's so easy to collect, you get a lot of diversity of environments and tasks. And you want the diversity, right, to so-called hopefully allow the model to see more things and then generalize better ultimately. But I also think that there's a pretty low threshold to what, like the performance of a model that you only train from video. Because these videos doesn't come with action labels. Unlike, let's say, teleoperation, it's hard to think about it. So you always have to use some techniques to infer actions. But those actions are always, you know, like approximation. Not only that, because every embodiment is different. Imagine you have an ecosystem video of some humans doing something. The thing is, all of us have hands, but all of our hands look similar, but they're actually also distinctly different. And I tend to think that in robotics, all the minor detail matters a lot. Millimeters, you're talking about 50 milliseconds latency. could be the difference between you falling down and standing up, you know? And so when you actually apply in the real physical, because the real physical world, like physics is just brutal, like gravity is brutal. Like if you just miss by that little bit, it's the difference between you're able to do it and not doing it. And so I think video is great, again, to give you that whole very broad concept, okay, what is playing tennis, what is playing basketball, things like that, right? But if you actually want to play basketball, I think you can't just watch basketball. Yeah, something, yeah.