Going Direct Conversations
00:27:46 · Chris Paxton, Jordan Wolfe
[00:27:46] S0: The robot guy, I walked by this booth, obviously, at this conference with lots of top researchers. And a lot of them were saying, this is one of the coolest robot manipulation things I've seen. And he's just getting blown away by the old man next door. Right. So like, that's just like, I want to just be clear. It was actually very good. The robot part was very good. It was just, yeah. [00:28:08] S2: Yeah. Yeah. We take it for granted how like, how unbelievable our hands are, right? Like just the dexterity and like what you can manipulate and how you can manipulate. It's pretty, it's pretty remarkable. All right, let's go to kind of one last one, which is sim to reel or reel to sim. So Chris, maybe kind of break that down for us, what that means and kind of your perspective on the level of use, like usability in training and training.
00:28:40 · Chris Paxton
[00:28:40] S0: Yeah, so this is actually something that I worked on for a very long time, like six plus years, I think. But so robotics. So yeah, SynthReal means that we train the robot how to do something in a simulation. And then it also has to work in the real world, right? So I take the robot, and maybe I create a digital twin of the task that I want it to perform. If it's an industrial task or an industrial kidding task, maybe I would just spawn all the objects that I need to place on a table and have a copy of the robot and have it sit and have and then maybe I create a whole bunch like a million of these robots in simulation I have them all practice picking and placing the object and into into the target and whatever that right and I use that data like I simulate cameras or whatever sensors that I may have access to and then I train a policy, train a neural network on that simulated data. And the sim-to-real part, that's the sim part. The sim-to-real part is then I have to set that up in such a way that I can then take that policy and use it in the real world. So this is the way that a lot of things work right now, particularly when it comes to all the dancing robot videos that you see are sim-to-real. all of them are trained this way. But there's a lot of other stuff like central manipulation is another, is an active research area, so.
00:30:04 · Jordan Wolfe, Michael Cho, Chris Paxton
[00:30:04] S2: Let's go over to RoboPapers. Thank you for that. So actually, before we do that, I wanna, I guess if we were to step back, when you think about all these different training methods, and obviously several folks working on a variety of both open source and their own closed source robot foundational models and multimodal models and world foundational models, whatever you wanna call it, I want to get each of your perspectives on, on, on, on how do we, how do we ultimately solve this training gap? Like what is the thing that we can do today to move the field forward in the most effective way, I guess, uh, around the training gap. If there's one thing that you pick, you pick, we could do. Gosh, [00:30:46] S1: uh, I think you need, yeah, it's very hard to pick one thing because like, it's a, it's very unclear which one is actually going to scale. Like all, all these, uh, [00:30:55] S0: I mean, if it's like wave your magic wand, infinite money. Sorry. Go ahead. I have an answer.
00:30:55 · Chris Paxton, Michael Cho, Jordan Wolfe
[00:30:55] S0: I mean, if it's like wave your magic wand, infinite money. Sorry. Go ahead. I have an answer. [00:31:01] S1: I have talked to so many different researchers now. Different researchers have different, like, kind of, let's say, I don't know, like, inclination of biases. Some people think, you know, TeddyWarp is really the way. Some people are really betting on, like, role model. I mean, I should say something about simulation, right? So to me, Simulation data. Simulation have kind of two types now. You have the traditional simulation engine, and then you have the wall model, which is the new stuff. Because the wall model is, in a way, a simulator as well. It just behaves very, very differently and trains very differently. I think simulation has got to... Okay, if I were to bet, I feel that simulation alone can get us 80%, 90% there. It's my guess. Okay. But because you need 99.9999% so it's not enough therefore. But it can probably move us all near to 90 something percent. The issue with simulation, I don't know whether Chris you agree with me, at least with the classical simulation engine, is that I believe contrary to popular belief, we're not bottlenecked by compute actually, I think. I think that in a way, the its capabilities is also its curse, right? Because with a very good simulation engine, you can literally simulate anything you want. And that's a big problem because the search space or the exploration space is nearly infinite. [00:32:26] S2: It's too wide, yeah.
00:32:28 · Michael Cho
[00:32:28] S1: Yeah, you can imagine, you can use a simulation engine to, let's say, simulate a robot frying egg, let's say. You can imagine in your head maybe an infinite way to set up that scene, that task. Different lighting, different robots, different eggs and whatnot. And so it will take a crack researchers like Chris to drastically shrink that search space into something that is tractable with the compute budget that he has. But as researchers do that, well, so, you know, they are missing a lot of actually potential edge cases that may happen in the real world that they could have simulated, but they just don't know. I mean, they would have messed up. And so I think that's a big part of it. So if I look at it that way, actually the bottleneck to a simulation, a big bottleneck is having more people like Chris, honestly, because that process is very curated. Like it's very handcrafted. Like you need, like Chris will have to come up with very like a man, like handcrafted, like the rewards functions for the reinforcement learning, things like that. It's all very handcrafted. So, which means actually the human in the loop, and unique researchers to do it. Like, you know, it's a very, very specialized skill set. That's a big bottleneck. World model, on the other hand, all this video generation, I think it's also very, very interesting. I think they potentially can move the needle a lot. But you can imagine to build a world model, you also need a lot of probably like video data and tele-op data too. So you still need a lot of data, a huge amount of data to build a decent world model, right? Yeah, so it's hard to pick one. I feel that tele-op data, video data, simulation, all are needed. Like we, we need all. And there's a lot of papers that we've come across, you know, that we've interviewed that do co-training of various sorts, right? They'll, they'll, they'll, they'll, they'll scale it using some video or like, you know, do the real to seem the real thing.
00:34:24 · Jordan Wolfe, Chris Paxton
[00:34:24] S2: Yeah. So, yep. Sure. What's your thought, Chris? If there was kind of what, if you have one thing that kind of stands out in your mind, if someone who's actually in there doing, you know, working on it every day, [00:34:34] S0: I mean, I think that if you were going to wave a magic wand and had infinite money, then I think you'd just have lots and lots of robots doing. I think that they worked. But practically speaking, I think Michael's correct. It seems like you need a little bit of everything, because they're all good at slightly different things. And they all have different pluses and minuses. Sure. Yeah. So the real solution is going to be a mix. So, Chris, I actually saw a recent tweet that you put out that caught my eye. [00:35:07] S2: And I don't have it in front of me, so I might butcher it a little bit. But basically, the premise was the reality of it is if we want low-cost generalized robots, we need a bunch of robots in the field learning and actually doing stuff. That's,