00:20:14 · Aditya Agarwal, Sergey Levine
[00:20:14] S1: Yes. [00:20:15] S0: Yes. And that's very, very hard. And, you know, I I I wish there there was a way to to do that better, but, you know, certainly, the most important thing is not the the the PhD or the credentials. It's the right mindset, the willingness to learn things, to admit that there's some things that you don't know, and most importantly in robotics, the willingness to have the attention to detail. And the attention to detail at at many levels in the stack, it's like easy to say, but it's hard to do because people, especially smart people with an engineering mindset, often have a natural tendency to specialize. And that is a bit antithetical to having that attention to detail to realize that, like, the reason your model is not running is not because you don't have the right learning rate. It's not because you didn't use the right dataset. It's because you ran on a robot where the fingers have a little gap between them when you command them to close fully because somebody didn't attach the connector correctly. So you have to go there, plug in the connector properly, test it yourself, and then go back to training your models.
00:21:10 · Aditya Agarwal, Sergey Levine
[00:21:10] S1: Maybe that's a good segue. Right? Kind of the common concern slash complaint that you hear about robotics is that the training data is too high dimensional to kind of be able to collect the amount of data that you need in sufficient kind of quantity to be able to train these models. And you're already kind of essentially starting from on the back foot because you don't have kind of you know, existing repository of data much like, you know, we had for text. So help us kind of shape this. Right? Like, what what are your latest views on either the scarcity or perhaps actually, like, you know, relative abundance of data for robotics. Right? Like, how should one think about kind of that general thought? [00:21:58] S0: So I'll I'll give maybe, like, a frustratingly big picture answer to this. [00:22:02] S1: I'll just Go from big... Yeah. Please.
00:22:03 · Sergey Levine
[00:22:03] S0: The the universe is really big, and almost none of the universe is inside computers. Almost none of it is on the Internet. The entirety of the universe, effectively up to a rounding error, is the physical world, which means that if you want a machine that understands the universe, that understands physical phenomena, it has to be a robot. There's no other way to get that information. You will not find it on the Internet because it is not there, because the Internet is very small and the universe is very big. Now, if we bring that big picture principle down to something a little bit more concrete, insofar as robots become useful, insofar as anybody wants a robot around them, well, there's probably gonna be lots of robots. Like, you know, cars are useful, and there's lots of cars. Robots become useful, there'll be lots of robots. When there are lots of robots, they are all collecting data. They are trying things, succeeding, failing Sure. Recording what's going on around them. So even if we step back from the... From how big the universe is and just consider, you know, economically relevant work that somebody wants done, if some fraction of that is done by robots, the data coming from robots, dwarfs everything else. Like, it's not even a close call. It's it's just enormous. Now, you can backpedal from that even more. You could say, well, okay, maybe they're not so useful. There's billions of them, but there's only nearly thousands. And you can do that calculation. You still end up with robots at any kind of commercial scale producing enormous amounts of data. So now that's that's very, very good news. That means that the sort of asymptotic picture for this Yeah. Looks extremely good.
00:23:32 · Aditya Agarwal, Sergey Levine
[00:23:32] S1: Correct. [00:23:32] S0: So then we have a bootstrapping problem. Like, how do we get there? [00:23:35] S1: Yep. [00:23:35] S0: Well, a bootstrapping problem is is... That's not like a good problem to have. If you know the asymptotic thing looks good, and you just have to somehow lift yourself up there. Okay. [00:23:42] S1: Do the You're an optimist. You're... This is why you're startup founder. This is great. [00:23:45] S0: Engineering stuff, do the industrial stuff, and you'll get there. And you already know that, like, that place you're gonna get to is a really good place. [00:23:52] S1: So, you know, it's interesting. So we had Dimitri from Waymo. He was the co CEO of Waymo here a couple of weeks ago. You know, a lot of what we were talking about is that, yeah, it's great to have Waymo operate, I think, I don't know, in 15 cities today. But I mean, I think there's probably both data to support your argument, because the number of cars that Waymo is running is still not that large. It's like maybe like 5,000, 6,000 across all of its cities. Right? So it's not... It's actually, like, low thousands, if you are... Mid thousands, I guess. But on the flip side, their bootstrapping took them seventeen years to kinda get to that point. So I... You know, it's kind of, I guess, making an argument for you. Asymptotically, you don't need a high end count to be able to collect the amount of data. And, you know, cars are operating in the most, in some ways, high risk environments alongside humans. So that's good. But how do you kinda cut down the bootstrap time?
00:24:43 · Sergey Levine, Aditya Agarwal
[00:24:43] S0: It's because cars are too specialized. So if if you were a person K. And you're, you know, maybe you're a child, you're growing up, you're learning all your physical skills. If the only thing you were allowed to do in life is drive cars, it'd be very difficult to acquire all the physical skills you need. If you're, you know... Not not even a a baby. If you're like a five year old, a seven year old, and you get put behind the wheel of a car, okay, we all know what's gonna happen. Like, that's not a good that's not a good, you know, child development strategy. The good thing about robots is that they are less specialized because, like, it seems counterintuitive. Like, a robot needs to be able to do more stuff than a car, but it's also a virtue because it can do more stuff than a car. Therefore, it has more things to learn from, some that are higher stakes, some that are lower stakes. And those lower stakes things will bootstrap the higher stakes things just like they do for people. Like, that is not at all a radical idea. So the problem with cars is that they're too specialized. [00:25:39] S1: It's actually a good answer because you're right that, like, actually, certainly operating a car as a human is pretty unnatural to kind of, like, have the interplay of using your hands and your legs to kinda do something. There's not much prior art that you can kind of... In your own personal experience that you can draw from. Whereas for a robot, you can kind of reuse and kinda recombine a lot of the the core underlying concepts. So that makes sense.
00:25:39 · Aditya Agarwal, Sergey Levine
[00:25:39] S1: It's actually a good answer because you're right that, like, actually, certainly operating a car as a human is pretty unnatural to kind of, like, have the interplay of using your hands and your legs to kinda do something. There's not much prior art that you can kind of... In your own personal experience that you can draw from. Whereas for a robot, you can kind of reuse and kinda recombine a lot of the the core underlying concepts. So that makes sense. [00:26:01] S0: Yeah. Also, a car can't, like, learn about the world by playing around because you're gonna get stopped by the cops or crash by something. [00:26:06] S1: Yeah. RL is hard. Well, yeah, walk us through, like, some of the accelerating feedback loops that you have seen when you are able to deploy your models on kind of, like, hardware into kind of, like, settings that then allows them to kind of, like, start collecting data in a particular setting, which then kind of, like, in some ways accelerates the capabilities of those models?