00:26:31 · Sergey Levine
[00:26:31] S0: So both deployments that we've done ourselves and deployments that other companies using our models have done, you know, oftentimes, the the theme there is actually pretty clear cut that you get to a level of autonomy that's sufficient that you can deploy something under human oversight and human interventions, and the interventions that fix the mistakes provide exactly the data that you need to get the model to very high performance. That's like... That... That's not a a novel strategy at this point. This is something that a lot of folks working on robotics robotic learning understand very well. I think, however, in the in the story arc of this stuff, that's actually just the beginning. I think eventually, what we'll get to is a situation where not only can you deploy and use human interventions to fix up the mistakes and provide exactly the right data, but the robot's own experience and also provides additional data to improve over time. And this is something that is not yet widely used in deployment, but it is working in laboratory experiments that we're doing. And it actually make... It's a very natural transition because when you have a robot deployed under partial autonomy with interventions, what the interventions do is they correct mistakes. So the robot is maybe sorting something at a logistics facility. It drops an item. The intervention is to pick up the item and put it back. So now the robot knows how to fix the mistake. But it doesn't know how to avoid that mistake in the first place, And that's where the reinforcement learning will come in and address those issues. So initially, the interventions give you a way to fix mistakes. That makes you more autonomous. So now you can be clumsy but autonomous, and then the RL comes in and fixes the clumsiness. And that's a slower process and is enabled by greater autonomy.
00:28:04 · Aditya Agarwal
[00:28:04] S1: So maybe to zoom out for a second. Right? We obviously have a number of companies that are really doing full stack kind of robotics development. Let's let's talk maybe, as an example, Optimus, right, at Tesla. And then we also have folks who are working much more on, like, broad based horizontal capabilities with then kind of essentially integrators or kind of like folks who are deploying, figuring out how to combine them. How do you think this plays out over the next five years? Right? Like, obviously, pros and cons of each approach, and you have a very strong, I guess, know, horse in the race slash point of view. But walk us through kind of, like, how you think about the vertical versus horizontal landscape for robotics today.
00:28:48 · Sergey Levine
[00:28:48] S0: Yeah. So I think that the... To me, the most important thing here is that the robotic problem is really difficult. And one of the really important lessons of, you know, the... All of the recent advances we've seen in AI is that when you're faced with a really difficult intelligence problem, somewhat counterintuitively, making the problem as big as possible actually makes it easier. So if your goal is to solve machine translation, you know, naively, we might think that the best way to do it is to build a machine translation model. Like, get lots of paired examples of, like, English sentences and Chinese sentences, train a model that maps one to the other. That's what the the early sequence to sequence models did, and that's your method. Turns out that's the wrong answer. The right answer is to solve all NLP problems all at once, and then you get a better machine translation system. And this is like... At this point, this is not a controversial statement. Like, every single NLP researcher will tell you this is exactly what you do. So what does that tell us about robotics? That if you want your humanoid to be really good, do you get lots of examples of a humanoid, or do you solve all of the robotics problems? I think the answer there is very clear, and I think that there's a good reason why AI systems have this blessing of generality because when you have generality, you get more data. You see more of the unusual situations, more of the exceptions, more of the edge cases. And in robotics, that's even more important than it is in natural language. So I think this lesson actually applies more to robotics than it does to NLP.
00:30:13 · Aditya Agarwal, Sergey Levine
[00:30:13] S1: So it's like the bitter lesson applied to, like, robotics. And I think it's actually an interesting answer, which I hadn't thought of, which is you are, I guess... You know, it's easy to kind of conflate humanoids with all robotics, but they're not actually the same thing. It's kind of like the point you're also making. Right? [00:30:28] S0: There's nothing wrong with humanoids. I think humanoids are are amazing, and I I would love to see humanoid robots all around us. But I also think that tech generally doesn't work that way. Take computers for instance. It used to be that computers were like the humanoid companies. You would go to IBM in the fifties and sixties and you would buy a computer and it was a vertically integrated business. IBM comes in, they install the computer, it's a room size thing. The IBM technicians come in, the IBM, you know, software engineers come in, everything is is is IBM. And now, how does it work? Now we have operating systems, we have hardware, we have software. A software company usually doesn't worry about like building their own, you know, CPUs because everything is nicely factorized. And I think we'll see that in robotics. We'll see this kind of explosion of creativity with all sorts of different robots, all sorts of different robotic applications. And the key to that is the right kind of physical intelligence layer that enables that factorization to happen.
00:31:25 · Aditya Agarwal, Sergey Levine
[00:31:25] S1: Yeah. It's interesting. I think one of the... You know, certainly, I I don't know how many of you have done this or remember doing this, but certainly, like, when I was younger, I would actually kind of hand assemble all of my CPUs. You basically go figure out what different components to buy. You assemble it. It was kind of fun. But I think one of the things that has happened even to myself, I haven't hand built a computer, I guess, for a long time. But I think our relationship to hardware over the last fifteen years, ever since this magical device came out, is that our only mental model is that great hardware almost exists in a vertically integrated way. And so it has made us, I think, almost forget that there are different ways of doing this. 85% of the world still uses the Android operating system with its insane variety of different components and parts and configurations. And so I think your answer there is great because both can happen. You will see vertical, you will see horizontal, you'll see innovation at almost every different layer in a bunch of different ways. Do you think that there is still time to create a new robotics foundation model company? [00:32:40] S0: That's an interesting question. It's hard to answer that question because it really depends on kind of an an answer to something that nobody really knows, which is how will the the Ultimate Robot Foundation model work? [00:32:52] S1: Yeah.
00:32:52 · Aditya Agarwal, Sergey Levine
[00:32:52] S1: Yeah. [00:32:52] S0: So I certainly have my own theory about that. I have my road map and and the the kind of research that physical intelligence is pursuing. But if I'm being very upfront about this, like, that is an open science problem. And when you have an open problem like that, it could be the answer somebody nobody expects. Like, you know, certainly, if you if you talk to Yann LeCun, he'll have a very different perspective on this than I do. And maybe he's right. Like, you know, the great thing about science is that it's not a, you know, a purely abstract philosophical debate Yeah. Where we just tell stories. Like, there is an actual truth out there, and somebody's right and somebody's wrong. [00:33:24] S1: Yeah. I'll see you out in the field. Right? So maybe, you know, speaking of Yan, probably the the... I was gonna say easiest, but certainly the... One of the hottest areas of, I would say, company formation and, hence, company, I would say, investments have been world models. World models as applied to robotics, world action models. What's your... Like, how does... And, you know, again, world models is a small term with, like, at least, like, you know, 20 different definitions of how can people end up, like, you know, applying it to the thing that they're doing. But how do you think about almost that kind of, like, strand, if you will, or that that kind of, like, you know, node of kind of the research tree, and how it intersects with how you think about the world at, like, you know, physical intelligence?