00:34:10 · Sergey Levine
[00:34:10] S0: I'll give a little bit of a roundabout answer because it's it's not that there's anything inherently good or bad about world models. It's that the most important thing for building foundation models paradoxically is not the model itself. It's the data. Mhmm. And if you want a foundation model that really works, especially in a new area, the most important thing to figure out is what should the data be, where is it gonna come from, the long term trajectory about how the model will get better and better, meaning where the additional data will come from, and you kinda have to subordinate the other choices to that. So I I talked in the beginning when you asked me about Pi Zero about why it's based on LLM and a diffusion action generation mechanism. I don't think those choices are fundamental. I think the fundamental thing is figuring out the trajectory for data. And I think that trajectory is gonna be a bit complex. Like, it's something where perhaps we start off with a lot of data that is produced by humans, eventually move towards more like intervention style data, eventually more autonomous data. And whether the thing that we train on that data is a video model, a world model, an LLM, a diffusion based action generation mechanism, is sort of a downstream choice. Now, my own hypothesis is that, actually, the ultimate models will do all of those things. Yep. That they will have a world model component because that is how you do reasoning that is below the semantic level of representation. So if you want to, you know, cook a recipe, maybe you think about it semantically. You say, oh, I'm gonna add the vegetables. I'm gonna add the salt. If you are rock climbing, you're not thinking I'm gonna grab the stone that is to the right of that thing over there. You're thinking of a spatial concept. Mhmm. And you probably need a robotic foundation model to be able to imagine things visually, and that is basically a world model. But again, these choices are downstream. The big question is how you're gonna get the data that will enable all this to function. And that's why I think that, you know, some of the world model focus is maybe a little bit distracting in that it kinda shifts more of the focus to what kind of model you have away from the most important question.
00:36:10 · Aditya Agarwal, Sergey Levine
[00:36:10] S1: Every almost successive era of company that comes out of Silicon Valley, they share traits with how the company is structured, what the company does, but they're also kind of different in their own ways. Right? Certainly, was... There were shades of Google that were quite similar to Microsoft, but then also pretty different. Certainly, when I was at Facebook, we were heavily inspired by a lot of the way Google was structured, but then we had our own crazy things like move fast and break things. Then certainly, the Frontier Model Labs look pretty different than the prior generation. So how... What kinds of things perhaps have you been, you know, borrowed from kind of the state of the art in your company? And what have you kind of tried to, you know, in some ways, innovate from first principles in how you run the company? [00:36:59] S0: It's like a little tricky for me to give you an answer that I can put into words because I think that, you know, the truth is that I am extremely happy with how physical intelligence functions, but I don't think I I or even my other cofounders can really take credit for that because it was to some degree an accident. But let me try to, like, reverse it. Like, I I think the truth is, like, we didn't plan on some, like [00:37:19] S1: You have the data. We have to create the
00:37:19 · Aditya Agarwal, Sergey Levine
[00:37:19] S1: You have the data. We have to create the [00:37:21] S0: brilliant strategy. We got, like, super lucky, and now I can sort of look back and reverse engineer maybe some of the things we might have done right. But one thing that I think is really important and uniquely important in robotics that is not, I think, as critical for Google or for OpenAI or or for Meta is that because robotics is so interdisciplinary and because it's so important to have that attention detail and see all parts of the stack, you kinda have to have a culture that both embraces heterogeneous skill sets and at the same time sort of encourages people to gain complete context. And that's a slightly paradoxical thing because specialization might imply that, like, oh, you're the hardware expert. I'm the model expert. You do your thing. I'll do mine, and let's just try to, like, not meet too much in the hallway because we're all doing separate things. And the right thing is, like, a very healthy respect for the expertise that other people have that you don't have, but at the same time, like, a burning desire to really get at least some kind of understanding of everything that's going on. And you you gotta strike that balance somehow. And if you do that, then you get this magical place where everyone has their superpower. Everyone is, like, very respecting of the different skill sets people have, but at the same time, has at least, like, some rough conception of what the whole stack is doing.
00:38:40 · Aditya Agarwal, Sergey Levine
[00:38:40] S1: Yeah. It is always surprising to me how that meta observation ends up being just a key part of, like, great great companies early on, which is that it's not that everyone is a generalist. It's that everyone actually shows up with, like, something that they're world class at, but then they don't constrain themselves to just the thing that they are world class at. There's both a curiosity, but then also just willingness to kind of go out and learn about the whole stack. And it's a weird mixture, and you have to strike a balance because... You know? But I think great early... Great founding teams often are able to find a great equilibrium on that. [00:39:17] S0: Yeah. One one thing that, like, really stands out to me in my memory, which kind of made me think like, oh, man, I'm really happy about how this company functions, is that we have a a person on our hardware team who works on, you know, very low level hardware stuff, like, you know, actuator sensors, that sort of thing. And he was giving a talk, kind of like a lunch tech talk type type of thing. And I could see that a disproportionate number of the people that showed up to the talk are the ones that are furthest away from the hardware, like, you know, the ML infrastructure, like folks that are, you know, doing the really esoteric reinforcement learning algorithms. Like, those are the people that most wanted to learn about this thing because they have this burning desire to understand the entire stack.
00:39:17 · Sergey Levine, Aditya Agarwal
[00:39:17] S0: Yeah. One one thing that, like, really stands out to me in my memory, which kind of made me think like, oh, man, I'm really happy about how this company functions, is that we have a a person on our hardware team who works on, you know, very low level hardware stuff, like, you know, actuator sensors, that sort of thing. And he was giving a talk, kind of like a lunch tech talk type type of thing. And I could see that a disproportionate number of the people that showed up to the talk are the ones that are furthest away from the hardware, like, you know, the ML infrastructure, like folks that are, you know, doing the really esoteric reinforcement learning algorithms. Like, those are the people that most wanted to learn about this thing because they have this burning desire to understand the entire stack. [00:39:52] S1: That's cool. I'm gonna give you some rapid fire questions. What are the tasks that surprised you most when a robot did it? [00:39:58] S0: Using a plastic bag to pick up simulated dog poop. [00:40:04] S1: That's good. Humanoid or wheelbase with two arms for the first form factor for the home? [00:40:11] S0: Wheelbase because then it's not gonna fall down and injure your dog. [00:40:16] S1: Or child. But, yes. One paper from the last year that is not yours that the room should read. There
00:40:16 · Aditya Agarwal, Sergey Levine
[00:40:16] S1: Or child. But, yes. One paper from the last year that is not yours that the room should read. There [00:40:27] S0: is a paper called Neural Thickets from Esolo's group. I unfortunately forget the name of the first author, which has a really interesting insight that as models get bigger, random perturbations to their weights are actually still very good, whereas if they're small, random are not good, and this can be seen as an explanation for why RL works that is like sort of a slightly subversive explanation. Maybe RL works because it just adds random noise rather than doing anything smart. [00:40:53] S1: And this is not specific to robotics, right, if [00:40:55] S0: I remember correctly? Not It's specific to robotics, but I I I think that it will... It's very interesting to me because I care a lot about reinforcement learning because I think it's essential to make robots work. So anything that sort of tells us more about why reinforcement learning works, even if it's a little subversive, is very valuable to me. Yeah. That was a good paper. [00:41:11] S1: One researcher under 30 this room should be paying attention to. It's a tricky one. That's an ageist question. Yeah. One Ageist question. One early career researcher. [00:41:24] S0: See, the the trouble is that since I'm a professor, I kinda have to, like, name one of my students. [00:41:29] S1: Go for it. Go for it. This is why, you know, you get the stage of exam. [00:41:31] S0: I I have a compulsion that [00:41:34] S1: I have work with.