Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: And do you think that's an approach that you'll pursue? [00:00:00] S1: I don't think that there's like one right answer, right? So I think there are some domains where deploying a system under human control makes a lot of sense. There are some domains where deploying a partially autonomous system is very reasonable. That's kind of domain dependent, because robots aren't just one thing. Some people might not want a robot in their home that is constantly being controlled by a person off-site, but maybe for some applications, that doesn't matter. [00:00:00] S0: If you mark the start of Physical Intelligence through today, what has been the most surprising thing to you that you've discovered or the nature of how the research has gone?
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: If you mark the start of Physical Intelligence through today, what has been the most surprising thing to you that you've discovered or the nature of how the research has gone? [00:00:00] S1: One of the things that's been surprising to me is that I think we've made a lot more progress on dexterity than I thought we would. We had good reason to believe that if we just collect more and more data, that just steadily gets better. What was surprising is that we could also get these systems to perform very dextrous behaviors without really doing anything particularly special for that. The same, by the way, also applied to getting systems to work on different embodiments, where we could get our models to work on all sorts of other robots, including robots with multi-fingered hands, robots with different numbers of degrees of freedom. And obviously, we needed to get data and we needed to fine-tune the model, but the model itself didn't need to change. It didn't even need to be told through any kind of prompt what the robot was. And that was also surprising to me, because I would have thought that we would need some fancy techniques to adapt the system to faster, more dextrous, more complex tasks, and also to different kinds of embodiments. But it actually seems to generalize pretty well. [00:00:00] S0: I'm always interested in the spectrum of capabilities and especially where the systems today are more advanced than you think people would probably expect and where they're less advanced than people might expect.
· Sergey Levine
[00:00:00] S1: This is something that's always been very tricky to understand in robotics. There's this idea that roboticists always talk about called Moravec's paradox. It's actually true in all areas of AI, but especially in robotics, this is a big deal. We kind of have a cognitive bias to think that things that are easy for us will be easy for the machine. Solving calculus problems is difficult for most people. Picking up a cup is easy for most people, so we think, oh, machines should be able to do this, but calculus, that requires some extra fancy stuff. But it's actually the other way around. There are things that are easy for us because they have to be, otherwise we wouldn't survive. We're very good at spotting the tiger in the jungle because the people that weren't so good at it got eaten by the tiger and they're not around anymore. Because of that, we have this cognitive bias and we think that there are things that should be very easy, but they're actually very difficult engineering challenges. However, something that is changing is that machine learning slightly changes that equation. Programming something by hand to pick up any cup anywhere, that's difficult. Getting a machine learning system to do it if you have data for it, it's actually not that difficult. And I think increasingly what we'll see is a shift where domains where collecting data is straightforward, they actually end up falling into the easy bucket over time, even if they are physically intricate. But there will be domains where collecting data is difficult, where you need to use more common sense, where you need to reason at multiple levels of abstraction, connect physical skills that you've learned in other areas to knowledge that you got from the web. And those will be tough, and that's where we'll need more technology advances.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: What is the science of common sense? When we say that, what does that mean? [00:00:00] S1: For the purpose of robotic learning, we can think of it as applying semantic inferences using knowledge learned from other domains to the current physical task at hand. You can think of common sense as the opposite of muscle memory. So muscle memory, like if you play a sport, you practice something a lot, you hardly think about it, you just kind of do it on autopilot. Common sense, in my mind – I don't know if this is the conventional definition, but I think it's a reasonable definition – is when you know something to be true, because you saw it, or you read about it, or you heard it. And now you are in a situation where that fact is highly pertinent to what you need to do. And you are able to make that connection, apply it to your situation, ground it in the environment that you're in and make the right decision. [00:00:00] S0: One of the other differences that's so interesting to me is people that have used – everyone's used chatbots now – you query it, you get an answer, query it, get an answer. We're now seeing what happens with Claude Code and other things where you give it something complicated and it's able to do it for very long without failing. What's the similar thing, long range in robotics?
· Sergey Levine
[00:00:00] S1: It's something that we're working on quite a bit right now, and the methodology is not that different at some level. The way that our models work now, as I mentioned, is they use this chain of thought process to reason about the task. When you have that, you can actually do very long horizon tasks. You can have a robot that goes and takes out all the dishes from the dishwasher, puts them in the correct cabinets, wipes down the counter, all that kind of stuff. The interesting thing here is that we found – maybe about six months ago – that our models had gotten to the point where they could be improved just from supervising them with high level instructions. You take a robot, you put it in a new kitchen, you ask it to clean the kitchen. It gets to work and then it fails somewhere. So now, okay, what do you do? Well, you add more data. Traditionally, what we would do in that situation is add more teleoperation data to cover a wider range of kitchens. But what we tried, kind of on a whim, is to see, okay, well, what if we don't add more teleoperation data? What if we just add more data labeled with the semantic command? So basically just take whatever the robot experienced and just label it with some semantic commands, but don't add any more low level actions. And that actually helps. That actually improves its ability to generalize. So what that means is that the bottleneck had actually shifted from the lowest level – meaning the robot's ability to physically do the task – to this middle level, where now the system is more bottlenecked by its ability to interpret the scene and select the correct next step, which can be supervised with language. That's a big deal because now that means that someone can literally talk to the robot.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: Coaching it, basically. [00:00:00] S1: Yeah, exactly. And make it better just by talking to it. [00:00:00] S0: If we're in 2050 and there's no robot in my kitchen doing my dishes for me, what do you think the most likely explanation is for it not having gotten there by that point?