Invest Like the Best with Patrick O'Shaughnessy
· Sergey Levine
[00:00:00] S1: This is something that's always been very tricky to understand in robotics. There's this idea that roboticists always talk about called Moravec's paradox. It's actually true in all areas of AI, but especially in robotics, this is a big deal. We kind of have a cognitive bias to think that things that are easy for us will be easy for the machine. Solving calculus problems is difficult for most people. Picking up a cup is easy for most people, so we think, oh, machines should be able to do this, but calculus, that requires some extra fancy stuff. But it's actually the other way around. There are things that are easy for us because they have to be, otherwise we wouldn't survive. We're very good at spotting the tiger in the jungle because the people that weren't so good at it got eaten by the tiger and they're not around anymore. Because of that, we have this cognitive bias and we think that there are things that should be very easy, but they're actually very difficult engineering challenges. However, something that is changing is that machine learning slightly changes that equation. Programming something by hand to pick up any cup anywhere, that's difficult. Getting a machine learning system to do it if you have data for it, it's actually not that difficult. And I think increasingly what we'll see is a shift where domains where collecting data is straightforward, they actually end up falling into the easy bucket over time, even if they are physically intricate. But there will be domains where collecting data is difficult, where you need to use more common sense, where you need to reason at multiple levels of abstraction, connect physical skills that you've learned in other areas to knowledge that you got from the web. And those will be tough, and that's where we'll need more technology advances.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: What is the science of common sense? When we say that, what does that mean? [00:00:00] S1: For the purpose of robotic learning, we can think of it as applying semantic inferences using knowledge learned from other domains to the current physical task at hand. You can think of common sense as the opposite of muscle memory. So muscle memory, like if you play a sport, you practice something a lot, you hardly think about it, you just kind of do it on autopilot. Common sense, in my mind – I don't know if this is the conventional definition, but I think it's a reasonable definition – is when you know something to be true, because you saw it, or you read about it, or you heard it. And now you are in a situation where that fact is highly pertinent to what you need to do. And you are able to make that connection, apply it to your situation, ground it in the environment that you're in and make the right decision. [00:00:00] S0: One of the other differences that's so interesting to me is people that have used – everyone's used chatbots now – you query it, you get an answer, query it, get an answer. We're now seeing what happens with Claude Code and other things where you give it something complicated and it's able to do it for very long without failing. What's the similar thing, long range in robotics?
· Sergey Levine
[00:00:00] S1: It's something that we're working on quite a bit right now, and the methodology is not that different at some level. The way that our models work now, as I mentioned, is they use this chain of thought process to reason about the task. When you have that, you can actually do very long horizon tasks. You can have a robot that goes and takes out all the dishes from the dishwasher, puts them in the correct cabinets, wipes down the counter, all that kind of stuff. The interesting thing here is that we found – maybe about six months ago – that our models had gotten to the point where they could be improved just from supervising them with high level instructions. You take a robot, you put it in a new kitchen, you ask it to clean the kitchen. It gets to work and then it fails somewhere. So now, okay, what do you do? Well, you add more data. Traditionally, what we would do in that situation is add more teleoperation data to cover a wider range of kitchens. But what we tried, kind of on a whim, is to see, okay, well, what if we don't add more teleoperation data? What if we just add more data labeled with the semantic command? So basically just take whatever the robot experienced and just label it with some semantic commands, but don't add any more low level actions. And that actually helps. That actually improves its ability to generalize. So what that means is that the bottleneck had actually shifted from the lowest level – meaning the robot's ability to physically do the task – to this middle level, where now the system is more bottlenecked by its ability to interpret the scene and select the correct next step, which can be supervised with language. That's a big deal because now that means that someone can literally talk to the robot.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: Coaching it, basically. [00:00:00] S1: Yeah, exactly. And make it better just by talking to it. [00:00:00] S0: If we're in 2050 and there's no robot in my kitchen doing my dishes for me, what do you think the most likely explanation is for it not having gotten there by that point?
· Sergey Levine, Patrick O'Shaughnessy
[00:00:00] S1: I can think of a few reasons. My suspicion is that there is a long tail of challenges that has to do with the interaction of technology and people. Autonomous cars aren't that different in this regard, where getting to a level of comfort with deploying autonomous vehicles on the road was a significant challenge that ran in parallel with getting the technology to that level. Early Tesla self-driving was a bit controversial because it wasn't perfect. There was a question like, are people comfortable with this level of imperfection? Probably there are some tasks for robots where people will be comfortable with something that's not perfect, something that needs to learn from its mistakes. There are some areas where people will not be comfortable. Are you comfortable with it occasionally breaking your dishes? Maybe in a few years it will stop breaking those dishes but maybe in the meantime it's not quite there. Are you comfortable with a robot like that in a home where there's small children? Maybe not and that's okay. I think that figuring out how those factors interact and what that means for the timeline and for how these systems get better with experience, I think that's a tricky question, and I think it needs to be approached very carefully with a lot of sensitivity. There may be some domains where it makes a lot more sense for these systems to be deployed and bootstrapped and collect more data, and maybe other domains require more care. [00:00:00] S0: Can you imagine a purely technical explanation for why something might not work?
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: Can you imagine a purely technical explanation for why something might not work? [00:00:00] S1: I think the place where I would see the biggest technical risk is dealing with the breadth of different situations. If we're talking about a well-defined but slightly chaotic environment like cleaning hotel rooms or assisting human cooks in a restaurant, I have a very good sense for how to get that under control. If you're imagining a robot going into a home – one place where I can anticipate a challenge is that there are a lot of other unexpected things that can happen and you need a system that's very good at inferring what's going on and adapting to it or reacting intelligently. And I think we have a lot of ideas for how we can approach it, but that is the hardest part of the problem because when you're in a situation where just about anything could happen and you're controlling a physical device that affects the world around it, then you really need to get things right, at least at some level, pretty much in every case. It doesn't mean that you always have to succeed, but it doesn't mean that you always have to do something sensible that people are okay with. And I think there are a lot of really good ideas for how to do that, but that is probably the most challenging part of the equation.