Invest Like the Best with Patrick O'Shaughnessy
· Sergey Levine
[00:00:00] S1: It's something that we're working on quite a bit right now, and the methodology is not that different at some level. The way that our models work now, as I mentioned, is they use this chain of thought process to reason about the task. When you have that, you can actually do very long horizon tasks. You can have a robot that goes and takes out all the dishes from the dishwasher, puts them in the correct cabinets, wipes down the counter, all that kind of stuff. The interesting thing here is that we found – maybe about six months ago – that our models had gotten to the point where they could be improved just from supervising them with high level instructions. You take a robot, you put it in a new kitchen, you ask it to clean the kitchen. It gets to work and then it fails somewhere. So now, okay, what do you do? Well, you add more data. Traditionally, what we would do in that situation is add more teleoperation data to cover a wider range of kitchens. But what we tried, kind of on a whim, is to see, okay, well, what if we don't add more teleoperation data? What if we just add more data labeled with the semantic command? So basically just take whatever the robot experienced and just label it with some semantic commands, but don't add any more low level actions. And that actually helps. That actually improves its ability to generalize. So what that means is that the bottleneck had actually shifted from the lowest level – meaning the robot's ability to physically do the task – to this middle level, where now the system is more bottlenecked by its ability to interpret the scene and select the correct next step, which can be supervised with language. That's a big deal because now that means that someone can literally talk to the robot.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: Coaching it, basically. [00:00:00] S1: Yeah, exactly. And make it better just by talking to it. [00:00:00] S0: If we're in 2050 and there's no robot in my kitchen doing my dishes for me, what do you think the most likely explanation is for it not having gotten there by that point?
· Sergey Levine, Patrick O'Shaughnessy
[00:00:00] S1: I can think of a few reasons. My suspicion is that there is a long tail of challenges that has to do with the interaction of technology and people. Autonomous cars aren't that different in this regard, where getting to a level of comfort with deploying autonomous vehicles on the road was a significant challenge that ran in parallel with getting the technology to that level. Early Tesla self-driving was a bit controversial because it wasn't perfect. There was a question like, are people comfortable with this level of imperfection? Probably there are some tasks for robots where people will be comfortable with something that's not perfect, something that needs to learn from its mistakes. There are some areas where people will not be comfortable. Are you comfortable with it occasionally breaking your dishes? Maybe in a few years it will stop breaking those dishes but maybe in the meantime it's not quite there. Are you comfortable with a robot like that in a home where there's small children? Maybe not and that's okay. I think that figuring out how those factors interact and what that means for the timeline and for how these systems get better with experience, I think that's a tricky question, and I think it needs to be approached very carefully with a lot of sensitivity. There may be some domains where it makes a lot more sense for these systems to be deployed and bootstrapped and collect more data, and maybe other domains require more care. [00:00:00] S0: Can you imagine a purely technical explanation for why something might not work?
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: Can you imagine a purely technical explanation for why something might not work? [00:00:00] S1: I think the place where I would see the biggest technical risk is dealing with the breadth of different situations. If we're talking about a well-defined but slightly chaotic environment like cleaning hotel rooms or assisting human cooks in a restaurant, I have a very good sense for how to get that under control. If you're imagining a robot going into a home – one place where I can anticipate a challenge is that there are a lot of other unexpected things that can happen and you need a system that's very good at inferring what's going on and adapting to it or reacting intelligently. And I think we have a lot of ideas for how we can approach it, but that is the hardest part of the problem because when you're in a situation where just about anything could happen and you're controlling a physical device that affects the world around it, then you really need to get things right, at least at some level, pretty much in every case. It doesn't mean that you always have to succeed, but it doesn't mean that you always have to do something sensible that people are okay with. And I think there are a lot of really good ideas for how to do that, but that is probably the most challenging part of the equation.
· Patrick O'Shaughnessy
[00:00:00] S0: If I go back to thinking about the right model to think about the Physical Intelligence approach to doing this whole exercise – help me make it as simple as possible. So one might be, we're going to build a whole variety of different kinds of form factors to do a whole variety of different kinds of things and match all this data together and start to experiment with how we can, on evals, make it better. Is that just the simplest way of doing it? Is there an even simpler way? And I'm asking because I'd love to then contrast it with some other approaches that you're interested in that you're not doing that others are doing.
· Sergey Levine, Patrick O'Shaughnessy
[00:00:00] S1: In my mind, the most important thing to get right is to get the system to be general, in particular, to get it to be general with respect to how it can be improved. For example, hand-designed robotic controllers are not very general with respect to how they can be improved because it requires a human engineer to go in and improve it. A learning-based perception system is more general because all it requires is human labelers to go in and label more data. A system that learns autonomously from data that it gathers through its own experience is even more general because you don't need the human labelers. The key is this generality, particularly with respect to improvement. And the decisions we make are, to a very large extent, centered around that. I don't know if the correct design for a robot is to have three cameras. I don't know if it needs a touch sensor. I think we're very agnostic to that. I think we'll try a lot of those different choices. I'm not even sure if in the long run it's going to have a language model. Maybe we'll have some other kind of model that's trained on very diverse data. The key is this level of generality. [00:00:00] S0: What other approaches are the most interesting to you?