Invest Like the Best with Patrick O'Shaughnessy
· Sergey Levine
[00:00:00] S1: There was a gentleman named Benjie Holson who used to work at Everyday Robots – part of Alphabet before it dissolved. He spends a lot of time thinking about tasks that robots could do. So he wrote a really interesting blog post a while back. There was this Robot Olympics that was held in China where robots would run around on a track and jump and so on. But maybe these aren't the real challenges we should worry about. How about a Robot Olympics centered around everyday tasks that people do? It's kind of a Moravec's paradox thing, where there's tasks that people find really easy but that robots struggle with. And he had things like opening a door, washing a frying pan with grease on it, using a plastic bag to pick up dog poop. Things that people don't find particularly challenging but that no current robotic system can do. And he listed maybe a dozen of these things. This wasn't part of a concerted research project. We had developed processes and systems for ingesting new tasks that we wanted to use for all sorts of tasks. We figured a good way to test this is to say, here's a big list of tasks. Let's just go through this process that we've developed and see if it works. So it's almost like a test of our internal operations and model training system. Then we tried these things and it actually turned out that we could solve almost all of them. Well, there's one we couldn't do which was turning a dress shirt inside out because the grippers on this thing wouldn't fit inside the sleeve. So we probably need to change the gripper. I think on a technicality, we didn't succeed at peeling an orange because he said, do it with the fingers, and our fingers weren't strong enough. So we had to use a little tool, like a little knife, basically. Everything else we could do. If anybody watches those videos, one thing that I think is important to keep in mind is, we didn't develop anything special for this. We literally used this as a test of our task onboarding process. There's something interesting there because it suggests the power of generality, that when you have this general system, you can really just onboard all these crazy tasks without really doing anything particularly sophisticated.
· Patrick O'Shaughnessy
[00:00:00] S0: I was curious before when you said superhuman ability, dexterity or something like that, where we're limited by what we can do or maybe by what we can control, even if it gets smaller. What are some of the other dimensions that we might surpass human ability on in terms of physical ability? What are the other trend lines?
· Sergey Levine
[00:00:00] S1: So here's a fun one. We were working on a task where our robot had to plug in things like power cables or ethernet cables or something like that. When a person does this – obviously if you practice it a lot, you'll get really good at it. But when a person does this without having practiced a lot, you pause frequently, right? Because it's not a physical thing. You just have to pause what's going on. You have to make sure that it's all aligned and all that stuff. So you do it very slowly. And if you're teleoperating a robot, you do it even more slowly because there's this level of indirection. It turns out to be pretty straightforward to go in and find all those pauses and remove them. You can speed things up further. So you can get to a task where a person demonstrates what it means to succeed. And then you can have the robot practice the task and succeed in the same way, but a lot more quickly, a lot more efficiently. The most general way to do this is with reinforcement learning. But there are also some simple tricks you can do if you just want speed. So that's one example of something where you can have a machine that does it a lot better. At some level, you have a processing bottleneck. That's why the person does it slowly, because they have to process what's going on. But speeding up processing is something that people understand quite well in computer science.
· Patrick O'Shaughnessy
[00:00:00] S0: There's this amazing Michael Crichton novel called Prey, where it seems like for a given problem, there may be an optimal or set of optimal shapes of the robot to perform the task. And that where you should do is analyze the problem, then have something that can almost like morph or transform into the right form factor. How do you think about that, the innovation on the form factor side rather than the data and model side?
· Sergey Levine
[00:00:00] S1: I think that in general, in robotics, the ability to innovate on form factors has been very constrained because of the AI challenge. If you have a traditional AI pipeline, like you're doing some motion planning and stuff like that, it's hard to just go and cobble together some new robot. Because when you do that, you have to like characterize the dynamics of the system, you have to do SysID, you have to build up all the stuff. If you could just put together a robot in your garage, load up a robotic foundation model and tell it to do a bunch of stuff – maybe it won't be perfect at it. Maybe it needs more data to really perfect it. But you can at least get the thing moving. I think that can be a really powerful engine just to get everybody to experiment with this stuff. I don't think that I'm the right person to design the perfect robot. There are people here of course who are a lot better at that. But in general, I think that it's just like with personal computers. I think the key is to let people experiment and play around with it and radically lower the barrier to entry for that. Then we'll see a lot of creativity. When we first started using computers, there was a limited number of form factors. Now you can have a computer in your phone, a computer in your car, embed a computer in your refrigerator. They're everywhere and they're very different. Generality, good software, good foundation on top of which you can build applications. Those are key to enabling that.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: Your co-founder Lachy once described to me the feeling of physical intelligence for a human is like learning how to ride a bike. There's that moment when you didn't know how to do it and then you do know how to do it. And that feeling is physical intelligence, that snap of understanding. [00:00:00] S1: There's actually a physiological explanation for this. There were studies that were done in monkeys using tools, and you can actually find where in the brain, which neurons activate for the monkey to figure out where its hand is. It turns out that if it's using a tool, they activate based on location of the tool tip, not based on location of the hand. The tool being an extension of your body is a real physiological thing. Your brain literally does that. [00:00:00] S0: Knowing that, what does that do to impact the approach to your research? [00:00:00] S1: It says that physical intelligence should be at some level agnostic to embodiment. A good foundation model should figure out how to manipulate whatever body it's controlling, whatever tools it has at hand. There's basically one problem, not many different problems. There isn't like a humanoid problem and a car problem and a bulldozer problem and a robot bolted to the table problem. There's one problem. And if you solve it at a full level of generality, that's really, really powerful.