Invest Like the Best with Patrick O'Shaughnessy
· Sergey Levine, Patrick O'Shaughnessy
[00:00:00] S1: In my mind, the most important thing to get right is to get the system to be general, in particular, to get it to be general with respect to how it can be improved. For example, hand-designed robotic controllers are not very general with respect to how they can be improved because it requires a human engineer to go in and improve it. A learning-based perception system is more general because all it requires is human labelers to go in and label more data. A system that learns autonomously from data that it gathers through its own experience is even more general because you don't need the human labelers. The key is this generality, particularly with respect to improvement. And the decisions we make are, to a very large extent, centered around that. I don't know if the correct design for a robot is to have three cameras. I don't know if it needs a touch sensor. I think we're very agnostic to that. I think we'll try a lot of those different choices. I'm not even sure if in the long run it's going to have a language model. Maybe we'll have some other kind of model that's trained on very diverse data. The key is this level of generality. [00:00:00] S0: What other approaches are the most interesting to you?
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: What other approaches are the most interesting to you? [00:00:00] S1: One thing that's a very important question in this area, and something that I think the research community and the tech community has not fully answered is the dichotomy between different data sources, particularly with respect to real data and simulation. It's a very controversial topic. I have a very strong opinion about it. But I think that – it's worth acknowledging that if we look, for example, at humanoids, you've seen videos of humanoids doing all these acrobatics. There's a particular pipeline that makes that work, which is very heavily reliant on simulation and very light on real-world data. Often actually, zero real-world data. And then there are the approaches that work well for robotic manipulation that often are the opposite. They often use very little simulated data, often use large amounts of real-world data and very large foundation models. And it is surprising that in these two robotic domains, the dominant approaches look so different. It may be that one will win out and there is a particular approach that can handle everything in the long run. Or maybe there's some sort of synthesis of these ideas. That's important. I don't know the answer to that. I have subjective opinions. I think the approach we're taking is a very good one. But I think that it's interesting to look at that and see why is it that these things are so different.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: Can you talk about the contrast between cool and useful? The Boston Dynamics robot is very cool. The backflip is super cool. Inverting the body. It all looks really good. I don't know what I need that requires a robot to do a backflip. So I'm curious how you think about optimizing around cool versus useful. [00:00:00] S1: The strategy we've taken is, subject to the constraint that it's useful, make it as cool as possible. We make decisions first and foremost based on our assessment of what will drive the tech forward towards this truly general, broadly applicable robotic foundation model. But in doing that, we try to stress test it against the toughest challenges we can throw at it. The toughest challenges are the ones that look cool. We didn't set out, for example, to build a robot that can make espresso or can fold laundry. But in the process of building these general systems, we figure like these would be particularly challenging, particularly exciting things to try with them to see how far we can push them. [00:00:00] S0: Can you talk about the Robot Olympics?
· Sergey Levine
[00:00:00] S1: There was a gentleman named Benjie Holson who used to work at Everyday Robots – part of Alphabet before it dissolved. He spends a lot of time thinking about tasks that robots could do. So he wrote a really interesting blog post a while back. There was this Robot Olympics that was held in China where robots would run around on a track and jump and so on. But maybe these aren't the real challenges we should worry about. How about a Robot Olympics centered around everyday tasks that people do? It's kind of a Moravec's paradox thing, where there's tasks that people find really easy but that robots struggle with. And he had things like opening a door, washing a frying pan with grease on it, using a plastic bag to pick up dog poop. Things that people don't find particularly challenging but that no current robotic system can do. And he listed maybe a dozen of these things. This wasn't part of a concerted research project. We had developed processes and systems for ingesting new tasks that we wanted to use for all sorts of tasks. We figured a good way to test this is to say, here's a big list of tasks. Let's just go through this process that we've developed and see if it works. So it's almost like a test of our internal operations and model training system. Then we tried these things and it actually turned out that we could solve almost all of them. Well, there's one we couldn't do which was turning a dress shirt inside out because the grippers on this thing wouldn't fit inside the sleeve. So we probably need to change the gripper. I think on a technicality, we didn't succeed at peeling an orange because he said, do it with the fingers, and our fingers weren't strong enough. So we had to use a little tool, like a little knife, basically. Everything else we could do. If anybody watches those videos, one thing that I think is important to keep in mind is, we didn't develop anything special for this. We literally used this as a test of our task onboarding process. There's something interesting there because it suggests the power of generality, that when you have this general system, you can really just onboard all these crazy tasks without really doing anything particularly sophisticated.
· Patrick O'Shaughnessy
[00:00:00] S0: I was curious before when you said superhuman ability, dexterity or something like that, where we're limited by what we can do or maybe by what we can control, even if it gets smaller. What are some of the other dimensions that we might surpass human ability on in terms of physical ability? What are the other trend lines?
· Sergey Levine
[00:00:00] S1: So here's a fun one. We were working on a task where our robot had to plug in things like power cables or ethernet cables or something like that. When a person does this – obviously if you practice it a lot, you'll get really good at it. But when a person does this without having practiced a lot, you pause frequently, right? Because it's not a physical thing. You just have to pause what's going on. You have to make sure that it's all aligned and all that stuff. So you do it very slowly. And if you're teleoperating a robot, you do it even more slowly because there's this level of indirection. It turns out to be pretty straightforward to go in and find all those pauses and remove them. You can speed things up further. So you can get to a task where a person demonstrates what it means to succeed. And then you can have the robot practice the task and succeed in the same way, but a lot more quickly, a lot more efficiently. The most general way to do this is with reinforcement learning. But there are also some simple tricks you can do if you just want speed. So that's one example of something where you can have a machine that does it a lot better. At some level, you have a processing bottleneck. That's why the person does it slowly, because they have to process what's going on. But speeding up processing is something that people understand quite well in computer science.