00:42:35 · Karol Hausman
[00:42:35] S0: I think we're still reasoning through it. I don't think we have all the answers yet. I think the important piece about how how to think about physical intelligence is we are not very dogmatic. It's not that we sit down and think very, very hard and then come up with the solution and this is our bet. I think the the better way to think about it is that we are really truth seeking, and we run experiments, and we don't know the solution. And and we know that we don't know the solution, so we want to follow the scientific method and really try to find the truth and really try to find what works and what doesn't. Because I think there is a true answer out there. We just need to be very humble in finding it. So that's, I think, how we arrive at the current set of answers. We run a lot of experiments. We try many different ideas, and we see which ones stick, and then we we double down on them. Now in terms of... Based on what we've seen, based on the evidence we've we've seen, how we think it's going to work or how I think it's going to work. These models will need to be able to absorb very diverse datasets, where it's less about picking the right data or picking the right way of collecting data, more about building the engine that allows you to absorb all kinds of data, whether it's videos of people, whether it's teleoperation data from robots, data from handheld devices, video data, really, or simulation data, really anything. And the more data they can absorb, the better the better the model will be. And I think we're kind of at this stage of of robot learning where we kind of try to throw anything we can at these models, build them in a way that they can absorb as much of it as possible, and get them to the threshold of being able... Being deployable, where you can actually deploy them in the in the world and have have robots out there collecting data for real doing economically valuable tasks. And I think once you're at that stage, once you're at that threshold of they can actually work and do valuable valuable tasks, That's when you enter this next stage of now deploying robots at scale, and deploying it in multiple verticals in diverse environments and actually delivering value. And I think that second stage is actually going to be the stage where we get most data from.
00:44:46 · Karol Hausman, Mario Gabriele
Where the more robots you deploy, the better the models should get, the more you can deploy them and there's a natural flywheel to it. And what's exciting about the moment today, this moment right now, is that I believe we're very close to this threshold. We're already above that threshold. And that's something that is really, really exciting because I think the models will keep on getting better. They will be able to absorb more and more data, but they will also have this sustainable source of data that is very, very valuable because that's the data that is the most real, that is the closest to how you actually want to deploy the robots. [00:45:23] S1: For folks that maybe haven't spent as much time digging into this or or sort of coming to it fresh, I would say that one of the experiments as you think about this data collection that physical intelligence has done maybe more than others is this real world data approach. Why is that so important and so valuable to to get and, you know, so valuable to enter that phase two that you talked about that can sort
00:45:47 · Karol Hausman
[00:45:47] S0: of start the the flywheel? We have few theories why this is really important. But I think maybe the meta point here is that either it was a different path that worked much better, like maybe for simulation or from videos or something else, we would happily pick that path. So again, it's not that we sat down and thought that this is the best path and this is the only thing we're going to do. We run a lot of experiments. We tried out a lot of ideas, and this is the one that seems to be working very, very well. Why do we think that that this is important? I think it's it's kind of difficult to describe it in the absolute. It's, I think, a little bit easier to compare it to to other alternatives. So one popular alternative is simulation. And this is, in particular, an area that that had a lot of success in... Especially in locomotion use cases where you have robots walking around or doing stunts like backflips and things like this. Most of these methods are training simulation first. And I think for those kind of tasks, the main complexity is about how you move your own body. It's less about interacting with the world and more about how do I move my legs correctly so that I can walk or I can run. And in that sense, as long as you model your own body accurately, you should be good. If you model it very very well, your own particular robot, and that transfers from simulation to the real world, you should be able to to learn that behavior in simulation, and then that's good enough to them to work in the real world. Now when it comes to the problem of manipulation where you're manipulating the world around you, The difficulty is less about modeling your own body, like how you move, you know, your arm from a to b, but it's more about modeling how the world will react to it. Right? So the problem of manipulation is more about this interaction with the world that you have or the the the interaction of the world that you're interacting with. And I think in this case, it's it's just much harder to simulate the world around you than it is to simulate your own body. It's no longer about just a single a single robot that you need to simulate. You need to simulate everything, And we don't really know how to simulate everything at scale, how to do this accurately enough and scalably enough so that it it would work for for anything.
00:48:01 · Karol Hausman, Mario Gabriele
For every single one of of of tasks or objects, it takes really long to get it exactly right, to get all the friction parameters right, to get the simulation behaviors right, and it's just not as scalable. That's been our finding so far. [00:48:14] S1: And so it's sort of the case to to try and distill that that, you know, with simulation, you can do some of these... I don't know. Maybe this isn't the perfect word, but sort of coarser, larger actions that are more self contained. But once you start trying to, yeah, implement picking up the coffee cup or the towel or whatever it might be, you're starting to rely on a simulation that would have to be so good, you'd effectively have to, you know, create a a simulation that is perfectly similar to reality, and that's where the the real world data starts to become so important. Yeah. You would need to simulate all of the external world Yeah. With any one of those tasks. And that's just too too costly, too difficult to do. But as long as you can get away with [00:49:00] S0: just simulating your own body, I think that works perfectly fine, and that's what we've seen in locomotion or, you know, backflips or dances or things like that.
00:49:00 · Karol Hausman, Mario Gabriele
[00:49:00] S0: just simulating your own body, I think that works perfectly fine, and that's what we've seen in locomotion or, you know, backflips or dances or things like that. [00:49:08] S1: There was a moment, I think, about a year and a little bit ago where you talked about reinforcement learning making a comeback, and that has since become a really interesting part of the way that physical intelligence seems to work with recap. What were you seeing at the time that made you think, you know, this this technique might have sort of a second or additional wind here? And why has that been so useful for you? Yeah. There's been a long history of applying reinforcement learning to robots. And the problem
00:49:42 · Karol Hausman
[00:49:42] S0: of reinforcement learning is such that you need to be learning from your own experience, and while gathering that experience you need to encounter some successes. And then you want to increase the probability of good actions that lead to those successes and decrease probability of the actions that lead to failures. What this means is that you need to have... As you as you explore the world, as you collect your own experiences, you need to have some of these successes because if you don't see any of them, you basically don't really know where to go. You're you're kind of lost. And if you start from scratch, if you just try to command random commands to a robot, it's very very unlikely that you'll encounter a success based on that. So let's say you're trying to teach a robot how to grasp an object, and you command basically random commands to all of the motors of the arms. It's very, very unlikely that some of these commands are going to lead to a single success of you grasping an object successfully. And that's what we often refer to as the exploration problem. But we don't really have a way of guiding the robot on how to explore so that you can encounter a success. And as soon as you encounter it, then you're on the flywheel. You can now increase probabilities of those actions. So just to get to the very first one, very, very hard. So... And that's been the problem of reinforcement learning for a very long time. But now with the models that we've been putting together, this exploration problem became much easier because robots now don't start from scratch. They start from a foundational like PI zero, PI zero five, or PI zero six, where they already have some intuitive understanding of how motions work and how you can explore the world around you. So even if you put a new object in front of the robot, the chances are that if you ask it to grasp it, it will either grasp it or it will it will fail in an interesting way. So the probability of you actually doing something useful starts to become very, very high, or at least much higher than it was before. So now you have a way to explore the world, and that allows you to apply reinforcement learning methods much more easily because they can encounter successes much quicker.