00:39:41 · Mario Gabriele
[00:39:41] S1: You had this insight that this was the right moment to to go and solve this problem. But at least from the outside, one could imagine many different form factors that might take. Like, another version of this company might be saying, hey. We're gonna try and build the the best humanoid robot ourselves, or, you know, we're gonna take this approach to getting the data we need to to run these models very effectively versus another approach. How did you sort of land on the version of physical intelligence as it is today?
00:40:13 · Karol Hausman
[00:40:13] S0: I think from the get go, we had an idea of... We had the thesis for the company. And that thesis was around all the research that we've done up until this point. That similarly to what happened with language, it's not gonna be specialist models that's really going to solve this problem. It's going to be generalist models that work across all kinds of different tasks, all kinds of different environments, and all kinds of different robots. And it was similar to lag... To what we've seen in language where, you know, you would think that the best translator would be would be just specialized for translation or the best coder would be just specialized or just trained on code. But it turned out that the way to be the best at all of those fields is to train one generalist and, you know, generalist that that takes in poetry data and the coding data and translation data, and then it turns out to be much better than all of those specialists at those specialist tasks. And we started seeing something similar in robot learning, in robotics, that if only we could collect enough data, if that data was very, very diverse, and if we do it the right way, we should be able to build the best generalists. If we solve that, that is really gonna allow us to have this world of many diverse form factors and robots that can be very intelligent. So I think the the kind of the two thoughts that we had from the beginning is that one is intelligence has always been the bottleneck for robotics. But rather than trying to start a robotics company that focuses on a specific robot, how can we tackle this problem head on and just focus on the intelligence? And then the second thought being that the way the way to solve intelligence is to take the lessons that we learn from vision and language and other fields and really take the foundation model approach, which includes things like cross embodiment learning, large diversity of data, a lot of real world data, and do the research necessary to figure out how how to build these models.
00:42:10 · Mario Gabriele
[00:42:10] S1: So you sort of land on this idea of, you know, the AI brain for these different robot types, put it sort of simply. How how did you start to think about the right way to gather the data? Because that's clearly such a big piece of it, and, obviously, there are different pieces... Different players that take different approaches there. So I imagine you must have had to reason through that in a million different ways to to land on the one you have.
00:42:35 · Karol Hausman
[00:42:35] S0: I think we're still reasoning through it. I don't think we have all the answers yet. I think the important piece about how how to think about physical intelligence is we are not very dogmatic. It's not that we sit down and think very, very hard and then come up with the solution and this is our bet. I think the the better way to think about it is that we are really truth seeking, and we run experiments, and we don't know the solution. And and we know that we don't know the solution, so we want to follow the scientific method and really try to find the truth and really try to find what works and what doesn't. Because I think there is a true answer out there. We just need to be very humble in finding it. So that's, I think, how we arrive at the current set of answers. We run a lot of experiments. We try many different ideas, and we see which ones stick, and then we we double down on them. Now in terms of... Based on what we've seen, based on the evidence we've we've seen, how we think it's going to work or how I think it's going to work. These models will need to be able to absorb very diverse datasets, where it's less about picking the right data or picking the right way of collecting data, more about building the engine that allows you to absorb all kinds of data, whether it's videos of people, whether it's teleoperation data from robots, data from handheld devices, video data, really, or simulation data, really anything. And the more data they can absorb, the better the better the model will be. And I think we're kind of at this stage of of robot learning where we kind of try to throw anything we can at these models, build them in a way that they can absorb as much of it as possible, and get them to the threshold of being able... Being deployable, where you can actually deploy them in the in the world and have have robots out there collecting data for real doing economically valuable tasks. And I think once you're at that stage, once you're at that threshold of they can actually work and do valuable valuable tasks, That's when you enter this next stage of now deploying robots at scale, and deploying it in multiple verticals in diverse environments and actually delivering value. And I think that second stage is actually going to be the stage where we get most data from.
00:44:46 · Karol Hausman, Mario Gabriele
Where the more robots you deploy, the better the models should get, the more you can deploy them and there's a natural flywheel to it. And what's exciting about the moment today, this moment right now, is that I believe we're very close to this threshold. We're already above that threshold. And that's something that is really, really exciting because I think the models will keep on getting better. They will be able to absorb more and more data, but they will also have this sustainable source of data that is very, very valuable because that's the data that is the most real, that is the closest to how you actually want to deploy the robots. [00:45:23] S1: For folks that maybe haven't spent as much time digging into this or or sort of coming to it fresh, I would say that one of the experiments as you think about this data collection that physical intelligence has done maybe more than others is this real world data approach. Why is that so important and so valuable to to get and, you know, so valuable to enter that phase two that you talked about that can sort
00:45:47 · Karol Hausman
[00:45:47] S0: of start the the flywheel? We have few theories why this is really important. But I think maybe the meta point here is that either it was a different path that worked much better, like maybe for simulation or from videos or something else, we would happily pick that path. So again, it's not that we sat down and thought that this is the best path and this is the only thing we're going to do. We run a lot of experiments. We tried out a lot of ideas, and this is the one that seems to be working very, very well. Why do we think that that this is important? I think it's it's kind of difficult to describe it in the absolute. It's, I think, a little bit easier to compare it to to other alternatives. So one popular alternative is simulation. And this is, in particular, an area that that had a lot of success in... Especially in locomotion use cases where you have robots walking around or doing stunts like backflips and things like this. Most of these methods are training simulation first. And I think for those kind of tasks, the main complexity is about how you move your own body. It's less about interacting with the world and more about how do I move my legs correctly so that I can walk or I can run. And in that sense, as long as you model your own body accurately, you should be good. If you model it very very well, your own particular robot, and that transfers from simulation to the real world, you should be able to to learn that behavior in simulation, and then that's good enough to them to work in the real world. Now when it comes to the problem of manipulation where you're manipulating the world around you, The difficulty is less about modeling your own body, like how you move, you know, your arm from a to b, but it's more about modeling how the world will react to it. Right? So the problem of manipulation is more about this interaction with the world that you have or the the the interaction of the world that you're interacting with. And I think in this case, it's it's just much harder to simulate the world around you than it is to simulate your own body. It's no longer about just a single a single robot that you need to simulate. You need to simulate everything, And we don't really know how to simulate everything at scale, how to do this accurately enough and scalably enough so that it it would work for for anything.