No Priors: Artificial Intelligence | Technology | Startups
00:19:34 · Sarah Guo, Cheng Chi, Tony Zhao
[00:19:34] S1: Tony, as you mentioned, like, it's an open question, actually, what the right way to scale data up is. And so there are strong theories around teleop, around, like, pure RL, around video and world models. Like, how did you think about all of these? [00:19:50] S2: Yeah. So from our perspective, actually, it's kinda somewhat surprising. So in the beginning, we worried that the data from GloVe or UMI like data that has higher quality but lower quality compared to TallyUp. Because for TallyUp, you're using exactly the same hardware and software stack between training and testing. It's perfectly distribution matched. But what we realized is actually this glow form factor encouraged people to do more text version, more natural movement, and those actually result in a more intelligent behavior on the modeling side. And in terms of, you know, data quality, we don't really see a difference in terms of, you know, how much... Like like, there's a gap between TidyOp and GloV data. [00:20:28] S0: After we did the 20 engineering, like, series method? Yeah. Yeah. Like... Because, like, apparently, there is a mismatch, right, that's in the camera frame, there's a human inside of the robot, and there are just a lot of things that we need to do to kind of convert a human data one to one to, like, as if it is robot data and have the model not being able to tell the difference. [00:20:52] S2: Yeah. And that kinda relies on, again, the whole... The full cycle iteration between hardware and software. [00:20:57] S1: What about RL?
00:20:57 · Sarah Guo, Cheng Chi, Tony Zhao
[00:20:57] S1: What about RL? [00:20:59] S2: We see a lot of great promise for RL in locomotion, and we think that will continue to be true for locomotion. So what we see really... RL as a method is very powerful, but it is much less sample efficient compared to imitation learning, and we see that to work great in environments where it's easy to simulate. For the case of locomotion, you only need worry about rigid body dynamics and rigid body contact between the robot and the ground. And, you know, because you engineer a robot, you know everything. But for manipulation, it's kinda hard for [00:21:31] S0: us to [00:21:31] S2: imagine, like, have... There's actually the same amount of diversity and the distribution of real object in terms of matching both appearance... Its its appearance and physical properties. And we think that it's gonna be challenging compared to global data collection and tally up.
00:21:31 · Cheng Chi, Tony Zhao, Sarah Guo
[00:21:31] S2: imagine, like, have... There's actually the same amount of diversity and the distribution of real object in terms of matching both appearance... Its its appearance and physical properties. And we think that it's gonna be challenging compared to global data collection and tally up. [00:21:49] S0: Yeah. I think it's really about which method can get us there faster. There might be different methods that will eventually get there. For example, like, you know, simulation world model. Right? And, like, it's almost a tautology to say that if I have a perfect world simulator, anything can be done there. Like, as long as you can do it in the real world, you can do it in a simulation, and you can, like, you know, cure cancer in a simulator. Right? But what it turns out for robotics is that just some some things are harder than others, and it really depends on the problem itself. So in the case of locomotion, as I mentioned, all we need to model in a simulator are point contacts with a somewhat flat ground. [00:22:33] S1: Mhmm. Like feet. [00:22:34] S0: Yes. [00:22:34] S2: Yeah.
00:22:34 · Cheng Chi, Tony Zhao, Sarah Guo
[00:22:34] S2: Yeah. [00:22:35] S0: But sort of the behavior we want out of it is actually very difficult to model. Mhmm. Like, it's all these, like, reactive behaviors that when you feel like your leg is hitting something, you should, like, retract and, you know, step again. These are very, very hard to describe or try to learn from demonstrations directly. But in the case of manipulation, I think the difficulty is flipped. That it's a lot easier to capture the behavior itself, and it's a lot harder to simulate the world. Mhmm. For example, if you were to grasp a transparent cup with some orange juice in it, it's ridiculously hard to simulate how, like, your hand deforms around the cup and how all those ripplings, how those... Like, the color of the juice results in, like, the rendering and what the policy end up seeing. Simulating that is very expensive and difficult. But all we need to learn is just to, like, get your hand to be in front of the cup and then close with the appropriate amount of force, and that's actually very easy to learn. That's why, like, we see so many success of imitation learning in the case of robotics manipulation is because the behavior itself is actually not as hard as simulating the world, and that's why we see faster progress there. [00:23:58] S1: Is there anything that you have changed your point of view on in data over the last year?
00:23:58 · Sarah Guo, Cheng Chi, Tony Zhao
[00:23:58] S1: Is there anything that you have changed your point of view on in data over the last year? [00:24:03] S2: Like, it's one thing... I wouldn't say changed, but just data quality really matters. Like, I think we know... I always knew data quality matters, but once you scale it up, like, it it really matters. And and then... Because, you know, just like the diversity of of behavior that you experience in the wild is very hard to control, and the hardware failures are hard to control. You need to constantly monitor them. Just... You need to spend a lot of... Huge amount of engineering effort just to make sure that, you know, the data is clean. [00:24:32] S0: Yeah. And also building all those automatic processes. Right? We have our own way of calibrating the glove before we ship it out, and we have this whole, like, software system to catch if a... Something is broken on the glove, and we can detect it automatically. It's, like the importance of data quality kinda translates to all these repeatable processes, and we don't need a human to be staring at the data to know that something is wrong. [00:24:55] S1: When you described the beta for next year, a lot of it sounded like, we just wanna understand behavior, like how people actually want to use it. We can make some design decisions for the actual product. What technical challenges do you still see?
00:24:55 · Sarah Guo, Cheng Chi
[00:24:55] S1: When you described the beta for next year, a lot of it sounded like, we just wanna understand behavior, like how people actually want to use it. We can make some design decisions for the actual product. What technical challenges do you still see? [00:25:11] S2: So to me, I think there's two kind. The number one is really figuring out the training recipe at scale. We as a field just entered the realm of scaling, and we just got the amount of data that we need. I think now is a perfect time to start doing research and actually figure out what exact training recipe we need to actually get robust behaviors. And I think we're in a unique position because of the amount of data and the entire pipeline we built around data. The second point, I think, is just really hardware is hard. We are still pushing the bound... The envelope, performance envelope of hardware. It's not really clear actually what is needed, what is necessary for the hardware to be reliable, because whenever the mechanical team build a hardware, the learning team will try to... Try harder to push it against the boundary, and then it'll break at some point. But I think what's interesting in this company is that everybody's under the same roof. So immediately after something breaks, it goes straight back into mechanical design, and then we have another iteration, like I said, for the hand parts very quickly. Hardware is hard, but it is important. And I think it's a hard but right thing to do. And I think we as a field shouldn't avoid doing the hard things just because they're hard. Yeah.