Y Combinator Startup Podcast
00:49:43 · S1, Chelsea Finn
[00:49:43] S1: The next question is, does the model output raw motor commands directly or does it output a target hand position and let a controller solve for the joint angles and what makes that the right level to learn at? [00:49:56] S0: So the model... All the models that I showed were outputting target joint positions. So like, what is the angle of this joint? What is the angle of this joint and so forth that you want to hit? And then there's a controller, like a PD controller that is trying to then hit that target position for those joints. The model actually is also trained to predict target gripper positions, like where in three d space should my gripper be. And you could also use that as well and back out the joint positions. You could also... Like, another thing you could do is you could go directly to, like, motor torques or to, like, voltages or efforts. There are pros and cons of different options. We have found controlling joints, and controlling in the three d space of the gripper to both work well. And so there's pros and cons. I think the one thing that would be nice about going directly to the voltages is that you could also get a more stiff output or a less stiff output, whereas the controller, if you have a fixed controller, then you're not letting your model control that aspect. And so there's different pros and cons. Think that what we're working seems to work. It doesn't seem to be a bottleneck. And I often like to focus on the things that seem to be bottlenecks versus things that don't seem to be bottlenecks. Okay.
00:49:56 · Chelsea Finn, S1
[00:49:56] S0: So the model... All the models that I showed were outputting target joint positions. So like, what is the angle of this joint? What is the angle of this joint and so forth that you want to hit? And then there's a controller, like a PD controller that is trying to then hit that target position for those joints. The model actually is also trained to predict target gripper positions, like where in three d space should my gripper be. And you could also use that as well and back out the joint positions. You could also... Like, another thing you could do is you could go directly to, like, motor torques or to, like, voltages or efforts. There are pros and cons of different options. We have found controlling joints, and controlling in the three d space of the gripper to both work well. And so there's pros and cons. I think the one thing that would be nice about going directly to the voltages is that you could also get a more stiff output or a less stiff output, whereas the controller, if you have a fixed controller, then you're not letting your model control that aspect. And so there's different pros and cons. Think that what we're working seems to work. It doesn't seem to be a bottleneck. And I often like to focus on the things that seem to be bottlenecks versus things that don't seem to be bottlenecks. Okay. [00:51:14] S1: Next question is, do robots need something like imagination? Well, the ability to picture what should happen next before they can become truly useful.
00:51:23 · Chelsea Finn
[00:51:23] S0: So the Pi0.7 model that I showed has something like this, where it can kind of imagine what a future image should look like and then try to accomplish that. We found that that leads to improvement, and we saw on the shirt folding example, we saw, like, a quantitative bump from using that sort of imagination compared to not using it. At the same time, I think that the model actually performed surprisingly well without that as well. And we're actually hoping... We're thinking about writing an entire paper, an entire technical report just about that capability in that model. But the the model without that was so good that we felt like we needed to actually have that play a bigger part of the story because it it... That seemed like that was really delivering, in terms of actually getting really strong results. And so, I it seems like one design choice. I think it's hard to say if it's going to be a critical component or not. I guess the good news with these kinds of models is that if you develop a good data set, you can run experiments and you can continue to test things with the data set that you have quite effectively. I also think that being able to predict the future seems like a very relevant objective compared to predicting future actions. So that should, I would imagine, help in terms of learning from all the data that you have available to you. Yeah, hard to say if this is going to be a critical component or not. It seems like empirically, so far, it seems to help, although perhaps not as much as you might expect. And even without that imagination, the robot can do pretty incredible things. Okay.
00:53:01 · S1, Chelsea Finn
[00:53:01] S1: Next is right now, it seems that robots are doing amazing tasks but in a very slow manner. What is needed to improve the speed? [00:53:08] S0: I'm really excited about improving the speed. The... And we we did see speed improvements from reinforcement learning. We also have another release, called the RL token that... Where we showed actually even faster speed and actually faster speed than human teleop. I think one of the bottlenecks is that when you teleoperate robots to do things, which is the easiest way to teach robot to do something, people are kind of slow at teleoperating the robot. We have a couple of projects in the pipeline that I think have really promising results in terms of getting fast policies. And so I think more to come there. Yeah, I think it's either you need to figure out how to make the data faster or you need to figure out how to be faster than the data. We've seen evidence of being able to be a little bit faster than the data. And in terms of the next steps, it's either to kind of go even further than that or make the data faster. [00:54:03] S1: Cool. What's the most surprising task you've seen a robot complete recently? What do you want to see it do next?
00:54:10 · Chelsea Finn
[00:54:10] S0: So the most surprising thing was not really a task, but when we were working on Pi0.7, personally trained one of the policies for some of the initial tests for assembling this pinwheel. And when I was working on training the... Training it to, to construct the pinwheel, one thing that really surprised me was, in all the data, we kind of carefully controlled controlled the strategy for how to assemble the pinwheel, where you, basically, will take the the kind of precut piece of paper and take a little pin and insert the pin into a hole in the paper. And in all of the data, we picked up the pin with the right hand and, picked up the paper with the left hand and inserted it. And the robot kinda started doing that. And then it actually made a mistake. And the paper ended up on the right side, and the pin ended up on the left side. And what the robot did is it picked up the paper, and it picked up the pin with its left gripper. And it put the pin with its left gripper and inserted it into the paper, with its right. And it had never seen data of inserting the pin with his left gripper. And it kind of showed that even that wasn't in the post training data at all, like, was only... Like, and it wasn't even in pretraining either. The robot essentially had learned this sort of equivariance between his left hand and his right hand so that it could actually transfer behaviors from one hand to another, despite the fact that that was never in the data. And so that was a really cool moment. I feel like I don't know if other people were as excited about it as I was when I shared it with some people, but it kind of shows this kind of emergent capability in these models that I hadn't seen before. And then, yeah, in terms of what I'd love to see, I don't know. I I love seeing robots do anything. And the... I guess, I think that there's still a lot of... A long way to push in terms of reliability for robots being able to do tasks for a really long periods of time. I don't necessarily think that much about individual tasks, but more so about capabilities and how to get the next capability from these models. The, yeah. And so I... Yeah. Anything, I guess the... Yeah. A robot doing anything always, gets me excited even if it's something that hasn't been done before. I guess one thing that we've been doing recently is, having robots, use knives to slice vegetables.
00:56:32 · Chelsea Finn, S1
I think that there's, a lot that you can do there once you actually can use knives safely, which is one thing that we've done recently. [00:56:40] S1: Okay. And then the last question, is how can someone break into robotics from a software engineering background? Great.