00:59:16 · Ademi Adeniji, Michael Cho
[00:59:16] S2: Maybe. Yeah. I don't know. I like to use the humans as like, to ground how I think about these things. Like... And so maybe, like, let's not think about unsupervised. Like, let's just take traditional supervised RL. You have a reward function, something you're trying to do. Like, let's say you're trying to, like, be really good at basketball or, like, maybe you're trying to perfect a free kick in soccer. Like like, there's there's maybe a few people that are very, very good at free kicks, like that can score every time or like close to every time. Right? But everyone knows how to pick the ball out of the back of the goal or, you know, walk, walk around, you know, like you may say you missed the goal. You can go walk, put it back on the, on the spot, try again. Right? So that reset that reset behavior is much, I would say, simpler in some sense or much more easy to learn than the very fine grained sort of, like, motor skill of, like, hitting the ball with the perfect angle, the perfect amount of force to get the curve right. So I think, like [01:00:10] S0: I see. [01:00:10] S2: Really what we're looking at is you wanna build a set of primitives that you can bootstrap on so that you can efficiently, like, reset and efficiently, like, be somewhat close to the the more complex behavior. And I think that's pretty I think that's pretty within scope. I think it's just how do you get those primitives and those inductive biases to begin with.
01:00:10 · Ademi Adeniji, Vincent Liu, Michael Cho
[01:00:10] S2: Really what we're looking at is you wanna build a set of primitives that you can bootstrap on so that you can efficiently, like, reset and efficiently, like, be somewhat close to the the more complex behavior. And I think that's pretty I think that's pretty within scope. I think it's just how do you get those primitives and those inductive biases to begin with. [01:00:28] S1: But you'd you'd also want, like, a, like, reasonably good policy for... In the freak example, right, to to like start off of. And so I guess you're still at the point you still go back to the fact that like, okay, if you can imitation learning your way or however however they do it these days, your way to like something good, then you sort of start from there. And then that that itself is maybe a chicken and egg as well. [01:00:49] S0: Yeah. Well, very fascinating stuff. Okay. Cool. I I think, you know, it's... Maybe we should do do a wrap up. Is there anything, like, you know, final one or two points that you guys wanna say? Like, you know, what is some of the biggest lesson that you think you guys have learned? I think you shared a couple, but if you can summarize for us. And, also, maybe if you... This is a common question we like to ask you. If you, let's say, have... You guys have another three months to work on this paper, like, what what would be something that you want to have done, like, to squeeze it into this paper?
01:01:21 · Vincent Liu
[01:01:21] S1: Oh, should I go first? Oh, yeah. Sure. Okay. And then maybe you could add stuff after I'm... After I wrap up. So I guess I'll start with the second question, which is what will we do in three more months? We had actually planned to sort of explore, like we mentioned, like like a VLA style training. But I guess the idea being, like, given what we've seen about the trade offs of using three d points, is there a way we can extract the benefits of that into, like, something like a VLM or like an LLM, which has a very strong reasoning prior, and maybe get like some generalist behavior. I think that's very interesting. And I think we've already seen a lot of work come out in the last month or so, pursuing this direction. In terms of like, you know, the one to two points, you know, I'll probably say something a little oversimplified, but I think it's sort of shaped by my sort of research in other domains more than robotics and also some time at Tesla, which is I feel like, you know, you basically for deep learning systems to really work in the real world, they've only really worked when the test set is basically the same domain as the training set. And so I kind of have this extreme perspective that you should start with that sort of hypothesis and then ask yourself, how can I basically collect data from every single, like, inference time scenario that I care about? And then you run massive self supervised learning or behavior cloning at scale, which is kind of what Tesla does. And in robotics, you have this morphology gap issue if you're doing this from the human perspective, but I feel like, you know, those algorithm... Algorithmic things will sort of fall away with time. And so I don't know. I've kind of maybe a little pessimistic view on on research where if you just do this at scale, at the the entire global scale, you will get very far to the point where maybe you even solve it.
01:03:07 · Ademi Adeniji
[01:03:07] S2: Yeah, I think, I think, yeah, mostly agree. I think like one thing that's interesting to me from this project is sort of, I think the learning, I guess, from a lot of the pain of some of the things we tried that didn't work is that ergonomics really matter. And a lot of times, simplicity is sort of the way to go. We tried a lot of things as far as getting motion tracking working, more mocap like solutions with gloves. We've tried chest mounted iPhone camera. We've tried real senses, depth estimation methods. A lot of these more bottom up approaches to getting something like this working and already at that level of scale, which is research level scale, right? Like we're not trying to get any like robust system marking. It's like, we just want to basically be able to solve a task. And your system has so many points of failure that even getting enough data to train something for a research paper becomes a bottleneck. It becomes a headwind. So I'm thinking how the learning here was adopt something much more simple and scalable, even at a research level. Like obviously we want, we care about ideas in research, but I think ideas, a lot of the really important ideas are very simple and kind of like make somewhat dumb assumptions at first, but then we sort of think about the assumptions like, oh, Hey, maybe that's not so dumb. Right. So I think simplicity. I think one thing that I'm really excited about moving forward that I would definitely do if I had more time is one. Yes. Totally. I think exploring the nexus of three d representations and all of their really powerful invariances with internet scale models like VLAs is a great direction that many people are already taking. I think the other thing that I think would be great to see more work in is actually like, there's a ton of really good hands coming out that are like really close to the human hand in terms of like size, let's say. And yeah, they're still expensive, I think largely, but it seems like they're becoming more available to the point that we should really revisit a lot of the kinematic retargeting that we've been doing. Let's say doing a lot of hand tuning because the hand is super big, Like if you're using an Allegro or a leap hand. And like, let's see how well things work with like a hand that's like more similar to the human hand.
01:05:29 · Ademi Adeniji, Chris Paxton, Michael Cho, Vincent Liu
There's a project out of Wells group called Ruka, which basically like a few students basically built an anthropomorphic like hand. That's like pretty close to the average hand size of a human and showed like, you you can do these, like it's tendon driven. So you have to be from learning to actually do like your inverse kinematics. But I think like just having something more one to one opens up so many more possibilities from learning from humans and doing things much more, in much more with much more dexterity. And so I think there's ongoing works in Lerrel group. And I think, like, you know, I think maybe we should really try and, like, see how far we can get with dexterity with better hardware. [01:06:12] S3: If you haven't seen it, you should check out the Sharpa hand, which should be [01:06:15] S2: on sale this year. [01:06:16] S3: 35 to 50 k. That thing is amazing. A really good piece of hardware. [01:06:21] S2: Exactly. [01:06:21] S0: Yeah. It was stealing cuts away. Yeah. It was stealing cuts. Was crazy. Yeah. It's [01:06:25] S1: pretty insane. Okay. [01:06:29] S0: Cool. So I I think maybe one final final question for you guys. Like, is there any papers that you come across recently that, you know, you... That kinda blew your mind or you wanna give a shout out to so that people listening to this can also maybe read those papers later? [01:06:42] S3: Or so we can invite them on later. [01:06:44] S2: Mhmm. I mean, you mentioned Seohong. I think I think you mentioned Seohong. [01:06:47] S0: Yeah. Yeah. That's right. Yeah. About, like, effective Horizon reduction. Yep.
01:06:47 · Michael Cho, Ademi Adeniji, Vincent Liu, Chris Paxton
[01:06:47] S0: Yeah. Yeah. That's right. Yeah. About, like, effective Horizon reduction. Yep. [01:06:51] S2: Horizon reduction. Yeah. I think I think his line of work is super compelling in this direction, like, really doing a thorough study of what are the limitations of RL and revisiting these things in a really principled way. It's a ton of experiments and really good theory. I'm really looking forward to having him on if that's coming up. [01:07:10] S1: Yeah. Finso? Yeah. Not so much for me, least not off the top of my head. Sorry. Yeah. [01:07:14] S0: It's all [01:07:15] S3: good. All good. Yeah. [01:07:16] S0: Yeah. It's all good, man. Anyway, you know, thank you again. You know, thank you guys for coming up. [01:07:20] S1: Yeah. Thanks for having [01:07:21] S2: the discussion. Yeah. [01:07:22] S0: Yeah. That's fine. Yeah. It's definitely a great discussion. Alright, Chris. What did you like? [01:07:30] S3: Yeah. I really... I thought it was really interesting work. So we learned about how you could just use your hands to give demonstrations for robots. Seems like a really nice scalable way of getting robot data collection to work. Yeah. Lots of... It's nice about... To hear about how you need to, like, keep it simple, make things work well. Yeah. Make things work well for users. What did you think? Yeah.