RoboPapers
00:45:23 · Michael Cho, Jiageng Mao
[00:45:23] S0: Good request. Maybe Jiageng as well? [00:45:28] S5: Yeah. I actually believe in the power of in the wild teachings because it's very hard to scale up the physical robot. So basically, every of the teleoperation tools, they need a physical robot with a human. And so you have to... The manufacturing of the physical robot is limited by the manufacturers, but the human's labors, it can be cross housed or they can be more scalable than those utility operators' physical robots. So I'm more interested in how do you apply those in the wild. One thing is... One dimension is how do you apply those in the wild data, especially you're using, like, egocentric manipulation, human manipulation videos to train your humanoid robots. So that is one thing I'm thinking about. And another thing is... So how do you design exoskeletons? Just like a UMI's, but it's UMI for humanoid robots. So they basically have arms and dexterous hand, exoskeletons about dexterous hands, also have cameras as an egocentric view. Humans can use those to collect data and just use those data to train humanoid robots. So in method, they will have a physical robot to teleoperate just using humans to test the data. I I believe that will be a more scalable ways, and I think we will spend more time in the future on these topics. Yeah.
00:47:03 · Michael Cho
[00:47:03] S0: Right. Perfect. Yeah. Maybe I I guess... I think we we are about to wrap up here. Maybe as a as a typical last question, Chris and I like to ask this. Like, is is is there any cool paper you guys came across recently you wanna give a shout out to? Doesn't have to be related to Humanoid Everyday. It could be computer vision or whatever. Yeah. In case there are some gems that me and Chris or the the listeners missed it, they can go and read up the... That paper. Yeah. Maybe, Yue Wang, you can... You can go first.
00:47:33 · Yue Wang, Michael Cho
[00:47:33] S1: Yeah. So as you... As we mentioned at the beginning, the Sonic paper from NVIDIA is a good one. And I guess it's pretty similar to the Twist two paper in terms of the device they use. They use a pickle two, I guess, to teleoperate the robot. That will unlock a lot of capabilities. And the other thing, as I mentioned at the beginning, which is another paper, is, Unitree's Teleoperation System, the... Which was released last week. That was very impressive because, I think they use, some motion tracking system, to do multi person tracking, and then they do, multi person, like a tidal operation with mocap system. So that one is also quite powerful. So I'm imagining in the future, if we can have a MoCap system that is mounted in a very large space on the ceiling and that can track multiple persons, then we can probably do some multi robot teleoperation to perform a even harder task. So that's... That one is very impressive. Yeah. Those are some interesting papers. And the other interesting paper is, I just want to give a shout to, Jiageng. So he released a paper called the phase word, which is trying to leverage some video generation model to to generate some demonstrations and then transfer those demonstrations from a general video domain to the robotic domain, which is a very interesting one from my perspective. [00:49:00] S0: What... What's it called? It's called the What's it called? [00:49:02] S1: Face Face word. Okay. Cool. [00:49:04] S0: Cool. Cool. Alright.
00:49:04 · Michael Cho, Jiageng Mao, Chris Paxton
[00:49:04] S0: Cool. Cool. Alright. [00:49:05] S5: So we also have some... Basically, we are are doing zero shot robotic manipulations. That means we do robot manipulation without any robotic data for training. And so where data is coming from is coming from a view generator. So basically, you are given an image and a task prompt and using view generators to generate the process of how the task was completed. But it is only from the pixel space. The instructions are only from pixel space. And our work is we're trying to simultaneously reconstructing the underlying, interactable scenes from the image and videos. And on top of these scenes, we can do the reinforcement learning to track the signals from the generated videos so that we can do the zero shot manipulations. And also, it's very generalizable without any robot data for training. This is the basic idea of this paper. And we also have a very cool, interactive demo that's running on the web based modules. So you can just click on the interactive demos, and you can play with that. I think it's a very cool demos. [00:50:14] S0: Yeah. I'm looking at the website. It looks super cool, man. [00:50:16] S6: Yeah. Well, let's just take a look. Maybe we'll [00:50:18] S0: Cool. Cool. [00:50:19] S6: Come back sometime. [00:50:20] S0: Yeah. Yeah. Maybe about about Xiawei Hongyi. Any any cool people or answer you?
00:50:20 · Michael Cho, Hongyi Jing, Chris Paxton, Zhenyu Zhao
[00:50:20] S0: Yeah. Yeah. Maybe about about Xiawei Hongyi. Any any cool people or answer you? [00:50:26] S3: Yeah. So so, yeah, like like we had mentioned, I think for the data teleoperation pipeline, I think this two is very impressive. And I think it's it's, you know, control of the whole... Like, mimicking the human body movement. I think it's really cool, and we wanna do... Maybe do some improvements on it in the future, like introducing some dexterous manipulation, wearing some, you know, manuscript, gloves, or something like that to do the... Some dexterous manipulation based on this whole body data collection pipeline. And, also, I think I think another interesting paper for me is the, like, the heater, you know, the u... The unit three g one playing ping pong. And I think that's also another impressive work for me and, like, you know, that the humanoids doing sports. And I think we also want to do some fancy things like... Fancy sports like that using GY in the future. Yeah. [00:51:19] S6: That was actually the last episode we published. [00:51:21] S0: Yeah. We just published. He was here, yeah, last month. Was a great episode. [00:51:25] S6: That was a Yeah. [00:51:26] S0: That was super cool. I guess, Xiawei and and Zhenyu? [00:51:29] S2: Like, the twist tube the twist tube paper thing. Yeah. One of the collaborators is also in our lab, Hong. The... Yeah. I think it'd be cool if you [00:51:38] S0: Cool. Cool. Cool. And then Xiawei [00:51:40] S6: everything okay where you are? It's been, like, nonstop sirens. [00:51:45] S2: Oh, sorry. Sorry about that.
00:51:45 · Zhenyu Zhao, Xiawei Liu, Chris Paxton, Michael Cho
[00:51:45] S2: Oh, sorry. Sorry about that. [00:51:47] S4: No. No worries. Yeah. [00:51:48] S6: It's good. It's [00:51:50] S4: Yeah. So for me, I also read the, hitter, the the ping pong one paper. I thought that was really interesting to me because, but I, as I remember, I think they used, like, a third view camera for calculating the, the the... Where where the ping pong ball would land. Yeah. I thought that was, a very smart approach, and I thought that... And me and Hongyi and Zhenyu also discussed that maybe in the future, we might do... We we are planning on doing something that's, you know, more egocentric based so that we don't need a third view camera, and we would like to do some interactive, tasks or, a multi robot task with, egocentric, camera. So that would definitely leverage the difficulty, but it would be really impressive if we are able to do that. Yeah. [00:52:50] S0: Well, hopefully, the g two, if it if it comes out, we will have a actuated neck. It will make things a lot easier to do ecocentric. It's a lot harder if you... If your neck can't move. But anyway, mean wider field [00:53:01] S6: of your camera is honestly like anything you could do. [00:53:04] S0: Yeah. Like, put a, yeah, put a three sixty. [00:53:06] S6: That's stuff you could do better. Really. Yeah.