RoboPapers
00:47:33 · Yue Wang, Michael Cho
[00:47:33] S1: Yeah. So as you... As we mentioned at the beginning, the Sonic paper from NVIDIA is a good one. And I guess it's pretty similar to the Twist two paper in terms of the device they use. They use a pickle two, I guess, to teleoperate the robot. That will unlock a lot of capabilities. And the other thing, as I mentioned at the beginning, which is another paper, is, Unitree's Teleoperation System, the... Which was released last week. That was very impressive because, I think they use, some motion tracking system, to do multi person tracking, and then they do, multi person, like a tidal operation with mocap system. So that one is also quite powerful. So I'm imagining in the future, if we can have a MoCap system that is mounted in a very large space on the ceiling and that can track multiple persons, then we can probably do some multi robot teleoperation to perform a even harder task. So that's... That one is very impressive. Yeah. Those are some interesting papers. And the other interesting paper is, I just want to give a shout to, Jiageng. So he released a paper called the phase word, which is trying to leverage some video generation model to to generate some demonstrations and then transfer those demonstrations from a general video domain to the robotic domain, which is a very interesting one from my perspective. [00:49:00] S0: What... What's it called? It's called the What's it called? [00:49:02] S1: Face Face word. Okay. Cool. [00:49:04] S0: Cool. Cool. Alright.
00:49:04 · Michael Cho, Jiageng Mao, Chris Paxton
[00:49:04] S0: Cool. Cool. Alright. [00:49:05] S5: So we also have some... Basically, we are are doing zero shot robotic manipulations. That means we do robot manipulation without any robotic data for training. And so where data is coming from is coming from a view generator. So basically, you are given an image and a task prompt and using view generators to generate the process of how the task was completed. But it is only from the pixel space. The instructions are only from pixel space. And our work is we're trying to simultaneously reconstructing the underlying, interactable scenes from the image and videos. And on top of these scenes, we can do the reinforcement learning to track the signals from the generated videos so that we can do the zero shot manipulations. And also, it's very generalizable without any robot data for training. This is the basic idea of this paper. And we also have a very cool, interactive demo that's running on the web based modules. So you can just click on the interactive demos, and you can play with that. I think it's a very cool demos. [00:50:14] S0: Yeah. I'm looking at the website. It looks super cool, man. [00:50:16] S6: Yeah. Well, let's just take a look. Maybe we'll [00:50:18] S0: Cool. Cool. [00:50:19] S6: Come back sometime. [00:50:20] S0: Yeah. Yeah. Maybe about about Xiawei Hongyi. Any any cool people or answer you?
00:50:20 · Michael Cho, Hongyi Jing, Chris Paxton, Zhenyu Zhao
[00:50:20] S0: Yeah. Yeah. Maybe about about Xiawei Hongyi. Any any cool people or answer you? [00:50:26] S3: Yeah. So so, yeah, like like we had mentioned, I think for the data teleoperation pipeline, I think this two is very impressive. And I think it's it's, you know, control of the whole... Like, mimicking the human body movement. I think it's really cool, and we wanna do... Maybe do some improvements on it in the future, like introducing some dexterous manipulation, wearing some, you know, manuscript, gloves, or something like that to do the... Some dexterous manipulation based on this whole body data collection pipeline. And, also, I think I think another interesting paper for me is the, like, the heater, you know, the u... The unit three g one playing ping pong. And I think that's also another impressive work for me and, like, you know, that the humanoids doing sports. And I think we also want to do some fancy things like... Fancy sports like that using GY in the future. Yeah. [00:51:19] S6: That was actually the last episode we published. [00:51:21] S0: Yeah. We just published. He was here, yeah, last month. Was a great episode. [00:51:25] S6: That was a Yeah. [00:51:26] S0: That was super cool. I guess, Xiawei and and Zhenyu? [00:51:29] S2: Like, the twist tube the twist tube paper thing. Yeah. One of the collaborators is also in our lab, Hong. The... Yeah. I think it'd be cool if you [00:51:38] S0: Cool. Cool. Cool. And then Xiawei [00:51:40] S6: everything okay where you are? It's been, like, nonstop sirens. [00:51:45] S2: Oh, sorry. Sorry about that.
00:51:45 · Zhenyu Zhao, Xiawei Liu, Chris Paxton, Michael Cho
[00:51:45] S2: Oh, sorry. Sorry about that. [00:51:47] S4: No. No worries. Yeah. [00:51:48] S6: It's good. It's [00:51:50] S4: Yeah. So for me, I also read the, hitter, the the ping pong one paper. I thought that was really interesting to me because, but I, as I remember, I think they used, like, a third view camera for calculating the, the the... Where where the ping pong ball would land. Yeah. I thought that was, a very smart approach, and I thought that... And me and Hongyi and Zhenyu also discussed that maybe in the future, we might do... We we are planning on doing something that's, you know, more egocentric based so that we don't need a third view camera, and we would like to do some interactive, tasks or, a multi robot task with, egocentric, camera. So that would definitely leverage the difficulty, but it would be really impressive if we are able to do that. Yeah. [00:52:50] S0: Well, hopefully, the g two, if it if it comes out, we will have a actuated neck. It will make things a lot easier to do ecocentric. It's a lot harder if you... If your neck can't move. But anyway, mean wider field [00:53:01] S6: of your camera is honestly like anything you could do. [00:53:04] S0: Yeah. Like, put a, yeah, put a three sixty. [00:53:06] S6: That's stuff you could do better. Really. Yeah.
00:53:06 · Chris Paxton, Michael Cho, Yue Wang
[00:53:06] S6: That's stuff you could do better. Really. Yeah. [00:53:09] S0: But... Yeah. Anyway, yeah, thank you... Well, thank you so much for coming on, guys. And, also, I I just wanna give, you know, you guys credit for putting so much work. I know it's incredibly, incredibly, incredibly hard to even get this, like, amount of data out there. I was, like, a bit shocked, to be honest. I was like, oh my gosh. How many how many robots are you bringing in the process to get it? But thanks thanks for putting out there. I think the community will benefit lot. And, clearly, you guys did the did ablation study. It will move the needle. So, yeah, thank thank you so much for the work. Yeah. Yeah. Looking look looking forward to more more papers from you guys. [00:53:44] S1: Thank you so much for giving us the opportunity to share the [00:53:47] S0: For sure. Okay. Cool. See you guys soon. Ciao.