No Priors: Artificial Intelligence | Technology | Startups
00:01:15 · Tony Zhao, Cheng Chi, Sarah Guo
[00:01:15] S0: I would say I think we're kind of in between the GPT moment and the ChatGPT moment. Like, in the context of LMs, what it means is that it seems like we have a recipe that can be scaled, but we haven't scaled it up yet. And we haven't scaled it up so much so that we can have a great consumer product out of it. So this is where I mean, like, GBT, which is like a technology, and chat GBT, which is a product. [00:01:41] S2: Yeah. And so we're seeing across academia, there's consensus around what's the method for manipulation, but everybody's talking about scaling up. It's like we know there's sign of life for the algorithms people are picking, but people don't know if we have more data, like what happened to g p t two, g p three, what will happen. And... But we see a clear trend that, you know, there's no reason to believe that robotic doesn't follow the trajectory of other AI fields that, you know, scaling up is gonna improve fullness. [00:02:11] S1: Maybe even if you took a step back, like, what was the process for deploying a robot into the world, like, ten years ago, like, preset of generalizable AI algorithms? Like, why why was it so slow as a field?
00:02:11 · Sarah Guo, Cheng Chi
[00:02:11] S1: Maybe even if you took a step back, like, what was the process for deploying a robot into the world, like, ten years ago, like, preset of generalizable AI algorithms? Like, why why was it so slow as a field? [00:02:24] S2: Yeah. So previously, you know, classical robotics have this sense, plan, act modular approach where there's a human design and interface between each of the modules, and those are need to be designed for each specific task and each specific environment. In academia, that means for every task, that means a paper. So a paper is you design a task, design an environment, and you design interfaces, and then you produce engineering work for that specific task. But once you move on to the next task, you throw in all your code, all your work, and you start over again. And that's also kinda what happened to industry. And so for each application, people build a very specific software and hardware system around it, but it's not really generalizable. And therefore, it's feel like we're just running in loops. We build one system and then we build the next one, but there's no synergy between them. And as a result, the progress has been somewhat slow. [00:03:13] S1: I feel like that's a good segue into some of the amazing research work that you guys have contributed over the last five years to the field. Should we start with diffusion policy? What was the impact of that?
00:03:13 · Sarah Guo, Cheng Chi
[00:03:13] S1: I feel like that's a good segue into some of the amazing research work that you guys have contributed over the last five years to the field. Should we start with diffusion policy? What was the impact of that? [00:03:23] S2: Yeah. So diffusion policy is a specific algorithm for a paradigm called imitation learning. That's really the most intuitive way of how to use machine learning for robotics. So you collect paired data, action and observation of what the robot should do, you use that to train a model with supervised learning, and then the robot do the same thing. The problem is that in the field, it's known to be very finicky. So when it comes to researchers, when I start into the field, people are like, the researcher themselves, the specific researcher, need to collect the data so that there's exactly one way to do everything. Otherwise, the robot... Either the model training will diverge or the robot will behave some weird way. And diffusion model really allows us to capture multiple modes of behavior for the same observation in a way that still preserves training stability, and that really kinda unlocked more scalable training and more scalable data collection. [00:04:17] S1: So it doesn't have to be you personally wearing, you know, a tele op set in order to make a robot learn. [00:04:23] S2: Yep. Yep. So, like, we can have multiple people, sometimes even untrained people, collecting data, and the result will still be great. [00:04:29] S1: Where do ALOHA and ACT play into
00:04:29 · Sarah Guo, Tony Zhao, Cheng Chi
[00:04:29] S1: Where do ALOHA and ACT play into [00:04:32] S0: this? Yeah. So these two papers are actually, like, super close to each other. They're, like, one month or two months away. That's actually how me and Cheng know each other. It was about looking at each other's paper, like... And we met on Twitter, I think, when Cheng's back in Colombia. Before ALOHA, I think the typical way people collect data is with a, like, polyoption setup with VR headsets. And it turns out to be very unintuitive to do, and it's hard to collect data that is actually dexterous. What ALOHA brings is a very simple and reproducible setup, so it's very intuitive. [00:05:04] S1: Sorry. In terms of just for most people who haven't worn a teleop setup, is it the lag? Is it, like, just, you know... How how should I compare it to, like, playing a video game or something? [00:05:14] S0: Yeah. I think ALOHA make it feel more like playing a video game. Normally, it feels kinda disconnected [00:05:19] S2: Mhmm.
00:05:19 · Cheng Chi, Tony Zhao, Sarah Guo
[00:05:19] S2: Mhmm. [00:05:19] S0: That you're just, like, moving in the free air, and the robot is moving with some delays. Mhmm. But ALOHA reduces that delay by a lot, and that contribute to the kind of smoothness and how fast human can react. Like, once we got those really dexterous data, what it allows us to do is to investigate on algorithms that are actually solving things that are difficult. In this case, it's through the introducing of transfer... Using transformers in the case of robotics. And there was a long period of time that I think robotics was stuck with three layer MLPs and ConvNets. And as you make it deeper, it works worse. But it turns out that once you have very strong and dextrous datasets, like, just throw a transformer at it, and it works quite well. [00:06:06] S1: Actually, like, just in terms of progress of the industry over time, transformers didn't make sense without certain level of data collection capability. Okay. [00:06:14] S0: And also other system around it, for example, action chunking, which is to predict a trajectory as opposed to predicting single samples of actions. All these things kind of combined to make textless task, bimanual tasks more scalable. [00:06:28] S1: Why is chunking important here if I think about, like, just the analogy to LMs and, like, text sequence prediction?
00:06:28 · Sarah Guo, Tony Zhao
[00:06:28] S1: Why is chunking important here if I think about, like, just the analogy to LMs and, like, text sequence prediction? [00:06:35] S0: I think it just kind of throws the model off if you're trying to force it to react every millisecond. That's not not not how human act. We perceive, and we can actually move quite a bit without looking at things again. And that turns out to make things... The motion a lot more consistent and our performance to be a lot better. [00:06:57] S1: And and you discovered that actually transformers architecturally did apply to robotics. Cheng, you felt then that data collection was still a problem, so enter UMI.