No Priors: Artificial Intelligence | Technology | Startups
00:05:19 · Cheng Chi, Tony Zhao, Sarah Guo
[00:05:19] S2: Mhmm. [00:05:19] S0: That you're just, like, moving in the free air, and the robot is moving with some delays. Mhmm. But ALOHA reduces that delay by a lot, and that contribute to the kind of smoothness and how fast human can react. Like, once we got those really dexterous data, what it allows us to do is to investigate on algorithms that are actually solving things that are difficult. In this case, it's through the introducing of transfer... Using transformers in the case of robotics. And there was a long period of time that I think robotics was stuck with three layer MLPs and ConvNets. And as you make it deeper, it works worse. But it turns out that once you have very strong and dextrous datasets, like, just throw a transformer at it, and it works quite well. [00:06:06] S1: Actually, like, just in terms of progress of the industry over time, transformers didn't make sense without certain level of data collection capability. Okay. [00:06:14] S0: And also other system around it, for example, action chunking, which is to predict a trajectory as opposed to predicting single samples of actions. All these things kind of combined to make textless task, bimanual tasks more scalable. [00:06:28] S1: Why is chunking important here if I think about, like, just the analogy to LMs and, like, text sequence prediction?
00:06:28 · Sarah Guo, Tony Zhao
[00:06:28] S1: Why is chunking important here if I think about, like, just the analogy to LMs and, like, text sequence prediction? [00:06:35] S0: I think it just kind of throws the model off if you're trying to force it to react every millisecond. That's not not not how human act. We perceive, and we can actually move quite a bit without looking at things again. And that turns out to make things... The motion a lot more consistent and our performance to be a lot better. [00:06:57] S1: And and you discovered that actually transformers architecturally did apply to robotics. Cheng, you felt then that data collection was still a problem, so enter UMI.
00:07:06 · Cheng Chi
[00:07:06] S2: Yeah. So after ALOHA and diffusion policy, I was super excited about imitation learning. But at the time, both of us started still doing teddy operation, and that just feels super limiting. I think the problem is that in your setup at a time like a teddy op setup, it takes a PhD student a couple of hours to set it up in the lab. They pretty much restrict data collection to a lab. But in order for a robot to actually work as a product, it needs to be worked in the wild, in unseen environments, and that requires data be also be collected in the wild. And at the time, I was thinking, okay, is there a way we can collect robotic data without actually using a robot? That forced me to think, okay, what's the actual most essential part of robotics data? And after different policy and act, actually, the paradigm is kinda simple. You just need paired observation and action data. In our case, observation is the video clip. The action is the movement of your hand plus how the finger moves. I realize that all of this information you can get from a GoPro. You can track the movement of GoPro in space, and you can track the motion motion of the gripper and also finger through images as well. And that's why I built this UMI gripper. It's three d printed. At the time, the project had three teacher students. We just took the grippers everywhere. I it was two weeks before the paper deadline. Every time I go to a restaurant, before the waiter would come in, we just collect some data. And very quickly, we got, I think, 1,500 video clips of this espresso cup serving task. And that turns out to be one of the biggest datasets in robotics and just simply by three people. And that's where the power kinda shines. And then with that amount of data, allow us to train the first end to end model that can actually generalize to unseen environments. So we can push the robot around in Stanford. Actually, Tony was there as well. You know, push the robot arm around the Stanford campus, and then anywhere,
00:09:04 · Tony Zhao, Cheng Chi, Sarah Guo
[00:09:04] S0: you know, the robot can serve you a drink. Yeah. I think that is the moment I was like, hey. Maybe we should start a company. This is actually working so well. I remember, like, just falling in tongue and But you have [00:09:16] S2: times that doesn't work well. [00:09:17] S0: Day. Yes. I think the only exception I saw was when it's under direct sunlight. Yeah. Right? And I think the reasoning was, like, over that whole, like, two, three weeks of data [00:09:26] S2: collection two weeks, it's all raining, so there's no sunlight data, so, like, it fails. That also demonstrates the importance of distribution matching. So in order for a robot to work in a sunny environment, it must have seen sunny environments during the training data. [00:09:39] S1: Yeah. It's really interesting because I remember when I first met you guys, it was like you spent like, I don't know, dollars 200,000 across all of your academic research. And yet the scale of data collection as translated to model capability is leading, right? So it's very interesting that, you know, we look at where we are, maybe going back to Tony's point of scaling and massive capital deployment. But that entire paradigm actually wasn't relevant before people realized, like, you should train on all of the internet data. And we just don't have that in robotics. So the entire field is just blocked on having any scale of data that's relevant.
00:09:39 · Sarah Guo, Tony Zhao
[00:09:39] S1: Yeah. It's really interesting because I remember when I first met you guys, it was like you spent like, I don't know, dollars 200,000 across all of your academic research. And yet the scale of data collection as translated to model capability is leading, right? So it's very interesting that, you know, we look at where we are, maybe going back to Tony's point of scaling and massive capital deployment. But that entire paradigm actually wasn't relevant before people realized, like, you should train on all of the internet data. And we just don't have that in robotics. So the entire field is just blocked on having any scale of data that's relevant. [00:10:16] S0: Yeah. I think these days, you're still, like, assuming debates about, like, what is even the right way to scale. There are, like, world models. There are simulations. There is teleoperation. There are, like, all these new ideas. And I think this is the the sort of area that we really want to innovate, that we want to differentiate, that we want to find out something that is both high quality and scalable. [00:10:38] S1: And then you guys... You decided to start a company pushing this cart around Stanford. Tell me about that decision, and congratulations on the launch and and sort of the the direction and team you've built.
00:10:38 · Sarah Guo, Tony Zhao
[00:10:38] S1: And then you guys... You decided to start a company pushing this cart around Stanford. Tell me about that decision, and congratulations on the launch and and sort of the the direction and team you've built. [00:10:48] S0: Yeah. It's a it's a very interesting journey. I remember in the beginning, it's actually two of us in Cheng's apartment. On his desk, we were, like, clamped a robot there and tried to do some tasks, and it soon becomes, like, I think, an eight person team towards the end of twenty twenty four. And now we're at around, like, 30 to 40 people. We're not the best at everything. Right? But starting a company allows us to find people who we really love working with and then bring all the expertise together from mechanical engineering, supply chain, like, software engineering, like, controls, and to build a system together that is not like a demo, but a real product. [00:11:27] S1: You've built this amazing team. What are people actually signing up for? What's the mission of Sunday? [00:11:31] S0: Yes. It is to put a home robot in everyone's home. I think there are a lot of AIs trying to make you more efficient during the work, but there is not enough AI that actually helps you with all these mundane things that are not creative, that really has nothing to do with what's making us more intensely human. What's ideal for people to spend more time on is actually with their hobbies, with their passions, as opposed to spending more time doing chores.