The People's AI: The Decentralized AI Podcast
00:06:44 · Jeff Wilser
[00:06:44] S0: Am I am I right that rules based approach, is that analogous to before kind of LLMs and the transformer architecture in Gen AI, the idea was we will label. Right? We'll say we'll say, hey, this is a cat. This is a dog. And so then they realize, okay, the kind of transformer breakthrough, realize you don't have to do that. You can just feed a shitload of text and data, and the transformer will kind of figure it out and make these connections on its own. Is it something similar happening in robotics that that that instead of having all of these kind of rules based logic that there's more elegant way to to get these robots to move their their hands and claws?
00:07:28 · Bayley Wang
[00:07:28] S1: Yeah. So modern rules based approaches are a little more sophisticated than the ones from like the early mid twenty tens because we do have LLMs and BLMs now, and they can pick up a little bit of the fuzziness fuzziness. Right? So if you want a robot to clean a kitchen, you can use the LLM to go plan out a set of actions and then use a more conventional approach to find the objects in the scene that those actions might correspond to and have the robot go like move its arms to the coordinate of the cup and then move it to the dishwasher. But the actual like planning was done by an LLM. So it does account to account for some level of fuzziness. But still fundamentally, this process is a rules based approach. The humans are writing rules and making assumptions about what the environment will look like. And those assumptions tend to break down. It's one of those unfortunate things that even the best teams of developers are nowhere near omniscient, so they can't plan for everything. And that's why the LLMs are so powerful. Right? No one writes the rules for the LLMs. They see a cross section consisting of, like, tens of percent of the entire Internet. And by doing so, they're able to capture a little bit of everything. So while LLMs sometimes hallucinate, they very rarely are horrifically incorrect. Like, they generate semantically and syntactically correct English. They generate plausible outputs. It's sort of like the difference in robotics between the robot picking up the dish and like throwing it on the ground versus the robot say picking up the wrong dish and putting it in the wrong container. Right? Like the the first one is not very good, but the second one you can at least sort of like put up with. Or in language modeling terms, it's the difference between getting the wrong answer versus getting something which is just a string of words that don't even make any sense next to each other.
00:09:14 · Jeff Wilser, Bayley Wang
[00:09:14] S0: I'd love to get a little more into your personal background and kinda what brought you to PrismaX. And I think that might give listeners a good sense of where are the particular pain points in robotics and and how data fits into this. Tell us a little more about your journey from MIT to what you're building now. [00:09:32] S1: Yeah. Yeah. So I was a robotics researcher at MIT, and I was part of one of the teams, the mini Cheetah team, that helped solve this hardware problem by making the actuators a heck of a lot cheaper and easier to get. But when it came time to decide whether we took that robot and we made a commercial spin off or we just, like, let it rest and went went back to doing doing research and doing other things. We made the very active decision to not try to sell the mini cheetah. The MIT robotics team is probably one of the only great academic robotics teams without a commercial spin off. And the reason we did that is because we we couldn't figure out how to how to get the robot to do anything useful. [00:10:11] S0: For those who are not familiar, what was the mini cheetah and talk about kinda what it did and and what it Yeah. Yes. Yeah.
00:10:11 · Jeff Wilser, Bayley Wang
[00:10:11] S0: For those who are not familiar, what was the mini cheetah and talk about kinda what it did and and what it Yeah. Yes. Yeah. [00:10:16] S1: So you've probably seen these robot dogs that have been running around. The mini cheetah was the was the first, like, commercially manufacturable robot dog. It was built using advanced, what they call quasi direct drive actuators. So high force, but also very, easy to measure, like, what force they're they're exerting. And it it was built using mass production techniques using parts off of, like, drone drone, drone supply chains. So it was a little little dog about the size of, like, a... I don't know. About about two feet long. And it could do all kinds of things. It could run. It could jump. It could do a backflip. But as I was saying, when it came time to figure out what this machine could actually do in the real world, all we could think of was very expensive robotic pet for rich people. And we floated we floated the idea of, like, putting it on Kickstarter for, several thousand dollars, but decided it just wasn't a scalable market. And and in the end, we were right. There were... There are a lot of companies in this crowded space building robot dogs, but they're just being marketed as as toys right now because no one can figure out the the the software to make them do it. [00:11:23] S0: In in fairness, I guess some of these designer lap dogs, you know, if you if you if you're wanting to buy a particular breed of, like, hypoallergenic dogs, you do spend, $4,000. So I guess, like, hey. Is it cheaper than Yeah.
00:11:23 · Jeff Wilser, Bayley Wang
[00:11:23] S0: In in fairness, I guess some of these designer lap dogs, you know, if you if you if you're wanting to buy a particular breed of, like, hypoallergenic dogs, you do spend, $4,000. So I guess, like, hey. Is it cheaper than Yeah. [00:11:36] S1: Yeah. There there is there is a [00:11:38] S0: require dog food. [00:11:39] S1: There is a small market that I would say is about the same size as people buy $5,000 hypoallergenic dogs, but that's also not a like venture scale market. Yeah, is Venture fund your dog. And you'd be the [00:11:51] S0: one North Star for an addressable universe. Yeah. Yeah. Okay. So you're just like, alright. This is a fairly hyper niche thing. We can probably find a... Or hopefully find a a more scalable, better use case and and and better kind of, like, type of robot that does more than just, like, freaking backflips for the hyper wealthy.
00:11:51 · Jeff Wilser, Bayley Wang
[00:11:51] S0: one North Star for an addressable universe. Yeah. Yeah. Okay. So you're just like, alright. This is a fairly hyper niche thing. We can probably find a... Or hopefully find a a more scalable, better use case and and and better kind of, like, type of robot that does more than just, like, freaking backflips for the hyper wealthy. [00:12:16] S1: Yeah. Yeah. Yeah. So after... Right. After the after that, we, just sort of went off and did did our own things. I went and did a... Did, like, worked on a company doing displays and optics. But robotics has always been top of my mind and things that I'm interested in and I... Things that I think are really high impact. So, you know, a couple years ago, g... GPT three showed the world that large scale unsupervised training on huge datasets was incredibly powerful, and the resulting emergent behavior could do crazy things. So everyone everyone started thinking about robotics again and how the same types of software and same types of workflow could make robotics better. But when you come and do robotics, you immediately run into a data problem, like a a data problem, a scaling problem, where if you wanna train a model which handles text, you go download the whole Internet and you go do your training. When you wanna train a robotics model, step one is to collect data and then you like stop because it's really annoying to collect data. It requires hiring hundreds or thousands of people to do the data collection to really do anything meaningful.