The People's AI: The Decentralized AI Podcast
00:16:40 · Jeff Wilser
[00:16:40] S0: So tell us about what brought you to and what kinda led to PrismaX your and light bulb of, okay, given the challenges of gathering data in the real world, we need some other solution.
00:16:56 · Bayley Wang
[00:16:56] S1: Yeah. So, fundamentally, what inspired PrismaX is this idea that diverse large datasets and diverse large operator, like, environments are what powers these models and makes them robust. If you if you... Like, a lot of the decentralized data projects are saying like, oh, we can provide data for cheaper. But for robotics, it's about providing the data that works. And I think that's a unique opportunity where the decentralized project has very strong potential to be the state of the art and, like, and beat out all of the centralized projects. Right? Because if if I'm a centralized project and even if I had the money to go hire a million people to collect data for me, I I still don't know who to hire. Like, I need to select the, like, split of million people that I think best represents the world. And I would argue that you will always wind up with, managers bias in a situation like this. It's it's like very subconscious in your mind that people will plan for the things that they're familiar with and the things that they find important. Right? And this is a problem. You wind up with a model and an AI and a robot which is tailored to the preferences of like a committee of 20, 30 people who maybe they tried their best to plan, but in the end it was not not successful. If you let the community involve organically, you wind up with a sampling of the world which much more closely corresponds to what people care about in the real world, and that gives you robustness. It's emergent behavior. Right? Decentralized networks have emergent behavior just like models trained on large scale data, And I I I think that's important. It gets rid of as many rules as possible and lets it lets it evolve according to like a very simple economic target of like this task has value or this task is one that people do more frequently. And that's what led that's what led to this idea that by using decentralization and by using protocols to coordinate millions of people, you could scale this... You could scale robotics up much more quickly than you would be able to with any sort of centralized project. And the rest of it just fits together.
00:19:00 · Jeff Wilser
[00:19:00] S0: That makes complete sense in theory. I love that that framework. Talk about what it looks like in kind of... In reality or kinda be concrete. So it's... As I understand it, it's a robotics data DAO. What does it look like? What are some examples of how people are are feeding, data to the system and getting compensated for it?
00:19:20 · Bayley Wang
[00:19:20] S1: Yeah. Yeah. So we we work with a few data modalities. So fundamentally, Prisma PrismaX addresses the three core needs of robotics, data, teleoperators, models. And what that means is that you can... People who are participating can either submit, more passive environmental videos of themselves completing tasks, like, the way down to just here are the videos on my phone and the network will... The protocol will discover which of these videos are important, which of these videos are useful, and provide rewards for them. Up to something like, here's a first person view video of myself completing some tasks either requested by a customer or just passively, all the way to, you can... People on the network can use the protocol to go remotely operate robots and that collects probably the highest quality modality of data. But you see there's like barriers to adoption. Right? You can either be a very passive reward hunter, just upload your videos. Some of it will be useful, get some rewards. All the way to like I'm putting in real work and like if when you operate a robot that that's not just like accumulate network tokens for later. That's like a business which can reward, much more much more valuable immediate incentives. Like right now, like robotics companies are paying $50 an hour for this kind of operator. So there's like a direct value translation for for guild operators, for robot operators to be able to immediately engage with the industry and, like, just make make a business, make a money out of... Make money out of it.
00:20:50 · Jeff Wilser, Bayley Wang
[00:20:50] S0: I heard about one concrete task where users are submitting videos of themselves like folding sheets. You know? Can you talk a little bit? Yeah. Yeah. Is that an accurate example? What are what are things people are doing? And then you kinda connect the dots of how that kind of video they're filming can eventually feed into robotics in action. [00:21:13] S1: Yeah. So the sheet the sheet folding the sheet folding project, we're doing it we're doing it because domestic robot... Robotics is a field which is just very interesting in the robotics industry right now. There are a lot of companies that are trying to tackle this. It aligns well with, you know, hospitality use cases, hotel use cases where there's like a pretty clean GTM for the robotics. And it's something which everyone has. Like, everyone has towels or sheets they can fold. So it's very accessible for the... [00:21:44] S0: It's also low stakes. Right? If the robot Yeah. Yeah. Screws up the sheet folding, no one's going to die. Right? Unlike self driving cars where if they make a mistake, oops, pedestrian is killed. So [00:21:55] S1: Yeah. Exactly. Yeah. Exactly. [00:21:56] S0: It's a great low hanging fruit. I agree.
00:21:58 · Bayley Wang
[00:21:58] S1: Yeah. So then then looking at how this exactly works, the idea is that if you had, say, ten thousand hours of videos folding sheets... Of people folding sheets, you get, a lot of diversity to be able to address like all kinds of environments. But at the same time, the task is repetitive enough that it's easy for the AI model to pick up on how to do it. Right? It's it's similar to like if if you if you watch a hundred hours of videos of someone doing the same task over and over again, you're... It's much easier to learn that task than if you went and watched a hundred hours of random tasks, you would learn basically nothing. And the exact way this works is not to get, like, too deeply technical is the model will go sort of watch those videos and learn the repetitive patterns in those videos without ever having them labeled. Like, it doesn't need to really know what a sheet is or what an arm is or what a hand is. It just sees the same things happening in the videos over and over again and builds a strong internal representation of those patterns. Those patterns, the the the the latents they call them are, like, very compact. They they contain a lot of information, and the model has learned to extract that information. That then gets decoded by, like, a a... They call it in body specific specific model, in order to directly output things like the gripper positions or the joint forces needed to actually complete the task. So that way you can take your huge dataset, train a big model once to generate these, like, representations. Then for each of the robots you want to ship against, you would train a much smaller model on a much smaller amount of robot specific data in order to be able to in order to be able to do the, like, robot specific action decoding.