The People's AI: The Decentralized AI Podcast
00:23:42 · Jeff Wilser, Bayley Wang
[00:23:42] S0: So minor follow-up on that. This sounds very maybe, like, silly question, but does the angle matter? I was imagining if most people when folding sheets... If they're... I imagine if I'm if I'm videoing myself holding sheets, maybe I like... If I hold it with my hand, I I do it one way or if I put it across the room their way. But if all of the videos the robots are seeing are filmed a certain way, are there blind spots? So as you have to get, like, do you encourage users? Say, we we are... What's really useful is videos from reverse angle and or bot below angle. And I know it's a kind of a a bizarre question, but I think it gets at some of the complications and nuance here of the the complications of the real world. Right? So like if if, you know, how much variety and richness do you need in different kinda like angles of folding sheets, for example? I think everything in the real world probably has similar types of of complexity of of how do you get enough, like, 360 degree views to get the robot a good sense of, okay, here's what's actually going on. [00:24:52] S1: I think sheet folding is a somewhat, I I would say, not not unique, but like in the minority of examples where the sheet is likely to obscure a large part of the [00:25:03] S0: Yeah.
00:25:03 · Bayley Wang
[00:25:03] S1: View for a while. So for this particular example, I think video from the front and back would definitely be helpful. But in general, the robot... The the... If the robot sees enough video, it should gain an understanding of how the world works in order to be able to sort of guess the obscured views from the, visible views. For for, like, the specific sheet folding example where we're collecting data to address a single task, you probably do need multiple views in order to be able to do the synthesis because you don't have a lot of that data in, like, of other tasks to be able to learn what the hidden parts look like. But if you can imagine if you had enough data, would have view like when the robot would learn when its arm was being obscured, what it was doing. Like, it would learn to guess from like videos where the... There... Its arms weren't obscured. So I think in general for a foundation scale modeling, you don't really need an explicit, like, every task needs to be recorded from every angle. And this sort of like jumps into another another interesting question of like, where does the camera need to be? Yeah. And a lot of people like, oh, yeah. You need egocentric video. You you need like specialized sensors. I I I don't think so. I think if you go set a camera on on a desk and then like do things, that contains a lot of information already. And sure, it doesn't match the point of view of the robot's cameras. But that's fine because the model should be able to learn how to extract the actions anyway from that point of view. Right? Like, I once again, like doing the biomimetic analogy, you don't need to watch a first person view video of someone completing a task to learn how to do it. You're smart enough to deduce where your arms go from like a video from any angle. And what's important is that the like the like view is relatively unobscured and that's like reasonably high resolution. Resolution.
00:26:54 · Jeff Wilser, Bayley Wang
[00:26:54] S0: Makes sense. And... Okay. So we've we've covered sheet folding. What are some other concrete examples of of use cases that that people are feeding data through? [00:27:03] S1: I think probably the most interesting one in the short term is going to be warehousing and restocking. It has really good alignment with what, the types of communities that engage with decentralized projects are are doing anywhere. Like, there's a lot of people who are like gig workers or like do... Working like convenience stores or retail. And it also has a very immediate use case that there are plenty of companies that are either trying to do last mile light warehousing or restocking or, like, some sort of, like, light mix of domestic commercial. Ease... Easy data to collect. You could wear a camera, like, on your chest as you go about doing your doing your work and just passively collect that collect that data. [00:27:42] S0: Tell us a little more about what this looks like as far as incentives and compensation and where are we in the timeline? To what extent is this already out there in the wild? And give us a sense of kinda what the adoption has been like so far.
00:27:42 · Jeff Wilser, Bayley Wang
[00:27:42] S0: Tell us a little more about what this looks like as far as incentives and compensation and where are we in the timeline? To what extent is this already out there in the wild? And give us a sense of kinda what the adoption has been like so far. [00:27:57] S1: Yeah. So for our project specifically, we've really been working more on the more on the demand side with robotics companies. And for the supply side, we're going... I I I think initially we're going to work with, you know you know, like there there's folks like folks like Vana that already have data enthusiast communities. There is also guilds guilds like YGG which were focused on gate... More gaming type things but want to move into something which has more real world value. And these people are already used to the structure of like I... Incent... Incentive hunting either for tokens or for points that convert to tokens in the future. And I think for for, for the data collection stuff, it's it's really no different. The interesting thing I would say is for robot teleoperation. Like, part of our network and our protocol also covers directly being able to connect to a robot and operate the robot to complete tasks. [00:28:51] S0: Is that remote for... I'm trying to visualize this remotely or literally they go to a place and they they put their hands
00:28:57 · Bayley Wang
[00:28:57] S1: on on a robot? Remotely. Remotely. Okay. You you remotely operate the robot through like it could be as simple as a keyboard and a joystick or VR or something. Okay. And that that type of that type of task has a more interesting incentive structure right now because there are customers that are willing to like they're paying $50 an hour for that service right now, but they can't scale. So you you... That's like a very robust business. Right? Like they're they're like not only do they not want to pay $50 an hour, but more to the point they do not know how to scale these robot operators. Like the team is like like you you have companies with a valuation of over $1,000,000,000 running a team of eight people to do their data collection right now. Like one of the people is like a founding engineer. That kind of situation. So being able to offer something which is a lot more scalable is really helpful. And there's a lot of interesting things you can do there. Like, we encourage guild formation because guilds cannot only go and like organize the operators on the ground. They can also do things like invest in a robot that can be operated to generate data and mine incentives. And that's really helpful for the robotics companies as well because that means that they can, like, at least break even on the hardware, like, sell it at cost to a guild for the data collection rather than having to, like, raise dilutive capital for a manufacturing round in order to, like, get to the point where they can even start collecting data.
00:30:24 · Jeff Wilser, Bayley Wang
[00:30:24] S0: Amazing. Say a little more about how this works as far as compensation. Is there a token? How is a DAO fit into this? What are kind of the the web three mechanics here? [00:30:35] S1: Yeah. Yeah. Of course of course, there's a token. Right? We're a decentralized project. And what what else would we do? But the the DAOs are sort of their organize... Their organizational units to manage access to the data and, sales to the data, so that we don't have to go manage all of the data ourselves. The way the tokenomics work is that it's it's similar to like any any any other data project I think or what the other data projects should be if they're not already already doing this. That new data contributed to the network generates new tokens because that's like new value being retained by the network. Then the transactions themselves happen through like a a token buy and burn mechanism. And that that's that's where the that's where the DAOs sort of come in as well. Right? Like like, central buy and burn is, somewhat regulatory suspicious in many countries. So by having the DAOs, you sort of you sort of allow that to happen. And it's it's up to the DAOs to go manage the manage the, data access. [00:31:37] S0: What... What's the token called? And where should listeners go to learn more if they wanna get involved with this?