00:03:53 · Jaron Lanier, Danny Fortson
[00:03:53] S7: Yeah. There are some. [00:03:54] S10: I sat in your house in your other room with all the instruments back in Berkeley in 2018. You were on this podcast when it was called Danny in the Valley. But the occasion, which is why I wanna start here, the occasion was you'd published this book, 10 arguments for deleting your social media accounts right now. And I remember just really enjoying that conversation, and it stuck with me because you were making an argument that, you know, we're eight years later, and it feels like the world has kind of caught up to that to a large degree. You're having these bans across the world, and, obviously, social media is still social media, but it does seem like there is a general awakening to, like, this extractive manipulative kind of business model is a problem. Is there a through line to your mind with what we are seeing right now with AI, the business models at the heart of this new industry, and, you know, how it is developing and, you know, obviously developing very quickly?
00:04:56 · Jaron Lanier, Danny Fortson
[00:04:56] S7: Yeah. Unfortunately, we're seeing much too strong of a through line, and it's kind of a tragic one because it's so unnecessary, and there would be better decisions. In my view, what happened, and I say this not as an outsider, but as an insider, and I'm speaking to my friends and of my friends and of myself and of my community, what happened is we became addicted to addiction as a business model. We ourselves became addicted to the thing which was addicting others. So if you look at the stated major benefits of AI algorithms, which is, you know, maybe we'll find cures to major diseases, maybe we'll transform the way we get energy to use and material science and all these things, and that could improve human life and prospects. So there's all that. But the way we make money, if you look at the very large numbers of people with subscriptions to AI services now, is extremely similar to the kind of emotional addiction paradigm that we saw with social media and that we continue to. And, there's a couple of things to say here, which are... You know, there's a lot of layers of irony and complexity and also, I think, lot of layers of hope. [00:06:30] S10: We're speaking, you know, in the week that Anthropic comes out with its 355 page IPO prospectus, you know, potentially the largest deal in the history of capitalism. So it's obviously very much on people's minds. And, you know, they spend almost a 100 pages on the various risks, including, hey. This new digital god we're creating might kill us all.
00:06:52 · Jaron Lanier
[00:06:52] S7: Yeah. That's right. I guess I should just take a moment here to say that I'm not an absolutist critic at all. What I had always thought is that the core issue is really the business model that if the way you make money from computation is by manipulating people, which is very, very easy to do with computation. It's the very easiest thing to do with it at all. Then your incentives become rather dark, and you tend to make money from doing things that are darker and darker. And in my view, that's why we have a kind of an unfortunately dark and depressed culture in younger generations all over the world. And it's why politics went off the rails in a lot of ways during this period and many other things. Just as we're saying that tech should be able to improve medicine, tech is actually probably doing more to harm people by making them reject modern medicine through cranky algorithms. So, you know, we make ourselves so absurd. Now the thing is, the business model for AI has a lot in common. It has the addiction dynamic in common. People definitely get addicted to using their chat. It definitely has the inherent ultra easy readily available illusions of computing in common. And what I mean by that, for those who don't know, way back in the mid sixties when computers were so small, they really couldn't do anything, there was an experiment called ELIZA where somebody made a chatbot that just kinda spit back what somebody had said, and people were convinced it was the most empathic, fully conscious, real therapist god ever. Like, you know, like, it takes no technical know how to make computing into this addictive manipulative thing because it's just so natural, just having this information system that can respond. It's just super easy to turn it into a manipulation box. This is going back to the social media era. If you say everything's free, then you're the product as the saying goes and as many have discovered it. The interesting thing about the AI era is people are actually paying for these subscriptions for the most... A lot of them are, not everybody. But... So in a way, what I wanted is happening. But here, the problem is a different problem where... Because we now have GenAI, we have a way of filtering all of the wealth and efforts and creativity of mankind through this scrambler so that it seems like it's just coming from the company.
00:09:12 · Jaron Lanier, Danny Fortson
So in a way, we're getting the worst of both worlds. We're getting the sort of weird anonymizer, dehumanizer thing that was happening with the algorithms on social media. But at this... But then we are paying for it, but it's not actually going to the people who are the sources of the information that's being rescrambled to turn into Gen AI. Instead, it's just going to these companies, And it suggests this idea where the only thing you need in the world is OpenAI and Anthropic, and everything else becomes dispensable, including their own customers. So it's very stupid, and I feel very embarrassed for my kind. I don't know what to say. I just... It's it's just very pathetic. [00:09:52] S10: So is what you're saying that effectively what we're doing with AI is basically we're paying for our own stuff because these systems have trained on all human creation. Anything on everything on the Internet, all the books, everything. And then as you say, it goes to the dehumanizer, which is a great term. And then we pay $20 a month to get access to that output
00:10:18 · Jaron Lanier
[00:10:18] S7: that Let me qualify this a little bit because that, I am sort of saying that, but not quite. There's an important difference here. So to me, the the large language models are cool things. Like, I... I'm not anti the technology itself. In fact, I think it's cool. I love having new tools to extend human efforts and collaboration. But the thing is, just like with a library, a library by itself just sits there. It only has any meaning if somebody goes in and does something with it. Like a library without a culture of scholarship, without a culture of library science, without librarians, it's it's just inert. And the same thing is true of these models, these large language models and other recent frontier models. There has to be something done with them in order for them to have any impact. And so the thing we chose to do is reproduce this Eliza illusion from the sixties or the thing that drove social media, which is, oh, let's make it a fake person, and then, people become addicted to it. They treat it as their therapist or girlfriend or partner in some way. And then, of course, you start to own their lives because now you own this intimate part of them. Just like with social media, I think sometimes it works out well. Sometimes it doesn't.
00:11:33 · Danny Fortson, Jaron Lanier
[00:11:33] S10: So question. One of the things we've been talking a lot about and writing about recently on the pod and in the paper is this notion that we, the leaders of this company, have created a new species. And therefore, they're, quote, unquote, moral agents that they have some equivalents and that we must treat them or really think about them in a way that is not just [00:11:59] S7: Yeah. [00:12:02] S10: One, do you agree with that? And two, if you don't, is that problematic, that framing? [00:12:06] S7: It embarrasses me so deeply. [00:12:08] S10: Why? [00:12:08] S7: So so... Okay. So let's unpack this. First of all, if we're talking about absolute truth, any question about what's a being and what's not a being is ultimately a matter of faith. We have latitude to believe different things. We have to live, and we have to make decisions. And I think as a pragmatic concern, we just have to have assumptions that we make about things. So I I feel a similar argument for me has to apply to AI, which is, you know, if we make a machine and we declare it god, what's the consequence? And the consequence is we have to let it run rampant, and letting a machine run rampant is really stupid because they... [00:12:51] S10: You wouldn't Well, and that's kind of what we're... That is what we're doing right now.