00:10:31 · Max Read, Erin Griffith
[00:10:31] S1: Yeah. And I would say too, it's funny reading them if you've been following AI for a while. Like, it's kind of like getting six years of Twitter and blog posts and Substack, like, compressed into, like, a document of legalese because they're listing everything from, you know, the resource constraints on building out compute to the various sort of AI safety problems to just sort of like your your basic business problems with, you know, customer satisfaction. But the ones that caught my eye, for example, are concerns have been raised both within our organization and by other researchers that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight and security measures, pose catastrophic or existential risks to humanity. This is a pretty rare kind of thing to do. Another one that I was really... It [00:11:16] S3: really fast. Can we just go back to that really quick, though? Because one thing that the great Matt Levine at Bloomberg pointed out was that this is written in passive voice. It's not Anthropic saying we think [00:11:27] S1: Right. [00:11:27] S3: That our products... It's it's... It says, like, some people may be saying. [00:11:32] S1: Within our organization, some people... I I don't know if it's our founder and president Dario Amodei or anybody, but some people have raised [00:11:39] S3: this question. Who can it be?
00:11:39 · Erin Griffith, Max Read
[00:11:39] S3: this question. Who can it be? [00:11:40] S1: Yeah. Another one that I I I like is, in addition, considerations relating to model safety, interpretability, alignment, and welfare may lead us to limit or delay the deprecation of older models. So just to translate that a little bit, when they're talking about model welfare, that's they think the models might be alive, and they're not ever gonna shut down some of the product lines because they want to make sure that they get taken care of. [00:12:01] S3: What? I missed this one. [00:12:02] S1: Wait. I I I have to credit Joe Weisenthal at Bloomberg for picking this one up. I'm... They're not they're not saying this out loud, but they're saying considerations relating to welfare may lead us to limit or delay the deprecation of older models. This is a little like Apple saying the iPhone 12 is alive, so we're just gonna keep making it forever. [00:12:19] S3: My god. Imagine trying to explain that to some, like, you know, analyst at Goldman Sachs and be like, we need to put this in here. [00:12:26] S1: I mean, the thing that I wonder about a lot is how much people who are spending their time talking to chatbots all day start to buy into and belong to this stuff. And so you actually will have activist investors who are saying, you know, we have to make sure that we're taking care of model welfare and whatever else. Else. This is a future coming soon. [00:12:43] S3: Well, the activist investors won't have a whole lot of ability to do much because of the governance of the company.
00:12:43 · Erin Griffith, Max Read
[00:12:43] S3: Well, the activist investors won't have a whole lot of ability to do much because of the governance of the company. [00:12:49] S1: Yeah. Let's see. I actually wanna get into this. Well, but let's start... Before we get there, let's start with just a quick word about the founders because this is a company that has a really specific philosophy around AI risk and AI safety that's sort of the core... Not just at the core of the business, but it's kind of their origin story. I'm sure a lot of people are familiar, but will you just give us the twenty second version of where Anthropic comes from? [00:13:08] S3: Sure. And this is kind of the story of AI also. You know, Google is working on AI. Elon Musk and Sam Altman see it, and they're like, man, I don't think they care enough about the future of humanity. So they start split off and start OpenAI, and they're like, we're gonna build AI, but we're gonna do it for the good of humanity. We're the principled ones. And then within OpenAI, Dario and all of his friends are looking around and saying, wait, Sam doesn't care enough about the future of humanity. We need to build the safe AI before they can. So they split off and do that. And that has like kind of continued on down the line. So that's where Anthropic is. Their roots are truly in like AI safety, and they are the probably most extreme on that spectrum of, like, the the fears inside the company at the top of the company and, you know, around it.
00:13:08 · Erin Griffith, Max Read
[00:13:08] S3: Sure. And this is kind of the story of AI also. You know, Google is working on AI. Elon Musk and Sam Altman see it, and they're like, man, I don't think they care enough about the future of humanity. So they start split off and start OpenAI, and they're like, we're gonna build AI, but we're gonna do it for the good of humanity. We're the principled ones. And then within OpenAI, Dario and all of his friends are looking around and saying, wait, Sam doesn't care enough about the future of humanity. We need to build the safe AI before they can. So they split off and do that. And that has like kind of continued on down the line. So that's where Anthropic is. Their roots are truly in like AI safety, and they are the probably most extreme on that spectrum of, like, the the fears inside the company at the top of the company and, you know, around it. [00:13:52] S1: Well, let's talk about how this relates to governance because they have this very unique governance structure.
00:13:52 · Max Read, Erin Griffith
[00:13:52] S1: Well, let's talk about how this relates to governance because they have this very unique governance structure. [00:13:58] S3: I'm like losing my voice now. And hopefully, they can use some AI to like make my voice sound less gravelly. Mhmm. Yeah. So Anthropic is kind of following a trend that has been happening for the last fifteen years at tech companies, which is that founders want to retain control even while selling shares, which would, in general, you know, the classic form of governance means that if you sell shares, those shareholders get more control. This started kind of goes back to Mark Zuckerberg and his multiple class of shares. But Anthropic has taken it even a little bit further. They have five classes of stock. Their seven founders have 50.1% voting share. You know, they're going to be in charge of electing all the directors to the board. And then they also have this, like, independent trust that controls a majority of the board seats. And so it's like a little bit convoluted. And I think it's... No one's gonna push back on this because over the years, every time startups have tried to do this, investors have made a big stink about it. Like, oh, no. We hate this. And then they just go along with it anyways. And so like, it hasn't... Yeah. There there has there has not been a a line in the sand. Even with SpaceX, they were a little bit more extreme about that. Some of the exchange... Or some of the indexes tried to put their foot down, and then at the end of the day, they're just like, ah, fine.
00:15:13 · Max Read, Erin Griffith
[00:15:13] S1: So this is not necessarily new in Silicon Valley, but it is for Anthropic seems to be distinctly sort of connected to the AI safety philosophy at the heart, which you should say. I'm not sure if we said it out loud, but, like, really specifically, Anthropic is the company that believes most strongly in the possibility of total AI annihilation, Armageddon, and additionally believes most strongly in the possibility that the LLMs are alive or conscious or sentient [00:15:37] S3: in some way. And they lay out that they believe their founders are best suited to sort of shepherd this into the world in a way that, you know, is is a little self aggrandizing when I was reading it. I was like, wow. You guys are special. And I guess you are because you're at the helm of a potentially $2,000,000,000,000 company. But, you know, they they are even able to, like, name their successors. They have a secret list of successors that they all have to approve if, like, somebody leaves the board. And so they're setting this up to be in control of it forever. They don't wanna lose control of the AI or of the company. [00:16:09] S1: Yeah. I wanna return to the philosophy, the sort of question of the philosophy, because one of the questions I have about Anthropic is, you know, beyond this governance structure here, to what extent is there... Are there philosophical commitments around AI safety? Like, how do they inform the company itself or inform the business decisions, or do they?