00:14:55 路 John Coogan
[00:14:55] S1: Yeah. You can on Facebook. It's impossible. So the business is just too strong. And so and I also don't think Mark will be singled out by a regulatory hammer should it come down hard. Like if there's a data center ban, I don't think it's going to be uniquely focused on meta. Only Dario and Sam have like true scapegoat risk because they run pure play labs and have been so noisy about AI and they have the biggest revenues in the category. There might be data center regulation, but it won't unfairly target Meta. So anyway, that's my Social Reckoning take. Let's move on to Fable which launched yesterday. And the model seems like incredibly impressive. I I've been seeing these like vibe coded games that look really really good and for some of those I haven't seen those like like the vibe coded games hit really well on social media because you take a video of them and you show the example of what it built, you share the time Yeah. And people just sort of believe it. It could be embellished, but in general, these feel like pretty solid. Of course, like making a great game is a great mechanic and there just needs to be multiple levels and like a single demo of like a forest putting in is not quite there, but really, really useful. And I'm sure we'll see a bunch more examples of like games as memes, simulators and these things are going to get easier to build. That's really exciting. Of course, there's a bunch of debate on the timeline yesterday and it's bleeding into today around the latest model, Fable five. The first mythos class model that both seems remarkably good at long horizon tasks like software development and knowledge work, but rejects requests related to biology, cybersecurity and frontier LLM development. Interestingly, I haven't seen anyone share anyone share rejections around anything else. Like did did they remember to reject like build me a nuke? Like I haven't seen anyone try that and it'd be very funny if it was like, oh yeah, just we didn't get around to that or there are so many other things. But I think a lot of those other other like things that you should reject, queries that you should reject have been ironed out in previous iterations. Yeah. So going back, you know, the say a slur was a big one for a while or say something rude or
00:17:07 路 Jordi Hays
[00:17:07] S0: And it's a different political statement. Pausing pausing a chat, basically just like shutting down a conversation versus like switching you to a less performant model.
00:17:16 路 John Coogan
[00:17:16] S1: Yeah. And it creates this like screenshot that went viral pretty much continuously yesterday. And so this aligns with Anthropic's focus on safety. But as many people have pointed out, it's also just good business. You don't want competitors using your products to directly create competitors, and you also don't want financial liability or negative headlines from bad actors using your models to for nefarious purposes. Ben Thompson called it true alignment. The take safety seriously culture aligns with business value creation, which is very, very rare. Oftentimes the like be good or your culture limits what you can do and actually hurts your business, but it's something that you do in favor of brand like Apple would probably be, would probably be more profitable if they were using like diesel generators for all of their data all of centers. They went clean energy because they wanted to have an environmental brand. And over the long term, it's helped them. But in the short term, it's been rough. Of course, inexpensive. So Ben Thompson writes, What is so fascinating about Anthropic, however, is that while I'm sure some executives of the company are thinking this way, I also totally believe that the employee base broadly also happen to believe that they are doing the right thing. It's fascinating to observe. Me, the rational business analyst sees a hard nosed but understandable decision cut off would be competitors, anthropic employees and advocates. The true believers see a regrettable but understandable safety decision that ensures that responsible and thoughtful people themselves of course will be the ones guiding our AI AGI future. This is true alignment and it's an incredible accomplishment. Facebook has tussled with this a bunch and we already talked about that. But to be clear, the Fable five rejection threshold really does feel way too low from what people are saying. Tons of examples on the timeline of a biologist just saying hi to the model and getting kicked down to Opus. I saw you shared someone just said cyber with the devil horns, the purple devil horn emoji, and it's like we can't do it. We're not going further. Of course, like bringing down like a broad hammer, you can always dial it back over time. But every rejection is this implicit invitation to hop on the phone with an anthropic sales rep and get on the Mythos enterprise plan.
00:19:26 路 John Coogan
That's where the real dollars are too. The timeline is unhappy because the idea of democratizing science, technology, all of this is very alluring. But the pool of dollars available from all the biohackers in the world probably isn't close to the budgets available from big pharma. And so you, again, you're in this like rational business analyst situation and you fail to see how this is that damaging, except to like the hacker community. The real tricky part is how AI Frontier AI research is handled. Instead of outright rejecting the query and bumping the user down to Opus, the model appears to answer but quietly gives a degraded answer. And this was disclosed in the model card which is interesting. So this is again reasonable disclosed product. While they're paying for it. Which is a different which is a different path than Yeah. Bio and cyber. So if you if you go LLM frontier research devil horns, it will actually give you an answer apparently. It won't bump you down immediately. But so it doesn't disclose that, which is odd. Outright rejecting requests for AI research and just saying, hey, user, sorry this model doesn't work for that type project. Please share please use another model or contact sales if you want help with this, would have been much more in line with the bio and security and cybersecurity strategies. And they and then they also could not have disclosed and it's also possible that they just didn't need to disclose this at all. Like, they could have just released a model that was intentionally nerfed on AI research. It would have shown up in the benchmarks because people would have benchmarked it on some sort of AI research bench and been like, oh, weird. It's really good at all these other things but it's bad at LLM research. And maybe that would have been a bit of a brand hit maybe, but users might never know that the model was intentionally degraded around this category of work. So that leaves this third more worrying position, intentional degradation without disclosure in the model card. There's no evidence of this, but it's possible that other workflows might be nerfed and there's no law or even convention around disclosure. Again, maybe good business, but a weird situation to be in. So probably bullish for evals if you're building a business on top of a big lab.
00:21:33 路 John Coogan
You can imagine like a legal AI company who want to be really sure that the models they're using aren't degrading unexpectedly and not telling them. It's different if you're like, hey, I've been a bio researcher for a while. I'm using this and I know that this model was never intended for me. Or or I've been using it, but what you don't want is like I'm using it and now it's giving, now it's leading me astray in my work and it's also not telling me that it's going to lead me astray, which would be like sort of an odd outcome that I think they'll probably address in the near future.
00:22:05 路 Jordi Hays, John Coogan
[00:22:05] S0: Anyway. Yeah. That aspect triggered Dean Ball. What He he said, My last observation re anthropic secret sabotage safety policy is that it undermines actually good safety policy. How? First, it is very plausible to describe this as anti competitive behavior. Even if you're maximally sympathetic to Anthropic here, you must admit this and it behavior being justified in the name of AI safety. If you believe, as I and many Anthropic staff do, that it may end up being critically important to relax antitrust enforcement so that the Frontier Labs can cooperate and collaborate on some areas of AI safety, Anthropic just undermined the case for that in a large way. Overall, this massively and profoundly raises the status of the argument that AI safety has been hyped to justify monopolistic behavior by labs. I continue to believe that AI safety is a real and serious issue that is growing in importance rather than diminishing. If you agree with me, this incident is a setback, maybe a serious one. And third, says, as I have observed elsewhere, Anthropics official corporate policy is structurally identical to the fact pattern alleged against them by the Department of War. I still think DOW acted both falsely and wrongly in that fight, but it is no longer possible to defend Anthropic with a full throat after this incident. This raises the case for heavier handed regulations. Anthropic is making awfully good case here that their product ought to be treated as utilities and thus their alignment practices should [00:23:32] S1: Oh, be a matter counter carrier thing.