SemiAnalysis Weekly
00:41:23 · Max Kan, Jordan Nanos, Andrew Megalaa
[00:41:23] S1: our partners. [00:41:24] S0: Yeah. If they have no ambition to go beyond what they were doing today or last year Yeah. Then, yeah, there's nothing useful for them to do. But if they have ambition to accomplish more with the tools that are at [00:41:37] S1: the disposal. Yeah. If the the person who would have been forced to just be, you know, kind of some PhD candidate grunt worker two years ago can now kind of leverage their infinite PhD grunt workers on tap to elevate themselves and, you know, just do way more interesting material science, that's great. We'll just have to see sort of... I I mean, I think I'm generally very optimistic kind of human agency and people being able to kind of, take their life in their own hands and, like, you know, do whatever makes them happy. And so I am just kind of bullish People kind of using these tools to be, just up leveling themselves essentially. We'll have to see what actually happened. [00:42:19] S2: It's interesting. I actually completely disagree with that. [00:42:22] S1: Andrew Andrew's a hater. [00:42:23] S2: No. Think I most people... Well, I mean, when you look at it, like most people in America, they just want to go in, put their hours in and go home. They like don't care that much about their careers and like... And like maybe maybe actually like this is what forces them to care. Cause otherwise, you won't be able to provide value, but we'll see. Yeah. Interesting.
00:42:23 · Andrew Megalaa, Jordan Nanos, Max Kan
[00:42:23] S2: No. Think I most people... Well, I mean, when you look at it, like most people in America, they just want to go in, put their hours in and go home. They like don't care that much about their careers and like... And like maybe maybe actually like this is what forces them to care. Cause otherwise, you won't be able to provide value, but we'll see. Yeah. Interesting. [00:42:43] S0: Okay. Let me go back and get some quick reactions to results that we kind of glossed over. So, you love Opus 5.5. Were you super happy to see it come out on top on a lot of these leaderboard charts or justifying me? What... Where'd you get that insight? [00:43:03] S2: I mean, obviously, like, you know, I'm I'm glad that my subjective experience was validated. Though, you know, some domains that I am not particularly involved in, like, it was very clear that Astra was just, like, ridiculous... Ridiculously strong on. Some of these, like, material science, but also, you know, like some of the engineering stuff is very clear that like Astra was like a step ahead there. And maybe that's just because we don't have Table 5.5 yet, but, you know, we'll see. But it's good to see that like different labs have different strengths because, know, there's spike here in different things so you can leverage whatever is best for what you're actually doing. [00:43:44] S0: Yeah. Makes sense. How about Gemini 4 Carbon? Quick take there. We saw Sundar quote tweeted everything there. You guys are excited about that. Like [00:43:55] S1: Yeah. That was sick. Shout out Sundar. I mean,
00:43:55 · Max Kan, Andrew Megalaa, Jordan Nanos
[00:43:55] S1: Yeah. That was sick. Shout out Sundar. I mean, [00:43:59] S2: we're excited about it too. Like, hey. More Frontier models is better for all of us. We definitely want, you know, better models from everybody. It only benefits the consumer as there are more competition. [00:44:10] S1: Yeah. I do I do however think that the right sort of apples to apples comparison for where the Frontier labs currently sit is if you exclude cyber and bio and alignment and look at that GDPX score. Maybe even switch to partial scoring too because that kinda gives you some more signal on legal and finance where no one's smart enough to do binary grading. So at least get reward for binary grading. And if you do that sort of ranking, you you see that Carbon is still kind of behind Astra and Opus, etcetera. But that being said, they... It is certainly sort of an inflection in the trend for where Google's model releases were going. Because I think at one point, they kinda felt that they were gonna fall farther and farther behind even, like, Meta in particular, potentially even XAI, and potentially even some of the Chinese open source guys. But they've, like, kind of really reestablished themselves as the third place player with Carbon. [00:45:15] S0: I'll come to this as a timing thing then. Like, sorry to interrupt, but Fable five five, you guys are teasing that might come out soon. Obviously, Opus, Sonnet are on the list
00:45:15 · Jordan Nanos, Max Kan, Andrew Megalaa
[00:45:15] S0: I'll come to this as a timing thing then. Like, sorry to interrupt, but Fable five five, you guys are teasing that might come out soon. Obviously, Opus, Sonnet are on the list [00:45:28] S1: of Yeah. So I I think I think the gap between Anthropic slash OpenAI versus Google today is likely still very big because, like, we're gonna have new flagship model releases with them quite soon. But, like, Muse Wonderful came out pretty recently. Right? And so I think you're sort of evaluating Google's position versus Meta and x AI. I think there's a pretty strong urban to me that actually they're, like, the top of that kind of tier two cohort. But I definitely still believe that there's, like, a pretty meaningful gap between open eye slash anthropic versus urban. [00:46:05] S2: And, you know, there's, like, the thing, like, you know, OpenAI and Anthropic are obviously not releasing their best models. So it's it's just a comparison between the public models. Well, like, Gemini and, like, Meta, like, they are releasing, like, the best that they have so far. Yeah. Anthropic could just be on, like, mythos four at this point. Who knows? Yeah. True.
00:46:05 · Andrew Megalaa, Max Kan, Jordan Nanos
[00:46:05] S2: And, you know, there's, like, the thing, like, you know, OpenAI and Anthropic are obviously not releasing their best models. So it's it's just a comparison between the public models. Well, like, Gemini and, like, Meta, like, they are releasing, like, the best that they have so far. Yeah. Anthropic could just be on, like, mythos four at this point. Who knows? Yeah. True. [00:46:24] S1: We're we're gonna get, like, Fable plus one because I I think Anthropic is still kind of rolling with Fable one level models essentially or, like, Mythos one. I guess it's called Fable five. Internally, they're probably, like, at least two generations above that. And then we're finally gonna get, you know, one generation above that hopefully this week. So I think the the gap between sort of the actual best model each lab has including internal versus well, actually, that gap is just sort of very wide across OpenAI, Anthropic, Google, Meta, SpaceX. [00:46:59] S0: Yeah. Well, maybe the biggest takeaway for me there is just how relentless the pace of model releases is. And, I mean, the trend is for it to get faster and faster, where they improve post training and improve the data mixes. [00:47:14] S1: RSI, bro. It's RSI.
00:47:14 · Max Kan, Jordan Nanos
[00:47:14] S1: RSI, bro. It's RSI. [00:47:17] S0: Yeah. Can you talk about next steps? Like, where do you go from here? You think it's possible... Actually, one thing we didn't talk about at all is the concept of releasing one task and one rollout for the category, for each of the categories or, well, for each of the data vendors, so multiple from some of the categories. You talk about that approach that you guys took there of of the mix of kind of partially public, partially private, and how other benchmarks approach it comparison?