SemiAnalysis Weekly
00:39:05 · Andrew Megalaa, Max Kan, Jordan Nanos
[00:39:05] S2: say it's definitely raised the bar for the kind of person you're looking for to like work like on stock. Like when you're when you're looking at like software engineering, for example, [00:39:17] S1: like I feel bad for the interns. [00:39:19] S2: Yes. Essentially. Like, you have to be really good to like provide value higher than just pure like Opus at this point as a software engineer. [00:39:26] S1: Yeah. Because I think a lot of people viewed interns as like, oh, I'm gonna... These are the people who I give my well specified, well defined problems I don't wanna do and they're gonna go do them for me. But obviously, now you just tell Claude to do that. And then sort of all the additional overhead of managing an intern oftentimes becomes sort of just like unbearable in those cases. [00:39:47] S2: I know one of the big quant trading firms had a post around this. It was like... Yeah. Really? Yeah. They they said their interns, like, finish their projects in the first few weeks this year compared to previous years. [00:39:59] S1: Hey, least interns, how do you say Ireland? That's good. Yeah. And [00:40:03] S0: I think the typical response from people is that you need to get more ambitious. It makes sense. I mean, going back to cybersecurity, where I've been doing some work recently. We saw this recently just with the increase in CVEs. I posted this on October 1. It's been a month. I don't have an update. There's a lot more than just this at this point.
00:40:03 · Jordan Nanos, Max Kan, Andrew Megalaa
[00:40:03] S0: I think the typical response from people is that you need to get more ambitious. It makes sense. I mean, going back to cybersecurity, where I've been doing some work recently. We saw this recently just with the increase in CVEs. I posted this on October 1. It's been a month. I don't have an update. There's a lot more than just this at this point. [00:40:26] S1: Yeah. Dude, is is the whole Internet just like broken now? Dude, what's what's gonna happen? What are you gonna do, Jordan? [00:40:33] S0: Well, I think it's a... The... Yeah. I can [00:40:36] S2: I mean? [00:40:36] S0: Do that in a lot more detail. Let's save that for another time, but let's use it as a way to make the point that there's, I think there's a lot more demand for cyber security experience right now, even when using the models. [00:40:51] S2: Yeah, for sure. [00:40:52] S0: And I think that's one of the first domains as a product of software engineering that's getting knocked down. But if you're pushing the frontiers of material science, for example, you're probably not automating away the jobs of the material sciences. You're probably starting to discover a whole bunch of new materials that need to go through the whole discovery phase and a [00:41:12] S1: lot Well, no, think it is very possible that you have automated the job of your PhD candidate assistant if you're a cracked material scientist. [00:41:20] S2: I think this is actually exactly what we've heard from [00:41:23] S1: our partners.
00:41:23 · Max Kan, Jordan Nanos, Andrew Megalaa
[00:41:23] S1: our partners. [00:41:24] S0: Yeah. If they have no ambition to go beyond what they were doing today or last year Yeah. Then, yeah, there's nothing useful for them to do. But if they have ambition to accomplish more with the tools that are at [00:41:37] S1: the disposal. Yeah. If the the person who would have been forced to just be, you know, kind of some PhD candidate grunt worker two years ago can now kind of leverage their infinite PhD grunt workers on tap to elevate themselves and, you know, just do way more interesting material science, that's great. We'll just have to see sort of... I I mean, I think I'm generally very optimistic kind of human agency and people being able to kind of, take their life in their own hands and, like, you know, do whatever makes them happy. And so I am just kind of bullish People kind of using these tools to be, just up leveling themselves essentially. We'll have to see what actually happened. [00:42:19] S2: It's interesting. I actually completely disagree with that. [00:42:22] S1: Andrew Andrew's a hater. [00:42:23] S2: No. Think I most people... Well, I mean, when you look at it, like most people in America, they just want to go in, put their hours in and go home. They like don't care that much about their careers and like... And like maybe maybe actually like this is what forces them to care. Cause otherwise, you won't be able to provide value, but we'll see. Yeah. Interesting.
00:42:23 · Andrew Megalaa, Jordan Nanos, Max Kan
[00:42:23] S2: No. Think I most people... Well, I mean, when you look at it, like most people in America, they just want to go in, put their hours in and go home. They like don't care that much about their careers and like... And like maybe maybe actually like this is what forces them to care. Cause otherwise, you won't be able to provide value, but we'll see. Yeah. Interesting. [00:42:43] S0: Okay. Let me go back and get some quick reactions to results that we kind of glossed over. So, you love Opus 5.5. Were you super happy to see it come out on top on a lot of these leaderboard charts or justifying me? What... Where'd you get that insight? [00:43:03] S2: I mean, obviously, like, you know, I'm I'm glad that my subjective experience was validated. Though, you know, some domains that I am not particularly involved in, like, it was very clear that Astra was just, like, ridiculous... Ridiculously strong on. Some of these, like, material science, but also, you know, like some of the engineering stuff is very clear that like Astra was like a step ahead there. And maybe that's just because we don't have Table 5.5 yet, but, you know, we'll see. But it's good to see that like different labs have different strengths because, know, there's spike here in different things so you can leverage whatever is best for what you're actually doing. [00:43:44] S0: Yeah. Makes sense. How about Gemini 4 Carbon? Quick take there. We saw Sundar quote tweeted everything there. You guys are excited about that. Like [00:43:55] S1: Yeah. That was sick. Shout out Sundar. I mean,
00:43:55 · Max Kan, Andrew Megalaa, Jordan Nanos
[00:43:55] S1: Yeah. That was sick. Shout out Sundar. I mean, [00:43:59] S2: we're excited about it too. Like, hey. More Frontier models is better for all of us. We definitely want, you know, better models from everybody. It only benefits the consumer as there are more competition. [00:44:10] S1: Yeah. I do I do however think that the right sort of apples to apples comparison for where the Frontier labs currently sit is if you exclude cyber and bio and alignment and look at that GDPX score. Maybe even switch to partial scoring too because that kinda gives you some more signal on legal and finance where no one's smart enough to do binary grading. So at least get reward for binary grading. And if you do that sort of ranking, you you see that Carbon is still kind of behind Astra and Opus, etcetera. But that being said, they... It is certainly sort of an inflection in the trend for where Google's model releases were going. Because I think at one point, they kinda felt that they were gonna fall farther and farther behind even, like, Meta in particular, potentially even XAI, and potentially even some of the Chinese open source guys. But they've, like, kind of really reestablished themselves as the third place player with Carbon. [00:45:15] S0: I'll come to this as a timing thing then. Like, sorry to interrupt, but Fable five five, you guys are teasing that might come out soon. Obviously, Opus, Sonnet are on the list
00:45:15 · Jordan Nanos, Max Kan, Andrew Megalaa
[00:45:15] S0: I'll come to this as a timing thing then. Like, sorry to interrupt, but Fable five five, you guys are teasing that might come out soon. Obviously, Opus, Sonnet are on the list [00:45:28] S1: of Yeah. So I I think I think the gap between Anthropic slash OpenAI versus Google today is likely still very big because, like, we're gonna have new flagship model releases with them quite soon. But, like, Muse Wonderful came out pretty recently. Right? And so I think you're sort of evaluating Google's position versus Meta and x AI. I think there's a pretty strong urban to me that actually they're, like, the top of that kind of tier two cohort. But I definitely still believe that there's, like, a pretty meaningful gap between open eye slash anthropic versus urban. [00:46:05] S2: And, you know, there's, like, the thing, like, you know, OpenAI and Anthropic are obviously not releasing their best models. So it's it's just a comparison between the public models. Well, like, Gemini and, like, Meta, like, they are releasing, like, the best that they have so far. Yeah. Anthropic could just be on, like, mythos four at this point. Who knows? Yeah. True.