The MAD Podcast: How AI Gets Built — with Matt Turck
00:37:24 · Nathan Benaich
[00:37:24] S0: Yeah. I think it's Yes. Pretty Yeah. Yeah. So, yeah. So that push really started. I mean, like post like the Ukraine war, like almost five years ago at this point, but more importantly, think it was like two years ago when JD Vance went to the Munich Security Conference and effectively said like, you need to invest in your own defense and get serious about this, you being, you know, NATO and Europe. And so ever since then, there's been, you know, a variety of different initiatives to give more capital to the new defense companies, to encourage governments to be providing loans to businesses to improve their supply chain. And that's, you know, created a set of like neo primes, like, defense companies that are trying to compete with prime contractors. So companies like Stark or Helsing being... Quantum Systems being like the the main ones in Europe. And then, you know, as as like warfare has changed, like it's gone from... I don't know, you know, developing strike systems over the... Over sort of mid range to, you know, hit tank, you know, battalions to now being like, well, getting pummeled like every day with hundreds and hundreds of FPV drones or various other kinds of aerial systems that are autonomous and, you know, now can no longer be stopped with jamming because they use computer vision or they're guided by these like fiber optic cables, which by the way have gotten much more expensive because of data centers. And so we need more, you know, novel like air defense systems. And so that's where companies like Alta Ares in France and others have really been pushing. And and as you said, like the supply chain is super important because, for example, like, you know, NATO wouldn't procure systems in in Europe that don't have at least 70% of the parts from from, you know, non China basically. And countries are really queasy about taking US protection or US guarantees at face value because they're shifting on a weekly basis. So, yeah, I think that's an example of a market which is sort of has to be domestic by by design and can breed new companies that are much more competitive than their US peers.
00:39:41 · Matt Turck, Nathan Benaich
[00:39:41] S1: So then if Frontier AI cannot be built, outside of The US, then, let's let's go back to The US, a part of your report on on research effectively calls, Anthropic OpenAI and Google as as as winners, as like the definite winners, which obviously, you know, has been, years in the making, but, there was a moment where it felt like there was a plurality of labs. Effectively, what you're saying is that no, like those three are the winners. And does that include, I guess nobody knows what they actually do, like SSI and thinking machines and like there seems to be like hundreds of neo labs. [00:40:17] S0: Yeah. Yeah. Maybe meta on the margin. Yes. Super intelligence lab. Yeah. Especially as now they're pushing towards like serving their models through API, but also having an enterprise business. So that one, you know, that might change over the next like six to twelve months depending on how their go to market works. But yeah, would say it's probably those three plus meta. [00:40:33] S1: So your view is that the the advantage is gonna keep compounding and that's gonna be the state of the market
00:40:33 · Matt Turck, Nathan Benaich
[00:40:33] S1: So your view is that the the advantage is gonna keep compounding and that's gonna be the state of the market [00:40:39] S0: versus I think so for like the general purpose frontier. Now there's there's like examples of perhaps like a really focused bet on a on a particular capability, which has like... Where you can make certain trade offs to the model in order to get much better performance at a task. Like I'm referring to, for example, these classification models effectively, like JEV, which can grow really, really quickly and become popular. Again, TBD, how long it lasts. But I think it's I think it's an encouraging sign for entrepreneurs that shows that, you know, you can take a focused resource bet and it can pay off. But over time, it's again like TBD if there are like these kind of gaps and capabilities that are sufficiently exploitable. [00:41:17] S1: So what do you think happens to the dozens of neo labs? I mean, not to keep talking about Europe, like just in London, there's three of them. Right? There's, the David Silver thing and, like, another two the... You know, in The US every few weeks. There's one, you know, Jeff Dean's new lab and so on and so forth. So what happens to those?
00:41:36 · Nathan Benaich
[00:41:36] S0: Well, I think it depends on the... Their kind of focus and modality. Like for example, on robotics and embodied AI, I think you can... I can see how an independent lab would still have a chance of competing and winning like because of the immense focus on, you know, enterprise AI, basically, that OpenAI and Anthropic has. Like, I have a slide in the report where I basically just, like, show all the different product lines that OpenAI and Anthropic had, or admittedly, OpenAI had lots and how they progressively just cut all of them. And there was a quote like, you know, this this moment is like too important to be distracted by side quests or some version of that, I'm paraphrasing, which meant that they broadly don't do dedicated robotics. Although maybe there's some of that kind of data in the in the Astra training recipe. Science is a little bit unclear. Anthropic is clearly leaning a lot into biology and drug discovery. OpenAI, maybe more on the model side, less on like we wanna own more of the stack. There's like kind of related to embodied AI, this whole like category of world models, whether that's, you know, a video model with some action in it or like some other definition of world model. But but in general, like Anthropic doesn't really do any of that or at least like from what we can see publicly. I mean, they don't even have an image generation model, for example. I think audio, pretty clearly, labs do have some of those capabilities, but, you know, if 11 labs can, like, keep accelerating the way it does, then clearly there's there's a need for a dedicated capability. I think those are probably the areas where I can see an independent lab, like, working out, but but ones that broadly say we're gonna do frontier intelligence for knowledge work, but just, make it better, faster, cheaper. I think that's tough. I think for me, like, I'll I'll believe it when I see it. But yeah, this idea of like you open your computer or the app looks exactly the same, but it's just better. I don't know if that Yep.
00:43:26 · Matt Turck
[00:43:26] S1: And in world model labs, are you seeing anything exciting? I mean, obviously, he's a Lee's lab was acquired for a lot of money and that's unclear what happens from there, given that the buyer is AMD. Because Yann LeCun started his company AMI in Paris since the
00:43:44 · Nathan Benaich
[00:43:44] S0: World Labs. Yeah. I mean, I've I've, you know, visibility into Odyssey where I was an early investor, like, doing world models. And I think the the interesting thing in that business is like their most latest... Their latest systems like Odyssey three. And there you can see that an autoregressive model trained to basically take all this video data and then have it conditioned on actions such that you know that, okay, I'm taking one action and then I generate the next frame as a result of the action. That paradigm applied to robotic manipulation, to self driving, to like video games, to like entertainment or something, can see generalization across all those different domains and already seeing that model being used for robotic manipulation training. So I I think at a at a broad level, I'm like kinda interested how image, audio, video ends up all mushing together into basically a world model. And then if that actually happens, then that could end up subsuming simulation and game engines. And then that's very important for moving robotics into the real world, but also designing physical systems and like hardware engineering. And to get back to like the AMD World Labs point, it's like if that happens, then the compute substrate that serves those kinds of environments and allows you to train and run inference is super, super important. And therefore having knowledge about how the world model research ecosystem is moving and what kind of hardware improvements you can make to be the number one choice is a big, big revenue pie. That's I think what motivated part... Well, part of the reason that AMD, bought WorldLabs. Sort of in the same way that NVIDIA over many, many years was like very close to research, labs that are doing shaders or three d vision or other things and, you know, giving compute out and constantly getting feedback from researchers such that they always knew like my hardware is the best for where the where the kind of frontier is going.