00:05:37 · Ruby Justice Thelot, Qasim Wani, Ruby Justice Thelot
[00:05:37] S1: Totally. Yeah. Yeah. Yeah. Mhmm. So how exactly would you define RSI? Like, what is your definition, personally? [00:05:44] S0: I would say it's really can an agent invent a new lab? [00:05:48] S1: A new lab? [00:05:48] S0: A new lab. A new lab. [00:05:50] S2: Interesting. Like, what can one agent invent a new lab? [00:05:53] S0: Or a swarm of agents. [00:05:54] S2: A swarm of agents. [00:05:55] S1: Yeah. Seems like a really high bar. Like, that seems like a lot higher than what most people describe as RSI. I mean, some people's definition is as simple as like, is the model useful to programmers or AI researchers in any way that would cause the cycle time of new model development to shorten?
00:06:17 · Qasim Wani, Ruby Justice Thelot
[00:06:17] S0: Yeah, I think that I would describe that as self improvement and not recursive self improvement. Like the nomenclature here is on the recursive piece, because the whole premise behind RSI is by virtue of models getting good at very specific capabilities, we should achieve super intelligence. And that's sort of why I really like GDPX as a benchmark because it's not just focused on RSI, it's focused on all the critical components that will yield to super intelligence. RSI just happens to be a direct north star towards measuring for superintelligence. So it is a fairly opinionated answer, but that's what you need. You need people that have an opinionated way of measuring for model progress instead of generic improvement in productivity. It's kind of like saying, what is more important, creating good PowerPoint slides or doing drug discovery? They're both important like, to to... Like, if you're a researcher who can't really talk to a lot of, nontechnical people and you have to do some, like, reporting on material science and drug discovery research, if you don't have good taste in PowerPoints, you're probably not gonna be able to, like, get your agenda out the door. And so our version of RSI is just like, can an agent or a swarm of agents invent the next version of a frontier lab? [00:07:42] S1: Well, why can't they do this already? [00:07:46] S0: They're not good at it. That's sort of how we Yeah. [00:07:49] S1: But like why? Like where where did they come up short?
00:07:49 · Ruby Justice Thelot, Qasim Wani
[00:07:49] S1: But like why? Like where where did they come up short? [00:07:51] S0: So there's so many places. So for GDP x, we observed that the average accuracy or the the mean reward for all the frontier models was not more than 24%, which means that there's still a lot of room to grow for these models to improve on. Part of the reason why these models are not good at it, we sort of talk about it in our blog post within the GDPX report, one of the big problems with models like Astra or Carbon and Opus is that they don't know what experiments to prioritize. Yeah. And as a human, you know, as like engineers and researchers ourselves, like we know that certain experiments are more valuable than other experiments. Now this is not to say that models don't have any sort of intuition on this. They certainly do. It's just that they're not opinionated enough. Opus and sort of the clot models tend to be more opinionated than others, but they still lack that That level sounds right. They still lack that level of like, this is a high taste experiment to run, And by doing this experiment, here's how long roughly I think it should take to finish. And like using that existing logs of information and background research to make the next bet. And that's basically what we were able to find as like, you know, capability gaps in the frontier models.
00:09:18 · S3, Ruby Justice Thelot
Ad[00:09:18] S3: I want to shout out our sponsor, Lovable. You know, the app you've been meaning to build, the internal tool, side project, or product you'd otherwise lose a weekend to? With Lovable, you get software you can ship today. That includes an editable codebase you can inspect and change, plus two way GitHub sync. Lovable also handles the backend and infrastructure, including managed Postgres, auth, storage, hosting, payments, and more, so you don't have to wire it all up yourself. And through Lovable's MCP server, you can create and deploy your project directly from the agents you already use. Turn ideas into software people love with Lovable. Lovable.dev. Now onto the episode. [00:09:56] S2: Wouldn't as a prerequisite for any of that to happen, like, a model would have to have perfect knowledge of all of it, the different, like, of its own weights being hosted within the company and then also of all of the teams, like the pretraining team and the post training team. And, like, do models currently have that level of oversight or sort of, like, omniscience over their current state within the company even before we get to RSI? Is that something that they can do?
00:10:22 · Qasim Wani
[00:10:22] S0: Yeah. So so this is actually like a really interesting question of like, why why do RSI outside or why work on this problem outside the Frontier lab and not directly inside the lab? I think that's sort of like a bet that we are taking, which I think is somewhat interesting. There's this really good book that I highly recommend called Why Greatness Cannot Be Planned. It's by Kenneth Stanley. It's a really fun read for the technical folks. All that the reader talks about in that book is to be able to solve the most ambitious ideas, you have to have some degree of nonlinear thinking. Yeah. If your ideas are the exact same as everyone else's, you're going have the same outcomes as everyone else. And part of what our hypothesis is that the labs are focused on superintelligence. Like is on, like our side is on the critical path towards the labs achieving superintelligence. The problem is that, and this is sort of what we discovered empirically at xAI is, RSI is fundamentally a data problem. It is not a model problem. It is a data problem. That's all it is. And this is sort of like my biggest qualm with the data industry is, as someone who personally evaluated dozens of vendors back when I was at xAI, the quality of samples is not that great. Majority of the vendors that you work with don't really know what it takes to build a frontier grade model. They have some ideas based on their interaction working with the labs. The labs usually won't tell them this was good because of so and so. They kind of just YOLO things and hope that it's good. And so our hypothesis is that if you take set of guys from the labs and instead of them working in the labs, but now creating high quality data and selling that to the labs, you actually get to do two things. One, you get to democratize superintelligence by leveling the playing field. Like I don't want to be in a world where Opus is the only model lab that has access to AGI or ASI. So that's like just healthy for the ecosystem. And second is that there's just like a lot more entropy. There's more interesting things that might happen by working on these kinds of problems and not being limited to a certain type of a model architecture or model recipe. And this is why, you know, that's sort of like how we think about why doing this outside and inside.
00:12:54 · Ruby Justice Thelot, Qasim Wani
[00:12:54] S1: And so why is data, like why is data the bottleneck for RSI, not model capabilities? [00:13:01] S0: I think having high quality data is what enables better model capabilities. The way that I think about this is that we have entered a new economy where the frontier labs have become the literal market exchange for goods and services. And from that perspective, there's only like two real ways of contributing to this economy, which is either you sell compute via energy or you sell data via like human intellect. Yeah. There's like nothing else that matters besides these two things. So both compute and data are important. The problem with the data industry is that I think it's extremely undervalued. We have multiple model companies that are worth over $1,000,000,000,000. We have multiple chip companies worth over $1,000,000,000,000. But the sum total of the data industry is not more than $150,000,000,000. And that to me feels like a very gross underestimate of how valuable access to high quality data is. Internally at the labs, we would spend, as we all know, ungodly amounts of money on data curation and coming up with high quality data. So it's literally all how good's your data is, how good's your model. [00:14:13] S1: It's kind of funny hearing you say that when a lot of other people in AI are like, oh, data doesn't even matter that much because of RSI, right? The model will just be able to RSI its way to overcome whatever data bottlenecks it has, which is like a thing that I've always been skeptical of. It seems like it's kind of magical thinking.