00:10:26 · Herbie Bradley
[00:10:26] S0: Yeah, mean, this reminds me of a lot of the things in software engineering where the actual quality of code or quality of decision making particularly and architectural decision making is below what an experienced senior engineer might feel happy with. And yet, because it's a highly verifiable domain, many people at the frontier of using LLMs for coding are just kind of like yellowing, the agent's form on your task, don't worry too much about code quality, just keep iterating, and you'll get to some output that is plausible or reasonable. So far, I think it has, in low stakes domains, been that you can in fact do this, but nonetheless, there have been many examples of some Vibe created failures and people's Vibe created apps for security holes as a result of this, and I think this is actually going to make it harder for the broader economy to adopt AI at scale, especially at the huge agents form level, if they are required to sort of go all in on this kind of ignore quality to some extent or sacrifice some level of some dimension of decision making that senior engineers or others would be happy with in order to just token max because it would improve overall productivity.
00:11:37 · Ruby Justice Thelot
[00:11:37] S1: Right. I think a counterargument to this whole AGI Centrism thing is straight lines on graphs, the model will just continue to get better, trust the exponential, and also the maximum possible capabilities maxes would have accurately predicted the last five or so years of AI progress and AI development. So I guess I need some epicentic humility with that in mind. But if they're not correct, like if AI capability does continue to improve in a jagged way rather than improving in a fully general way and leading to a real for real superintelligence within a few years, what do you think would be the most likely reasons that that doesn't happen?
00:12:28 · Herbie Bradley, Ruby Justice Thelot
[00:12:28] S0: I think probably it's that the at some point, the straight line on the graph diverges with, you know, actual what we would consider desirable qualities of a superintelligence, like the ability to make human like decisions across almost every domain, but better even to be able to do persuasion or other big tasks, which are very hard to quantify or define, as well as humans. You can define arbitrary benchmarks and evals to hill climb on, but in some cases that may diverge from real world qualitative utility. I think that is part of the reason why we're saying, for example, doctors' subjective experience or how good models are at diagnosing medical cases to be that the models have not improved a huge amount often in the last nine months. It's because, you know, you can construct many different benchmarks and evals for this task, but, ultimately, they don't quite capture everything. And so the question really for the diverges, you know, capabilities, Maxis, and what you might consider AI centrists is something like, you know, what are the problems of generalizing good hearting, and what happens if you sort of extrapolate the train on every quantifiable thing to the limit, and then what remains? [00:13:51] S1: Well, what remains? [00:13:52] S0: Well, that's the question. My take is still that there are a number of things in human activity, especially economically valuable things, that are very hard to quantify. And one reason why, for example, I don't expect creative writing capability to surpass the best humans or other things like this.
00:13:52 · Herbie Bradley, Ruby Justice Thelot
[00:13:52] S0: Well, that's the question. My take is still that there are a number of things in human activity, especially economically valuable things, that are very hard to quantify. And one reason why, for example, I don't expect creative writing capability to surpass the best humans or other things like this. [00:14:11] S1: If the AGI centrists are right, then does that imply that we will see some sort of market crash or correction, or have the markets not fully priced in the capabilities and axi perspective yet? [00:14:24] S0: Oh, yeah. I think these two things are mostly divorced, and the markets themselves are mostly dependent on the rate of diffusion. And right now, there's a huge overhang in terms of the amount of capabilities that exist and what people are actually using them for, what companies are using them for in the real economy. And so that diffusion cycle will continue even if we froze capabilities progress, And then the markets would depend on like how big is the level of demand from that. [00:14:49] S1: How true is that? Like, what would you consider current capabilities that would be able to have economically beneficial, useful applications that haven't been diffused? Like one clear example I can think of is the school system, I think could be massively improved with current AI. And this hasn't happened. But this to me is more political constraints because public schools are not subject to economic incentives in the way private businesses are. [00:15:17] S0: So could
00:15:17 · Herbie Bradley, Ruby Justice Thelot
[00:15:17] S0: So could [00:15:18] S1: we imagine that, you know, there are just like insurance companies run by boomers who don't understand the latest tech who aren't using it where it would be helpful? [00:15:27] S0: I mean, more or less, yes. I think it is just across all of the big, let's say, white collar domains, where there are large companies. AI adoption has progressed to the point where many of these companies have bought Claude Code seats or GPT seats for all their employees, but the intensity of use is not there. And there is still the fact that you ultimately need some degree of creativity or, you know, open thinking in order to be able to figure out what are the best applications of AI for your company, for your team, for your job, and a lot of people are just sort of not really doing that. And as a result, you see situations like, you know, a Fortune 500 company has ChatGPT seats for everyone, but only 1% of the company is using it by... Is, like, maxing out the subscriptions, and everyone else is, like, sort of not really using it very much, or they're not really using the most useful applications. It's just like we're still very, very early in this diffusion cycle, I think, and so demand could easily, you know, 10 x, a 100 x from here just with current capabilities.
00:16:23 · Ruby Justice Thelot, Ruby Justice Thelot
Ad[00:16:23] S1: We'll continue monitoring right after this message from our sponsors. ElevenLabs, AI that communicates at human level across every channel and modality. ElevenLabs dot I o slash m t s. Scale your startup on Neon. Millions of developers and startups have already chosen Neon for their back end. Start on the free plan or get up to a 100,000 in credits for your startup at neon.com/mts. Special thanks to our sponsor, Kong, the AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. Konghq.com. Ad[00:17:00] S2: Finally, prove what your AI investment is worth with Larridin. Head to lairdon.com today to start measuring your AI's bottom line. MongoDB, trusted to run critical applications by 75% of the Fortune one hundred, the intelligent data platform for the era. Apply today for access to up to $15,000 in Atlas credits, dedicated technical support, and exclusive events and go to market opportunities. Mongodb.com. [00:17:33] S1: Forgetting about aggregate economic productivity for a second, going back to what you said about employees in white collar businesses having and using these AI subscriptions, Why haven't we seen power law distributions start to form within companies or within sectors where some employees or some firms who are effectively leveraging AI are becoming massively more productive than others that are not? I think maybe we're seeing this a little bit in software, but not really in any other industries, from what I can tell. Even in software, it's not as pronounced as what I would expect.