Lenny's Podcast: Product | Career | Growth
00:20:20 · Lenny Rachitsky, Anish Acharya
[00:20:20] S0: And this begs the question a bit of just what will separate the companies that win in this world where AI is kind of doing a lot of this? I imagine part of the answer is the human in this local maxima coming in with a better idea. Is there anything else that you think becomes a differentiator in this world where AI is doing so much of the work that humans are currently doing? [00:20:39] S1: I mean, I think one thing that's under discussed is that it's unclear that the sort of competitive equilibria that exists in a lot of industries will really change. So, let's say Pizza Hut, and Domino's, and Papa John's, and Round Table all get a data center of PhDs, and they're all going to either adopt it or have a CEO change and adopt it. So, it'll be a rocky period and there'll be some relative shuffling, but I think that they're all going to embrace the new technology because most companies do. I don't know that one of them is going to have 99% of the market. So, I think what's under discussed is that yes, in the near term I think there'll be winners and losers based on adoption of the technology and how ambitiously you adopt it. But I do think there's a lot of industries that will sort of maintain their current competitive dynamics because they're not intelligence bound. I think the most useful question to ask yourself as a founder CEO is just, "Hey, if we assume these things are infinitely intelligent and astonishingly cheap, how would we reorganize the company?" Because that's where we're going?
00:20:39 · Anish Acharya, Lenny Rachitsky
[00:20:39] S1: I mean, I think one thing that's under discussed is that it's unclear that the sort of competitive equilibria that exists in a lot of industries will really change. So, let's say Pizza Hut, and Domino's, and Papa John's, and Round Table all get a data center of PhDs, and they're all going to either adopt it or have a CEO change and adopt it. So, it'll be a rocky period and there'll be some relative shuffling, but I think that they're all going to embrace the new technology because most companies do. I don't know that one of them is going to have 99% of the market. So, I think what's under discussed is that yes, in the near term I think there'll be winners and losers based on adoption of the technology and how ambitiously you adopt it. But I do think there's a lot of industries that will sort of maintain their current competitive dynamics because they're not intelligence bound. I think the most useful question to ask yourself as a founder CEO is just, "Hey, if we assume these things are infinitely intelligent and astonishingly cheap, how would we reorganize the company?" Because that's where we're going? [00:21:42] S0: Kind of along those lines, you have this interesting take about this kind of split that might be coming within companies between these generalists and specialists, and how AI plays into that. Talk about that. [00:21:52] S1: Yeah. Well, I think that there's so many interesting things that this touches on. One is the question of open weight versus frontier. So I'm sure, are you familiar with Pareto efficiency? I'm sure you are.
00:21:52 · Anish Acharya, Lenny Rachitsky
[00:21:52] S1: Yeah. Well, I think that there's so many interesting things that this touches on. One is the question of open weight versus frontier. So I'm sure, are you familiar with Pareto efficiency? I'm sure you are. [00:22:03] S0: Please explain.
00:22:04 · Anish Acharya
[00:22:04] S1: Yeah, so Pareto efficiency is just for price performance, for example, the efficient frontier is what's considered the optimal trade-off of a unit of performance for a unit of price. And if you're along that curve, you're always paying the rational amount for the performance. And what's interesting is that frontier models are actually irrationally priced in that, first of all, Mythos is infinite dollars a token, you can't use it. But if you look at even something like Fable 5, for one IQ, conceptually one IQ of extra intelligence you're paying 100X more than Opus 4.8. So it's not rational, but I think there's a lot of jobs in which you have unbounded upside, like drug discovery. And if that one IQ point lets you discover the next statin, it's a trillion dollar outcome. So, it's sort of rational to pay for the highest intelligence in these types of jobs and industries. Now on the other hand, I'm going to pick on maybe legal. There's perhaps only so much upside to be had in legal or finance. And for those jobs, you actually do want to be very Pareto efficient and probably pay for good performance at a good price, not infinitely priced, infinite potential upside performance. So, I think what we're going to see is a split between job functions that demand kind of mid-IQ intelligence, and those will often be open weight, sort of biased with reinforcement learning, things that make the models even cheaper, more performant for a narrow job, along with incredibly "expensive but performant" frontier tokens for sales, support, research, engineering. So, I think you're going to end up having both architectures, and we can touch on this in a little bit but I do think there are these sort of comparative advantages amongst model families, and then also amongst, of course, individual models that we're seeing more and more of. It's not going to be one or the other.
00:24:01 · Lenny Rachitsky, Anish Acharya
[00:24:01] S0: Such an interesting insight. So, just to make sure I understand what you're describing here, you're thinking there's going to be this split between within an org of function and model, where specific functions that have a lot more upside and potential leverage go for the frontier models, and you described these kind of product sales, engineering research roles. And then, for other functions you don't need Mythos, you don't need Astra, I think is that the latest one that's coming? [00:24:29] S1: Yes, Astra. Yeah, yeah. [00:24:30] S0: And so it's both, you're saying the AI doesn't need to be the frontier model and the people don't have to be the smartest people in the world to do really well in that world. [00:24:37] S1: Yeah, and I don't want to be diminutive. There's extraordinary people in those job functions. I just think that they're bounded upside problems that they work on. You can only kind of close the books correctly, you can't close the books 100X better. So yeah, that's exactly what I'm saying. [00:24:53] S0: I wonder if it's connected back to that discussion we had earlier about it's verifiable. If it's a lot more verifiable, you don't need the frontier model versus, I don't know, the potential upside.
00:24:53 · Lenny Rachitsky, Anish Acharya
[00:24:53] S0: I wonder if it's connected back to that discussion we had earlier about it's verifiable. If it's a lot more verifiable, you don't need the frontier model versus, I don't know, the potential upside. [00:25:04] S1: I'm not sure. I mean, I think for a ... Verifiability is a good question because I think for a drug development company, you have infinite upside, it is verifiable, but closing the books is also a verifiable problem, which has limited upside. I think the question is really how much upside is there and how hard is it to calculate it? Here's a nuance. Customer support, a customer may call in and report a bug. That bug may actually be the first breadcrumb to a thing that changes our entire organization. And if the CEO was on that call or the smartest person, they could follow the trail. But if we actually have this "mid-IQ generalist," then not, that actually makes the case for as a basket of problems always use frontier intelligence. And I can of course make the case for the mid-IQ basket as well, which is simply that we've crossed an intelligence threshold for almost every economically useful problem, and anything beyond that threshold is simply waste. [00:25:57] S0: Yeah. And I think the point people forget is that the models that are not the frontier models, that was what the frontier was, I don't know, six months ago, and we were so impressed and we loved it. It was like, "Holy shit, it can do all this." And now just because there's something better, we don't give those models as much credit.