00:27:18 · Ev Randle, Molly O'Shea
[00:27:18] S0: That's right. I'm like asking Opus four eight what I should have for lunch. You know, it's like, it's probably not that's probably not the best use of of our inference budget. [00:27:25] S1: But it's like, it's interesting. So I know you guys are investors in Legora. Was at Harvey the other week, and and they were talking about the same thing. Was like, it's kinda similar to to how text messaging was before. You would pay per text. [00:27:37] S0: That's [00:27:38] S1: right. And like, so now it's happening within our chatbots, like within the different kind of prompts that we have. Like how are you seeing, like do you, how do you think the business model of that will play out, and like, will it get more efficient? [00:27:51] S0: Yeah, yeah, think it all depends on, it sort of depends on two different things. So there's, I have something that I call, and there's many other mom tests out there, and so it might even be trademarked, I don't know. [00:28:05] S1: A mom test?
00:28:06 · Ev Randle
[00:28:06] S0: So my personal AI mom test is, you know, not every mom is like this, but my mom is from, lives in this rural suburban town in Colorado, and she's not exactly an AI native, or she doesn't know how to reset her password on her iPhone, for example. She's not the most tech savvy person. And so the thing I always ask is like, okay for my mom, what is the amount of queries or the amount of things that she needs out of AI that can't be done by a really, really, really cost effective open source model? And you know, I started asking this two years ago, and there was actually like, oh, well I don't know, now it's like 100%. There's nothing that my mom actually asks of her AI products that needs to be done by the frontier or even a near frontier model, and you can start layering that out. It's like, well okay, what about a high school student? Or like what about this person or this person or this professional in this area? And depending on your use case and who you are and what you're trying to get out of AI, you either, you might need the frontier. Like again, like I think what we're seeing in the market right now is people are willing to pay a huge premium to access frontier intelligence to this day, but there's a growing portion of tasks in the economy and within these enterprises where you just don't need a frontier model, and oftentimes you don't even need anything close to a frontier model. And so I think what you're seeing, and Cognition published some work on this as well where they actually trained, don't know if they, I think they post trained an open source model on tasks that they saw done within Cognition where there was low task complexity, like it didn't eat frontier intelligence, but they were extremely popular things that were happening within Cognition. So it was like people were maybe just defaulting to using a frontier model for this thing, and they're like wow, you'd actually save an immense amount of money if you took it away from the frontier because it's a very simple kind of gated task that doesn't need frontier intelligence, and you're gonna save 95% on that action or that query. So I think you're seeing more and more as the models get better and better, there's a growing proportion of queries or actions or whatever you wanna call the use of inference that's going on that doesn't need the frontier.
00:30:17 · Ev Randle
That said, again, I think what's also growing is uses for frontier intelligence because again, I don't think we had unbelievable quality coding models until Opus four point five until last winter, which is why you saw Anthropix revenue and why you saw Cloud Code go so parabolic because there was a genuine breakthrough in the usability of those models, and so it's not that frontier intelligence isn't growing and the demand for that and the ability and willingness to pay a premium for that isn't growing, it's just that there's a lot of non frontier tasks as well, and the demand for those is also growing immensely. And so I think the, my partner Eric Visria had this really great tweet where a lot of people try to make this very zero something. It's like, okay, is open source gonna win, or are Anthropic and OpenAI going to win? And I forget all the things that he went down, but it was basically like on device inference, yes. Open source inference, yes. Proprietary models, yes. Like, it seems like there's a lot of demand for all of this stuff, and the demand is all going parabolic, and so it's not this zero sum game, at least yet, in terms of like, well, either it's all gonna be open source or all gonna be a frontier model from one of the three frontier providers, it turns out that there's use for all of it and it's all growing very, very quickly.
00:31:33 · Molly O'Shea
[00:31:33] S1: So I guess to push on that a little bit, so to your point earlier this year and even to today, OpenAI has crazy amounts of revenue. Anthropic was like last rumored and reported around 45,000,000,000. And now people are speculating around 60,000,000,000. Both are trying to go public with the same with SpaceX and XAI. What happens once those models become efficient? Like, what what do you think is gonna happen to the scale? Do you think it's gonna continue? Do you think what do you think is gonna happen?
00:32:04 · Ev Randle, Molly O'Shea
[00:32:04] S0: Yeah. Yeah. It's I mean, it's it's it's the $1,000,000,000,000 question or the multi trillion dollar question if you kind of combine SpaceX AI, OpenAI, Anthropic all into that equation, and I think it's left, like it depends on what the next twelve to many years look like, and what I mean by that is like, do you believe that we're on a path to recursive self improvement and this future of a bunch of geniuses in a data center, whereas if you do have a bunch of geniuses in a data center, if the frontier goes that high, then you legitimately do have probably a lot of pricing power and a lot of ability to continue to grow and continue to monetize and probably even reaccelerate growth if you're a frontier lab. On the other hand, if at any point it seems like capabilities are actually hitting an absolute ceiling and distillation continues as it has historically and the open source actually gets 95% as good as wherever the ceiling of capabilities tops out, that's a really scary situation for the Frontier Labs. I don't think it's like a death nail for them because, again, most of the users of ChatGPT would use ChatGPT whether or not it was a GPT model in there or not. Like they like the product. Most of them don't even know, they wouldn't be able to tell you if the model's different. In our little bubble in San Francisco, people would be able to tell you, but like the vast majority of the 900,000,000 weekly active users were They [00:33:31] S1: no idea what 5.5 is.
00:33:31 · Molly O'Shea, Ev Randle
[00:33:31] S1: no idea what 5.5 is. [00:33:33] S0: No idea, and like they wouldn't be able to tell you, if you swapped it out for like 5.2 or something, they wouldn't be able to tell you, and so I think they're, it wouldn't, like they're not, like that scenario wouldn't kill them because again, people like the products, and they've done, both OpenAI and Topic have done a tremendous job of building amazing products on top of their models, but it would certainly be a very, very different situation because you'd have a much, much greater impact from open source depressing their ability to have pricing power and charge a premium for the tokens that they're producing. And it's like, I mean, open source inference also costs money. It's not like open source means free. Someone still has to run the GPUs, someone still has to build the data centers, someone has to still operate the data centers. It's more just about how much premium margin that the Frontier Labs can create, and I don't think we'll know until it's very clear whether or not we actually have RSI and there's going to be geniuses or god in a data center or whatever you wanna call it, at which case we'll have different problems to be thinking about, or if we do end up at some point topping out in capabilities sometime in the next few years and distillation continues, I think it's much harder to garner a premium margin if you're a frontier model company. It becomes much harder. [00:34:44] S1: Given the scale and how fast things are running, another common question is how do we fund this? [00:34:50] S0: Mhmm.