00:26:10 · Ev Randle, Molly O'Shea
Although now a lot of people think GPT 5.5 is actually better than the latest Opus model at coding. And then you have open source models which typically have immense value for the cost that you're paying for the inference of those models. And so there's great models for all these different tasks, and you do need to have a third party vendor that's sort of watching out for the customer and not just saying, know, in some of these enterprises, you have employees checking the weather with Opus four point eight. [00:26:40] S1: I know. I well, I was talking I've talked to a couple of companies about this already, and, like, how do you control spend on on your tokens and the token maxing debate and all that kind of stuff? Because, like, I know when I'm like prompting Claude or whatever I'm using, it's usually the dumbest, like it's not, it's a small fix. I could probably just go in myself, but look, the tool is magic. So I want to do it myself and make that small fix. But you do that over huge pieces of content or whatever [00:27:12] S0: And thousands of employees. [00:27:13] S1: And thousands of employees, and it clearly aggregates very large bills and great revenue sources for something. [00:27:18] S0: That's right. I'm like asking Opus four eight what I should have for lunch. You know, it's like, it's probably not that's probably not the best use of of our inference budget.
00:27:18 · Ev Randle, Molly O'Shea
[00:27:18] S0: That's right. I'm like asking Opus four eight what I should have for lunch. You know, it's like, it's probably not that's probably not the best use of of our inference budget. [00:27:25] S1: But it's like, it's interesting. So I know you guys are investors in Legora. Was at Harvey the other week, and and they were talking about the same thing. Was like, it's kinda similar to to how text messaging was before. You would pay per text. [00:27:37] S0: That's [00:27:38] S1: right. And like, so now it's happening within our chatbots, like within the different kind of prompts that we have. Like how are you seeing, like do you, how do you think the business model of that will play out, and like, will it get more efficient? [00:27:51] S0: Yeah, yeah, think it all depends on, it sort of depends on two different things. So there's, I have something that I call, and there's many other mom tests out there, and so it might even be trademarked, I don't know. [00:28:05] S1: A mom test?
00:28:06 · Ev Randle
[00:28:06] S0: So my personal AI mom test is, you know, not every mom is like this, but my mom is from, lives in this rural suburban town in Colorado, and she's not exactly an AI native, or she doesn't know how to reset her password on her iPhone, for example. She's not the most tech savvy person. And so the thing I always ask is like, okay for my mom, what is the amount of queries or the amount of things that she needs out of AI that can't be done by a really, really, really cost effective open source model? And you know, I started asking this two years ago, and there was actually like, oh, well I don't know, now it's like 100%. There's nothing that my mom actually asks of her AI products that needs to be done by the frontier or even a near frontier model, and you can start layering that out. It's like, well okay, what about a high school student? Or like what about this person or this person or this professional in this area? And depending on your use case and who you are and what you're trying to get out of AI, you either, you might need the frontier. Like again, like I think what we're seeing in the market right now is people are willing to pay a huge premium to access frontier intelligence to this day, but there's a growing portion of tasks in the economy and within these enterprises where you just don't need a frontier model, and oftentimes you don't even need anything close to a frontier model. And so I think what you're seeing, and Cognition published some work on this as well where they actually trained, don't know if they, I think they post trained an open source model on tasks that they saw done within Cognition where there was low task complexity, like it didn't eat frontier intelligence, but they were extremely popular things that were happening within Cognition. So it was like people were maybe just defaulting to using a frontier model for this thing, and they're like wow, you'd actually save an immense amount of money if you took it away from the frontier because it's a very simple kind of gated task that doesn't need frontier intelligence, and you're gonna save 95% on that action or that query. So I think you're seeing more and more as the models get better and better, there's a growing proportion of queries or actions or whatever you wanna call the use of inference that's going on that doesn't need the frontier.
00:30:17 · Ev Randle
That said, again, I think what's also growing is uses for frontier intelligence because again, I don't think we had unbelievable quality coding models until Opus four point five until last winter, which is why you saw Anthropix revenue and why you saw Cloud Code go so parabolic because there was a genuine breakthrough in the usability of those models, and so it's not that frontier intelligence isn't growing and the demand for that and the ability and willingness to pay a premium for that isn't growing, it's just that there's a lot of non frontier tasks as well, and the demand for those is also growing immensely. And so I think the, my partner Eric Visria had this really great tweet where a lot of people try to make this very zero something. It's like, okay, is open source gonna win, or are Anthropic and OpenAI going to win? And I forget all the things that he went down, but it was basically like on device inference, yes. Open source inference, yes. Proprietary models, yes. Like, it seems like there's a lot of demand for all of this stuff, and the demand is all going parabolic, and so it's not this zero sum game, at least yet, in terms of like, well, either it's all gonna be open source or all gonna be a frontier model from one of the three frontier providers, it turns out that there's use for all of it and it's all growing very, very quickly.
00:31:33 · Molly O'Shea
[00:31:33] S1: So I guess to push on that a little bit, so to your point earlier this year and even to today, OpenAI has crazy amounts of revenue. Anthropic was like last rumored and reported around 45,000,000,000. And now people are speculating around 60,000,000,000. Both are trying to go public with the same with SpaceX and XAI. What happens once those models become efficient? Like, what what do you think is gonna happen to the scale? Do you think it's gonna continue? Do you think what do you think is gonna happen?
00:32:04 · Ev Randle, Molly O'Shea
[00:32:04] S0: Yeah. Yeah. It's I mean, it's it's it's the $1,000,000,000,000 question or the multi trillion dollar question if you kind of combine SpaceX AI, OpenAI, Anthropic all into that equation, and I think it's left, like it depends on what the next twelve to many years look like, and what I mean by that is like, do you believe that we're on a path to recursive self improvement and this future of a bunch of geniuses in a data center, whereas if you do have a bunch of geniuses in a data center, if the frontier goes that high, then you legitimately do have probably a lot of pricing power and a lot of ability to continue to grow and continue to monetize and probably even reaccelerate growth if you're a frontier lab. On the other hand, if at any point it seems like capabilities are actually hitting an absolute ceiling and distillation continues as it has historically and the open source actually gets 95% as good as wherever the ceiling of capabilities tops out, that's a really scary situation for the Frontier Labs. I don't think it's like a death nail for them because, again, most of the users of ChatGPT would use ChatGPT whether or not it was a GPT model in there or not. Like they like the product. Most of them don't even know, they wouldn't be able to tell you if the model's different. In our little bubble in San Francisco, people would be able to tell you, but like the vast majority of the 900,000,000 weekly active users were They [00:33:31] S1: no idea what 5.5 is.