Y Combinator Startup Podcast
00:29:16 · Jeffrey Morgan
[00:29:16] S0: This new class of models, like DeepSeek Flash is a great example, and I think there'll be quite a few more, where it's ultra low cost per token. It's also low cost per task, which is a really important metric. And that class of models, in my mind will be the first ones that come down to this idea of like unlimited tokens. We all remember ChatGPT, you didn't really have to think about how many tokens you were using. You would just use it every day, you had unlimited. Ultimately, I think we return to that, but it's gonna take a lot of work in the model, the architecture to be custom trained for high volume token usage. And if you think about the start of the year, we really want, the open models got to the frontier of intelligence. We bridged the gap. We're maybe like less than three months behind between the frontier closed models and the open models. But the next problem to solve is extreme efficiency. Seeing for example, the GBT Luna model become very, very price effective for customers has been a huge boom. We talked a ton of customers where that kind of pricing enables widespread adoption within a team. I think we're gonna see that with open models. We already are seeing that with open models. I think the DeepSeek flash model is leading that charge. If we go back to the model breakdown on Ollama's cloud, the highest growth area is definitely the DeepSeek model and this is largely powered by the DeepSeek flash adoption. And so this new class of flash models where they're good enough for 80% of the tasks, they're really fast and they're ultra cheap. This new class of model that I think will enable some of those use cases.
00:30:40 · Jared Friedman, Jeffrey Morgan
[00:30:40] S3: Yeah, those are gonna be like the workhorse model to do like all the grunt work. [00:30:43] S0: Exactly, yeah. You won't have to be thinking about how many requests am I making, how many tokens. You'll be much more inclined to consume as much as you can because, you know, it's able to solve the hardest... Not the hardest, but you know, difficult problems. If you think back to the coordination layer we were talking about earlier too, being able to coordinate these flash models together to do different tasks can also yield great results that a bigger model can. So by having these cheaper models, not only are they more accessible, they can run faster and you can access them at higher volume, but you can start to chain them together and build new problems that are solved by orchestration on top. And that's a really exciting area for new startups, for existing inference providers, for some of the larger businesses today that solve workflow problems. Ultimately, being able to chain these models together is gonna be super helpful. And you won't have to think about the underlying costs.
00:31:35 · Garry Tan
[00:31:35] S2: Yeah. I guess when we first started talking about AGI, even on this podcast, there was sort of debate about, you know... And I think a lot of AI researchers would come out and say, like, there's just gonna be a giant god model, and it's gonna do everything. But, you know, I think so far, like, it hasn't quite worked out that way. Like, obviously, you still... You know, if you're if you have to literally hack the NSA, maybe you need mythos or something. But for the majority of use cases, like, you're talking about orchestration and you're talking about, like, smaller models, you you you know, that the task composition actually probably gives you a bunch of ways to make it more repeatable. It's more trustworthy. Like, it actually does work at a cost that is, like, possible. So, you know, if if it was gonna be a god model versus, like, lots of, you know, smaller special purpose or even just simpler models. It's turning out to be the latter so far.
00:32:29 · Jeffrey Morgan, Harj Taggar
[00:32:29] S0: I think for most customer use cases, there's a level at which a model becomes good enough. And then they can continue using that level of intelligence. Maybe the model will get faster, it'll have better architecture, it'll have new capabilities. But they won't have to reach for the god model. But I do think there are use cases where the most powerful models unlock them and that'll continue to be a thing. It'll be really exciting, sometimes scary as well on what they can do. But for the run of the mill use cases where open models really shine, I think that's where we're hitting a point where you're not solving necessarily the hardest problems within the business. But they're hard enough where it's now unlocked by open models. Will there be an open model that becomes a God tier model? I think it's possible. And we're seeing really exciting developments from Zhipu AI and GLM where some of the tasks they are frontier. And we all saw with the Kimi model how for web development it became the best model. And that sent this new shockwave across the market, which is it's less about a gap and it's more about a head to head competition, which I think makes all of this even much more exciting. [00:33:37] S4: This is a bit of a sensitive question, but what do you think about this and the geopolitics around it?
00:33:41 · Jeffrey Morgan
[00:33:41] S0: I think a lot of the geopolitical angles around this start with where the model's from. And the more we spend time with customers and users, a lot of it's actually how the model's run, where it's run, how it's run. Is it run-in a secure environment? And that starts to matter a lot more. But I do think, look, it's super important that a customer in The US can use a model trained in The US. And we have two kinds of classes of customers we speak to. One is they don't really care where the model's from, they care about where it's run. But for every one of those, there's a customer that's saying, I really care about where the model's from. Because it's data the way it, it's not even just a security issue as much as how does the model speak? We all go through and communicate. And we all go through, a lot of the models go through phases where they sound more robotic, they sound more friendly. And a lot of that matters too. But I think the highest order bet is obviously making sure that you have a model that end to end, you understand where the data's from, which is great from the Nemotron models. Then you can go and introspect what made this model. Because if you're putting in a mission critical task, which people are absolutely using open models for mission critical tasks. There's a post online about how Ollama powers the analytics of a power plant to detect surges in Finland to make sure that the lights stay on. That's where these models, the model origin really matters.
00:35:07 · Jared Friedman, Garry Tan, Jeffrey Morgan
[00:35:07] S3: For for like critical tasks like that, how do you ensure that a Chinese model, even if it's hosted in The US, isn't basically, like, booby trapped to, like, cause problems? [00:35:19] S2: Yeah. The Manchurian candidate problem. Exactly. Have there any been any like known cases of the Manchurian Candidate yet? [00:35:26] S0: I think not that I can think of off the [00:35:28] S2: top of haven't my head. Heard. I feel like I would have heard about it, but... [00:35:31] S0: You know, what you don't see a lot on some of the press articles is how robust some of the the IT and security teams are at the businesses that we know of, the top Fortune 500 businesses. They're really used to this already because open source software, if you think the average application, has thousands of dependencies. This isn't a new problem. And all it takes is one dependency for there to be a major security issue in the entire application. [00:35:55] S2: The blockchain poisoning is insane. [00:35:57] S0: It's a thing. Been a thing for decades. And it's not new in that sense. It's a little more opaque because you can't, like, dig into the model. It's [00:36:06] S3: It it is deterministic. [00:36:07] S0: But it's deterministic. And if you screen the model properly with safety checks, by and large, at least what we're hearing is from customers is that can be solved.