Y Combinator Startup Podcast
00:16:46 · Jeffrey Morgan, Aaron Epstein
[00:16:46] S0: Yeah. I think a lot of the... You know, what we know of as harnesses today, a lot of those pieces will go down into the model. But as you kind of push a lot of the core loop of the model and hooks down to the model itself, there are these pieces that kinda come out. Like memory's a great one. And the general guidance we're using is if it's a stateful problem, like there's storage involved, that's something that in the end can't go into the model. Because the model's trained and it's as we know training runs are now happening on like a monthly basis. But it's still not up to date with the latest data. So generally storing data is like a huge problem space that I don't think will ever make its way down into the model layer. [00:17:28] S1: Like managing credentials and that kind of stuff [00:17:29] S0: isn't gonna Security, credentials, safety. Open models don't have all the safety tooling that closed model providers give you out of the box. But that's super important, especially for businesses who adopt them.
00:17:29 · Jeffrey Morgan, Aaron Epstein
[00:17:29] S0: isn't gonna Security, credentials, safety. Open models don't have all the safety tooling that closed model providers give you out of the box. But that's super important, especially for businesses who adopt them. [00:17:40] S1: I'm curious where you see sort of the end or future state for enterprise on the balance between frontier closed models and open source model, especially on spend. It feels like, you know, initially it was just everyone's just allocating all of their budget to Anthropic or OpenAI. My sense now is, yes, open a... Open source is clearly growing, but, like, so are... So is, like, Anthropic's friend. So, like, the things seem to be growing together. Like, does that continue? Or do you think there's sort of, a steady state where it's, I don't know, like, half the budget's gonna be on the closed source frontier model, and half's going be an open source or something different. [00:18:19] S0: The super majority of tokens and this is our take will be open models within a business. Call it 8090%. That doesn't mean 8090% of the budget will go to open models. In fact, I think what the open model community is doing incredibly well together is lowering the cost to make it more accessible. And so maybe you'll only pay 10% to 20% of the cost towards open models, but your token Most of [00:18:41] S1: your tokens will be going through the open models.
00:18:41 · Aaron Epstein, Jeffrey Morgan
[00:18:41] S1: your tokens will be going through the open models. [00:18:42] S0: Right, which will enable a whole bunch of use cases on top, because you have this abundance of tokens. You're not thinking about taking away token access from your team, you're giving more and more access. I think for the hardest tasks, that's reserved for these frontier labs, where a lot of the best researchers are. And then from there, there's a whole bunch of problems in the middle, where maybe it's a combination of open and closed models working together. But I think the steady state is that most of the models are open. [00:19:07] S1: It's kind of an interesting idea, because as the models get more powerful, ideally you just wanna delegate to your smartest model to figure out when to go to an open model. But the labs who are in the models presumably don't want [00:19:19] S0: that. And I think, look, I think that all the labs are aligned in many ways to one thing, which is how do you serve the customer? And I think it will be up to the customer to decide. If I have a router where some of the scheduling and harder orchestration happens through frontier model, so be it. But a lot of the kind of line item work can happen through open models and the collaboration of the two together. I think we've seen a ton of projects, whether it's from Sakana AI or Open Router, that have combined the two and seen really good results.
00:19:19 · Jeffrey Morgan, Jared Friedman
[00:19:19] S0: that. And I think, look, I think that all the labs are aligned in many ways to one thing, which is how do you serve the customer? And I think it will be up to the customer to decide. If I have a router where some of the scheduling and harder orchestration happens through frontier model, so be it. But a lot of the kind of line item work can happen through open models and the collaboration of the two together. I think we've seen a ton of projects, whether it's from Sakana AI or Open Router, that have combined the two and seen really good results. [00:19:47] S3: This is not too dissimilar from a human organization, right? Like you have a law firm. There's like a partner, and then there's like a bunch of associates. And like the partner farms out the work to the associates, right? It's like the same. [00:19:59] S0: We saw the same thing with cloud computing, where it was really a blend of proprietary software, some of them provided by the cloud providers themselves. For example, AWS had DynamoDB, which was kind of their proprietary scale out database. But then a lot of customers use that in conjunction with PostgresDB. And in the end, what we see is customers will use a combination of the two.
00:19:59 · Jeffrey Morgan, Harj Taggar, Jared Friedman
[00:19:59] S0: We saw the same thing with cloud computing, where it was really a blend of proprietary software, some of them provided by the cloud providers themselves. For example, AWS had DynamoDB, which was kind of their proprietary scale out database. But then a lot of customers use that in conjunction with PostgresDB. And in the end, what we see is customers will use a combination of the two. [00:20:20] S4: Think it's a very common pattern. Mean, this is also the same design for why the Apple silicon is actually more superior, the special special special accelerators for different kinds of workloads for, let's say, image processing, supposed to audio, that's a spin. Or even go way back in in in the PC era. You had, like, your standalone audio card. Right? Graphics card and all that. [00:20:41] S3: And speaking of Apple Silicon, should we talk about local models? Because you're you're in a bit of a unique position and be because you have large businesses both in cloud hosted models and locally hosted models that will run on your laptop. What are you seeing in those two worlds, and what do you
00:20:41 · Jared Friedman, Jeffrey Morgan, Garry Tan, Aaron Epstein
[00:20:41] S3: And speaking of Apple Silicon, should we talk about local models? Because you're you're in a bit of a unique position and be because you have large businesses both in cloud hosted models and locally hosted models that will run on your laptop. What are you seeing in those two worlds, and what do you [00:20:56] S0: think is gonna happen? I think it's incredibly exciting because it's similar to the closed versus open model question. It'll be a mix in our mind. And that's what we hear from customers as well, where for easier tasks, could run them locally and with lower latency and of course lower costs when it comes to the per token costs. Ultimately you're buying hardware upfront. And you'll use that in conjunction with these cloud models. What's exciting about this next generation of hardware, which we've had for a few years now, is just how good they are at running the 20,000,000,000 parameter to 40,000,000,000 parameter range of models, sometimes up to 120,000,000,000 parameters. [00:21:32] S2: Yeah. Qwen 3.838 b is now as good as OPUS 4.6 for coding. Is that right? [00:21:38] S0: That's what the benchmark showed. [00:21:40] S2: Yeah. That's wild. [00:21:40] S0: It's incredibly exciting because you can run that on not the the lowest memory MacBook, but the second lowest memory MacBook you can find from this store. So it's incredible. [00:21:50] S3: And are you seeing your users do that? [00:21:52] S1: Like, how are you [00:21:52] S3: seeing people use the local models versus the cloud hosted models for in practice?