Y Combinator Startup Podcast
00:13:58 · Jared Friedman, Jeffrey Morgan
[00:13:58] S3: That's interesting. Do you think any of those hidden layers might get unbundled and become their own companies or providers? [00:14:06] S0: Absolutely. There was a really good talk from some of the Anthropic team, the platform team, and they talked about three big things. One was knowledge. How do you connect your company's data and context to the model? One is coordination. So as you know, when you make a request to in your Cloud app or Codex, it goes off and spins out a bunch of sub agents, some of those in the cloud, some locally. There's a coordination problem. And then lastly, there's an execution problem, which we talk a lot about as sandboxes. But these agents, more and more of them are moving to the cloud. There's a huge compute problem to be solved there. And if you look at what's happened classically in the cloud business, open source has meant that there are best of breed companies for each of those problems. Whereas you might have had kind of the, if you look back to the original generation cloud products, you have like the Heroku of the world, you have Google App Engine, where all those things were bundled together. But what developers ended up preferring is best of breed products for each one.
00:15:00 · Garry Tan, Jeffrey Morgan
[00:15:00] S2: It's the nivolism, which is all things are just bundling or unbundling. Exactly. Like the frontier model. Frontier Model Labs want you to be totally in their walled garden of manage agents and their context, their memory layer. And then meanwhile, Little Tech and all the founders out there and all the open source developers don't wanna be caged in. So we're gonna make all this other stuff. And it'll it'll be an interesting moment to figure out like what ends up winning. I mean, it'll probably be some mix of both. [00:15:27] S0: I think so. And you've got this abundance of open model tokens that's being created. There are just dozens of open model providers that are able to serve these tokens. The new scarcity, the problems now are what's above the tokens, right? How do you orchestrate an agent from A to B? These are problems that are tons of new systems and engineering problems that are just really hard to solve for an individual dev. Like there's no way they're going to build all those layers.
00:15:27 · Jeffrey Morgan, Garry Tan
[00:15:27] S0: I think so. And you've got this abundance of open model tokens that's being created. There are just dozens of open model providers that are able to serve these tokens. The new scarcity, the problems now are what's above the tokens, right? How do you orchestrate an agent from A to B? These are problems that are tons of new systems and engineering problems that are just really hard to solve for an individual dev. Like there's no way they're going to build all those layers. [00:15:53] S2: Well, the interesting thing now is because the coding agents themselves are getting a lot better, and ostensibly, this is the worst the models will ever be, the classic reason why there was a moat here was it was just too hard to have really well maintained software that was properly tested, that actually satisfied user need. And what if that goes away? Like, we're literally at this moment where, actually, maybe the a... You know, you'll just have a cron, and it runs a markdown file in some TypeScript, and it'll just... You know, there's no lock in anymore. Right? Like, you could be using OpenAI's memory system one day, and then actually, you know, you could have an agent be constantly syncing that against your own memory system. And actually it just works. It's fine. Like it works over MCP. Like there's all... You know, there's plenty of like runtime testing, and then there's no lock in.
00:16:46 · Jeffrey Morgan, Aaron Epstein
[00:16:46] S0: Yeah. I think a lot of the... You know, what we know of as harnesses today, a lot of those pieces will go down into the model. But as you kind of push a lot of the core loop of the model and hooks down to the model itself, there are these pieces that kinda come out. Like memory's a great one. And the general guidance we're using is if it's a stateful problem, like there's storage involved, that's something that in the end can't go into the model. Because the model's trained and it's as we know training runs are now happening on like a monthly basis. But it's still not up to date with the latest data. So generally storing data is like a huge problem space that I don't think will ever make its way down into the model layer. [00:17:28] S1: Like managing credentials and that kind of stuff [00:17:29] S0: isn't gonna Security, credentials, safety. Open models don't have all the safety tooling that closed model providers give you out of the box. But that's super important, especially for businesses who adopt them.
00:17:29 · Jeffrey Morgan, Aaron Epstein
[00:17:29] S0: isn't gonna Security, credentials, safety. Open models don't have all the safety tooling that closed model providers give you out of the box. But that's super important, especially for businesses who adopt them. [00:17:40] S1: I'm curious where you see sort of the end or future state for enterprise on the balance between frontier closed models and open source model, especially on spend. It feels like, you know, initially it was just everyone's just allocating all of their budget to Anthropic or OpenAI. My sense now is, yes, open a... Open source is clearly growing, but, like, so are... So is, like, Anthropic's friend. So, like, the things seem to be growing together. Like, does that continue? Or do you think there's sort of, a steady state where it's, I don't know, like, half the budget's gonna be on the closed source frontier model, and half's going be an open source or something different. [00:18:19] S0: The super majority of tokens and this is our take will be open models within a business. Call it 8090%. That doesn't mean 8090% of the budget will go to open models. In fact, I think what the open model community is doing incredibly well together is lowering the cost to make it more accessible. And so maybe you'll only pay 10% to 20% of the cost towards open models, but your token Most of [00:18:41] S1: your tokens will be going through the open models.
00:18:41 · Aaron Epstein, Jeffrey Morgan
[00:18:41] S1: your tokens will be going through the open models. [00:18:42] S0: Right, which will enable a whole bunch of use cases on top, because you have this abundance of tokens. You're not thinking about taking away token access from your team, you're giving more and more access. I think for the hardest tasks, that's reserved for these frontier labs, where a lot of the best researchers are. And then from there, there's a whole bunch of problems in the middle, where maybe it's a combination of open and closed models working together. But I think the steady state is that most of the models are open. [00:19:07] S1: It's kind of an interesting idea, because as the models get more powerful, ideally you just wanna delegate to your smartest model to figure out when to go to an open model. But the labs who are in the models presumably don't want [00:19:19] S0: that. And I think, look, I think that all the labs are aligned in many ways to one thing, which is how do you serve the customer? And I think it will be up to the customer to decide. If I have a router where some of the scheduling and harder orchestration happens through frontier model, so be it. But a lot of the kind of line item work can happen through open models and the collaboration of the two together. I think we've seen a ton of projects, whether it's from Sakana AI or Open Router, that have combined the two and seen really good results.