Y Combinator Startup Podcast
00:52:52 · Jeffrey Morgan, Garry Tan, Harj Taggar, Aaron Epstein
[00:52:52] S0: Yeah. I think it it provides you a set of values you can work around. Especially when you have so much power in your hands with AI, there are just parts of it that AI can help you with, but it won't hold you accountable to it. And And how does software work? And to look at Ollama for example, I'm sure there's versions of Ollama running in the wild from two years ago. How will your software work when somebody falls in love with it and continues using it for two years? Is it still gonna be working? Well hopefully they update to the latest software or it's a cloud service. But I think you build that muscle memory. And we definitely have that from a lot of our more senior engineers on the team who are at VMware or Nicira, for example. But at the same time, I think there are a lot of lessons we learned in the previous generation of DevOps and infrastructure that aren't valid anymore in [00:53:33] S2: the AI world. Oh, Tell us about it. What have you found? [00:53:36] S4: What is not valid anymore? [00:53:37] S0: I think a good example that I classically used is there was this generation of companies called Platform as a Service And and like the 2010s. The Herokus of the world was a great example of this. [00:53:50] S1: I mean, Heroku started out
00:53:50 · Aaron Epstein, Jeffrey Morgan, Garry Tan
[00:53:50] S1: I mean, Heroku started out [00:53:50] S0: as that. Started out as a Platform as a Service. And there's this concept that if you're a layer on top of something else, that you're in kind of a vulnerable position as a startup, which is absolutely not true in the eye world. And in fact, going up the stack can sometimes be even better because you're closer to the customer. In an infrastructure world, that's also the case. And that was like an analogy that we had to like so many of these muscles, we actually had to break building a llama. Another one was, you know, these LMs are never perfect. And like in the systems world, you want everything to be exactly as it's designed to run. It's tested. It's validated. But LMs by definition are not [00:54:25] S2: That's a feature, not a bug. Exactly. [00:54:27] S0: Yeah. It's a feature. [00:54:27] S2: You want it to be a little nondeterministic, I suppose. [00:54:30] S0: And I think from building a team too, it's that with AI now, there are just problems that you don't need to staff as heavily, whereas you did ten years ago, right? If you think about what does your customer support pipeline look like? What does it look like to deliver a cloud service? Like it's a very different world with AI because how do you build a service where no engineer knows exactly how all the code works, which is obviously the case now? So there's just new lessons we're learning going from like a... Some of our team from infrastructure 1.0 in the 2000s to cloud in the 2010s to now the AI space. There are a lot of rules that break.
00:55:07 · Garry Tan
[00:55:07] S2: I mean, you're probably actually doing an incredible service to like, both sides of the ecosystem, and that, like, the end users get this, like, very clean thing that just works, especially, like, the tokens just come out, and they're very clean, and the API makes sense, and it's rational and logical. And then on the flip side, like, I mean, if you don't have a layer like Ollama, I've directly experienced this, where it's like, oh, yeah, the underlying inference provider has a weird error for, you know, if you put this parameter in this way, or it expects JSON, and, you know, it's not documented. It's just like this insane minefield. Like, the agents can kinda figure it out, but you're gonna bang your head into the wall for a couple hours before the agent figures it out. And in the meantime, you're like, this is a terrible experience, you know? And so you're in there probably helping the inference providers fix all these fundamental bugs too.
00:55:57 · Jeffrey Morgan, Garry Tan
[00:55:57] S0: Yeah. And it's part of the the job we do. And I think one of the big opportunities in the open model landscape is curation and taking a fragmented universe of models and inference technology and cloud services and harnesses and making that actually just work is a really valuable problem. Because the end developer, to your point, they just wanna build their software, They just wanna build stuff, they wanna build their next company, their next application. And I think that's where we come in, but it's where a ton of great services also come in. And we saw OpenRouter obviously is a good example of that from a wide model selection, so developer doesn't have to sign up for 100 different providers, they can just go to one, they can pay in one place. I think we've seen with open code, you can have one harness that integrates with any model. It's a really powerful experience for developers just looking to try the next model to see if it solves their use case better. So this curation and when there's an abundance of models and providers, now there's a scarcity in bringing that together into something that works. [00:56:58] S2: Thank you so much for joining us. That's all we have time for. [00:57:01] S0: Thank you guys for having me.