Y Combinator Startup Podcast
00:04:39 · Harj Taggar, Jeffrey Morgan, Aaron Epstein
[00:04:39] S4: So it went from roughly five x, so under somewhere around, I don't know, 15,000,000 tokens before before all these co work type of use cases. [00:04:48] S0: I think that's about right for that OpenClaw jump we saw in April. Obviously, in aggregate, it's in the 10 to 20x, if not more. As a whole, through Ollama's Cloud, saw 150x since the start of the year. And so it's just the big interesting thing here is this huge surge in demand for open models. Whereas open models, I'd say in 2024, 2025, from the large models being served were mostly being served as custom models. So you'd take an off the shelf model like DeepSeek or Kimi, and you'd fine tune it for your use case. Like for example, Cursor had famously done. And from there, you could serve that at scale. But seeing out of the box open models being served, that really only took off at the start of this year. [00:05:31] S1: Even since we started this podcast, these things sort of come in cycles maybe. Like I feel like early twenty twenty four, there was lots of interest in fine tuning your own custom models, then it sort of went away and it's like, all that will just be wasted effort or get stomped by the next model release. Seems like it's coming back now. You have a front seat to all of it. Like what's... Do you think we're going through like another cycle or is it here to stay this time? [00:05:55] S0: I think the release of these models, the cadence is only speeding up. Which makes it ever more harder to stay on top of that and have a post
00:05:55 · Jeffrey Morgan, Aaron Epstein, Garry Tan
[00:05:55] S0: I think the release of these models, the cadence is only speeding up. Which makes it ever more harder to stay on top of that and have a post [00:06:04] S1: You mean that the latest closed frontier models are releasing faster [00:06:08] S2: than that? [00:06:08] S0: I think on the open source side, it's getting faster and faster. This summer, we've already seen three iterations of the DeepSeek flash model What as an used to be more of a six month cycle. [00:06:20] S1: Yeah, the gap is like closing essentially. [00:06:22] S0: It's closing. And I think that makes it even harder to custom train models. On the flip side, I think the tooling is getting better. And so it allows teams that wanna fine tune their models to stay on top of it. [00:06:31] S2: I mean, we're also entering this moment where AI safety is becoming more and more of an issue at the Frontier labs. So the Frontier may well slow down to figure out its alignment and containment issues. And then meanwhile, the open source models and open weight models are continuing to grow and get better.
00:06:31 · Garry Tan, Jeffrey Morgan
[00:06:31] S2: I mean, we're also entering this moment where AI safety is becoming more and more of an issue at the Frontier labs. So the Frontier may well slow down to figure out its alignment and containment issues. And then meanwhile, the open source models and open weight models are continuing to grow and get better. [00:06:51] S0: Yeah. And we saw the announcement and the release of the GLM 5.3 model and its capabilities from a cybersecurity standpoint being extremely impressive. It creates a big opportunity for whether it's startups or existing businesses in the security and governance space to really jump in and help. Because I think if you look at that AT&T article I was speaking about, largely, the blocker to adopting open models is largely around security and safety. But from our experience talking to customers, whether it's in Europe, whether it's here in The US, if you can solve the safety problems, by and large, adopting the Chinese origin model labs is completely on the table and it's really exciting for these businesses. [00:07:27] S2: There's sort of this interesting moment right now where a Hugging Face had to use open weight models to actually even detect the hack from the frontier. [00:07:36] S0: A common question we get is like, well, where where can I use open models that are leaps and bounds of an advantage over using a frontier closed model? And one of the key use cases is security testing, and making sure that your software's secure.
00:07:36 · Jeffrey Morgan, Jared Friedman, Garry Tan
[00:07:36] S0: A common question we get is like, well, where where can I use open models that are leaps and bounds of an advantage over using a frontier closed model? And one of the key use cases is security testing, and making sure that your software's secure. [00:07:51] S3: Yeah, because like if you try to get Claude to like pen test your product, it will just refuse to do that. [00:07:56] S0: Correct, yeah, by and large. [00:07:57] S2: Whereas there are literally obliterated security researcher models that you can find on Hugging Face that allow you to do it. [00:08:04] S0: That is true. There are ones that are custom trained to be even more liberal to go and attack these problems. But even the out of the box models, they do come with safety training, but they're a little better understanding if you're doing this for a good use case versus one that's more of a negative or malicious use case. [00:08:26] S3: A cool thing about Ollama is that because you guys are such a key distribution channel for these models, my understanding is that typically the model developers are contacting you before the general release to like coordinate launches and stuff like that, and you often get sort of like previews of what's about to about to happen, and you get these incredible growth spikes when a new model drops. Like, I wonder if you could like tell us a bit about like, what it's like to operate this thing at scale.
00:08:52 · Jeffrey Morgan
[00:08:52] S0: Yeah, absolutely. And like I said before, the models are coming out faster, and faster, and faster. And so we've developed a playbook to what does a successful day zero model launch look like? And there are a lot of things to get right. There's making sure that it's supported and your favorite inference engine, which is generally sometimes a multi week process to make sure it's fast, to make sure it's accurate, to make sure it's up to spec with the reference. There's also finding the right use cases and harnesses for developers and users to make use of this model. Generally these models have new capabilities. This morning, there's announced DeepSeek launched their first multimodal model from a large model LLM standpoint. They had previously had some smaller OCR models, which unlocks a whole bunch of use cases. But what they also changed was the DeepSeek harness which launched recently so that it could support this capability. So step two is then to find the harnesses, make sure they're prepared to actually run this model and to do that effectively. But every model is different, they all have different challenges and architecture changes and tool calling mechanics and getting all this right is really hard. I think the key thing to do at the end of the day is to run benchmarks against the final product ahead of release and make sure that it's running as the research team at the lab specified.
00:10:03 · Harj Taggar
[00:10:03] S4: So I think one very important part that you play in the whole ecosystem is you kinda create a very legible standard to be able to make sure you have the best way to use the hardness for each new model, each new tool call, and all of it is consistent across all the different models, which is pretty hard to do.