Y Combinator Startup Podcast
00:02:17 · Jeffrey Morgan, Harj Taggar
[00:02:17] S0: Is there anything more to it? Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star. Cost is something they can solve in the short term, but it then enables them to then go and customize these models for their unique use case. [00:02:39] S4: Is there a particular large enterprise that you can name that has done this? [00:02:44] S0: I think there was a great article in the information yesterday from AT&T, and it ends up they've already shifted 40% of their token consumption to open models. And that's right now predominantly through US and Europe models, but they're also evaluating the Chinese models. [00:03:01] S4: What kind of workflows do they run? [00:03:03] S0: Predominantly coding agents. I think what we've seen just from the extreme growth and per developer or per user token usage has predominantly been from coding agents. And then earlier in March and April, saw OpenClaw take off and subsequently the Hermes agent project take off, which has then opened up that ability to automate a huge chunk of work over a long span of time to non developers too, whether it's finance or support or marketing or sales. [00:03:31] S4: You had this actually very cool graph on the takeoff exponential for OpenClaw. [00:03:37] S0: Yeah. So earlier this year, this is a graph of token usage by developer on Ollama's cloud, the average amount of tokens they use per week.
00:03:37 · Jeffrey Morgan, Jared Friedman, Harj Taggar
[00:03:37] S0: Yeah. So earlier this year, this is a graph of token usage by developer on Ollama's cloud, the average amount of tokens they use per week. [00:03:47] S3: We kind Which of is per developer. So like, it looks... This graph looks like it should be an aggregate of Ollama's growth, that this is actually the per user. [00:03:55] S0: Exactly. Right. This is on an individual user fit... User basis. How many tokens are they using a week? And so there's two big inflection points. One is that initial run up at the start of the year, which was driven by coding agents. So we saw Kimi, the GLM models, MiniMax launch. Finally, had open models that could power coding agents. And then in April, we saw this incredible growth from OpenClaw really, which was then not just developers, but the rest of the world could take a hard problem, give it to an open model and let it go complete the task, which obviously consumes a ton of tokens as it's figuring out what tools to use, what data to go fetch. We went from a context window of 128 to a million plus with open models, and so all that enabled this explosive growth. [00:04:39] S4: So it went from roughly five x, so under somewhere around, I don't know, 15,000,000 tokens before before all these co work type of use cases.
00:04:39 · Harj Taggar, Jeffrey Morgan, Aaron Epstein
[00:04:39] S4: So it went from roughly five x, so under somewhere around, I don't know, 15,000,000 tokens before before all these co work type of use cases. [00:04:48] S0: I think that's about right for that OpenClaw jump we saw in April. Obviously, in aggregate, it's in the 10 to 20x, if not more. As a whole, through Ollama's Cloud, saw 150x since the start of the year. And so it's just the big interesting thing here is this huge surge in demand for open models. Whereas open models, I'd say in 2024, 2025, from the large models being served were mostly being served as custom models. So you'd take an off the shelf model like DeepSeek or Kimi, and you'd fine tune it for your use case. Like for example, Cursor had famously done. And from there, you could serve that at scale. But seeing out of the box open models being served, that really only took off at the start of this year. [00:05:31] S1: Even since we started this podcast, these things sort of come in cycles maybe. Like I feel like early twenty twenty four, there was lots of interest in fine tuning your own custom models, then it sort of went away and it's like, all that will just be wasted effort or get stomped by the next model release. Seems like it's coming back now. You have a front seat to all of it. Like what's... Do you think we're going through like another cycle or is it here to stay this time? [00:05:55] S0: I think the release of these models, the cadence is only speeding up. Which makes it ever more harder to stay on top of that and have a post
00:05:55 · Jeffrey Morgan, Aaron Epstein, Garry Tan
[00:05:55] S0: I think the release of these models, the cadence is only speeding up. Which makes it ever more harder to stay on top of that and have a post [00:06:04] S1: You mean that the latest closed frontier models are releasing faster [00:06:08] S2: than that? [00:06:08] S0: I think on the open source side, it's getting faster and faster. This summer, we've already seen three iterations of the DeepSeek flash model What as an used to be more of a six month cycle. [00:06:20] S1: Yeah, the gap is like closing essentially. [00:06:22] S0: It's closing. And I think that makes it even harder to custom train models. On the flip side, I think the tooling is getting better. And so it allows teams that wanna fine tune their models to stay on top of it. [00:06:31] S2: I mean, we're also entering this moment where AI safety is becoming more and more of an issue at the Frontier labs. So the Frontier may well slow down to figure out its alignment and containment issues. And then meanwhile, the open source models and open weight models are continuing to grow and get better.
00:06:31 · Garry Tan, Jeffrey Morgan
[00:06:31] S2: I mean, we're also entering this moment where AI safety is becoming more and more of an issue at the Frontier labs. So the Frontier may well slow down to figure out its alignment and containment issues. And then meanwhile, the open source models and open weight models are continuing to grow and get better. [00:06:51] S0: Yeah. And we saw the announcement and the release of the GLM 5.3 model and its capabilities from a cybersecurity standpoint being extremely impressive. It creates a big opportunity for whether it's startups or existing businesses in the security and governance space to really jump in and help. Because I think if you look at that AT&T article I was speaking about, largely, the blocker to adopting open models is largely around security and safety. But from our experience talking to customers, whether it's in Europe, whether it's here in The US, if you can solve the safety problems, by and large, adopting the Chinese origin model labs is completely on the table and it's really exciting for these businesses. [00:07:27] S2: There's sort of this interesting moment right now where a Hugging Face had to use open weight models to actually even detect the hack from the frontier. [00:07:36] S0: A common question we get is like, well, where where can I use open models that are leaps and bounds of an advantage over using a frontier closed model? And one of the key use cases is security testing, and making sure that your software's secure.
00:07:36 · Jeffrey Morgan, Jared Friedman, Garry Tan
[00:07:36] S0: A common question we get is like, well, where where can I use open models that are leaps and bounds of an advantage over using a frontier closed model? And one of the key use cases is security testing, and making sure that your software's secure. [00:07:51] S3: Yeah, because like if you try to get Claude to like pen test your product, it will just refuse to do that. [00:07:56] S0: Correct, yeah, by and large. [00:07:57] S2: Whereas there are literally obliterated security researcher models that you can find on Hugging Face that allow you to do it. [00:08:04] S0: That is true. There are ones that are custom trained to be even more liberal to go and attack these problems. But even the out of the box models, they do come with safety training, but they're a little better understanding if you're doing this for a good use case versus one that's more of a negative or malicious use case. [00:08:26] S3: A cool thing about Ollama is that because you guys are such a key distribution channel for these models, my understanding is that typically the model developers are contacting you before the general release to like coordinate launches and stuff like that, and you often get sort of like previews of what's about to about to happen, and you get these incredible growth spikes when a new model drops. Like, I wonder if you could like tell us a bit about like, what it's like to operate this thing at scale.