Y Combinator Startup Podcast
00:00:00 · Jeffrey Morgan, Aaron Epstein, Garry Tan
[00:00:00] S0: Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star. Cost is something they can solve in the short term, but it then enables them to then go and customize these models for their unique use case. [00:00:20] S1: Early twenty twenty four, there was lots of interest in fine tuning your own custom models. Then it sort of went away and all of that will just be wasted effort or get stomped by the next model release. Seems like it's coming back now. You have a front seat to all of it. Do you think we're going through like another cycle or is it here to stay this time? [00:00:43] S2: Welcome back to another episode of the Lightcone. Today, we're talking to Jeffrey Morgan, cofounder and CEO of Ollama, the easiest way to run open source AI models locally and in the cloud. Ollama is used by 9,000,000 developers, has a 178,000 GitHub stars, and is used by 85% of the Fortune 500, which means Jeff knows a lot about the state of the art of AI, what models score highest on benchmarks, and what developers actually download and keep using. Jeff, welcome to the Lightcone. [00:01:16] S0: Thank you for having me. [00:01:17] S2: We're down to here. Like, what is the state of the art? What are you seeing out there?
00:01:17 · Garry Tan, Jeffrey Morgan, Jared Friedman, Aaron Epstein
[00:01:17] S2: We're down to here. Like, what is the state of the art? What are you seeing out there? [00:01:20] S0: Well, I think the biggest thing we're seeing is a shift to open models, especially in enterprise. And that's from a mix of US and Chinese origin models. And it's predominantly driven by coding agents and also AI assistance, more co work use cases like OpenClaw and Hermes. [00:01:36] S3: And because you sit in the token flow of like so many tokens, you have really good data on what models people are actually using, and how it's changing. What are the trends that you're seeing? [00:01:46] S0: Yeah, you know, Ollama started as a way to run open models on your MacBook or other hardware, Nvidia, AMD, Intel, And earlier this year we launched Ollama's Cloud. And what we're seeing there is that it's predominantly Chinese models right now of Chinese origin, but they're being accessed by businesses all over the world. Especially US and Germany is actually a big source of where open model tokens are being accessed. [00:02:13] S1: Is it all about cost? Are enterprises coming because they just want to get the cost down? [00:02:17] S0: Is there anything more to it? Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star. Cost is something they can solve in the short term, but it then enables them to then go and customize these models for their unique use case.
00:02:17 · Jeffrey Morgan, Harj Taggar
[00:02:17] S0: Is there anything more to it? Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star. Cost is something they can solve in the short term, but it then enables them to then go and customize these models for their unique use case. [00:02:39] S4: Is there a particular large enterprise that you can name that has done this? [00:02:44] S0: I think there was a great article in the information yesterday from AT&T, and it ends up they've already shifted 40% of their token consumption to open models. And that's right now predominantly through US and Europe models, but they're also evaluating the Chinese models. [00:03:01] S4: What kind of workflows do they run? [00:03:03] S0: Predominantly coding agents. I think what we've seen just from the extreme growth and per developer or per user token usage has predominantly been from coding agents. And then earlier in March and April, saw OpenClaw take off and subsequently the Hermes agent project take off, which has then opened up that ability to automate a huge chunk of work over a long span of time to non developers too, whether it's finance or support or marketing or sales. [00:03:31] S4: You had this actually very cool graph on the takeoff exponential for OpenClaw. [00:03:37] S0: Yeah. So earlier this year, this is a graph of token usage by developer on Ollama's cloud, the average amount of tokens they use per week.
00:03:37 · Jeffrey Morgan, Jared Friedman, Harj Taggar
[00:03:37] S0: Yeah. So earlier this year, this is a graph of token usage by developer on Ollama's cloud, the average amount of tokens they use per week. [00:03:47] S3: We kind Which of is per developer. So like, it looks... This graph looks like it should be an aggregate of Ollama's growth, that this is actually the per user. [00:03:55] S0: Exactly. Right. This is on an individual user fit... User basis. How many tokens are they using a week? And so there's two big inflection points. One is that initial run up at the start of the year, which was driven by coding agents. So we saw Kimi, the GLM models, MiniMax launch. Finally, had open models that could power coding agents. And then in April, we saw this incredible growth from OpenClaw really, which was then not just developers, but the rest of the world could take a hard problem, give it to an open model and let it go complete the task, which obviously consumes a ton of tokens as it's figuring out what tools to use, what data to go fetch. We went from a context window of 128 to a million plus with open models, and so all that enabled this explosive growth. [00:04:39] S4: So it went from roughly five x, so under somewhere around, I don't know, 15,000,000 tokens before before all these co work type of use cases.
00:04:39 · Harj Taggar, Jeffrey Morgan, Aaron Epstein
[00:04:39] S4: So it went from roughly five x, so under somewhere around, I don't know, 15,000,000 tokens before before all these co work type of use cases. [00:04:48] S0: I think that's about right for that OpenClaw jump we saw in April. Obviously, in aggregate, it's in the 10 to 20x, if not more. As a whole, through Ollama's Cloud, saw 150x since the start of the year. And so it's just the big interesting thing here is this huge surge in demand for open models. Whereas open models, I'd say in 2024, 2025, from the large models being served were mostly being served as custom models. So you'd take an off the shelf model like DeepSeek or Kimi, and you'd fine tune it for your use case. Like for example, Cursor had famously done. And from there, you could serve that at scale. But seeing out of the box open models being served, that really only took off at the start of this year. [00:05:31] S1: Even since we started this podcast, these things sort of come in cycles maybe. Like I feel like early twenty twenty four, there was lots of interest in fine tuning your own custom models, then it sort of went away and it's like, all that will just be wasted effort or get stomped by the next model release. Seems like it's coming back now. You have a front seat to all of it. Like what's... Do you think we're going through like another cycle or is it here to stay this time? [00:05:55] S0: I think the release of these models, the cadence is only speeding up. Which makes it ever more harder to stay on top of that and have a post
00:05:55 · Jeffrey Morgan, Aaron Epstein, Garry Tan
[00:05:55] S0: I think the release of these models, the cadence is only speeding up. Which makes it ever more harder to stay on top of that and have a post [00:06:04] S1: You mean that the latest closed frontier models are releasing faster [00:06:08] S2: than that? [00:06:08] S0: I think on the open source side, it's getting faster and faster. This summer, we've already seen three iterations of the DeepSeek flash model What as an used to be more of a six month cycle. [00:06:20] S1: Yeah, the gap is like closing essentially. [00:06:22] S0: It's closing. And I think that makes it even harder to custom train models. On the flip side, I think the tooling is getting better. And so it allows teams that wanna fine tune their models to stay on top of it. [00:06:31] S2: I mean, we're also entering this moment where AI safety is becoming more and more of an issue at the Frontier labs. So the Frontier may well slow down to figure out its alignment and containment issues. And then meanwhile, the open source models and open weight models are continuing to grow and get better.