The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:10:57 · Parag Agrawal, Harry Stebbings
[00:10:57] S0: There is internal data, but there is case law. There is facts. There's facts about companies, facts about people, and you have to go exclude information. So you have to be very comprehensive in law to say, I want to be confident that despite a lot of effort, you can't find this. Insurance underwriting has the same flavor. So there's... Sales is very web search heavy. AI science is very web search heavy. So on a relative basis, inference goes more in compute. But when you think of web search, all of these others start popping too. [00:11:26] S1: We were talking before about muse and instinct. If there's anything that needs web search, it's personal assistance. It's great. How does that... Yeah. It's great. This is like, woo hoo. This is the greatest thing for my business. How does that change your business?
00:11:26 · Harry Stebbings, Parag Agrawal
[00:11:26] S1: We were talking before about muse and instinct. If there's anything that needs web search, it's personal assistance. It's great. How does that... Yeah. It's great. This is like, woo hoo. This is the greatest thing for my business. How does that change your business? [00:11:40] S0: In our business, right, anytime agents start becoming more useful for more use cases because models either get better or cheaper, it's great for us. Right? Because we bet that agents will be the consumers for the web, and we're we've been building tech for agents. So when agent do more, it's great for our business. So we want models to keep getting better and cheaper so that agents do more and more and more. And if that happens, it's great for our business. If models get smarter, don't the agents beneath them do fewer searches, and then it's worse for your business? I don't think so. If you think of models, there is the trend around models being smarter and then models having more memorized than parametric memory. Like, those are two slightly different dimensions. Today, by and large, I would say models have good recall from parametric memory on, let's call them head facts. Facts. It's like for somebody famous, everything about them, Wikipedia, it can memorize. So it'll tell you who the president was in a certain year. Right? Because a model can memorize those things. The model can tell you what year I graduated from college. Maybe for me it can, but it can't tell you for somebody who works at parallel. [00:12:56] S1: Do you know things if I if I... I mean, if I were
00:12:56 · Harry Stebbings, Parag Agrawal
[00:12:56] S1: Do you know things if I if I... I mean, if I were [00:12:58] S0: to Even if it was in the pretrained data. Seriously? Because it's lossy compression. So what a model's parametric memory is doing? It's lossily compressing to understand patterns in the world. And so it can't memorize every fact in pretraining data. So one, not every fact is in pretraining data. Two, it can't... For even stuff that's in pretraining data, the model's actually trying to find patterns rather than memorize them. And then further, as you make models efficient, which is you make them smaller and smaller while keeping the performance by distilling them or whatever, you lose more of the parametric memory where you try to keep the reasoning. [00:13:33] S1: Do you think we will see models become smaller and smaller than every company have their own model with their own data and the fireworks theory of, you know, own your own intelligence being true? [00:13:44] S0: There are two questions in there. So one, think we're gonna see two things. We're gonna see the biggest or the frontier models be bigger and bigger over time. We are also going to see smaller and smaller models being able to reach any fixed level of performance. So if you say, okay, I want opus 4.8 level of performance. Okay? That's good enough for my use case. Every six months, a much smaller model will be able to deliver that to you. The useful range of sizes of models will be way, way, way different. What does it [00:14:14] S1: mean if the frontier get bigger and bigger? What are the ramifications of that?
00:14:14 · Harry Stebbings, Parag Agrawal
[00:14:14] S1: mean if the frontier get bigger and bigger? What are the ramifications of that? [00:14:18] S0: They are better. All of the ramifications you imagine. The reason the frontier will get bigger and bigger is ultimately the gap between the value for certain use cases incremental quality can provide you. Can be so high in certain use cases, which can be so valuable that it's worth paying for. So if you can build it, there will be use cases for it as long as by making it bigger, you can make it better. It appears there is no end to the scaling law that we can perceive so far. Again, I'm conflating bigger with like models are getting bigger, but they're also able to think longer. So you're just able to throw more compute at the same problem. That will keep happening. I think we'll be able to throw more and more compute at same problem and make the answer be marginally better over time. And so we're gonna just spend a lot of money on extremely large models solving really hard problems. [00:15:11] S1: Do you agree with the consensus view that you'll have 90% of token activity go through open models, but 90% of dollars go through frontier models?
00:15:11 · Harry Stebbings, Parag Agrawal
[00:15:11] S1: Do you agree with the consensus view that you'll have 90% of token activity go through open models, but 90% of dollars go through frontier models? [00:15:21] S0: I don't think it'll be 90% on either of those two, actually. Why? So if you believe my claim that a useful model and the frontier model will be 100x, 1000x often price from each other. So this model is still useful. It is hard to know which use cases over time will get optimized to which scale of model in between. And I think that is a real path dependency in terms of where open models end up. I do think if we had strong confidence that an American built open model was going to be state of the art as far as open models went, I would have more confidence in saying that they'll be pretty good. Do you have confidence in American open models? I want them to exist. So far, it's not clear what's going to happen. But I'm hoping that there will be a great series of American open models and perhaps even competition to have the best American open model. I think what you need is actually not just one person motivated to build an American open model. You need two people competing against each other to build the best American open model. Do you [00:16:28] S1: think there's value in the model routing layer, routing layer as Americans call it? Some people think immense value. Some think commoditization.
00:16:28 · Harry Stebbings, Parag Agrawal
[00:16:28] S1: think there's value in the model routing layer, routing layer as Americans call it? Some people think immense value. Some think commoditization. [00:16:37] S0: Again, there's path dependency there. So today, there is real value. Today, there's real value because when we say routing, we talk about two different dimensions. Which model and which GPU running that model by which vendor? When you're in a world where the demand supply is very feared, People are like hunting for GPUs and people need capacity to serve their customers and people. All of these are like us, like relatively early stage startups growing really rapidly, sometimes beyond what your forecast or predictions say. And then all of a sudden you're looking for capacity. And so you want to be able to get it where you can. And so that creates real value in having some of these routers as log in, they can solve these two problems saying, give me flexibility if I need to get a different source of tokens and push comes to shove. There's no tokens on this model in the SLA I want. I'll switch models as well. So in this moment, it's really valuable. Now, I don't know what happens to overall demand supply on GPUs and tokens. But if it remains this way, the routing layer is really valuable. What do [00:17:44] S1: you think is not so valuable today that will be incredibly valuable in three to five years time? Perhaps data.