The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:11:26 · Harry Stebbings, Parag Agrawal
[00:11:26] S1: We were talking before about muse and instinct. If there's anything that needs web search, it's personal assistance. It's great. How does that... Yeah. It's great. This is like, woo hoo. This is the greatest thing for my business. How does that change your business? [00:11:40] S0: In our business, right, anytime agents start becoming more useful for more use cases because models either get better or cheaper, it's great for us. Right? Because we bet that agents will be the consumers for the web, and we're we've been building tech for agents. So when agent do more, it's great for our business. So we want models to keep getting better and cheaper so that agents do more and more and more. And if that happens, it's great for our business. If models get smarter, don't the agents beneath them do fewer searches, and then it's worse for your business? I don't think so. If you think of models, there is the trend around models being smarter and then models having more memorized than parametric memory. Like, those are two slightly different dimensions. Today, by and large, I would say models have good recall from parametric memory on, let's call them head facts. Facts. It's like for somebody famous, everything about them, Wikipedia, it can memorize. So it'll tell you who the president was in a certain year. Right? Because a model can memorize those things. The model can tell you what year I graduated from college. Maybe for me it can, but it can't tell you for somebody who works at parallel. [00:12:56] S1: Do you know things if I if I... I mean, if I were
00:12:56 · Harry Stebbings, Parag Agrawal
[00:12:56] S1: Do you know things if I if I... I mean, if I were [00:12:58] S0: to Even if it was in the pretrained data. Seriously? Because it's lossy compression. So what a model's parametric memory is doing? It's lossily compressing to understand patterns in the world. And so it can't memorize every fact in pretraining data. So one, not every fact is in pretraining data. Two, it can't... For even stuff that's in pretraining data, the model's actually trying to find patterns rather than memorize them. And then further, as you make models efficient, which is you make them smaller and smaller while keeping the performance by distilling them or whatever, you lose more of the parametric memory where you try to keep the reasoning. [00:13:33] S1: Do you think we will see models become smaller and smaller than every company have their own model with their own data and the fireworks theory of, you know, own your own intelligence being true? [00:13:44] S0: There are two questions in there. So one, think we're gonna see two things. We're gonna see the biggest or the frontier models be bigger and bigger over time. We are also going to see smaller and smaller models being able to reach any fixed level of performance. So if you say, okay, I want opus 4.8 level of performance. Okay? That's good enough for my use case. Every six months, a much smaller model will be able to deliver that to you. The useful range of sizes of models will be way, way, way different. What does it [00:14:14] S1: mean if the frontier get bigger and bigger? What are the ramifications of that?
00:14:14 · Harry Stebbings, Parag Agrawal
[00:14:14] S1: mean if the frontier get bigger and bigger? What are the ramifications of that? [00:14:18] S0: They are better. All of the ramifications you imagine. The reason the frontier will get bigger and bigger is ultimately the gap between the value for certain use cases incremental quality can provide you. Can be so high in certain use cases, which can be so valuable that it's worth paying for. So if you can build it, there will be use cases for it as long as by making it bigger, you can make it better. It appears there is no end to the scaling law that we can perceive so far. Again, I'm conflating bigger with like models are getting bigger, but they're also able to think longer. So you're just able to throw more compute at the same problem. That will keep happening. I think we'll be able to throw more and more compute at same problem and make the answer be marginally better over time. And so we're gonna just spend a lot of money on extremely large models solving really hard problems. [00:15:11] S1: Do you agree with the consensus view that you'll have 90% of token activity go through open models, but 90% of dollars go through frontier models?
00:15:11 · Harry Stebbings, Parag Agrawal
[00:15:11] S1: Do you agree with the consensus view that you'll have 90% of token activity go through open models, but 90% of dollars go through frontier models? [00:15:21] S0: I don't think it'll be 90% on either of those two, actually. Why? So if you believe my claim that a useful model and the frontier model will be 100x, 1000x often price from each other. So this model is still useful. It is hard to know which use cases over time will get optimized to which scale of model in between. And I think that is a real path dependency in terms of where open models end up. I do think if we had strong confidence that an American built open model was going to be state of the art as far as open models went, I would have more confidence in saying that they'll be pretty good. Do you have confidence in American open models? I want them to exist. So far, it's not clear what's going to happen. But I'm hoping that there will be a great series of American open models and perhaps even competition to have the best American open model. I think what you need is actually not just one person motivated to build an American open model. You need two people competing against each other to build the best American open model. Do you [00:16:28] S1: think there's value in the model routing layer, routing layer as Americans call it? Some people think immense value. Some think commoditization.
00:16:28 · Harry Stebbings, Parag Agrawal
[00:16:28] S1: think there's value in the model routing layer, routing layer as Americans call it? Some people think immense value. Some think commoditization. [00:16:37] S0: Again, there's path dependency there. So today, there is real value. Today, there's real value because when we say routing, we talk about two different dimensions. Which model and which GPU running that model by which vendor? When you're in a world where the demand supply is very feared, People are like hunting for GPUs and people need capacity to serve their customers and people. All of these are like us, like relatively early stage startups growing really rapidly, sometimes beyond what your forecast or predictions say. And then all of a sudden you're looking for capacity. And so you want to be able to get it where you can. And so that creates real value in having some of these routers as log in, they can solve these two problems saying, give me flexibility if I need to get a different source of tokens and push comes to shove. There's no tokens on this model in the SLA I want. I'll switch models as well. So in this moment, it's really valuable. Now, I don't know what happens to overall demand supply on GPUs and tokens. But if it remains this way, the routing layer is really valuable. What do [00:17:44] S1: you think is not so valuable today that will be incredibly valuable in three to five years time? Perhaps data.
00:17:51 · Parag Agrawal
[00:17:51] S0: I think today we don't know how to pay for unique valuable insight or data. Because as intelligence is cheaper, you're going to want to build upon either data or insight that comes from somewhere else with your unique data. Right? So what... Like, if you think of like an abstract notion that, okay, I have intelligence here. I want some data. You want some data and there's some data in the public domain. If we can pull together your insights and my insights and all of this data in the public domain and intelligence on top, we can create something bigger than what you could have done by yourself, what I could have done by myself. So now how do you transact to create this sort of whole that is bigger than the sum of what you could have done yourself and what I could have done or what OpenData was good at. And so that transaction feels like a data transaction to me or an inside transaction to me. And so we don't yet know how to transact that way. So we fall down to, okay, all I can do is use my data and use OpenData and let's see what I can do with it. But if we figured out better ways of pricing data and good things will happen, but it's not something that's yet a market. But what I'm talking about is data transactions at inference time. So it's not necessarily training time. We are building knowledge. Let's let's take an example of today's world, for example. Let's say in your job since I'm sitting with the VC, you probably have access to a pitch book or like a product like that to collect data. And so you you get grounded in knowledge of what's happening and you get to use that data to figure out how to make decisions in addition to all the the notes you have and deal memos you've written perhaps over the years or insights you've had about, like, how to choose founders. And you're essentially composing insights from your personal lived experience, your notes, public data to make decisions. Now you pay PitchBook by the seat, but now you're running a bunch of agents. And your agents can perhaps not access everything you can on PitchBook. All you're doing, like, sort of this to get a browser, perhaps against terms of service to send agent via browser to PitchBook using your auth credentials. And it's inefficient. Inefficient. It's clunky. Right? But clearly, their data is valuable. And clearly, your agent should have convenient access to it.