The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:36:51 · Harry Stebbings
[00:36:51] S1: Do you buy Yan Lakun's post transformer world theory? And how does that change everything?
00:36:57 · Naveen Rao
[00:36:57] S0: I do think we will be in a post transformer world. I don't think the transformer is the be all end all. I don't think I truly understood this until maybe a couple of years ago because transformers conflate a lot of things. But what we care about is reasoning, being able to reason through a problem. I don't know who said it, but it's like intelligence is what you do when you don't have the information, not what you do when you do have the information. So being able to reason through a problem from some sort of mechanistic or first principles understanding is what intelligence is. So transformers kind of sort of have that, but they really do it through this agentic kind of flow where you're taking multiple samples and you're deciding what's good. So I take one trace and then compare it to another trace, and I say, okay, this trace is better. So I'm gonna call that my reasoning trace. And so the way we kind of implement this reasoning thing is through a retrieval mechanism. So attention basically has this this context, which is a retrieval thing, and then we we can sample off of that and eventually get to something that looks like reasoning. I think this is extremely inefficient for how we do reasoning. We don't need to build reasoning this way. Our our brains do not work in this way. We don't build a huge trace, pump it into the input of another model, and then produce another trace. That's not how our brain fork. So there are way more efficient ways to do this and way more high performance from an algorithmic stand... Standpoint ways to do this. And so we will move past the transformer.
00:38:18 · Harry Stebbings, Naveen Rao
[00:38:18] S1: Do you think what it takes to be a great founder has changed or not? [00:38:22] S0: I don't think it's changed. I think the the basic principles are still the same. I think having a viewpoint on the future, the... Your vision actually may matter more now than it did before. You know, I think, in the pre AI world, you could be a a pretty good founder by just being able to execute. You don't have to have a very good vision. You just have to know how to execute. And, you know, if you just go out execute a big company, which isn't that hard because big companies are slow and, you know, lumbering, think I you could actually build a startup, and you could, you could sell it or or build a company out of it. I think these days, if you're just based on execution, I I don't think you're gonna get very far. [00:38:56] S1: We spoke about the different layers. You said, I don't think this is gonna get commoditized in data. I think specialization, I think there is value there. Which layer does get commoditized over time?
00:38:56 · Harry Stebbings, Naveen Rao
[00:38:56] S1: We spoke about the different layers. You said, I don't think this is gonna get commoditized in data. I think specialization, I think there is value there. Which layer does get commoditized over time? [00:39:05] S0: Yeah. It's a good question. I mean, arguably, energy is already a commodity. Right? So this is already commoditized. I think a certain amount of intelligence does actually start to become commoditized. We're already seeing it, to be honest. Like, models that are six months old, let's say the Chinese open models, you know, that level, they're pretty useful. Right? Something at six months old is still pretty darn good. So I would argue that's that's a hallmark of, commoditization. Right? The frontier is not the only thing that matters. For certain problems, is. But in six months, that frontier will be in the open models. [00:39:36] S1: So then help me out. What happens if the delivery mechanism or the output of all the infrastructure is then commoditized?
00:39:36 · Harry Stebbings, Naveen Rao
[00:39:36] S1: So then help me out. What happens if the delivery mechanism or the output of all the infrastructure is then commoditized? [00:39:43] S0: Well, I I think commoditization doesn't necessarily mean it's bad. It just means that margin, yeah, falls quite lot, but it also means that the market is very big. So you you could argue that tomatoes are commoditized, but we still eat them. We still need them. You know? So we will have a huge market for intelligence, and that market will continue to grow, but the margins may not be huge. I I actually don't think the model companies will be the ones who own the margin of the whole stack. It's not even true today. Like, NVIDIA owns the margin of the stack today. Right? I would say they're probably the one who take the lion's share of the of the margin. [00:40:17] S1: A A lot of people say to me the cost of inference will actually go massively down over the next few years, and we'll see the commoditization potentially of the inference market. Would you agree with that or not not agree with that? [00:40:28] S0: Well, let's break it down. What is in... What is commoditization of the inference market mean? What that means is that the operator who's who's running inference, I assume. Yeah? [00:40:37] S1: Yeah. Absolutely. Your Fireworks, your model, your base TANs, and the services that they provide.
00:40:37 · Harry Stebbings, Naveen Rao
[00:40:37] S1: Yeah. Absolutely. Your Fireworks, your model, your base TANs, and the services that they provide. [00:40:42] S0: Because if you look at their value prop, what they've done, and they've done very well, is take a model that exists, an open model, say, and then make it work well, you know, with uptime guarantees and make it, you know, relatively cost efficient to run on infrastructure. So they're an infra play. Right? They're basically mapping the model onto physical hardware and doing that well. Now I think there will be value there in the future, but the price of inference, like what people are willing to pay per token, will drop. And so they're basically... They have to squeeze more margin out of the hardware's capabilities to the delivery of that token. And that, you know, the amount of juice in that layer, I think, will decrease. So I think it'll result in them having lower margins. And and I think the model itself, like, is commoditized. You have several different ways to get it now. But the hardware side of it probably won't won't collapse. I think that part of it is still gonna have an advantage because you can still squeeze more juice out of, hey. I convert energy to intelligence more efficiently. The... We have more... A lot more juice to squeeze there versus the infrastructure layer in my view. I actually think the hardware is the place to be over the next ten years. [00:41:45] S1: Dude, it's quite fucking hard doing hardware investing. It's a very different mindset. Is this generation of investors equipped to be hardware investors?