The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:34:15 · Harry Stebbings, Naveen Rao
[00:34:15] S1: Do you think they can afford to in the day by day race that is this current ecosystem? [00:34:21] S0: That's the problem. It's really fast. I mean, you've gotta be on top of it. I mean, like, if you're one month late on a model, that's that's disaster if you're OpenAI or Anthropic or Grok or Google. They can't afford to take those big risks just yet. You know, you're seeing this, like, you know, when Elon spun up, x AI, he basically took, okay. Here's the paradigm. Let's go let's go hire the people, and boom. He got to the frontier pretty fast because he had the resources. So what that points out to me was that there wasn't a ton of, like, very deep innovation. It was taking the pieces, putting them together with a team that knows how to build them and executing very fast. I think that's very different than trying to come up with a new paradigm. And, you know, you look at, like, Yanl Kun's company and, you know, Fei Li's company, they were they were trying to do deeper innovations. And I think that's what... That might be in the realm of startups, and it starts starts to look like drug development. Right? Like, all the big risks are taken by startups, and all the incremental stuff is done by big companies. The big companies buy the startups when they show promise. [00:35:16] S1: I sometimes ask simple questions, but I don't know the answers, and you're very smart. Why does everyone get so excited by RL today? You mentioned it kind of the mid learning there, and I just have so many people. I I just... It's all about RL. I'm like, oh, I I agree. Yeah. I agree.
00:35:32 · Naveen Rao
[00:35:32] S0: There's several reasons. And if you think about the way we learn, it's primarily reinforcement learning. If I were to teach you a new skill, you know, learn how to serve a tennis ball or something, it's it's really an RL loop. I would give you a paradigm of what it should look like. Here's how your serve should look, maybe demonstrate that or show you a video or something, and it will allow you to build an internal model of what good is or great is. And then you keep judging yourself against that that internal model and say, yes, I'm getting there. I'm getting better or I'm not, and I need to, like, ask for help. So that is RL, and I think the reason it's important is we cannot gather enough data to solve every task in the world. We simply can't. You can't pre train your way out of this by just gathering data. It's just too hard. Models, frankly, aren't efficient enough to to make use of it. So you need to have a a paradigm where you can start to say, okay. These are the 10 most important tasks. And then I have some sort of what we call a verifiable reward. I have some... Something that's that internal model. Like, a good serve is I hit it really hard and it went in. It's very... It's verifiable. I can measure the speed. I can measure whether it was in or out of bounds. When I have that verifiable reward, then I can start to say, okay, did the actions I took, did those actions lead to that good outcome or not? So now I I have a structure that allows me to solve any set of general tasks, and that... That's why RL is so exciting.
00:36:51 · Harry Stebbings
[00:36:51] S1: Do you buy Yan Lakun's post transformer world theory? And how does that change everything?
00:36:57 · Naveen Rao
[00:36:57] S0: I do think we will be in a post transformer world. I don't think the transformer is the be all end all. I don't think I truly understood this until maybe a couple of years ago because transformers conflate a lot of things. But what we care about is reasoning, being able to reason through a problem. I don't know who said it, but it's like intelligence is what you do when you don't have the information, not what you do when you do have the information. So being able to reason through a problem from some sort of mechanistic or first principles understanding is what intelligence is. So transformers kind of sort of have that, but they really do it through this agentic kind of flow where you're taking multiple samples and you're deciding what's good. So I take one trace and then compare it to another trace, and I say, okay, this trace is better. So I'm gonna call that my reasoning trace. And so the way we kind of implement this reasoning thing is through a retrieval mechanism. So attention basically has this this context, which is a retrieval thing, and then we we can sample off of that and eventually get to something that looks like reasoning. I think this is extremely inefficient for how we do reasoning. We don't need to build reasoning this way. Our our brains do not work in this way. We don't build a huge trace, pump it into the input of another model, and then produce another trace. That's not how our brain fork. So there are way more efficient ways to do this and way more high performance from an algorithmic stand... Standpoint ways to do this. And so we will move past the transformer.
00:38:18 · Harry Stebbings, Naveen Rao
[00:38:18] S1: Do you think what it takes to be a great founder has changed or not? [00:38:22] S0: I don't think it's changed. I think the the basic principles are still the same. I think having a viewpoint on the future, the... Your vision actually may matter more now than it did before. You know, I think, in the pre AI world, you could be a a pretty good founder by just being able to execute. You don't have to have a very good vision. You just have to know how to execute. And, you know, if you just go out execute a big company, which isn't that hard because big companies are slow and, you know, lumbering, think I you could actually build a startup, and you could, you could sell it or or build a company out of it. I think these days, if you're just based on execution, I I don't think you're gonna get very far. [00:38:56] S1: We spoke about the different layers. You said, I don't think this is gonna get commoditized in data. I think specialization, I think there is value there. Which layer does get commoditized over time?
00:38:56 · Harry Stebbings, Naveen Rao
[00:38:56] S1: We spoke about the different layers. You said, I don't think this is gonna get commoditized in data. I think specialization, I think there is value there. Which layer does get commoditized over time? [00:39:05] S0: Yeah. It's a good question. I mean, arguably, energy is already a commodity. Right? So this is already commoditized. I think a certain amount of intelligence does actually start to become commoditized. We're already seeing it, to be honest. Like, models that are six months old, let's say the Chinese open models, you know, that level, they're pretty useful. Right? Something at six months old is still pretty darn good. So I would argue that's that's a hallmark of, commoditization. Right? The frontier is not the only thing that matters. For certain problems, is. But in six months, that frontier will be in the open models. [00:39:36] S1: So then help me out. What happens if the delivery mechanism or the output of all the infrastructure is then commoditized?