The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:15:44 · Harry Stebbings, Naveen Rao
[00:15:44] S1: So you sat down here pre unconventional in the idea maze, and you were like, problem set that I'm gonna focus on is dot dot dot. What was that? [00:15:53] S0: Well, I kinda look at it like if I were to solve a problem, what is the effect? Like, what what what does it unlock? In fact, it's kind of like being a scientist. Like, being a good scientist is all about asking the right question that if you answer it, it unlocks many more questions. That's almost a hallmark of a good paper. So for me in this in this endeavor, it was clearly like, okay. Well, we're gonna run out of energy. That's clear. The paradigm we have hasn't really been focused on this. I think this is gonna be where the the the focus will be because we need to keep scaling. Right? Like, we need to deliver intelligence in many new form factors and greatly disrupt the cost in order to make it ubiquitous. So no one's thinking about this yet. I should start it. [00:16:32] S1: Can I ask, we were talking about, like, model efficiency and how it can deliver in terms of energy? Well, we see models getting more and more efficient today. How do we think about where we are on that model efficiency curve? Is there a lot of room to run there? Or are we kind of at the asymptotic point where you're like, we're eking out the final increments?
00:16:32 · Harry Stebbings, Naveen Rao
[00:16:32] S1: Can I ask, we were talking about, like, model efficiency and how it can deliver in terms of energy? Well, we see models getting more and more efficient today. How do we think about where we are on that model efficiency curve? Is there a lot of room to run there? Or are we kind of at the asymptotic point where you're like, we're eking out the final increments? [00:16:50] S0: No. No. No. We have a lot to go. And I think the reason is is we've not done this true co design between the model and the physical side of it. So you can come up with a concept and you may be able to get, say, let's call it five percent efficiency gain just from software on a standard GPU. That sounds great. However, you can only exploit whatever concept so much because the physical thing is built in a certain way. If you look at our brain, they coevolve. We coevolve the intelligence layer and the physical layer, and we're able to exploit things in the intelligence layer in the physical layer. So sparsity, for instance. Sparsity means, let's say, just not having connections between computing units. I can get some kind of gain out of using sparsity in a GPU, 100%. But when I have a standard off chip random access memory, the limit of those gains is not... Is capped. I can't go too far because that physical thing is not designed necessarily to exploit this kind of sparsity. So if I can build something that exploits it, I can get massive gains. That's really what the brain did. That's how it coevolved.
00:16:50 · Naveen Rao, Harry Stebbings
[00:16:50] S0: No. No. No. We have a lot to go. And I think the reason is is we've not done this true co design between the model and the physical side of it. So you can come up with a concept and you may be able to get, say, let's call it five percent efficiency gain just from software on a standard GPU. That sounds great. However, you can only exploit whatever concept so much because the physical thing is built in a certain way. If you look at our brain, they coevolve. We coevolve the intelligence layer and the physical layer, and we're able to exploit things in the intelligence layer in the physical layer. So sparsity, for instance. Sparsity means, let's say, just not having connections between computing units. I can get some kind of gain out of using sparsity in a GPU, 100%. But when I have a standard off chip random access memory, the limit of those gains is not... Is capped. I can't go too far because that physical thing is not designed necessarily to exploit this kind of sparsity. So if I can build something that exploits it, I can get massive gains. That's really what the brain did. That's how it coevolved. [00:17:52] S1: When we think about that, like, coevolution or that specialization of architecture to the task almost, does that mean that we'll unwaveringly have the specialization of chips? We see etched and we see fractal and olex and cerebris, obviously, but a number of people going after specific workloads. Is that the future of compute? How do you think about that?
00:18:14 · Naveen Rao
[00:18:14] S0: A 100% it is. And, you know, this actually goes back to even Turing's original essay. He wanted to build an intelligent machine. Yeah. I mean, that was that was his stated goal right at the beginning. So we went down this path of we took his machine, which was a theoretical construct, and we built physical versions of it. So anything we build is not truly quote unquote Turing complete because, you know, they have finite memory and stuff like this. But we were able to build applications that were super useful, right? I mean, we could run businesses off of them. We could edit videos and pictures and text and all kinds of great things. That's a huge productivity gains over the last sixty years. But it deviated from that original goal. And now we've kind of come back. So the purpose of a computer, I think, is to provide intelligence. To me, it's just a physical manifestation of intelligence. And so now we have to figure out how to make that better. And what's interesting is that if you make a better physical manifestation of intelligence, you made something general. Our brains in some ways are general. However, hardware is fixed in its workload. It doesn't do programmability in the same way that we think about it in a CPU. So the GPU is already down this path of building customized architectures for workloads, and I think we're just gonna hyper optimize around this. And that's a good thing because now we can build intelligent machines that are actually capable of building anything at some point.
00:19:30 · Harry Stebbings
[00:19:30] S1: Listen. Forgive my financial mind. You're a wonderful technologist. I am a delinquent trader for a living in many respects. But we mentioned, like, yes, unwaveringly specialization of chips. To what extent does that massively reduce the TAM for a horizontal provider, NVIDIA specifically, versus, dude, do not be so small minded. It is still so early. We have 50 extra room to run.
00:19:54 · Naveen Rao
[00:19:54] S0: It is that. The the market is gonna be huge. I mean, look. I I think NVIDIA executed extremely well on the path to getting us here. I mean, you know, when I started Nirvana in 2014, we didn't really understand how big the market was gonna be. It was clear to me that we needed a new computing architecture around neural networks. That was a new way to describe compute. And to me, it was like the most important thing a computer could do. I think Jensen recognized this relatively early on as well. I mean, he he started to talk about artificial intelligence at the, GTC in 2014. I remember it vividly because a lot of the graphics guys were pissed off about this. They were like, why did you spend so much time on AI? It's nothing. It's it's it's not a important workload, and graphics is. Right? And, he correctly recognized that this was gonna be important, invested a ton in that direction, and actually executed very well. We had to go through this evolution of computing architectures from what we use those machines for to what they are eventually going to become. But I think now it's pretty clear to me that I don't need to care about anything else. I only need to care about running a neural network very, very efficiently, very, very scalably. That's gonna be the figure of merit.