The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:13:26 · Harry Stebbings, Naveen Rao
[00:13:26] S1: How does that design mindset change when moving across the stack with that transition from pre model focus to post model focus? [00:13:34] S0: Yeah. I mean, it it means we don't treat the model as sacrosanct, and we're energy first in our in our mindset, meaning that everything we do has to give us an energy advantage. It sounds like a simple thing, but it's not how anyone has designed these things in the past. It's always been about speed. And this actually has its roots that go all the way back to ENIAC, you know, in the nineteen forties because, you know, computers were designed to be faster than the alternative. That was always the case. And the ENIAC's case... By the way, actually, do you know why ENIAC was built? [00:14:02] S1: No.
00:14:02 · Harry Stebbings, Naveen Rao
[00:14:02] S1: No. [00:14:03] S0: It's an acronym. Let's say, Electrical Numerical Integrator and Calculator, I think is what it stands for. So this was a computer that was built by the US government in 1945. It was one of the first large scale digital computers built, and its purpose, like every other machine that's ever been built has a purpose, its purpose was to do artillery trajectory calculation and do it faster and more accurately than the alternative. The alternative at that time were humans who literally did the maths. So, you have to be faster. That was the key. Next generation comes out, it's gotta be faster than that. You know, GPUs came out, they had to be faster at rendering than a CPU. Each thing successively has tried to be faster than the previous thing. That has led to efficiency for several reasons. I mean, making things faster in the same power envelope is more efficient, but also Moore's law was chugging along and, you know, we went from vacuum tubes to transistors. Transistors kept getting smaller and smaller and smaller. And now we're in a place where that physical side of things is not really changing a whole lot. Transistors haven't gotten much better. However, we have a lot of room left on the model and architectural levels. [00:15:06] S1: Does that mean we're seeing a commoditization of models, or does that mean we're seeing actually a separation of model quality in those that are able to save energy and have efficiency gains versus those that aren't?
00:15:06 · Harry Stebbings, Naveen Rao
[00:15:06] S1: Does that mean we're seeing a commoditization of models, or does that mean we're seeing actually a separation of model quality in those that are able to save energy and have efficiency gains versus those that aren't? [00:15:18] S0: I think that's exactly right. So, you know, one thing I've been pretty good at is kind of picking where people are gonna be focusing in the next couple of years, and I think this is gonna be [00:15:26] S1: a big part of Naveen. And, really, like, credit to you. Did you walk through when you're picking a market and go, like, this is where I think the world's gonna go, and so I'm gonna position myself here? [00:15:36] S0: Yeah. Absolutely. And, you know, it it also comes down just to interest. You know, this is, for me, one of the most interesting problems there is to build a better computer. [00:15:44] S1: So you sat down here pre unconventional in the idea maze, and you were like, problem set that I'm gonna focus on is dot dot dot. What was that?
00:15:44 · Harry Stebbings, Naveen Rao
[00:15:44] S1: So you sat down here pre unconventional in the idea maze, and you were like, problem set that I'm gonna focus on is dot dot dot. What was that? [00:15:53] S0: Well, I kinda look at it like if I were to solve a problem, what is the effect? Like, what what what does it unlock? In fact, it's kind of like being a scientist. Like, being a good scientist is all about asking the right question that if you answer it, it unlocks many more questions. That's almost a hallmark of a good paper. So for me in this in this endeavor, it was clearly like, okay. Well, we're gonna run out of energy. That's clear. The paradigm we have hasn't really been focused on this. I think this is gonna be where the the the focus will be because we need to keep scaling. Right? Like, we need to deliver intelligence in many new form factors and greatly disrupt the cost in order to make it ubiquitous. So no one's thinking about this yet. I should start it. [00:16:32] S1: Can I ask, we were talking about, like, model efficiency and how it can deliver in terms of energy? Well, we see models getting more and more efficient today. How do we think about where we are on that model efficiency curve? Is there a lot of room to run there? Or are we kind of at the asymptotic point where you're like, we're eking out the final increments?
00:16:32 · Harry Stebbings, Naveen Rao
[00:16:32] S1: Can I ask, we were talking about, like, model efficiency and how it can deliver in terms of energy? Well, we see models getting more and more efficient today. How do we think about where we are on that model efficiency curve? Is there a lot of room to run there? Or are we kind of at the asymptotic point where you're like, we're eking out the final increments? [00:16:50] S0: No. No. No. We have a lot to go. And I think the reason is is we've not done this true co design between the model and the physical side of it. So you can come up with a concept and you may be able to get, say, let's call it five percent efficiency gain just from software on a standard GPU. That sounds great. However, you can only exploit whatever concept so much because the physical thing is built in a certain way. If you look at our brain, they coevolve. We coevolve the intelligence layer and the physical layer, and we're able to exploit things in the intelligence layer in the physical layer. So sparsity, for instance. Sparsity means, let's say, just not having connections between computing units. I can get some kind of gain out of using sparsity in a GPU, 100%. But when I have a standard off chip random access memory, the limit of those gains is not... Is capped. I can't go too far because that physical thing is not designed necessarily to exploit this kind of sparsity. So if I can build something that exploits it, I can get massive gains. That's really what the brain did. That's how it coevolved.
00:16:50 · Naveen Rao, Harry Stebbings
[00:16:50] S0: No. No. No. We have a lot to go. And I think the reason is is we've not done this true co design between the model and the physical side of it. So you can come up with a concept and you may be able to get, say, let's call it five percent efficiency gain just from software on a standard GPU. That sounds great. However, you can only exploit whatever concept so much because the physical thing is built in a certain way. If you look at our brain, they coevolve. We coevolve the intelligence layer and the physical layer, and we're able to exploit things in the intelligence layer in the physical layer. So sparsity, for instance. Sparsity means, let's say, just not having connections between computing units. I can get some kind of gain out of using sparsity in a GPU, 100%. But when I have a standard off chip random access memory, the limit of those gains is not... Is capped. I can't go too far because that physical thing is not designed necessarily to exploit this kind of sparsity. So if I can build something that exploits it, I can get massive gains. That's really what the brain did. That's how it coevolved. [00:17:52] S1: When we think about that, like, coevolution or that specialization of architecture to the task almost, does that mean that we'll unwaveringly have the specialization of chips? We see etched and we see fractal and olex and cerebris, obviously, but a number of people going after specific workloads. Is that the future of compute? How do you think about that?