The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:09:58 · Naveen Rao
[00:09:58] S0: I mean, you could argue that. I suppose that's up to the the energy markets. Right? That's how they should be pricing and moving up. I I do see evidence through anecdotes at least. I don't have specific numbers to quote around this because every one of these behind the meter deals is bespoke, and people are paying way more per kilowatt hour than they would off the standard grid. So I think we're already seeing a two tiered pricing occurring, where the value of electricity for data centers is sort of higher than it is for general use. Right? So there's a lot of behind the meter stuff going on right now. Companies are are are building new sort of technologies like Bloom Energy. There's a Bloom Energy deal for building, you know, a couple of gigawatts in, in Arizona. If that works, amazing. That might be the technological advances we need to keep going, but it hasn't really really been proven out yet. And those economics, by the way, are very different than what you see on the grid. How so? I think if you just boil it down to how much people are willing to pay per kilowatt hour for energy for data centers, it's just higher. You know? You can tolerate a higher price because the margin structure is just so much better. Like, you know, if I have a a gigawatt of compute and I wanna sell that, I I can sell it for, let's call it five x the price. I'm making numbers up, but it's on the on the order of five x the price of what I pay for that electricity. So I got a lot of room, wiggle room there. So I could pay 30%, 40% more for that electricity.
00:11:19 · Harry Stebbings
[00:11:19] S1: When we can go slightly further down from energy, which is kind of the core foundational power to the compute that uses it. Yeah. What do we know now about compute that you wish you'd known two years ago?
00:11:32 · Naveen Rao
[00:11:32] S0: I think one thing I wish I'd known is the fact that we don't really see physical efficiency gains anymore. So you can separate out the way we build a computer and let's call it three levels. So the first level is the physical. So if I can send a bit from this point to that point and do it cheaper, you know, let's say I I represent zero as being here and one as being here, My phone has a certain weight. So there's energy required to move between these two states. If I make a lighter phone, it changes the amount of energy required. So that's just a pure physical thing. And Moore's law, as we generally refer to it, or Dennard scaling, you may have heard, where you make devices smaller, is really a physical thing. So you're making the physical layer more efficient and denser. So those innovations have largely stopped. And if we look at the last few generations of GPUs or other chips, you don't see efficiency gains much anymore. You see efficiency gains from the next layer up, which is kind of the computing architecture. So I would say most of the efficiency gains we've seen over the last several years or several generations have come from smaller data types, going from 32 bit to 16 bit to eight bit to four bit. So you get pretty good efficiency gains because you're moving fewer bits, you're computing with fewer bits. Then the next level, which is not really exploited that well, and we started this a little bit mosaic, is about how do I start enforcing locality and reduction of data movement at the model level? And five, six years ago, we didn't think about it like this because the model was not the primary driver of compute or even... Yeah. Maybe six years ago. It was still... Compute was a general purpose thing. I run websites and databases and all that kind of stuff. However, now if I look at the computer solely as a physical device designed to deliver intelligence, it changes the mindset a bit. Now I can co design all the way from the model through the architecture to the physical. And that's what I wish I'd known how much sort of alpha there is left in the top layers, not just the physical. We focus on the physical for a long time.
00:13:26 · Harry Stebbings, Naveen Rao
[00:13:26] S1: How does that design mindset change when moving across the stack with that transition from pre model focus to post model focus? [00:13:34] S0: Yeah. I mean, it it means we don't treat the model as sacrosanct, and we're energy first in our in our mindset, meaning that everything we do has to give us an energy advantage. It sounds like a simple thing, but it's not how anyone has designed these things in the past. It's always been about speed. And this actually has its roots that go all the way back to ENIAC, you know, in the nineteen forties because, you know, computers were designed to be faster than the alternative. That was always the case. And the ENIAC's case... By the way, actually, do you know why ENIAC was built? [00:14:02] S1: No.
00:14:02 · Harry Stebbings, Naveen Rao
[00:14:02] S1: No. [00:14:03] S0: It's an acronym. Let's say, Electrical Numerical Integrator and Calculator, I think is what it stands for. So this was a computer that was built by the US government in 1945. It was one of the first large scale digital computers built, and its purpose, like every other machine that's ever been built has a purpose, its purpose was to do artillery trajectory calculation and do it faster and more accurately than the alternative. The alternative at that time were humans who literally did the maths. So, you have to be faster. That was the key. Next generation comes out, it's gotta be faster than that. You know, GPUs came out, they had to be faster at rendering than a CPU. Each thing successively has tried to be faster than the previous thing. That has led to efficiency for several reasons. I mean, making things faster in the same power envelope is more efficient, but also Moore's law was chugging along and, you know, we went from vacuum tubes to transistors. Transistors kept getting smaller and smaller and smaller. And now we're in a place where that physical side of things is not really changing a whole lot. Transistors haven't gotten much better. However, we have a lot of room left on the model and architectural levels. [00:15:06] S1: Does that mean we're seeing a commoditization of models, or does that mean we're seeing actually a separation of model quality in those that are able to save energy and have efficiency gains versus those that aren't?
00:15:06 · Harry Stebbings, Naveen Rao
[00:15:06] S1: Does that mean we're seeing a commoditization of models, or does that mean we're seeing actually a separation of model quality in those that are able to save energy and have efficiency gains versus those that aren't? [00:15:18] S0: I think that's exactly right. So, you know, one thing I've been pretty good at is kind of picking where people are gonna be focusing in the next couple of years, and I think this is gonna be [00:15:26] S1: a big part of Naveen. And, really, like, credit to you. Did you walk through when you're picking a market and go, like, this is where I think the world's gonna go, and so I'm gonna position myself here? [00:15:36] S0: Yeah. Absolutely. And, you know, it it also comes down just to interest. You know, this is, for me, one of the most interesting problems there is to build a better computer. [00:15:44] S1: So you sat down here pre unconventional in the idea maze, and you were like, problem set that I'm gonna focus on is dot dot dot. What was that?