The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:06:53 · Harry Stebbings, Naveen Rao
[00:06:53] S1: Dude, what can we learn from China being so much further ahead of us that we should be, like, way more loose on policy and regulation? What are the takeaways? [00:07:02] S0: Well, I I think some of this is hard because it it hits the very core of our culture and and their culture. I think China, you know, can centrally plan. We don't. And there are advantages to both of these things. Right? And and these things where you have these giant infrastructural rollouts, yeah, there's an advantage. No doubt about it. However, I think when it comes to innovation, tip of the sphere sort of things, like, The US is far and away better. AI was born here, not there. You know, like, all of these these latest innovations have all come from here. Now they're doing some really great work in China also. I don't wanna diminish that because it's not just distillation. They're really doing some true innovation. However, I think that a tip of the spear stuff where you really open up, you know, new areas comes from here, not from there. So, you know, I I think we have to leverage our advantages of what it comes down to. [00:07:48] S1: When you look at what's scarce today, what do you think is not scarce today that will be in three to five years?
00:07:48 · Harry Stebbings, Naveen Rao
[00:07:48] S1: When you look at what's scarce today, what do you think is not scarce today that will be in three to five years? [00:07:55] S0: I think that excess capacity is gonna start getting very scarce. You know, if we look at growth rates, even even the kind of much more conservative growth rates of AI, we're gonna be out of energy pretty soon in the next few years. And it's not easy just to say, okay. I'm gonna find some more capacity. You will have used up all of that excess. So then we're in a place where we kinda converge to the rate of building. And, you know, I... I've heard people say, oh, we're gonna build 40 gigawatts in a year. There's no way. We have never done that. I just don't see it as being possible, at least not right now. Maybe in a few years, we'll come up with some new inventions, we will build our way out. But as it stands today, we're not gonna build 40 gigawatts in a year. Maybe we build 10. If we build 10, that's amazing. That's $500,000,000,000, just to put it in perspective. [00:08:38] S1: So when we look at the debt cycle that is already being talked about as being incredibly toppy, and we're seeing that now starting to cascade or unpack in CoolWeave and a lot of concerns around that. Is it actually just at the start, or does the music turn off at the party at 7PM?
00:08:38 · Harry Stebbings, Naveen Rao
[00:08:38] S1: So when we look at the debt cycle that is already being talked about as being incredibly toppy, and we're seeing that now starting to cascade or unpack in CoolWeave and a lot of concerns around that. Is it actually just at the start, or does the music turn off at the party at 7PM? [00:08:56] S0: The biggest deterrent of that will be, is there demand for AI services and therefore for GPUs and therefore for this excess, alright, this new energy build out? Obviously, I... I'm in this industry. I'm I'm pretty AI pilled. I think there's plenty of demand. I don't think we even scratched the surface. You know, there's a lot of problems we have to solve in AI still, but as we solve those, and we will over the next couple of years, it's just going to open up new types of demand that we don't even really know about today. So my sense is that this is a blip. It's not some kind of bubble being burst. It's sort of like if you said, hey, is there demand for people curing cancer? Of course there is. Right? Like, if you if you cure cancer, people are gonna want that. Like, you build an intelligent machine capable of doing anything, yeah, people are gonna want that. [00:09:39] S1: I always remember Vinyl Khosla saying to me, wanna take as much technology risk and as little adoption risk as possible. And it's exactly that. I have a cure for cancer, very difficult to do, but zero adoption risk. But then does that mean that we're mispricing the excess supply today if we're literally staring down the bowel of eighteen months running out?
00:09:58 · Naveen Rao
[00:09:58] S0: I mean, you could argue that. I suppose that's up to the the energy markets. Right? That's how they should be pricing and moving up. I I do see evidence through anecdotes at least. I don't have specific numbers to quote around this because every one of these behind the meter deals is bespoke, and people are paying way more per kilowatt hour than they would off the standard grid. So I think we're already seeing a two tiered pricing occurring, where the value of electricity for data centers is sort of higher than it is for general use. Right? So there's a lot of behind the meter stuff going on right now. Companies are are are building new sort of technologies like Bloom Energy. There's a Bloom Energy deal for building, you know, a couple of gigawatts in, in Arizona. If that works, amazing. That might be the technological advances we need to keep going, but it hasn't really really been proven out yet. And those economics, by the way, are very different than what you see on the grid. How so? I think if you just boil it down to how much people are willing to pay per kilowatt hour for energy for data centers, it's just higher. You know? You can tolerate a higher price because the margin structure is just so much better. Like, you know, if I have a a gigawatt of compute and I wanna sell that, I I can sell it for, let's call it five x the price. I'm making numbers up, but it's on the on the order of five x the price of what I pay for that electricity. So I got a lot of room, wiggle room there. So I could pay 30%, 40% more for that electricity.
00:11:19 · Harry Stebbings
[00:11:19] S1: When we can go slightly further down from energy, which is kind of the core foundational power to the compute that uses it. Yeah. What do we know now about compute that you wish you'd known two years ago?
00:11:32 · Naveen Rao
[00:11:32] S0: I think one thing I wish I'd known is the fact that we don't really see physical efficiency gains anymore. So you can separate out the way we build a computer and let's call it three levels. So the first level is the physical. So if I can send a bit from this point to that point and do it cheaper, you know, let's say I I represent zero as being here and one as being here, My phone has a certain weight. So there's energy required to move between these two states. If I make a lighter phone, it changes the amount of energy required. So that's just a pure physical thing. And Moore's law, as we generally refer to it, or Dennard scaling, you may have heard, where you make devices smaller, is really a physical thing. So you're making the physical layer more efficient and denser. So those innovations have largely stopped. And if we look at the last few generations of GPUs or other chips, you don't see efficiency gains much anymore. You see efficiency gains from the next layer up, which is kind of the computing architecture. So I would say most of the efficiency gains we've seen over the last several years or several generations have come from smaller data types, going from 32 bit to 16 bit to eight bit to four bit. So you get pretty good efficiency gains because you're moving fewer bits, you're computing with fewer bits. Then the next level, which is not really exploited that well, and we started this a little bit mosaic, is about how do I start enforcing locality and reduction of data movement at the model level? And five, six years ago, we didn't think about it like this because the model was not the primary driver of compute or even... Yeah. Maybe six years ago. It was still... Compute was a general purpose thing. I run websites and databases and all that kind of stuff. However, now if I look at the computer solely as a physical device designed to deliver intelligence, it changes the mindset a bit. Now I can co design all the way from the model through the architecture to the physical. And that's what I wish I'd known how much sort of alpha there is left in the top layers, not just the physical. We focus on the physical for a long time.