The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:27:37 · Harry Stebbings, Thomas Sohmers
[00:27:37] S1: Do you believe that space is a viable alternative truly, or is it conference talk and lip service to justify a market cap? [00:27:44] S0: I think something can start as one thing and turn into something else. I I would never ever bet against Elon. I I I mean, I I I primarily bet for Elon. If you asked me a year ago, I just would not have thought that there would be a good reason for it in the near term because it's gonna be cheaper, easier, etcetera, to build on land. I also think there's great alternative technologies, company we're partnered with, and I'm good friends with, the the CEO, is is a company called Panthalassa that's building ocean based data centers. Basically, a very interesting pumped hydro solution in the middle of the ocean. So there are alternatives that don't require going to space, I think, long term. Part of the the reason I'm I'm long term big believer in space data centers is I just think we're going to need to have a space economy for humanity to live up to its long term potential. [00:28:31] S1: Love that. Totally agree. I'll never bet against Elon. If we think about, like, the cost of intelligence being tied to the cost of energy, how should we think about energy as a bottleneck moving forwards? To what extent is it? We mentioned policy and regulation being a core bottleneck. Is energy a bottleneck moving forwards or less than people consider?
00:28:51 · Thomas Sohmers
[00:28:51] S0: I think there's two pieces to it. I mean, one, positron is trying to deliver more compute, more capabilities per watt per megawatt. And so, you know, sort of on our base case, we can turn, what you would have spent 500 megawatts with NVIDIA equipment and do that on a 100 megawatt. I don't think that's actually going to mean that you're only going to build a 100 megawatt facility. You're still going to build the maximum amount of compute that you can. You're just getting more tokens, more intelligence per, jewel. And so if if I go back to the long term thinking, you know, I think assuming humanity continues for hundreds, thousands of years, everything turns into an energy problem. And you can go back to thousands of years and and just look at the progression of mankind. Fundamentally, that is a perfect track of our ability to produce and use energy, discovery of fire up to, you know, nuclear power plants. The the simple tongue in cheek answer to your question is all progress is gated by energy. And even if there's energy available, it may not be economical, and so it won't be done. So I... Actually, I would say the bigger limiter than just saying that energy, like our ability to build and produce energy, we've got plenty of technologies and capability to do it. I would say we have got way more economic limitations. It's like, how much debt is the world willing to take on to build out everything over the next couple years? It ties into energy. It ties into the infrastructure itself, etcetera. So I think economics is a much, easier sort of scapegoat to pit.
00:30:16 · Harry Stebbings
[00:30:16] S1: I mean, people are already very concerned by the levels of debt being taken out in the debt cycle. Do you think their concerns are justified, and then do you share them?
00:30:24 · Thomas Sohmers
[00:30:24] S0: I think we've got a major sovereign debt problem that masks a huge amount of second and third third order elements in the financial system. Just the the inflationary consequences of government that that can print, you know, infinite amounts of its own currency. And the fact that we are... As we're already seeing the treasuries and, you know, the greater bond bond markets that there is greater and greater perceived risk of the most, quote unquote, risk free asset, I think will trickle down to all elements of the financial system. And so when people worry about Oracle's debt and and credit rating, I'm like, I believe in Oracle's business model and and ability to execute and and do everything a whole lot more than United States government. It's just the United States government can, issue its own currency and also has guns and nukes to, take tax revenue. So my biggest economic concern there is that there will be a more acute specific crisis that that arises out of the compounding of national debt leading to devaluation of the currency that has all of the consequences downstream rather than, like, I'm really not worried about any of the companies in the AI debt stream not hitting their revenue targets. I'd... Like, if the past three, four years have shown, we're accelerating every aspect of these businesses in terms of revenue profits and how they are improving the productivity and value downstream.
00:31:52 · Harry Stebbings
[00:31:52] S1: I'm jumping around, but fuck it. When I was doing the research, I was reading about KV caching and compression as part of this, and I was honestly getting lost. But I was intrigued and digging deeper and deeper. And I was like, why did I not know this before? And so I didn't think many will know this. What should we know about KV caching? Why is this important? Can you explain it to me a little bit?
00:32:13 · Thomas Sohmers
[00:32:13] S0: Yeah. So there's, always a, I would say, a, you know, give and take relationship with with innovation. I guess one element I'll I'll just have to explain to make all this this clear is the concept of a a sequence. I kinda already talked about a token. You know, a token effectively, as part of the training process, when any of these big, you know, model apps are are developing a new model, they have a vocabulary that they define. So they take their big giant corpus, and they do some statistical work to figure out what is the best encoding method to take all of the text in this and break it into chunks that get reused frequently to have things be more efficient. And so what you end up doing is if you took a English dictionary, you'll find that there are common prefixes and suffixes and and, you know, groupings of words. And if you just try to think of how would I best compress this, if if I just had symbols for these prefixes, suffixes, etcetera, compress this into a thing. And basically, what what ends up happening is a token, you know, if you're using ChatGPT and you see text streaming out, if that's going particularly slow or you're you're quite a keen eye, you'll see that it's portions of words that come out at times. Sometimes a full word, sometimes a a small fraction of word. And each of those little flashes that you see is a token. And roughly speaking, it's... A token is equivalent to a half to, like, 75% of a word on average in in large English corpuses. So that's token. A sequence or, you know, the thing that builds up to be in context in a model is the grouping of all those tokens in n order. And what happens when you're running inference, you you give it a prompt. You have... What is the capital of France as as your input prompt? That is tokenized. You know, that is four or five, six tokens and that that go into the model. And when you do inference, it's going to say the capital of France is Paris and the city of lights, you know, some thing after that. And so when you have that entire sequence, when transformers originally came out, for every token that was generated, you were doing the computation for generating all of those tokens including the ones that you've already processed.