The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:39:46 · Chase Lochmiller, Harry Stebbings
And so, you know, abstracting away some some of the difficulties by actually, you know, having a suite of services that can monetize these chips for longer will ultimately, we believe, extend the depreciation cycle beyond six years. [00:39:59] S1: What do you [00:39:59] S0: think everyone gets wrong about chip buying, chip depreciation that they should know? I think when we started to make really substantial purchases with hoppers, and this was in 2023, the feedback we got was like, well, we don't even know if this is gonna be valuable after year three. Right? Here we are three years later, and the prices being charged for utilizing hoppers is higher than the rates that were being charged three years ago when they were when they were brand new. So early financings was, you know, you had to get paid back very, very quickly, and you had to have like a lot of lot of debt service coverage. So I think people underestimate the ingenuity of applications and application developers of turning compute capacity into value and into valuable services for the economy. [00:40:48] S1: I spoke to Lee Jacobs before, and he was talking to me about the risk mindset that you had to ordering millions of dollars of GPUs from NVIDIA well before it was cool, as you put it. My question to you is how on earth do you think about forecasting demand today? Is it just I'll take as much of it as I can?
00:40:48 · Harry Stebbings, Chase Lochmiller
[00:40:48] S1: I spoke to Lee Jacobs before, and he was talking to me about the risk mindset that you had to ordering millions of dollars of GPUs from NVIDIA well before it was cool, as you put it. My question to you is how on earth do you think about forecasting demand today? Is it just I'll take as much of it as I can? [00:41:04] S0: No. I mean, we build up forecasts from having conversations with customers. I think we are a very customer centric organization. We we wanna deliver ultimately solutions for customers. The challenge today is that increasingly customers are being asked to forecast their demand further and further out because the timeline to bring online AI infrastructure is extending. And, you know, that's problematic for the industry writ large. One of the things that Crusoe is focused on is actually trying to reduce that time to token or time to delivery of a managed cluster of GPUs by actually deploying small modular manufactured AI data centers that can deliver more just in time AI infrastructure, particularly for smaller clusters. And we think that's gonna be transformative for the industry. We think it's a very important shift in terms of... You know, especially, like, when you look at earlier stage companies, earlier stage startups that are being asked to commit to compute in 2028, it's like that is an eternity from now. Right? You know? This is this is like a [00:42:02] S1: We might have RSI by then. Right? That might change it. Exactly. Exactly. So Have you ever had a point moment in your career where the future is so unknown?
00:42:02 · Harry Stebbings, Chase Lochmiller
[00:42:02] S1: We might have RSI by then. Right? That might change it. Exactly. Exactly. So Have you ever had a point moment in your career where the future is so unknown? [00:42:13] S0: I feel... I actually don't feel like it's super unknown. I actually feel like, you know, we've had this breakthrough in overall model development cycles. You know, I think the scaling laws have continued to prove out. I think we've reached... You know, we keep moving the goalposts of what is what is artificial intelligence, what is AGI. You know, you think about the the Turing test. When I took my very first computer science class in high school, the Turing test was heralded as this as this critical notion of true machine intelligence, and it seemed like an impossible task at that point in time. We blew past the Turing test, and almost no one even celebrated. You think about, you know, the model capabilities you have today, and now, you know, we're solving Millennium Prize problems. Right? This was something, you know, when I was an undergrad at MIT, these Millennium Prize problems sort of came about, and it was like this, wow, these are these unsolved problems where there's a real financial bounty out there. Like, you can earn a million dollars if you can solve one of these problems. And they sat there for decades unsolved. Now we have AI solving these these these critical problems.
00:42:13 · Chase Lochmiller, Harry Stebbings
[00:42:13] S0: I feel... I actually don't feel like it's super unknown. I actually feel like, you know, we've had this breakthrough in overall model development cycles. You know, I think the scaling laws have continued to prove out. I think we've reached... You know, we keep moving the goalposts of what is what is artificial intelligence, what is AGI. You know, you think about the the Turing test. When I took my very first computer science class in high school, the Turing test was heralded as this as this critical notion of true machine intelligence, and it seemed like an impossible task at that point in time. We blew past the Turing test, and almost no one even celebrated. You think about, you know, the model capabilities you have today, and now, you know, we're solving Millennium Prize problems. Right? This was something, you know, when I was an undergrad at MIT, these Millennium Prize problems sort of came about, and it was like this, wow, these are these unsolved problems where there's a real financial bounty out there. Like, you can earn a million dollars if you can solve one of these problems. And they sat there for decades unsolved. Now we have AI solving these these these critical problems. [00:43:19] S1: I agree with you. My question to you is we see these swarms of rogue agents doing things that are nefarious. We see whole industries that are being put to question. My girlfriend's a lawyer. Six months ago, she didn't use the legora. Now she hasn't written a document in five months. There is just such uncertainty across everything. We have RSI that could
00:43:19 · Harry Stebbings, Chase Lochmiller
[00:43:19] S1: I agree with you. My question to you is we see these swarms of rogue agents doing things that are nefarious. We see whole industries that are being put to question. My girlfriend's a lawyer. Six months ago, she didn't use the legora. Now she hasn't written a document in five months. There is just such uncertainty across everything. We have RSI that could [00:43:36] S0: come and and change absolutely everything. So, sure, I get the comment that there's uncertainty around what the future of work will look like in certain ways, but I also feel like it's it's actually not uncertain that AI is gonna be a critical underlying engine for work and productivity that is gonna shift things quite a bit... Quite dramatically. And so it's more about that there is certainty that AI is establishing capabilities that are at human level or greater than human level in terms of overall productive capacity. So sure. Is that gonna change things? Yes. That is gonna change things. So so there's uncertainty in the change, but not... To me, it's certain that there will be change. Maybe that's a different way of putting it. [00:44:18] S1: Can I ask, we we mentioned the managed inference being a great part of your business? [00:44:22] S0: Mhmm. [00:44:22] S1: Gavin Baker, one of your investors, has said that the lowest cost producer of intelligence wins. When you think about that, like, what's the unit? Is it, like, dollars per token, per training run? What is that unit fee?
00:44:35 · Chase Lochmiller
[00:44:35] S0: Dollars per token is a is a good measure in terms of this, but it's it's not necessarily... You know, there is token efficiency, so, you know, just not all tokens are created equal. So it's not like every single token has the same, you know, unit of value, but I think it is a good indicator. I do think that, you know, there are different things that folks optimize for in terms of utilization, whether it's if throughput, so the amount of tokens per second you're able to generate for for for a very specific model use case, as well as time to first token, so latency, time to first token, time to last token, you know, how quickly you're able to actually serve this. And a lot of that comes down to it's a confluence of of different things. Right? So there's, you know, what is the cost that go into that? Obviously, by being vertically integrated and owning the infrastructure, you're able to stack a lot of margin across the process. So it gives you more flexibility in terms of how you actually, you know, bill for that. Additionally, I think utilization of the GPU itself is a critical aspect of being able to deliver tokens ultra efficiently and shine on metrics like latency and throughput. And so how does that actually manifest? Well, you know, the GPU is actually the most valuable thing in the entire data center, right? It is like the most expensive thing. So it's the thing you want to make sure you're keeping busy. If it's sitting idle, that's just money burning, right? If you think about that from that perspective, I think one of the key aspects is actually the management of memory and the management of the, you know, of the KV cache specifically. So the KV cache is the key value cache. It's really like, you know, you can kind of think about, you know, when tokens are fed into a a large neural network, you can compute the output tokens by basically running this feed forward process and running all these matrix multiplications that takes time. You may also already know the answer of that output token before you put the... So if you... If you've actually put put those same tokens in before, you actually know the answers and you can look them up in this look up table. The KBCash can get quite large and so it can expand well beyond the amount of, you know, memory you have on chip in the HBM.