This Week in Startups
00:37:36 · Jason Calacanis, Thomas Tunguz
[00:37:36] S0: Thomas, these were software projects internally that I wanted to do. I tried to do one of them, blew through like $100,000 and I gave up because I just couldn't make forward motion on it. I was like, this is to wind up costing me $1,000,000 and I'm going to get two fifty ks worth of value from it. So something has changed dramatically here. What's your take? [00:37:54] S3: Yeah, I'll take the other side, which is you've seen GPU costs go up 120% from about $5 to about $13 in the last six months. I think as a result of the election and [00:38:05] S0: just That's the the hourly cost of a GPU you're talking to. [00:38:07] S3: Yeah, for Blackwell 300, which is state of the art today. [00:38:11] S0: I think we're going get to [00:38:12] S3: a place where we won't have enough AI in 'twenty seven. I think for Muse, we won't have it. For other products, we won't have it. We will start to buy AI the way that we buy Internet. When you buy Internet, you buy a certain amount of bandwidth. How much stuff can I download? Then I buy it as a function of speed. How quickly does it come? We are going to start to buy AI in the same way, which is like effectively, how many questions can I ask and how quick is the response? [00:38:40] S0: Wow, it's going to be rationed is what you're saying. You're going to get an allotment, it's going to be rationed, and this all you can eat stuff is maybe going to go away.
00:38:40 · Jason Calacanis, Thomas Tunguz, Elana Gold
[00:38:40] S0: Wow, it's going to be rationed is what you're saying. You're going to get an allotment, it's going to be rationed, and this all you can eat stuff is maybe going to go away. [00:38:47] S3: Right, and so that goes back to the Canva point, which is, sure, today when inference is cheap and you can get as much of it as you want, vibe code whatever I like. But in an environment where GPU and AI is constrained, not everybody will be able to afford state of the art AI, not everybody will be able to afford a soda performance. And so instead of spending $100,000 to vibe code a custom app, I'll spend $30,000 and I'll make do with an app that I have to kind of shim and wedge in. And so I think we'll ultimately see a pretty big resurgence at the app layer as a result, just because of the supply demand will make bespoke application development and maintenance too expensive for most people. [00:39:28] S2: So do you think we're not going to find a cheaper GPU and a cheaper chip? Because right now, obviously, NVIDIA owns most of the chips or a cheaper way to get power.
00:39:28 · Elana Gold, Thomas Tunguz
[00:39:28] S2: So do you think we're not going to find a cheaper GPU and a cheaper chip? Because right now, obviously, NVIDIA owns most of the chips or a cheaper way to get power. [00:39:36] S3: Yeah, I mean, we're seeing a doubling in intelligence per watt year over year. That's what the head of Google data centers has said. We've seen a 400x reduction in overall cost. But the reality is there is a finite supply. Next year, we'll have probably around 81 gigawatts. We have to decide what fraction of that is inference, how much of that are we selling to each other in order to do stuff for the enterprise, and about 60% of it is for model training. And if we continue to train these models at $3.04, $500,000,000 a run, there's just not that much inference to go around, even with the overall efficiency and the next generation chips. As you point out, Elana, like you have the Vera Rubins coming online and they're twice as productive, but the demand is growing way faster than a 200% per year.
00:40:17 · Jason Calacanis, D.A. Wallach
[00:40:17] S0: You know what's even harder than writing great code? Consistently recruiting people who can write good code. And if you're working on a project and you need to hire great engineers, but you don't want to go through a painful, time consuming hiring process, G2i can help. G2i has network of more than 8,000 pre vetted software engineers, the creme de la creme, all with more than five years of coding experience. These are the elite developers you need. And these aren't people who hack some automated coding test. The team at G2i puts engineers through live technical interviews to ensure these are folks who really know how to get their hands dirty building stuff. And the best part for founders, they make the entire process fast and easy. You get a dedicated Slack channel where you communicate directly with the G2i team so that you can share exactly what you need and start getting matched with engineers today. So go right now to g2i.co/twist and get $1,500 off your first hire. That's g2i.co/twist. [00:41:17] S1: Yeah. When you say the demand is growing so much faster, I mean, this is what all the companies in the space say too. Right? Yeah. And and my question is, well, then why don't the prices just go up? I mean, they could raise the price to the point where there is only the amount of demand that they can meet.
00:41:17 · D.A. Wallach, Thomas Tunguz, Jason Calacanis
[00:41:17] S1: Yeah. When you say the demand is growing so much faster, I mean, this is what all the companies in the space say too. Right? Yeah. And and my question is, well, then why don't the prices just go up? I mean, they could raise the price to the point where there is only the amount of demand that they can meet. [00:41:34] S3: Well, they're already at 85 percent margin. They're 85% You can look at the Anthropic S1, approximately 81%, 82% gross margin compared to a publicly traded software company of about 71%. And we've seen a price war. So you look at what OpenAI did with Luna. They cut prices 50%, then they cut prices 80%. You have open source models that are significantly less expensive, 80 to 85% less expensive than closed source. You have a lot of deflationary pressures that are going here. The problem is, think about it like a pharmaceutical company. If it takes me $500,000,000 or a billion dollars to make a drug, if I'm a pharma company, have seventeen years to commercialize that with a patent. If I'm a state of the art AI lab, I have forty one days. Because in forty one days, someone else is coming out with another soda model and then all of a sudden share starts to shift. And so I need to generate a pretty significant and healthy margin in those forty one days to be able to finance the development of the next model. [00:42:24] S0: Capitalism is so brutal, D. A. It's so brutal.
00:42:24 · Jason Calacanis, D.A. Wallach
[00:42:24] S0: Capitalism is so brutal, D. A. It's so brutal. [00:42:28] S1: Speculate? I I I would love to get your speculation, Thomas, about what the steady state margin ought to be for these companies. What you just said sounds like an argument for Frontier Labs having no sustainable margin in steady state. Tell me one, is that a correct representation? But two, if you start to think through these other layers, are the memory guys gonna be doing 80% gross margins four years from now, five years from now? I mean, Micron's trading at a forward P of four and a half, Hynix trading at a forward P of 2.8. I mean, if you believe what you're saying about the scale of this, how do you not put all of your retirement funds into Micron? [00:43:08] S0: I'll be right back. Jay trading like a lunatic. Well, as the market came down, I just bought a million dollars at Uber and Tesla, and I totally hit the bottom. But, Thomas, answer the question here about this. This steady state because we're kind of evoking Jevon's paradox here, I think.