This Week in Startups
00:36:25 · Elana Gold
[00:36:25] S2: Yeah. I mean, if your business relies on third party inference, what happens when a lot of these customers demand lower prices and you have to continue to cut your margins? And I think a lot of these... I mean, Canva is great, but like they're almost a part of their business is this AI wrapper and they're relying on other AI tools and LLMs to be able to create what they want. They have to be able to cut their costs. And I think it's going be really interesting because in the future, I think a lot of AI is going to be discussing these economics. We've spent years talking about what these models are capable of, but now companies are starting to ask what is actually the cost to deploy them at scale. And so I think companies have been writing this, okay, we need to deploy as much AI as possible in all the different features and aspects, but now they really have to hone in on, hey, if they are using AI, how many people are really necessary to build the same thing that they're building today? And I think you were just saying it where you're spending 40 ks to do, what, 2,000,000 worth of work. I don't think AI could do everything, but I think AI helps a lot of these shops do things quicker and easier and not as many people.
00:37:36 · Jason Calacanis, Thomas Tunguz
[00:37:36] S0: Thomas, these were software projects internally that I wanted to do. I tried to do one of them, blew through like $100,000 and I gave up because I just couldn't make forward motion on it. I was like, this is to wind up costing me $1,000,000 and I'm going to get two fifty ks worth of value from it. So something has changed dramatically here. What's your take? [00:37:54] S3: Yeah, I'll take the other side, which is you've seen GPU costs go up 120% from about $5 to about $13 in the last six months. I think as a result of the election and [00:38:05] S0: just That's the the hourly cost of a GPU you're talking to. [00:38:07] S3: Yeah, for Blackwell 300, which is state of the art today. [00:38:11] S0: I think we're going get to [00:38:12] S3: a place where we won't have enough AI in 'twenty seven. I think for Muse, we won't have it. For other products, we won't have it. We will start to buy AI the way that we buy Internet. When you buy Internet, you buy a certain amount of bandwidth. How much stuff can I download? Then I buy it as a function of speed. How quickly does it come? We are going to start to buy AI in the same way, which is like effectively, how many questions can I ask and how quick is the response? [00:38:40] S0: Wow, it's going to be rationed is what you're saying. You're going to get an allotment, it's going to be rationed, and this all you can eat stuff is maybe going to go away.
00:38:40 · Jason Calacanis, Thomas Tunguz, Elana Gold
[00:38:40] S0: Wow, it's going to be rationed is what you're saying. You're going to get an allotment, it's going to be rationed, and this all you can eat stuff is maybe going to go away. [00:38:47] S3: Right, and so that goes back to the Canva point, which is, sure, today when inference is cheap and you can get as much of it as you want, vibe code whatever I like. But in an environment where GPU and AI is constrained, not everybody will be able to afford state of the art AI, not everybody will be able to afford a soda performance. And so instead of spending $100,000 to vibe code a custom app, I'll spend $30,000 and I'll make do with an app that I have to kind of shim and wedge in. And so I think we'll ultimately see a pretty big resurgence at the app layer as a result, just because of the supply demand will make bespoke application development and maintenance too expensive for most people. [00:39:28] S2: So do you think we're not going to find a cheaper GPU and a cheaper chip? Because right now, obviously, NVIDIA owns most of the chips or a cheaper way to get power.
00:39:28 · Elana Gold, Thomas Tunguz
[00:39:28] S2: So do you think we're not going to find a cheaper GPU and a cheaper chip? Because right now, obviously, NVIDIA owns most of the chips or a cheaper way to get power. [00:39:36] S3: Yeah, I mean, we're seeing a doubling in intelligence per watt year over year. That's what the head of Google data centers has said. We've seen a 400x reduction in overall cost. But the reality is there is a finite supply. Next year, we'll have probably around 81 gigawatts. We have to decide what fraction of that is inference, how much of that are we selling to each other in order to do stuff for the enterprise, and about 60% of it is for model training. And if we continue to train these models at $3.04, $500,000,000 a run, there's just not that much inference to go around, even with the overall efficiency and the next generation chips. As you point out, Elana, like you have the Vera Rubins coming online and they're twice as productive, but the demand is growing way faster than a 200% per year.
00:40:17 · Jason Calacanis, D.A. Wallach
[00:40:17] S0: You know what's even harder than writing great code? Consistently recruiting people who can write good code. And if you're working on a project and you need to hire great engineers, but you don't want to go through a painful, time consuming hiring process, G2i can help. G2i has network of more than 8,000 pre vetted software engineers, the creme de la creme, all with more than five years of coding experience. These are the elite developers you need. And these aren't people who hack some automated coding test. The team at G2i puts engineers through live technical interviews to ensure these are folks who really know how to get their hands dirty building stuff. And the best part for founders, they make the entire process fast and easy. You get a dedicated Slack channel where you communicate directly with the G2i team so that you can share exactly what you need and start getting matched with engineers today. So go right now to g2i.co/twist and get $1,500 off your first hire. That's g2i.co/twist. [00:41:17] S1: Yeah. When you say the demand is growing so much faster, I mean, this is what all the companies in the space say too. Right? Yeah. And and my question is, well, then why don't the prices just go up? I mean, they could raise the price to the point where there is only the amount of demand that they can meet.
00:41:17 · D.A. Wallach, Thomas Tunguz, Jason Calacanis
[00:41:17] S1: Yeah. When you say the demand is growing so much faster, I mean, this is what all the companies in the space say too. Right? Yeah. And and my question is, well, then why don't the prices just go up? I mean, they could raise the price to the point where there is only the amount of demand that they can meet. [00:41:34] S3: Well, they're already at 85 percent margin. They're 85% You can look at the Anthropic S1, approximately 81%, 82% gross margin compared to a publicly traded software company of about 71%. And we've seen a price war. So you look at what OpenAI did with Luna. They cut prices 50%, then they cut prices 80%. You have open source models that are significantly less expensive, 80 to 85% less expensive than closed source. You have a lot of deflationary pressures that are going here. The problem is, think about it like a pharmaceutical company. If it takes me $500,000,000 or a billion dollars to make a drug, if I'm a pharma company, have seventeen years to commercialize that with a patent. If I'm a state of the art AI lab, I have forty one days. Because in forty one days, someone else is coming out with another soda model and then all of a sudden share starts to shift. And so I need to generate a pretty significant and healthy margin in those forty one days to be able to finance the development of the next model. [00:42:24] S0: Capitalism is so brutal, D. A. It's so brutal.