This Week in Startups
00:35:02 · Jason Calacanis, D.A. Wallach
[00:35:02] S0: Yeah. [00:35:03] S1: And that's not how capitalism is supposed to work. When you're... When you got to those fat margins, it's supposed to attract competition, the margins get pushed down, your competitive advantage period wanes. What's going on here? And, you know, we alluded to Elana Kahn earlier, like, is there anything that needs to happen here to either spur more competition or otherwise bring down these margins, which just seem unwarranted? [00:35:32] S0: Yeah. This is key question. And I think what we saw from Canva just this week, Canva cut 22% of their staff. That is a 100% AI taking and compressing down things people are paying for. How many people do you know who pay for Canva, who are moms and dads at home and using it? 1,200 people, I think it's 20% of their staff. I can't believe they have 5,000 people working at that company or 6,000 people working there. That's a large number. It feels like if Bending Spoons was running Canva, they'd have 400 people, like 10% or something. So yeah, I think a lot of this software is going to become free, Elana, and then a lot of this margin is going to get taken away, don't you think?
00:36:25 · Elana Gold
[00:36:25] S2: Yeah. I mean, if your business relies on third party inference, what happens when a lot of these customers demand lower prices and you have to continue to cut your margins? And I think a lot of these... I mean, Canva is great, but like they're almost a part of their business is this AI wrapper and they're relying on other AI tools and LLMs to be able to create what they want. They have to be able to cut their costs. And I think it's going be really interesting because in the future, I think a lot of AI is going to be discussing these economics. We've spent years talking about what these models are capable of, but now companies are starting to ask what is actually the cost to deploy them at scale. And so I think companies have been writing this, okay, we need to deploy as much AI as possible in all the different features and aspects, but now they really have to hone in on, hey, if they are using AI, how many people are really necessary to build the same thing that they're building today? And I think you were just saying it where you're spending 40 ks to do, what, 2,000,000 worth of work. I don't think AI could do everything, but I think AI helps a lot of these shops do things quicker and easier and not as many people.
00:37:36 · Jason Calacanis, Thomas Tunguz
[00:37:36] S0: Thomas, these were software projects internally that I wanted to do. I tried to do one of them, blew through like $100,000 and I gave up because I just couldn't make forward motion on it. I was like, this is to wind up costing me $1,000,000 and I'm going to get two fifty ks worth of value from it. So something has changed dramatically here. What's your take? [00:37:54] S3: Yeah, I'll take the other side, which is you've seen GPU costs go up 120% from about $5 to about $13 in the last six months. I think as a result of the election and [00:38:05] S0: just That's the the hourly cost of a GPU you're talking to. [00:38:07] S3: Yeah, for Blackwell 300, which is state of the art today. [00:38:11] S0: I think we're going get to [00:38:12] S3: a place where we won't have enough AI in 'twenty seven. I think for Muse, we won't have it. For other products, we won't have it. We will start to buy AI the way that we buy Internet. When you buy Internet, you buy a certain amount of bandwidth. How much stuff can I download? Then I buy it as a function of speed. How quickly does it come? We are going to start to buy AI in the same way, which is like effectively, how many questions can I ask and how quick is the response? [00:38:40] S0: Wow, it's going to be rationed is what you're saying. You're going to get an allotment, it's going to be rationed, and this all you can eat stuff is maybe going to go away.
00:38:40 · Jason Calacanis, Thomas Tunguz, Elana Gold
[00:38:40] S0: Wow, it's going to be rationed is what you're saying. You're going to get an allotment, it's going to be rationed, and this all you can eat stuff is maybe going to go away. [00:38:47] S3: Right, and so that goes back to the Canva point, which is, sure, today when inference is cheap and you can get as much of it as you want, vibe code whatever I like. But in an environment where GPU and AI is constrained, not everybody will be able to afford state of the art AI, not everybody will be able to afford a soda performance. And so instead of spending $100,000 to vibe code a custom app, I'll spend $30,000 and I'll make do with an app that I have to kind of shim and wedge in. And so I think we'll ultimately see a pretty big resurgence at the app layer as a result, just because of the supply demand will make bespoke application development and maintenance too expensive for most people. [00:39:28] S2: So do you think we're not going to find a cheaper GPU and a cheaper chip? Because right now, obviously, NVIDIA owns most of the chips or a cheaper way to get power.
00:39:28 · Elana Gold, Thomas Tunguz
[00:39:28] S2: So do you think we're not going to find a cheaper GPU and a cheaper chip? Because right now, obviously, NVIDIA owns most of the chips or a cheaper way to get power. [00:39:36] S3: Yeah, I mean, we're seeing a doubling in intelligence per watt year over year. That's what the head of Google data centers has said. We've seen a 400x reduction in overall cost. But the reality is there is a finite supply. Next year, we'll have probably around 81 gigawatts. We have to decide what fraction of that is inference, how much of that are we selling to each other in order to do stuff for the enterprise, and about 60% of it is for model training. And if we continue to train these models at $3.04, $500,000,000 a run, there's just not that much inference to go around, even with the overall efficiency and the next generation chips. As you point out, Elana, like you have the Vera Rubins coming online and they're twice as productive, but the demand is growing way faster than a 200% per year.
00:40:17 · Jason Calacanis, D.A. Wallach
[00:40:17] S0: You know what's even harder than writing great code? Consistently recruiting people who can write good code. And if you're working on a project and you need to hire great engineers, but you don't want to go through a painful, time consuming hiring process, G2i can help. G2i has network of more than 8,000 pre vetted software engineers, the creme de la creme, all with more than five years of coding experience. These are the elite developers you need. And these aren't people who hack some automated coding test. The team at G2i puts engineers through live technical interviews to ensure these are folks who really know how to get their hands dirty building stuff. And the best part for founders, they make the entire process fast and easy. You get a dedicated Slack channel where you communicate directly with the G2i team so that you can share exactly what you need and start getting matched with engineers today. So go right now to g2i.co/twist and get $1,500 off your first hire. That's g2i.co/twist. [00:41:17] S1: Yeah. When you say the demand is growing so much faster, I mean, this is what all the companies in the space say too. Right? Yeah. And and my question is, well, then why don't the prices just go up? I mean, they could raise the price to the point where there is only the amount of demand that they can meet.