This Week in Startups
00:39:28 · Elana Gold, Thomas Tunguz
[00:39:28] S2: So do you think we're not going to find a cheaper GPU and a cheaper chip? Because right now, obviously, NVIDIA owns most of the chips or a cheaper way to get power. [00:39:36] S3: Yeah, I mean, we're seeing a doubling in intelligence per watt year over year. That's what the head of Google data centers has said. We've seen a 400x reduction in overall cost. But the reality is there is a finite supply. Next year, we'll have probably around 81 gigawatts. We have to decide what fraction of that is inference, how much of that are we selling to each other in order to do stuff for the enterprise, and about 60% of it is for model training. And if we continue to train these models at $3.04, $500,000,000 a run, there's just not that much inference to go around, even with the overall efficiency and the next generation chips. As you point out, Elana, like you have the Vera Rubins coming online and they're twice as productive, but the demand is growing way faster than a 200% per year.
00:40:17 · Jason Calacanis, D.A. Wallach
[00:40:17] S0: You know what's even harder than writing great code? Consistently recruiting people who can write good code. And if you're working on a project and you need to hire great engineers, but you don't want to go through a painful, time consuming hiring process, G2i can help. G2i has network of more than 8,000 pre vetted software engineers, the creme de la creme, all with more than five years of coding experience. These are the elite developers you need. And these aren't people who hack some automated coding test. The team at G2i puts engineers through live technical interviews to ensure these are folks who really know how to get their hands dirty building stuff. And the best part for founders, they make the entire process fast and easy. You get a dedicated Slack channel where you communicate directly with the G2i team so that you can share exactly what you need and start getting matched with engineers today. So go right now to g2i.co/twist and get $1,500 off your first hire. That's g2i.co/twist. [00:41:17] S1: Yeah. When you say the demand is growing so much faster, I mean, this is what all the companies in the space say too. Right? Yeah. And and my question is, well, then why don't the prices just go up? I mean, they could raise the price to the point where there is only the amount of demand that they can meet.
00:41:17 · D.A. Wallach, Thomas Tunguz, Jason Calacanis
[00:41:17] S1: Yeah. When you say the demand is growing so much faster, I mean, this is what all the companies in the space say too. Right? Yeah. And and my question is, well, then why don't the prices just go up? I mean, they could raise the price to the point where there is only the amount of demand that they can meet. [00:41:34] S3: Well, they're already at 85 percent margin. They're 85% You can look at the Anthropic S1, approximately 81%, 82% gross margin compared to a publicly traded software company of about 71%. And we've seen a price war. So you look at what OpenAI did with Luna. They cut prices 50%, then they cut prices 80%. You have open source models that are significantly less expensive, 80 to 85% less expensive than closed source. You have a lot of deflationary pressures that are going here. The problem is, think about it like a pharmaceutical company. If it takes me $500,000,000 or a billion dollars to make a drug, if I'm a pharma company, have seventeen years to commercialize that with a patent. If I'm a state of the art AI lab, I have forty one days. Because in forty one days, someone else is coming out with another soda model and then all of a sudden share starts to shift. And so I need to generate a pretty significant and healthy margin in those forty one days to be able to finance the development of the next model. [00:42:24] S0: Capitalism is so brutal, D. A. It's so brutal.
00:42:24 · Jason Calacanis, D.A. Wallach
[00:42:24] S0: Capitalism is so brutal, D. A. It's so brutal. [00:42:28] S1: Speculate? I I I would love to get your speculation, Thomas, about what the steady state margin ought to be for these companies. What you just said sounds like an argument for Frontier Labs having no sustainable margin in steady state. Tell me one, is that a correct representation? But two, if you start to think through these other layers, are the memory guys gonna be doing 80% gross margins four years from now, five years from now? I mean, Micron's trading at a forward P of four and a half, Hynix trading at a forward P of 2.8. I mean, if you believe what you're saying about the scale of this, how do you not put all of your retirement funds into Micron? [00:43:08] S0: I'll be right back. Jay trading like a lunatic. Well, as the market came down, I just bought a million dollars at Uber and Tesla, and I totally hit the bottom. But, Thomas, answer the question here about this. This steady state because we're kind of evoking Jevon's paradox here, I think.
00:43:25 · Thomas Tunguz
[00:43:25] S3: Yeah. Look, think the problem... Okay, so the problem with the hardware companies and why I wouldn't load up on those stocks is primarily because there are GPUs lying fallow in warehouses. We don't have enough electricity to light them up. So at some point, NVIDIA bookings plateau just because we can't deploy them within The US, and then you have a seismic shock to those stocks. I think on the other question, D. A, what is the terminal margin for some of these businesses? You look at AWS, right? AWS, massive, the largest hyperscaler. A lot of the business before the AI wave was selling commodity compute. If you rent a server on AWS, it's a server just like there is on Amazon... Sorry, on Azure, just the way there is on GCP. The operating margins not the gross margins, but the operating margins of that business in other words, the profitability of that particular business unit is 35% to 36 net. So even if they're selling a commodity good, even if you're an oil company and you're selling crude or diesel, you can still have a really nice healthy business with competition. Coca Cola's gross margins and net margins are not zero because they're competing with Pepsi. They just continuously try to develop new products and innovate. So I think having even for the broader macro economy geopolitically, having these companies generate 70 to 80% gross margins is very healthy, sustainable, and it's probably the right thing because we want to encourage them to continue to develop and innovate.
00:44:44 · Jason Calacanis, Elana Gold, D.A. Wallach
[00:44:44] S0: Yeah, at the $50 to one ratio I put my spend, Elana, at $50 in value for every dollar. Now, I might not have built that stuff, so maybe it's really 10 to one or something. But I see the revenue going up in the projects I'm doing. I'm seeing the ambition, the dollars invested. Everything's going up into the right because when we deploy the software, we kind of get more efficient. That argues to me that they're undercharging me. And I think everybody's getting undercharged here massively. And plus, I just ordered a 12,000 Mac studio to run local models, which is exactly what I paid for my PC Junior, PC XT, PC AT in the eighties. They were $4,000 computers, dollars 5,000 computers, maybe you can get down as low as I think $2,000 back then. In today's dollars, that would be three or four X. So here we are, it used to cost the equivalent of $15,000 to buy a desktop computer. You just go back to $15,000 desktop computers and you got unlimited unmetered, well not unlimited, but unmetered local compute. [00:45:49] S2: And then just to add to that, you're still hiring at your podcast. So it's not like you're doing everything, it's all automated and you're completely done. Like I saw on your Twitter, you're hiring for 10 jobs, 14. [00:46:01] S1: Yes. [00:46:02] S2: I had to check the numbers and it's like, okay, sure, you're able to automate a lot of the manual tasks, maybe some of the editing that's made for your podcast, some of the preparation, but there still needs to be hands on. And I think some people forget that, especially as we continue to grow.