Invest Like the Best with Patrick O'Shaughnessy
· Gavin Baker
[00:00:00] S1: Yeah. Just any data center that can handle those, you can slot in the GB300s, and now everybody is good at making those racks. You know how to get the heat out. You know how to cool them. You're going to put those GB300s in, and then the companies that use the GB300s, they're going to be the low-cost producer of tokens, particularly, if you're vertically integrated. If you're paying a margin to someone else to make those tokens, you're probably not going to be. This has pretty profound implications. I think it has to change Google's strategic calculus. If you have a decisive cost advantage, and you're Google, and you have search and all these other businesses, why not run AI at a negative 30% margin? It is, by far, the rational decision, take the economic oxygen out of the environment. You eventually make it hard for your competitors, who need funding, unlike you, to raise the capital they need. And then on the other side of that, maybe you have an extremely dominant share position. Well, all that calculus changes once Google is no longer the low-cost producer, which I think will be the case. The Blackwells are not being used for training. And then when that model is trained, you start shifting Blackwell clusters over to Inference. And then all these cost calculations and these dynamics change. It's very interesting. Like, during the strategic and economic calculations between the players, I've never seen anything like it. Everyone understands their position on the board, what the prize is, what play their opponents are running, and it's really interesting to watch. If Google changes its behavior, because it's going to be really painful for them as a higher-cost producer to run that negative 30% margin, it might start to impact their stock, that has pretty profound implications for the economics of AI. And then when Rubin comes out, the gap is going to expand significantly.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Versus TPUs? [00:00:00] S1: Versus TPUs and all other ASICs. Now I think Trainium3 is probably going to be pretty good, and Trainium4 are going to be good. [00:00:00] S0: Why is that the case? Why won't TPU v9 be every bit as good?
· Gavin Baker
[00:00:00] S1: A couple of things. So, one, for whatever reason, Google made more conservative design decisions. And part of that is, so, Google, let's say, the TPU ... So, there's front end and back end of semiconductor design, and then there's dealing with Taiwan Semi. And you can make an ASIC in a lot of ways. What Google does is they do mostly the front end for the TPU, and then Broadcom does the backend, and manages Taiwan Semi and everything. It's a crude analogy, but the front end is, like, the architect of a house. They design a house. The back end is the person who builds the house. And then managing Taiwan Semi is stamping out that house like Lennar or D.R. Horton. And for doing those two latter parts, Broadcom runs a 50% to 55% gross margin. We don't know what on TPUs. Let's say in 2027, TPU I think consensus estimates may be somewhere around $30 billion. Again, who knows? $30 billion I think is a reasonable estimate. 50% to 55% gross margins. So, Google is paying Broadcom $15 billion. That's a lot of money. At a certain point, it makes sense to bring a semiconductor program entirely in-house. So, in other words, Apple does not have an ASIC partner for their chips. They do the front end themselves, the back end, and they manage Taiwan Semi. And the reason is they don't want to pay that 50% margin. So, at a certain point, it becomes rational to re-negotiate this, and just as perspective, the entire opex of Broadcom's semiconductor division is, round numbers, $5 billion. So, it would be economically rational now that Google is paying ... If it's $30 billion, we're paying them $15 billion. Google can go to every person who works in Broadcom semi, double their comp, and make an extra $5 billion. In 2028, let's just say it does $50 billion. Now it's $25 billion. You get triple their comp. And, by the way, you don't need them all.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Yeah. [00:00:00] S1: And, of course, they're not going to do that, because of competitive concerns. But with TPU v8, and v9, all of this is beginning to have an impact, because Google is bringing in MediaTek. This is maybe the first way you send a warning shot to Broadcom, "We're really not happy about"- [00:00:00] S0: All this money we're paying you.
· Gavin Baker
[00:00:00] S1: But they did bring MediaTek in, and the Taiwanese basic companies have much lower gross margins. So, this is the first shot against the bow. And then there's all this stuff, people say, "But Broadcom has the best SerDes." Broadcom has really good SerDes, and SerDes is an extremely foundational technology, because it's how the chips communicate with each other. You have to serialize and deserialize. But there are other good SerDes providers in the world. A really good SerDes is maybe it's worth $10 billion or $15 billion a year, but it's probably worth about $25 billion a year. So, because of that friction, and I think conservative design choices on the part of Google, and maybe the reason they made those conservative design choices is because they were going to a bifurcated supply. TPU is slowing down I would say as the GPUs are accelerating. This is the first competitive response of Lisa and Jensen to everybody saying, "We're going to have our own ASIC", is to say, "We're just going to accelerate. We're going to do a GPU every year, and you cannot keep up with this." And then I think what everybody is learning is like, "Oh, wow. That's so cool. You made your own accelerator. It has an ASIC. Wow. What's the NIC going to be? What's the CPU going to be? What's the scale-up switch going to be? What's the scale-up protocol? What's the scale-out switch? What kind of optics are you going to use? What's the software that's going to make all this work together?" And then it's like, "Oh, shit. I made this tiny little chip, and ..." Whether it's admitted or not, I'm sure the GPU makers don't love it when their customers make ASIC and try and compete with them, and like, "Whoops. What did I do? I thought this was easy." It takes, at least, three generations to make a good chip. Like, the TPU v1. It was an achievement in that they made it. It was really not until TPU v3 or v4 that the TPUs started to become even vaguely competitive.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Is that just a classic learning by doing thing? [00:00:00] S1: 100%. And even if you've made, from my perspective, the best ASIC team at any semiconductor company is actually the Amazon ASIC team. They were the first one to make the Graviton CPU. They have this Nitro, it was called SuperNIC. They've been extremely innovative, really clever, and like Trainium and Inferentia 1, maybe they're a little better than the TPU v1, but only a little. Trainium2, you get a little better. Trainium3 is I think the first time it's like, "Okay." And then I think Trainium4 will probably be good. I will be surprised if there are a lot of ASICs other than Trainium and TPU. And, by the way, Trainium and TPU will both run on customer-owned tooling at some point. We can debate when that will happen, but the economics of success that I just described mean it's inevitable. Like, no matter what the companies say, just the economics make it, and Reasoning from first principles make it absolutely inevitable.