Invest Like the Best with Patrick O'Shaughnessy
· Gavin Baker
[00:00:00] S1: A couple of things. So, one, for whatever reason, Google made more conservative design decisions. And part of that is, so, Google, let's say, the TPU ... So, there's front end and back end of semiconductor design, and then there's dealing with Taiwan Semi. And you can make an ASIC in a lot of ways. What Google does is they do mostly the front end for the TPU, and then Broadcom does the backend, and manages Taiwan Semi and everything. It's a crude analogy, but the front end is, like, the architect of a house. They design a house. The back end is the person who builds the house. And then managing Taiwan Semi is stamping out that house like Lennar or D.R. Horton. And for doing those two latter parts, Broadcom runs a 50% to 55% gross margin. We don't know what on TPUs. Let's say in 2027, TPU I think consensus estimates may be somewhere around $30 billion. Again, who knows? $30 billion I think is a reasonable estimate. 50% to 55% gross margins. So, Google is paying Broadcom $15 billion. That's a lot of money. At a certain point, it makes sense to bring a semiconductor program entirely in-house. So, in other words, Apple does not have an ASIC partner for their chips. They do the front end themselves, the back end, and they manage Taiwan Semi. And the reason is they don't want to pay that 50% margin. So, at a certain point, it becomes rational to re-negotiate this, and just as perspective, the entire opex of Broadcom's semiconductor division is, round numbers, $5 billion. So, it would be economically rational now that Google is paying ... If it's $30 billion, we're paying them $15 billion. Google can go to every person who works in Broadcom semi, double their comp, and make an extra $5 billion. In 2028, let's just say it does $50 billion. Now it's $25 billion. You get triple their comp. And, by the way, you don't need them all.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Yeah. [00:00:00] S1: And, of course, they're not going to do that, because of competitive concerns. But with TPU v8, and v9, all of this is beginning to have an impact, because Google is bringing in MediaTek. This is maybe the first way you send a warning shot to Broadcom, "We're really not happy about"- [00:00:00] S0: All this money we're paying you.
· Gavin Baker
[00:00:00] S1: But they did bring MediaTek in, and the Taiwanese basic companies have much lower gross margins. So, this is the first shot against the bow. And then there's all this stuff, people say, "But Broadcom has the best SerDes." Broadcom has really good SerDes, and SerDes is an extremely foundational technology, because it's how the chips communicate with each other. You have to serialize and deserialize. But there are other good SerDes providers in the world. A really good SerDes is maybe it's worth $10 billion or $15 billion a year, but it's probably worth about $25 billion a year. So, because of that friction, and I think conservative design choices on the part of Google, and maybe the reason they made those conservative design choices is because they were going to a bifurcated supply. TPU is slowing down I would say as the GPUs are accelerating. This is the first competitive response of Lisa and Jensen to everybody saying, "We're going to have our own ASIC", is to say, "We're just going to accelerate. We're going to do a GPU every year, and you cannot keep up with this." And then I think what everybody is learning is like, "Oh, wow. That's so cool. You made your own accelerator. It has an ASIC. Wow. What's the NIC going to be? What's the CPU going to be? What's the scale-up switch going to be? What's the scale-up protocol? What's the scale-out switch? What kind of optics are you going to use? What's the software that's going to make all this work together?" And then it's like, "Oh, shit. I made this tiny little chip, and ..." Whether it's admitted or not, I'm sure the GPU makers don't love it when their customers make ASIC and try and compete with them, and like, "Whoops. What did I do? I thought this was easy." It takes, at least, three generations to make a good chip. Like, the TPU v1. It was an achievement in that they made it. It was really not until TPU v3 or v4 that the TPUs started to become even vaguely competitive.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: Is that just a classic learning by doing thing? [00:00:00] S1: 100%. And even if you've made, from my perspective, the best ASIC team at any semiconductor company is actually the Amazon ASIC team. They were the first one to make the Graviton CPU. They have this Nitro, it was called SuperNIC. They've been extremely innovative, really clever, and like Trainium and Inferentia 1, maybe they're a little better than the TPU v1, but only a little. Trainium2, you get a little better. Trainium3 is I think the first time it's like, "Okay." And then I think Trainium4 will probably be good. I will be surprised if there are a lot of ASICs other than Trainium and TPU. And, by the way, Trainium and TPU will both run on customer-owned tooling at some point. We can debate when that will happen, but the economics of success that I just described mean it's inevitable. Like, no matter what the companies say, just the economics make it, and Reasoning from first principles make it absolutely inevitable.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: If I were to zoom all the way out on this stuff, I find these details unbelievably interesting, and it's, like, the grandest game that's ever been played. It's so crazy and so fun to follow sometimes I forget to zoom out and say, "Well, so what?" Like, okay, so, project this forward three generations past Rubin, or whatever, what is the global human dividend of all this crazy development where we keep making the loss lower on these pre-training scaling models like, "who cares?" It's been a while since I've asked this thing something that I wasn't blown away by the answer for me personally. What are the next couple of things that all this crazy infrastructure war allows us to unlock because they're so successful? [00:00:00] S1: If I were to posit like an event path, I think the Blackwell models are going to be amazing. The dramatic reduction in per token cost enabled by the GB300, and probably more the MI450 than the MI355, will lead to these models being allowed to think for much longer, which means they're going to be able to do new things. I was very impressed, Gemini 3 made me a restaurant reservation. This is the first time it's done something for me. Other than go research something and teach me stuff. If you can make a restaurant reservation, you're not that far from being able to make a hotel reservation, and an airplane reservation, and order me an Uber, and- [00:00:00] S0: All of a sudden, you've got an assistant.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: All of a sudden, you've got an assistant. [00:00:00] S1: Yeah. And you can just imagine ... Everybody talks about that, but you can just imagine it's on your phone. I think that's pretty near-term, but some big companies that are very tech-forward, 50%-plus of customer support is already done by AI, and that's a $400 billion industry. Then what AI is great about is persuasion. That's sales and customer support. Of the functions of a company, if you think about them, they're to make stuff, sell stuff, and then support the customers. Right now, maybe in late '26, you're going to be pretty good at two of them. I do think it's going to have a big impact on media. I think robotics, like we talked about the last time, are going to finally start to be real. There's an explosion in exciting robotic startups. I do still think that the main battle's going to be between Tesla's Optimus and the Chinese, because it's easy to make prototypes, it's hard to mass-produce them. Then it goes back to that, what Andrej Karpathy said, about AI can automate anything that can be verified. Any function where there's a right or wrong answer or right or wrong outcome, you can apply reinforcement learning and make the AI really good at that. [00:00:00] S0: What are your favorite examples of that so far, or theoretically?