00:25:13 · John Coogan, Jordi Hays
[00:25:13] S1: to a [00:25:13] S0: point where they've they're gonna be able to support SSI. SSI. Which is crazy. Not just their own needs. [00:25:18] S1: And then and then there's also another question [00:25:20] S0: in the sorry. The dynamic that's interesting is, like, OpenAI needs to massively scale infrastructure and and as we covered earlier this week commit to spending tens of billions of dollars with with Oracle. Yep. And their revenue today doesn't support that. Obviously, their their growth trajectory is absolutely insane. But OpenAI being in a position where they are competing head on with Google for this very very critical consumer use case, search knowledge retrieval [00:25:50] S1: Yep. [00:25:51] S0: And ultimately usefulness. It is a extremely tough position for OpenAI to be in.
00:25:57 · John Coogan
[00:25:57] S1: Yeah. So the other hyperscalers are working on ASICs that are alternative to NVIDIA GPUs, but they all seem to be following the similar path, trying to get line time at TSMC, not fully at scale. Then there's also the question, you know, I think a lot of people's mind is like, is this the end of history? Is there something that's coming down the pipe that could disrupt the NVIDIA GPU monopoly, the CUDA ecosystem, or even the Google TPU? And semi analysis has a funny meme here way down in the stack about the next generation of chips that were that that have been pitched from startups like Etched and a few others. So SemiAnalysis says, although baking transformers into silicon may sound cool, it's most mostly just a marketing slogan. 90% of transformers flops are just GEMMs and two fifty six by two fifty six or one twenty eight by one twenty eight systolic arrays in TPU and in TPU tranium are already optimized for these. Even modern GP GPUs with tensor cores are optimized well for GEMMs. Even if you bake transformers into silicon, aka just create a giant systolic array, most of your die area will still be taken up with SRAM cells. That's the memory. And you will still face the memory wall since your HBM memory bandwidth will be the same as GPGPUs, TPUs, and Trainium. And so the meme is transformers are just 90% matmoles. George Hotz had a similar analysis or take just saying that if you look at the actual energy use of GPUs right now, there isn't that much opportunity to squeeze more value out of them. They're pretty efficient and most of the most of the energy that goes in goes into these very, very specific math calculations that are already pretty optimized. We did hear from someone something about the this is from Martin Scarelli that Jane Street or was it was it Jane Street? One of the high frequency trading firms figured out how to run one of those very, very basic map mole calculations on a GPU more efficiently, and I guess kind of open sourced it or something or got out into the into the research world. So it it doesn't it doesn't feel like at this moment, I still think that there's a interesting bull case for the etched crew in some ways, in some niches or something, some specific models.
00:28:12 · John Coogan, Jordi Hays
But it doesn't seem like there's some new disruptive chip architecture that's gonna come out and and obviate the need for both Nvidia GPUs and Google TPUs. And so Google's very well positioned as Ben Thompson is writing. So he says this should give them their position that they have an at scale ASIC alternative to NVIDIA GPUs should give them a meaningful cost advantage to that end to the extent that cloud compute becomes commoditized is the extent to which Google actually has a margin advantage. Because if all of the clouds are commoditized, but all of them are on Nvidia, and then Google is taking the Nvidia margin basically from their TPU, they should be in a very good spot. This is to be sure a bit weird. Porat gained these plaudits those plaudits because there was so much waste in Google to cut which was downstream of the company's amazing monopoly and margins and search which hardly seems like the recipe for a structural cost advantage. But here we are. [00:29:11] S0: Sundar is gonna tell every employee, we're cutting your daily massage allowance from three massages down to one. We're deeply sorry, but we need to get fit. [00:29:20] S1: Gotta get fit. And you gotta get on Vanta. Automate compliance, manage risk, improve trust continuously. Vanta's trust management platform takes the manual work out of your security and compliance process and replaces it with continuous automation whether you're pursuing your first framework or managing a [00:29:34] S0: company's framework. Talking with Christina yesterday. She's an absolute dog. [00:29:40] S1: Boardroom general.
00:29:40 · John Coogan, Jordi Hays
[00:29:40] S1: Boardroom general. [00:29:40] S0: Boardroom general. [00:29:41] S1: She put the KPIs in orbit. [00:29:43] S0: She she really has. [00:29:44] S1: She really has. [00:29:45] S0: Did you see that? Did you see that? [00:29:47] S1: It. The Reed shirt. [00:29:49] S0: Yeah. The Reed's chart. She says It's just the Andrew said Christina often talks about stacking rice on the chessboard. Yeah. I didn't see this until after the show but I was like it seems like you guys are just chopping wood. Like Yeah. There like [00:30:03] S1: That's what it is. [00:30:04] S0: Putting in the work.
00:30:04 · John Coogan
[00:30:04] S1: Just you know, a couple percent here, a couple percent here. Just very consistent growth, very consistent growth, very smooth growth curve into a monster of business. So, congrats to everyone over at Vanta on the new round. And we will go back to Google. So, more broadly, the way in which Google seems to have flipped the switch in terms of going all in on AI along with Meta's spending on AI talent really does strongly suggest that AI in the end is a sustaining technology that favors the incumbents most of all. Both see a line of sight to new revenue streams and critically, both can fund AI from profits, not speculative investment or debt. And of course, both have distribution including search. Sometimes the Empire Strikes Back. I love that. So there were other takes from from search. You'll have to subscribe to Stratechery to get the full analysis. You gotta go and subscribe to Stratechery. If you're not subscribed, what are you doing? But the interesting takeaway from from search is that there it seems like Google's being a little squishy about hey, we don't want to report certain KPIs anymore. So they used to report paid clicks, and now the chief business officer, Philip Schindler, is saying and and on your paid click question, look, to be very clear, I think we said this before, we manage the business to drive great outcomes for our users and attract attractive ROI for our advertisers. We don't We actually don't manage to pay clicks or CPC targets which is fair and good, but it's funny because Ben Thompson is like, but I want that data. That would be helpful to me.
00:31:35 · Jordi Hays, John Coogan
[00:31:35] S0: I mean, when a business has a metric that they tell you is important for a long time and then they suddenly tell you it's not important. [00:31:42] S1: Don't worry about it. [00:31:42] S0: You gotta But [00:31:43] S1: it does make sense. They're a huge business. They've moved on to to bigger and like higher levels of abstract. [00:31:48] S0: Just look at the growth in tokens. Yeah. Yeah. But Don't worry about anything else. [00:31:52] S1: What what he says is that there are there are product changes and policy changes that actually drive better monetization but at the expense of paid clicks and that's probably AI searches and stuff like that. You'll see in the 10 Q paid clicks were up 4% year on year, but a number of factors affecting this these metrics from quarter to quarter such as advertiser spending, product changes, policy changes, user engagement. So it's really important when it comes to pay clicks and CPC to avoid drawing overly broad conclusions solely based on these metrics. Don't put me in the truth zone on this, he says. So Ben Thompson, this is exactly what you said. I don't even think you pre read this but you basically have Ben Thompson working in your head because [00:32:32] S0: This is what I said when Ben Thompson came on the show. Was like, I think your way of thinking has been so ingrained into my mind Yep. That I I think that I have an original thought and you've already thought it.