The Information's TITV
00:12:28 · Akash Pasricha, Ben Pouladian
[00:12:28] S0: Great. Okay. Well, Shiv, thanks for coming on. That is Shiv Verma, the CFO of Robinhood here on TITV. TPUs are taking center stage for Google right now. The Information has exclusive reporting that OpenAI, a Longtime Nvidia customer, is planning to buy Google's rival TPU chips. That scoop came the same day that Oracle said that it will also make TPUs available to its cloud customers. For more analysis, I want to bring on Umesh Padval, Managing Partner at Seligman Ventures, and Ben Pouladian, Founder and CEO of BEP Research. Gentlemen, welcome to you both. It is great to have you both back. Ben, I want to start with you. So why are Oracle and OpenAI turning to TPUs right now? [00:13:14] S3: I think they... You know, good morning. I think they wanna meet where their customer is. At the end of the day, I've I've written about this before. Google itself, like Google Cloud, buys a lot of NVIDIA compute, and they do a great job of building out their cloud and leasing it out to customers through GCP. And because Oracle is going through this transition of going from a software company to this major cloud provider, they need to follow suit to do the same thing. It's just another product offering in their inventory where they can now offer customers TPU compute. What percentage or what cost is yet to be determined. [00:13:57] S0: Okay. So, Umesh, Ben's making the point, the more variety, the better. But aren't there some benefits here that the TPU has compared to what NVIDIA offers?
00:13:57 · Akash Pasricha, Umesh Padval, Shiv Verma
[00:13:57] S0: Okay. So, Umesh, Ben's making the point, the more variety, the better. But aren't there some benefits here that the TPU has compared to what NVIDIA offers? [00:14:07] S2: So these are two brilliant moves by Google as well as by Oracle. Let me focus on Google. Google has two businesses, the infrastructure business where the TPUs are made and the training happens, and the TK's business which provides cloud services. So this is called cooperation and competition at the best to serve the best customers. So what did Google Cloud do? They used to use their own GPUs and g... And GPUs from NVIDIA, but mostly have their own models hosted on that for their customers. So now the deal with OpenAI is Google Cloud is now gonna provide a variety of proprietary models on their stack to serve their customers. On the other hand, Amin Vahdat, who heads up their infrastructure, did a brilliant move because he feels his TPU is as good or better and give competition to NVIDIA. So what does he do? He strikes a deal with Oracle, which is a cloud competitor of Google Cloud, and provides him access to his best technology. [00:15:18] S1: Right.
00:15:18 · Shiv Verma, Umesh Padval, Akash Pasricha
[00:15:18] S1: Right. [00:15:18] S2: Not only that, as you followed, I think Google is now going into the merchant market for TPUs. This is Oracle is one, but you know a neo cloud called FluidStack, which provides TPUs to Anthropic. So this is called competition between Google and NVIDIA and AMD. And then, regards to Oracle, they're trying to serve their customers with the best possible solutions, whether it's TPUs, NVIDIA GPUs, or the inferencing solutions like SambaNova in a disaggregated stack. It's brilliant. [00:15:59] S0: Okay. So I hear you. Let's talk about what the TPUs are good at compared to GPUs. Mean, Umesh is pointing to inference here. What are some of the advantages that the TPU has over NVIDIA right now in insofar as the work that it's trying to focus on?
00:15:59 · Akash Pasricha, Ben Pouladian
[00:15:59] S0: Okay. So I hear you. Let's talk about what the TPUs are good at compared to GPUs. Mean, Umesh is pointing to inference here. What are some of the advantages that the TPU has over NVIDIA right now in insofar as the work that it's trying to focus on? [00:16:19] S3: Obviously, the number one reason people turn to it is cost. And then from a business standpoint, the risk is being locked into one vendor, NVIDIA. So the TPU, obviously, has the same type of, you know, matrix multiplication and action that a GPU has, but at the same time, it's not as general purpose. So it's you have to basically design it for specific workloads. So Right. They... If if they use it for inference, that's great. I I think even Meta has, you know, entertained using some TPUs. So if you can find a specific workload or a case that you wanna use it for, it works well. And obviously, has proved it working well horizontally in its stack with its search, its ad business, YouTube overviews, and what it's building with Gemini. So it's a formidable competitor to what the NVIDIA GPU is. And I agree with Umesh that... I mean I mean, Vahdat has definitely taken the front seat and wants to show the world how great the TPU is, and they've ramped up. And they're making deals and working with Blackstone to finance TPUs for other customers. So it's definitely a a multichip world out there for customers. [00:17:40] S0: So, Umesh, I may
00:17:40 · Akash Pasricha, Umesh Padval, Gopal Srinivasan
[00:17:40] S0: So, Umesh, I may [00:17:42] S2: Let me add to this. Yeah. Look. The... People talk about TPU, but TPU has two solutions. One is for inferencing, and one is for training. So people only mistake TPUs for just training going after GPUs. It's both. They have two solutions, one for inferencing and one for training. Now what is the most important trend, I think, which is I'm seeing in the last six months since GTC, Jensen announced disaggregated solution. What does that mean? Initially, it was TPU or g p... XPU, a CPU, and inferencing engine all provided by one supplier, NVIDIA. It's disaggregated now, and what is happening is the best of breed. So you use the best of breed XPU for training, use the best of breed CPU for the orchestration, and use the lowest cost, highest performance inferencing engine from different suppliers. So it's disaggregating. This is where SambaNova plays tremendously well. Because what they have done is... You saw the deal where they have high performance but at extreme low power. And why is power important? You know, these data centers, when you have massive power of these chips, you need complex liquid cooling technology. SambaNova, because of the power envelope being so low, it does not need liquid cooling. It can do air cooling. Right. What does that do? [00:19:23] S4: Me
00:19:23 · Gopal Srinivasan, Akash Pasricha
[00:19:23] S4: Me [00:19:23] S0: jump in here for one second here. You know, I'm thinking about the projections that some analysts have put out here. So TITV Cowen is saying that TPU revenue by 2028 could be a third of Google Cloud's revenue, which is pretty staggering. Ben, I'll come to you for this one. Mean, how do you think we can put aside whether or not it hits that goal, that margin or that percentage or not? That's a projection. That's not the company saying it. But if it gets there, how does it affect the margins for the Google Cloud business?