Invest Like the Best with Patrick O'Shaughnessy
00:42:42 · Patrick O'Shaughnessy, Neil Movva
[00:42:42] S0: Because of heat. [00:42:43] S2: Exactly, thermals. In practice, you don't spend the majority of your time in a transformer in that happy path where you're doing a large batch. Matrix will apply. Our job is to basically build the engine around the chip such that we are feeding the GPU these large batches of work at all times. One of the most profound transitions we've had in the GPU world in the last year has been this moving of you don't program one GPU at a time anymore. You should think about the whole rack and maybe you should think about the whole cluster, the entire data center at a time. With NVIDIA, they've started shipping not just a single GPU or a single motherboard, but actually the whole rack system is something that they prescribe. They call it NVL 72. Their latest chip, the Grace Blackwell 300, that chips as a rack of 72 units and it's an open race to figure out who can program the whole rack scale computer as efficiently as possible. And my belief is that that shape of compute is the future of efficiency and speed. In fact, NVIDIA does a great job of if you want the lowest possible latency, you should be using that chip. And if you want the highest possible throughput, you should probably also be using that chip as of right now. And it all comes down to like, this is a very new paradigm of programming. [00:43:43] S0: One of the things you hear is that the market for the best chips, Blackwell's let's say [00:43:48] S2: Yes.
00:43:48 · Neil Movva, Patrick O'Shaughnessy
[00:43:48] S2: Yes. [00:43:48] S0: Is like a drug market or something right now. There's all sorts of fascinating things happening to get as many of them as possible because everyone's so short. It would be to react to that analogy, is that what it feels like? Then also to talk about what the market is like for, like, not the bleeding edge chips. If I am willing to accept a slightly or moderately inferior chip, what's that market like? Let us into that world. Media has
00:44:10 · Neil Movva
[00:44:10] S2: a long term view on all their chips. They see this immense demand for the black hole chips. They can do what other suppliers have done in the past, which is crank prices, made the market supply and demand curves well correct. They'll intersect at some point and everyone will be technically happier. But NVIDIA sees if they just let the most deep pockets buy all the chips that maybe hurts them in the long term if that customer ends up accruing a lot of more power. They understand that compute is power today. They're quite strategic about how they allocate compute. That's the first thought. The second thought is that relationships matter a lot. Nobody wants to have a huge order of a chip rental come in from this new startup that says, oh yeah, I'm gonna rent 10,000 black wells for three years or five years. The startup has only been operating for months, typically. Who knows that they're good for the money? The way you convince someone to give you access to compute is quite challenging these days and requires some pretty great relationships or just incredible financial backing to make this happen on the NVIDIA side. And it's all because the scarcity is so high and demand is just off the charts. Now for other chips, I wouldn't even call them inferior. I like to say there's no bad chips, there's really bad pricing. I will make any chip work at the right price. That's one of the ethos of the company. Let's talk about AMD. AMD, I think great chips overall. The challenge is that people don't understand how to program them very well. I've been talking to you about how we have such a great kernel team. We're so serious about squeezing the performance out of the hardware. NVIDIA is pretty good at doing that for their own chips, frankly. There's some alpha that we can squeeze out, but there's a lot more to be done other chips because the vendor does a little bit less work than NVIDIA does to make the best kernels out of the box. Or even better, there is alpha and just other people have this perception that AMD is not as good as NVIDIA. That's music to my ears. I'm very happy for them to sleep on this chip and for me to buy as much as I can. Now I think that that's not actually super true anymore. I think AMD is actually somewhat popular amongst some large buyers. I think publicly, Meta and OpenAI have bought a ton of AMD chips.
00:46:04 · Neil Movva, Patrick O'Shaughnessy
We're increasingly seeing that all the AMD supply is also being allocated. But there's a long tail of other companies that are popping up. Net new companies are great, such as Etched or SambaNova or D Matrix. All these companies are popping up. I think the main challenge for them is scale. Can they actually get enough wafer allocation for TSMC to pump out chips to make an end to the market? If there's a new chip on the market, I'd like to know about it as quickly as possible and evaluate whether we can buy a good fraction of that supply. [00:46:30] S0: And so it's fundamentally an arbitrage for you. Like, if you can be much better at eking out performance from chips that have received less attention, you can then resell that at a margin and it could be a great business.
00:46:30 · Patrick O'Shaughnessy, Neil Movva
[00:46:30] S0: And so it's fundamentally an arbitrage for you. Like, if you can be much better at eking out performance from chips that have received less attention, you can then resell that at a margin and it could be a great business. [00:46:41] S2: Exactly. And I think that it's not the case that everyone else has a skill issue that they can't make these chips work as well. I think we're quite confident in this. I think we're probably one of the best teams in the world to use multiple silicon architectures and be quite aggressive in chasing down performance in unlikely places. But, yeah, I think it's the speed at which we're willing to kind of build our stack around a new chip. We don't have a huge amount of incumbency around, well, data center providers are only stuck with this class of chip and it's gonna be a huge pain for us to deploy these net new chips. We have some very creative data center partners who are willing to move very quickly and there's a new class of this that we can talk about. Most importantly, we don't shy away from the challenge. That's frankly a big part of this is just saying, yes, we love TPUs, we're gonna make TPUs work. Yes, we love Tranium, we're gonna make Tranium work. And if it doesn't work easily, we're gonna find a way to fit it in with the heterogeneous serving system. It will have a place. Every chip has a comparative advantage. We had to find that advantage and then squeeze it in that direction.
00:47:36 · Patrick O'Shaughnessy
[00:47:36] S0: Your finance team isn't losing money on big mistakes. It's leaking through a thousand tiny decisions nobody's watching. Ramp puts guardrails on spending before it happens. Real time limits, automatic rules, zero firefighting. Try it at ramp.com/invest. As your business grows, Vanta scales with you, automating compliance and giving you a single source of truth for security and risk. Learn more at vanta.com/invest. Ridgeline offers one unified platform that automates away the complexity across portfolio accounting, reconciliation, reporting, trading, compliance, and more, all at scale. Schedule a demo at ridgeline.ai. Every investment firm is unique, and generic AI chatbots don't understand your process. Rogo is built by finance professionals. It's connected to your data, learns your process, and quickly produces client ready outputs. Check them out at rogo.com/invest. The best AI and software companies from OpenAI to Cursor to Perplexity use WorkOS to become enterprise ready overnight, not in months. Visit workos.com to skip the unglamorous infrastructure work and focus on your product. Just as an interlude before we get to hardware, data centers, energy, etcetera, which will be really fun part of the conversation, I'd love you to talk about your perception of the investor classes worry. If you look at memory stocks or my current favorite is you look at the chart that plots the percent of the S and P 500 that's semiconductors. Historically, it was like two, three, 4%. Now it's 19, 20, 21%. And it just sort of looks like if you're a student of market history, you get all these things through time that have sort of reached some crazy near term peak and then collapsed back to long term norms. That has all investors worried. A lot of people made a lot of money in Micron and SK Hynix and companies like this. But everyone feels like long term like compute's a commodity, they will not represent a quarter or fifth of the entire market capitalization of the world. They're scared and that's the setup. Everyone acknowledges that there's a huge shortage. Everyone sort of feels like, we'll figure it out and these things will revert back down to their normal place in capital markets. I'm curious what you think about that narrative. I'm less of