Invest Like the Best with Patrick O'Shaughnessy
00:46:04 · Neil Movva, Patrick O'Shaughnessy
We're increasingly seeing that all the AMD supply is also being allocated. But there's a long tail of other companies that are popping up. Net new companies are great, such as Etched or SambaNova or D Matrix. All these companies are popping up. I think the main challenge for them is scale. Can they actually get enough wafer allocation for TSMC to pump out chips to make an end to the market? If there's a new chip on the market, I'd like to know about it as quickly as possible and evaluate whether we can buy a good fraction of that supply. [00:46:30] S0: And so it's fundamentally an arbitrage for you. Like, if you can be much better at eking out performance from chips that have received less attention, you can then resell that at a margin and it could be a great business.
00:46:30 · Patrick O'Shaughnessy, Neil Movva
[00:46:30] S0: And so it's fundamentally an arbitrage for you. Like, if you can be much better at eking out performance from chips that have received less attention, you can then resell that at a margin and it could be a great business. [00:46:41] S2: Exactly. And I think that it's not the case that everyone else has a skill issue that they can't make these chips work as well. I think we're quite confident in this. I think we're probably one of the best teams in the world to use multiple silicon architectures and be quite aggressive in chasing down performance in unlikely places. But, yeah, I think it's the speed at which we're willing to kind of build our stack around a new chip. We don't have a huge amount of incumbency around, well, data center providers are only stuck with this class of chip and it's gonna be a huge pain for us to deploy these net new chips. We have some very creative data center partners who are willing to move very quickly and there's a new class of this that we can talk about. Most importantly, we don't shy away from the challenge. That's frankly a big part of this is just saying, yes, we love TPUs, we're gonna make TPUs work. Yes, we love Tranium, we're gonna make Tranium work. And if it doesn't work easily, we're gonna find a way to fit it in with the heterogeneous serving system. It will have a place. Every chip has a comparative advantage. We had to find that advantage and then squeeze it in that direction.
00:47:36 · Patrick O'Shaughnessy
[00:47:36] S0: Your finance team isn't losing money on big mistakes. It's leaking through a thousand tiny decisions nobody's watching. Ramp puts guardrails on spending before it happens. Real time limits, automatic rules, zero firefighting. Try it at ramp.com/invest. As your business grows, Vanta scales with you, automating compliance and giving you a single source of truth for security and risk. Learn more at vanta.com/invest. Ridgeline offers one unified platform that automates away the complexity across portfolio accounting, reconciliation, reporting, trading, compliance, and more, all at scale. Schedule a demo at ridgeline.ai. Every investment firm is unique, and generic AI chatbots don't understand your process. Rogo is built by finance professionals. It's connected to your data, learns your process, and quickly produces client ready outputs. Check them out at rogo.com/invest. The best AI and software companies from OpenAI to Cursor to Perplexity use WorkOS to become enterprise ready overnight, not in months. Visit workos.com to skip the unglamorous infrastructure work and focus on your product. Just as an interlude before we get to hardware, data centers, energy, etcetera, which will be really fun part of the conversation, I'd love you to talk about your perception of the investor classes worry. If you look at memory stocks or my current favorite is you look at the chart that plots the percent of the S and P 500 that's semiconductors. Historically, it was like two, three, 4%. Now it's 19, 20, 21%. And it just sort of looks like if you're a student of market history, you get all these things through time that have sort of reached some crazy near term peak and then collapsed back to long term norms. That has all investors worried. A lot of people made a lot of money in Micron and SK Hynix and companies like this. But everyone feels like long term like compute's a commodity, they will not represent a quarter or fifth of the entire market capitalization of the world. They're scared and that's the setup. Everyone acknowledges that there's a huge shortage. Everyone sort of feels like, we'll figure it out and these things will revert back down to their normal place in capital markets. I'm curious what you think about that narrative. I'm less of
00:49:45 · Neil Movva
[00:49:45] S2: a student of history as more of a member of history. I was born in 1997 and my mom worked at Intel in the run up to the year 2000 and the dot com boom and crash. I remember the time where Cisco was the most valuable company in the world and Intel was close behind. And mostly draw parallels to that period of history from twenty five years ago to today. And I think the main difference is that a lot of the investment in networking equipment historically was speculative. We anticipated this future demand for users that never came. I think what's interesting about token consumption or AI consumption broadly is that it's no longer speculative. People buy tokens because they're immediately valuable to them. You don't hoard tokens, you use them immediately. This is also even different from what we had two years ago where there was a supply crunch for hopper generation chips in 2023, 2024. In that period, it was all training oriented spend and training is inherently speculative. Now it's everyone instituting caps on how much you can spend on Cloud Code. It's a very, very different world to be talking about inference spend and predicting inference spend to go up. I do think inference spend monotonically increases. There's no speculation on inference.
00:50:50 · Patrick O'Shaughnessy
[00:50:50] S0: Coming back now to your take on hardware, the unit level is interesting to me. Talk about chips, talk about racks, talk about clusters. I'd love to talk about data centers. You said you've had some interesting partners doing some cool things. Talk us about the present and future of data centers as you see it, because this seems like obviously a critical thing being able to serve all this inference as lots of innovation in this part of the world and obviously you're focused on it.
00:51:15 · Neil Movva
[00:51:15] S2: I think one of the themes in our conversation has come back to what is training versus inference? Like, what is the difference between these two categories? And, you know, what was different about two years ago being training oriented and today being inference oriented? And I think the most conservative players in the entire AI stack have got to be the infra players, whether it's data centers or even more conservatives TSMC, that chip in for people. Data centers, they were built for training. Training is a super set workload over inference. You can make any training cluster work for inference, but maybe not vice versa. The difference there is networking. How much do you invest in bandwidth between chips and how large of a cluster do you need? There's actually a diseconomy of scale to data centers in some way. It's way more expensive and difficult to build a 100,000 GPUs in one data center than it is to build 10,000 than it is to build 1,000. Now we just talk about how many megawatts or gigawatts do you have. And basically, there's no way to build a gigawatt data center in The United States easily anymore. Even a 100 megawatts is increasingly hard. It's basically impossible unless you're a very special set of customers. 10 megawatts is probably on the edge of what's possible today. And one megawatt, I would argue, is plentiful. So there's this incredible floor on the market where you can find lots of aggregate power, but it will not be concentrated. And that was not interesting to anyone who's building training data centers because you just assume...