Invest Like the Best with Patrick O'Shaughnessy
00:37:00 · Gavin Uberti
[00:37:00] S1: For me, think there are two key tricks to this. The first one is that you can't build a chip alone. It's got to be a team problem, and your most important job is to go get great people to go with you and great people to go ahead and be inspired and excited to go ahead and do crazy things like this. It is a huge ask to go say, hey, guys, uproot your lives for six months or in one case, twelve months that we had sent one guy out well ahead. It sucks, but we're lucky to have team members who are in it for the right reasons. But think the second big thing too is being able to make decisions very fast, that one of the worst items is when there's a factory or there's a vendor who is waiting for you to go ahead and make some call and has been just stalled. And this happens all the time, even for very small things. So, send folks, delegate a big amount of responsibility to them and say, make a reasonable call. Okay, if you're wrong every now and then, but I would much, much rather be right both of the time and give an answer immediately than wait every time for the perfect response.
00:38:03 · Patrick O'Shaughnessy
[00:38:03] S0: Speed wins. What about spending money to go faster? There's this learn by doing thing which has become so interesting, and as the world has gone away from software and towards more hardware again in the world of technology, that we've outsourced so much of the learn by doing by shipping stuff overseas and effectively just being the ideal guys here in The US. Seems like that obviously is reversing, and you've adopted this way of learning by doing, like you want to be in that iteration learning loop. Absolutely. And part of that is willingness to spend and take risk with dollars. Yeah. Can you talk about that a little bit?
00:38:34 · Rob Wachen
[00:38:34] S3: I think there's a great quote of, like, the biggest risk is not taking risk. Very similar here, which is, like, every day, there's over a billion dollars of revenue in this category, and a lot of it's inference. So every day we don't ship, we're just leaving tons of opportunity on the table. So your willingness to spend money should be extremely high if you can get a very clear ROI out of it. So we have this concept that we call prefetching, which is when you're waiting for one thing to get done, when you know you're going to do other things once you have is there ways that you can paralyze the entire schedule? So for example, like, we know our chip is going to come back on a certain date. We want it to be that everything possible that could be done without the chip is done before the chip lands. And this costs a lot of money. This means that, like, we wanna build our entire software stack beforehand. Like, shipped racks to customer data centers without our chips in them with all the networking, all the CPUs, all the storage all set up so we could bring all that data center software up before the chips came back. It meant that we took over 700 FPGAs and put the entire full reticle chip on an FPGA cluster and ran a dozen different models with our full inference stack on them before the chips came back. It means that we built a thermal chip to mock the thermal profile of our chip and build cold plates based on that before the chip came back. It means we had the entire production line ready. It means we did many revs of the circuit board. It means the entire product was ready to go before the chips came back. And this is what it gives you. There is another very famous AI chip company that took ten months to go from getting their silicon back to having them running inference in a rack. And this was, like, publicly announced to their investors, and it was a really big deal. We were able to do it in forty days. And it's because by the time the chip came back, everything was boring. The software was already written. The rack was already there. The production line was already set up. We were just go, go, go, go, go, get everything together. You don't always catch everything. You make some tweaks on the fly, and then off you go.
00:40:24 · Gavin Uberti, Rob Wachen, Patrick O'Shaughnessy
[00:40:24] S1: In that particular case too, that was a big part of it. Also, like, shift, I think, made a big difference too. Totally. We went out and literally had a day shift and a night shift. There were team members who would come in at 10AM and leave at around midnight. She would come in at midnight and leave at 10AM. You're running around the clock to get to that forty days. [00:40:43] S3: Yeah, I mean over half the company lives next to the office, so it makes it easier to do that type of thing. [00:40:47] S0: You pay them to do that, right? Pay them extra, do you still do that? [00:40:50] S3: The invisible hand does wonders. [00:40:52] S1: I mean, works for me too. [00:40:54] S3: We're both there. [00:40:55] S0: I'd love to take one big step back and talk a bit about just the broader ecosystem here. The amount of shortages on the supply side, the exposure of risks in the global system and the supply chain around this stuff has become like everyday Wall Street Journal front page news. Like the stocks that people are watching and investing in and excited about, about the memory stocks. These were boring commodity nothing burgers five years ago. And now they're the center of global attention. If you just assess because you've been building in it, the global connected supply chain that's required to make stuff like this possible, just riff on it. Like what scares you? What's working well? What needs to change? What do you hope you change by virtue of how you build this thing? What's your assessment of this story right now?
00:41:42 · Gavin Uberti
[00:41:42] S1: I think that one of the most undervalued pieces of a supply chain story is almost none of these things are you buy them and they don't talk to the vendor again. You have to go collaborate. That is the most important part, being successful, I think, in ships with TSMC or with memory vendors. You need that partnership. I think that for TSMC in particular, people don't understand why it is so valuable. People look at the tech and the tech is the best in the world. But for me, the real value is all in the service. TSMC customer service is way, way better than I have seen at any other company in any other industry. It's the kind of thing where if you say, hey, you can improve your yield by making this change, You can go make them a recommendation and then we'll go run an experiment on their own dime in our case to see if they could actually get the higher yield. And when we found that we were right and the experiment worked, they moved over the rest of the line. And that kind of thing doesn't happen in most industries. If I go to, like, the steelworks plant and say, hey, I want you to change the composition of the steel, and they'll say, screw you. Not TSMC. It is why they are the number one and why they're going to win.
00:42:51 · Rob Wachen, Patrick O'Shaughnessy
[00:42:51] S3: One of the things that matters a ton is power availability and time to power. And the problem is the more power you want, the more shortage there is. It's actually very similar to chip clusters, which is like, why is Colossus charging $12 an hour for black holes? It's because they're the only place you can buy 20,000 of them at once. Right? Why is the 500 megawatt data center so hard to find? It's the exact same reason. You know, one of the things we need to think about is, like, how do we get way more juice out of each megawatt? People are looking throughout the entire stack, whether it's just improving the PUE, but also entirely new hardware to get the most tokens per megawatt to solve this problem. But fundamentally, building new buildings is hard. It's much easier to go from a 100 megawatt to a gigawatt from a gigawatt to ten and ten to a 100, and we are pushing the limits of what's possible on these timelines. So there's a lot of people trying to scale in their data centers as much as trying to scale them out. [00:43:41] S0: One of the interesting things about a system like this is what it replaces. Yeah. If I think about a rack like this versus, I don't know, a set of Blackwells or something or Rubens or whatever is coming next, how should I conceptualize that? It's not just Watts. It's also physical space. The Rebbe's talked about this in their recent reviews called that literally this is a problem, that there's literally no space to put the systems. How should I conceptualize what this represents or replaces in terms of other units of compute?