Invest Like the Best with Patrick O'Shaughnessy
00:33:59 · Patrick O'Shaughnessy
[00:33:59] S0: One of the very first things you and I talked about, Rob, was I started asking about Sohu, which is the name of the first product here. And you said we can talk about that in great detail, but the thing you should know is that what we're really focused on is building a machine that can, at scale, produce these things and generations of them as efficiently at the highest possible quality level. So we want to build like the company or the machine that is the company is the thing that will produce this thing and then subsequent things. So I'd like to talk about a few principles or cornerstones of the company. You've alluded to some of them. You've said velocity. You've said vertical integration. These have become more popular topics. Parallelization is something maybe that we should talk about. But I'm especially interested in your guys' willingness to take huge risk to go faster. Maybe tell your favorite story about why this is the philosophy, what it's allowed you to do that maybe other companies haven't done. There's a number of stories here,
00:34:52 · Gavin Uberti
[00:34:52] S1: but one of my favorites is there was a time where we were getting close to taping out the chip. We realized, wait a minute, one of our vendors is way, way behind schedule. And we had two very bad options. One option, just keep the current vendor and push our timelines up by on the order of a year. Another option was to push vendors, start over and also get push timelines out by a year. Neither of these was a good option. So we had to go look for option number three. And what that was, was we figured out they're all in Bangalore. They think they're actually going and doing the work. We went out and shipped a dozen of our top engineers across the world to Bangalore for six months. So I was there as well. I lived in Bangalore for four and a half months personally and every morning, we'd go ahead and walk across the crazy busy Bangalore streets into the office, We'd be the first ones in. We'd go out and build a wide variety of tools, both things like, hey, auditing a huge amount of the code that was going in, building a bunch of tools as well to make this go even faster, making sure we're making the right design decisions on the spot right there. No to allow back and forth, go ahead and decide immediately. And then at 1AM, we'd go walk back through the now empty Bangalore streets and do it all again the next day.
00:36:03 · Rob Wachen, Patrick O'Shaughnessy
[00:36:03] S3: We still had a bunch of the team in The US. We ran these twelve hour on each side handoffs where we had twenty four hour development cycle where at 8AM and 8PM every day, we'd all get on the Zoom, we'd share all the data, and we'd say, when I wake up, like, this must be done. Like, we must get this chip out. And it was extremely intense. At the same time, we saw other chips at the same stage as us with that same vendor that ended up taking years that still aren't out today. They still haven't even taped out today. And it's that level of extreme urgency that's required to bring products to market. What is the key [00:36:34] S0: to doing this? Well, this has become a trope because of Elon mostly that, like, his special skill and others that seek to emulate him would try to do this too, is figure out what the binding constraint is and just flood the zone personally on that thing, which is kind of like going to Bangalore or something. It seems like this is a central tenet of the business and of any business that's going to do this kind of vertical integration. What's the key to doing that well? Like, again, what have you learned about that specific act?
00:37:00 · Gavin Uberti
[00:37:00] S1: For me, think there are two key tricks to this. The first one is that you can't build a chip alone. It's got to be a team problem, and your most important job is to go get great people to go with you and great people to go ahead and be inspired and excited to go ahead and do crazy things like this. It is a huge ask to go say, hey, guys, uproot your lives for six months or in one case, twelve months that we had sent one guy out well ahead. It sucks, but we're lucky to have team members who are in it for the right reasons. But think the second big thing too is being able to make decisions very fast, that one of the worst items is when there's a factory or there's a vendor who is waiting for you to go ahead and make some call and has been just stalled. And this happens all the time, even for very small things. So, send folks, delegate a big amount of responsibility to them and say, make a reasonable call. Okay, if you're wrong every now and then, but I would much, much rather be right both of the time and give an answer immediately than wait every time for the perfect response.
00:38:03 · Patrick O'Shaughnessy
[00:38:03] S0: Speed wins. What about spending money to go faster? There's this learn by doing thing which has become so interesting, and as the world has gone away from software and towards more hardware again in the world of technology, that we've outsourced so much of the learn by doing by shipping stuff overseas and effectively just being the ideal guys here in The US. Seems like that obviously is reversing, and you've adopted this way of learning by doing, like you want to be in that iteration learning loop. Absolutely. And part of that is willingness to spend and take risk with dollars. Yeah. Can you talk about that a little bit?
00:38:34 · Rob Wachen
[00:38:34] S3: I think there's a great quote of, like, the biggest risk is not taking risk. Very similar here, which is, like, every day, there's over a billion dollars of revenue in this category, and a lot of it's inference. So every day we don't ship, we're just leaving tons of opportunity on the table. So your willingness to spend money should be extremely high if you can get a very clear ROI out of it. So we have this concept that we call prefetching, which is when you're waiting for one thing to get done, when you know you're going to do other things once you have is there ways that you can paralyze the entire schedule? So for example, like, we know our chip is going to come back on a certain date. We want it to be that everything possible that could be done without the chip is done before the chip lands. And this costs a lot of money. This means that, like, we wanna build our entire software stack beforehand. Like, shipped racks to customer data centers without our chips in them with all the networking, all the CPUs, all the storage all set up so we could bring all that data center software up before the chips came back. It meant that we took over 700 FPGAs and put the entire full reticle chip on an FPGA cluster and ran a dozen different models with our full inference stack on them before the chips came back. It means that we built a thermal chip to mock the thermal profile of our chip and build cold plates based on that before the chip came back. It means we had the entire production line ready. It means we did many revs of the circuit board. It means the entire product was ready to go before the chips came back. And this is what it gives you. There is another very famous AI chip company that took ten months to go from getting their silicon back to having them running inference in a rack. And this was, like, publicly announced to their investors, and it was a really big deal. We were able to do it in forty days. And it's because by the time the chip came back, everything was boring. The software was already written. The rack was already there. The production line was already set up. We were just go, go, go, go, go, get everything together. You don't always catch everything. You make some tweaks on the fly, and then off you go.