Invest Like the Best with Patrick O'Shaughnessy
01:09:21 · Gavin Uberti, Patrick O'Shaughnessy, Rob Wachen
[01:09:21] S1: There's always the index fund. [01:09:22] S0: Yeah, there's always an index fund, exactly. So it's actually never been scary for me. I think probably most of that is because there's a lot I don't know. And if I knew more about what you guys have done and the difficulty, I probably wouldn't have done it. I don't know what that says about maturing as an investor. Maybe I don't want you know, a lot more and and have some of that healthy naivete. Don't know. [01:09:41] S3: Funny. I mean, a lot of the traditional semiconductor funds missed the entire AI chip, like, all the AI chip companies. Yeah. And like, all the coding experts missed all the coding companies. I think it's very hard to realize the constraints have changed. And when you've looked at tape outs for twenty years, and you've seen so many of them not work on the first try or the second try or the third try, like you couldn't even run a workload. You totally forget that EDA tools are way better and that FPGAs exist today in a way that they didn't exist before. And all the types of validation you can do today just wasn't possible. For us, a lot of our believers either they were kind of on two sides of it. They were just believers in the market and the team, or they were building chips today in extremely technical, like the high 50 trading firms [01:10:25] S1: Yeah.
01:10:25 · Gavin Uberti, Rob Wachen, Patrick O'Shaughnessy
[01:10:25] S1: Yeah. [01:10:25] S3: Where they would literally audit everything from the micro architecture and the RTL to like the board designs and like the schedule and the software stack. And we would sit down with like 10 of their people who build their own chips and they're asking us such detailed questions that we're wondering, are they gonna build the chip? It was really on either of those sides. And if you were anywhere in the middle, like, you just wouldn't understand it. [01:10:43] S0: In the investing world, they all often talk about variant perception, something that you see or believe that others don't. Right? And that perception creates the opportunity. I think I've invested, I don't know, five or so times in Etched. And every time when you do it, stakes are getting bigger and bigger. It does get a little scarier and scarier. And because you guys have been so quiet in the marketplace, I think it's very easy to dismiss you. As the stakes get bigger and bigger, those dismissals are harder to hear. I do think betting on something that you see when what you hear from the outside world is very different. That perception gap equals opportunity. Exactly. The last thing I would say is the accumulated evidence of your guys' and your team's ability to solve seemingly impossible problems is one of the most interesting things a company can have. It's like a binary, like companies do this or they don't.
01:10:43 · Patrick O'Shaughnessy, Gavin Uberti
[01:10:43] S0: In the investing world, they all often talk about variant perception, something that you see or believe that others don't. Right? And that perception creates the opportunity. I think I've invested, I don't know, five or so times in Etched. And every time when you do it, stakes are getting bigger and bigger. It does get a little scarier and scarier. And because you guys have been so quiet in the marketplace, I think it's very easy to dismiss you. As the stakes get bigger and bigger, those dismissals are harder to hear. I do think betting on something that you see when what you hear from the outside world is very different. That perception gap equals opportunity. Exactly. The last thing I would say is the accumulated evidence of your guys' and your team's ability to solve seemingly impossible problems is one of the most interesting things a company can have. It's like a binary, like companies do this or they don't. [01:11:35] S1: That's the thing that's a big advantage of people who have been here for a long time. You get some new joiners who are scared shitless. You see a thing like this, and there are old timers who've been here for all of two years, smoking cigars in [01:11:47] S0: the trenches. [01:11:49] S1: Another one. Another one.
01:11:49 · Gavin Uberti, Rob Wachen, Patrick O'Shaughnessy
[01:11:49] S1: Another one. Another one. [01:11:51] S3: Yeah. There's definitely a find a way mentality. If you're here, you're here because you assume it's possible. So, like, we can't be saying it's impossible. Everything is solvable, and we're just going to work at it until we figure it out. A favorite story I have, this guy who's kind of a legend in silicon validation, who joined our team and we were doing the early stages of what's called wafer sort. When your chips are coming out of fab, they go out on these wafers and you have this thing called a probe card that attaches to the wafer before you dice it with these probe pads and you send these electrical signals to basically test which chips are good and bad. So when I slice the wafer into a bunch of chips, I can package it and only package the good ones. We go through our first wafer. It's like two or 3AM because we're doing it with TSMC over the phone in Taiwan. We have the screen with the wafer that's all gray and each ship is gray. And then as you start running the patterns, the squares are supposed to turn green or red. And they all turn red. We're like, fuck. Like, this is really bad. Like, everybody's like, guys, take a breath. He leads back. He's like, the puzzle begins. [01:12:52] S1: Like that you have to have the attitude of like, yes, you will go out and you will go stare into the abyss. You will go see scary things, and we'll solve them. [01:13:01] S0: When did you see the first Green Square?
01:13:01 · Patrick O'Shaughnessy, Rob Wachen
[01:13:01] S0: When did you see the first Green Square? [01:13:03] S3: Within a day of that. But in the moment, it's extremely scary, and there's a certain type of person who just like is addicted to that feeling, of just feeling the fear and solving it, and we're lucky to have lot of those people here. [01:13:15] S0: If you think about applying all of this earned know how from this last several years and now thinking ahead to Gen two, Gen three, beyond, what will you be doing most differently as a result of everything that you've learned? Just like from a conceptual standpoint, like the way that you will attack designing and producing this next one, based on what you learned doing it the first time?
01:13:36 · Rob Wachen
[01:13:36] S3: It took us a while to get to the primitives that we think are really what matters for scaling inference. We tried a bunch of things early on from compilers that would turn different models into FPGAs, to burning weights in silicon, to splitting your HBM, to KV cache and weights, and all of these different things. And there was a lot of cycles of learning till we got to the point that we realized that like, fundamentally, if you want to run a majority of tokens in the world, you need to do three things. You need to build a chip with the most flops and given power budget, you need to build a chip that has the lowest latency between other chips, so the biggest scale up domain possible, and you need to produce as much of it as possible. And I think probably in the first half of our journey so far, we learned the first two and that informed the design a lot and that informs a lot about the bets we're making in the future with the low voltage inference and the cluster scale memory. But the production part, I think, in the past year has become extremely obvious how much people want to deploy this stuff if you can have it available today. The best ability is availability. If I have a thousand chips a day, someone's gonna use them. And, you know, we need to build a chip that's not just like way better than what's been built before, but it needs to be available at many gigawatt scale. We need to be able to be building a product that is producible at gigawatts per month than the limit. As we think about that, a lot of the design decisions we're making with our next gen, which you've seen already, is just about simplicity. Removing tons of parts, trying to assemble and disassemble a thing again and again, and learning how to make it as quick in the cycle times as possible in production, making sure it's gonna be reliable, making sure it's gonna be serviceable, and making sure it's gonna be producible at gigantic scales.