Invest Like the Best with Patrick O'Shaughnessy
00:57:51 · Gavin Uberti, Rob Wachen
[00:57:51] S1: We built this massive, massive FPGA cluster to go out and verify that full chip worked as is. And FPGAs are digital entities. You can go test digital logic, but not analog logic. And it turns out that when the chip came back, we began to go see issues in our retention of adaptation of incorrect results. And we realized, wait a minute, there's a problem with a back pressuring logic across a clock domain crossing is failing, and this is going to cause the ship to produce wrong results and it is very, very hard to solve. We realized there was one and only one way to solve it, as we had to go line up two clock signals on our chip to within fifty picoseconds, that is literally fifty trillionths of a second, and we had to go get these signals aligned to this super small granularity and do it on every chip 2,000,000,000 times a second. [00:58:42] S3: A lot of people said this was impossible. We had people quit.
00:58:42 · Rob Wachen, Gavin Uberti, Patrick O'Shaughnessy
[00:58:42] S3: A lot of people said this was impossible. We had people quit. [00:58:45] S1: Yeah. That people literally were like, this problem is unsolvable, and that's the luck, guys. Well, when you have a problem like that, step one is, okay, let's assume the problem is solvable. How would it be solved? Well, we realized what we have to be able to do is find a way to go ahead and move our clock phase by a picosecond, ten picoseconds. And we had an idea, what if we had these two clocks, they had them just a little bit apart from each other. We figured out that, hey, if you go out and figure out the phase, if we then go out and use the drifting mechanism go ahead and wait for just the right amount of time to get that always lined up, we can do this extremely reliably and then lock the phases exactly where they have to be. We can guarantee this never happens. People were, I think, blown away that this worked, and it worked as well as it actually did, but we made it work. [00:59:35] S0: How long did that take? [00:59:37] S1: It was actually about two weeks. [00:59:39] S3: It was dark two weeks. [00:59:40] S1: It was a very scary two weeks, but it was the kind of thing where when that kind of thing happens, that is the most important time to go ahead and be investing effort, that is the hardest time to go do it when you feel like things are hopeless, but like the sooner you solve that problem, the sooner you can get back to building and scaling up production to mass volumes.
00:59:40 · Gavin Uberti, Rob Wachen
[00:59:40] S1: It was a very scary two weeks, but it was the kind of thing where when that kind of thing happens, that is the most important time to go ahead and be investing effort, that is the hardest time to go do it when you feel like things are hopeless, but like the sooner you solve that problem, the sooner you can get back to building and scaling up production to mass volumes. [00:59:57] S3: I think a lot of our story is like, as Gavin says, assume it is possible. Like, assume it is possible to have a chip with way more flops on it. Assume it is possible to have a system with way lower latency between chips. Assume it is possible to create a shared memory pool that can run at way higher bandwidths, like, how would one do it? A lot of the time when we do experiments, we will do dozens of experiments and all of them will fail. But we only need one to work. There's multiple times I mean, Gavin, I think, was leading the charge and our Chip bring up with, I think, 30 different board experiments, and three of them worked. And all three of them are worth their weight in gold. [01:00:30] S1: This is one of the things that people come to me and say, Gavin, almost none of your experiments works. I only got to get lucky once.
01:00:30 · Gavin Uberti, Patrick O'Shaughnessy
[01:00:30] S1: This is one of the things that people come to me and say, Gavin, almost none of your experiments works. I only got to get lucky once. [01:00:35] S0: So one idea for one of these stories that I'm asking about, you know, difficult moments in the company's history is around the ability to raise capital to fund the thing. I think when you started it, you knew you'd need capital, but you did not know you'd need the quantum of capital that you've ultimately raised and are spending to build the solution. And you hadn't raised money before. These are all new things, right? There was moments where it was really, really difficult because I was there, I saw it moments where it was extremely difficult to raise the money that you did, without which the company would not exist. It would have died. And like many great stories, there are many near death moments. But money specifically in this new world, this isn't software. You don't just need a little bit of money. Maybe you could tell the story about the true hardest part about raising money early on, before you had something that you could show people and be so proud of in performance that you could show them blow their socks off. It was just you guys talking about an idea. But talk about the early difficulties raising money because it was pretty hardcore.
01:01:30 · Rob Wachen, Gavin Uberti
[01:01:30] S3: We've had some intense moments. It reminds me of probably early twenty twenty four before we raised our series a. And we were at this point where we had done enough of the architecture, done enough of the design that we knew that the chip architecture was sound. We had to go build it. There was a lot more to do. We were ready to go into what's called the physical design stage. We needed to sign an agreement with a physical design vendor, which it will cost you at least $4,050,000,000 dollars. And then we had this realization as the models were getting bigger and bigger and you're seeing these giant MOE models come out, that we were going to need to build the entire cluster, not just the chip, but we were gonna need to build boards, we're gonna need to build interconnects, we need to build cold plates, we're gonna need to figure out all of the networking and everything, and that this was gonna cost a lot more than the $15,000,000 we had in the bank. [01:02:17] S1: And you're like, man, that was scary. That like, you're sitting in that moment, and you think, holy crap, we can't afford this, and I began looking at it like, how hard is it to go back to Harvard?
01:02:17 · Gavin Uberti, Rob Wachen
[01:02:17] S1: And you're like, man, that was scary. That like, you're sitting in that moment, and you think, holy crap, we can't afford this, and I began looking at it like, how hard is it to go back to Harvard? [01:02:30] S3: At the end of twenty three, we put together this like memo. We spent like a hundred hours on this because we're like, we have no idea how people are gonna believe us when we ask for the amount of money we're about to ask for. And it was like 30 pages. It was like extremely technical and in-depth of all the different things we needed to build and like all the milestones we needed to hit and how the market was gonna evolve and all the new use cases and the cost per token and all this modeling. And then we went and talked to investors and every major investor in the Valley passed immediately. They were just like, okay, two kids that just finished Harvard, haven't taped out a chip, no test chip. Inference, who knows if this is gonna be a big market? Everything's gonna be training. The model's still hallucinating. This could all be a bubble. You know, at the time, the biggest semiconductor fundraisers for a series a was around like $4,050,000,000 dollars. We were looking at this and we were just like tallying the bill. And we're like, we think we're gonna spend a $100,000,000 in the next twelve months. We really wanna do this. Like, if we want to actually get to scale and, like, actually get the performance we're talking about, like, this is going to be extremely capital intensive. How the hell are we going to pull this off?