No Priors: Artificial Intelligence | Technology | Startups
00:07:06 · Elad Gil, Andrew Feldman
[00:07:06] S1: Mhmm. [00:07:07] S0: And they turned out to be dead right. [00:07:09] S1: Were there moments where you just doubted whether this would work given the [00:07:12] S0: different for sure. Yeah. We had a period you know, we're solving a problem that had never been solved before. I mean, there'd been efforts across the entire seventy year history of the compute industry to build a wafer scale product. In fact, Gene Amdahl, sort of one of fathers of our field, one of the guys on Mount Rushmore of compute, failed miserably to do it. We had a period between about 2017, middle of twenty seventeen and middle of twenty nineteen, where we couldn't build it. We were spending about 8,000,000 a month. You're having board meetings every six weeks saying, I can't build it. No, it's still not working. And right, Oof is right. I mean, that's a huge amount of money and a huge amount of conviction your investors have. And each time we did a failure analysis, we got a little bit better at it. We got a little bit better at it. And then in the summer of of nineteen, we we yielded it, and it began to work. And the first time, we were sitting in a a little makeshift shift office in Downtown Los Altos in a building that was not designed for hardware guys, And we're staring at a computer, which is about as exciting as watching paint dry Mhmm. And it's working. And we just we couldn't speak for half an hour. Right? It's like, nobody had been able to do this, and it's working. And we did this.
00:08:31 · Elad Gil, Andrew Feldman, Sarah Guo
[00:08:31] S1: That's amazing. Because that that's the technical side of it, then there's a market side. Right? And also on the market side, to your point, it took time to get to the point where these workloads were really important. So were there moments where you doubted whether the market existed? [00:08:42] S0: You know, we we solved it, and we solved this sort of the hardest problem in the computer industry, and nobody cared. Nobody. It was like, you know, the first gen, we might have sold a dozen. The second gen, we probably sold 300, and now we're selling in a sense of tens of thousands in the third gen. We had a two or three year period where we were ahead of the market, and absolutely nobody cared that we were blisteringly fast. [00:09:08] S2: And you found some pioneering customers that were like atypical in terms of a starting point, right? Were some sovereigns who really bought ahead. Like, how did you think about being resilient to this period of being ahead of demand?
00:09:21 · Andrew Feldman
[00:09:21] S0: I think there's a path that has been laid down by new computer architectures. And often you begin in the supercompute world because those guys love speed, they don't care if your software is immature. And so we sort of ran the table there. We won at Argonne National Labs and at Lawrence Livermore and at Sandia and in Europe, the European Parallel Computing Center at LRZ. So we ran the table there, and then we we won some guys in in the oil and gas space, and we won some guys in pharma, all of whom have long histories of using extraordinary amounts of compute. But then historically, there's this giant chasm because none of them provide the volume to get to mainstream. And we won a sovereign, a g 42, and they became strategic partner and close friends, and they placed a billion dollar order on us. And with that, we were able to sort of transform the company. We're able to change our supply chain. We're able to deploy equipment in big enough clusters that that we could battle test at scale. You know, one of the challenges in hardware is your QA lab can't be as big as some of the customers you wanna deploy to. Mhmm. Right? But you can't put a $100,000,000 in your QA lab worth of your own gear. And they worked with us, and we began training models for them. We began doing inference with for them. They've been an extraordinary partner. This is Peng, who's CEO G42, and his chairman, Shi Tak Nun. We couldn't ask for better partners. And so we we were able to when OpenAI came along, when AWS came along, we we had the capacity. We were ready. Right? We'd battle tested. We'd sort of gotten over the the chasm. We we'd had a bridge, and so we could meet the demand. Yeah.
00:11:09 · Sarah Guo, Andrew Feldman
[00:11:09] S2: I think that kind of path dependence is sometimes undervalued in this field because the ability for you to go from a, you know, like tens, $100,000,000 order to 20,000,000,000 of backlog, Like, there's gotta be there's gotta be something in the middle as somebody work. Yeah. [00:11:24] S0: It's years of work. And and, you know, it's I I think often, and I'm sure many of your listeners are in the software world, and and you guys can scale so fast. Mhmm. Right? But but when you're building things, right, you you have to you wanna double, you gotta call your manufacturing partner, your Centimeters. You gotta they have to find power. They have to win the building. They have to add more lines. They have to make test fixtures. Right? Each step takes real time and effort to to grow. We're gonna try to increase manufacturing 10 x this year. Mhmm. That's about as fast as anybody in the history of hardware. [00:12:01] S2: It's also majority of the software stack for you guys. That's more scale. Right? [00:12:04] S0: You know, when when we started the company, Sarah, our one of my cofounders I do remember. I know. We we presented to you. One of my cofounders said, Andrew, it's gonna take about ten years to build a compiler. I said, no. That's crazy. That's big company talk. We can do it in five. It takes about ten years. Right? It turns out. Takes a long time to build a compiler. It is an extraordinarily difficult piece of software. And now we've got a good software stack.
00:12:04 · Andrew Feldman, Sarah Guo
[00:12:04] S0: You know, when when we started the company, Sarah, our one of my cofounders I do remember. I know. We we presented to you. One of my cofounders said, Andrew, it's gonna take about ten years to build a compiler. I said, no. That's crazy. That's big company talk. We can do it in five. It takes about ten years. Right? It turns out. Takes a long time to build a compiler. It is an extraordinarily difficult piece of software. And now we've got a good software stack. [00:12:34] S2: Can I ask you as an aside, actually, just because you have, for more than a decade, believed that this revolution is going to happen? How much is all of this AI generated coding relevant for Cerberus internally? Hugely.
00:12:34 · Sarah Guo, Andrew Feldman
[00:12:34] S2: Can I ask you as an aside, actually, just because you have, for more than a decade, believed that this revolution is going to happen? How much is all of this AI generated coding relevant for Cerberus internally? Hugely. [00:12:49] S0: I would say that that, you know, eight months ago, we weren't spending a thousand dollars in engineering on tokens, and we're we're probably at 25 or 30,000 right now, and it's ripping. I I think it's not useful for everybody. I I think that's the truth. I I think there are some some people who have sort of the perfect mindset for it. Right? And you you they are running eight or 10 agents, seven by 24. They've moved their coding style to being one in which they govern agents, whether they think about how to QA. So they've got a QA agent running. They think about how to sort of remedy some of the weaknesses in the coding models. Right? They're often verbose. They often cut out comments. They've really thought about, and it's a type of puzzle that the perfect fit for their mind. And they've gone from being sort of 10x guys to being 100x guys. I think the rest of us, myself included, we're sort of limping along. We're trying to figure out how we can make it work for our different jobs, for being the CEO, for being the CFO, for being accountants, for being in marketing. But for a small number, it is such a tool. And then the rest, we try and show them what others are doing, what best practices are. [00:14:02] S2: You're about 800 people now? [00:14:04] S0: 800, eight fifty, yeah. [00:14:05] S2: It's a lot of market cap per person.