Invest Like the Best with Patrick O'Shaughnessy
00:32:18 · Eric Vishria
And I think the first chip was seven nanometer or something, you That in software, if you have that logical block diagram of like why it works and everything else, you're kind of eighty percent of the way there, and it's a matter of go-to-market execution. In hardware, you're like two percent of the way there. There's things like physics, entire supply chain of vendors. There's of course TSMC, which everyone knows, but it's not just TSMC, there's thirty other vendors that matter in putting all this stuff together and everything else. I didn't know any of that. Fast forward from twenty sixteen to like twenty nineteen, I think Got their first parts back. You go through this like bring up, and then it's like, "Bring up, oh yes, we got a part back. I mean, it's like, "Bring up, yeah, then you gotta go through like, "Wait, bring up, " and there's like fourteen steps of bring up and everything else. And then by like twenty twenty, we had our like first thing that works. Then it's just this march of actually getting it to work. One of the lessons that I've learned on this stuff is basically, you go through all these sims and everything else in hardware and semis in particular, that basically is your roofline. The best it's ever gonna be is what that is. And then every bit of software and reality and compilers and kernels takes away from that roof line. You might start at ten percent of the roof line, once you bring it up, these guys are grinding for months and years to like get closer and closer and closer to the roof line. It's really different, it's really hard. I'm astonishingly bullish if I kind of rewind, part of the reason we made the investment was if you looked at the four prior generations of compute in my lifetime, so you had CPUs You had graphics, then networking, then mobile. There was a new workload each time, so you had multipurpose compute and it led to the CPU. You had massive parallelism, led to the graphics processor, graphics processor offered massive parallelism, which led to graphics. Then with networking, you needed really low latency chips, and so they had low latency chips. And then with mobile, you needed really power efficient chips, and in each case, we ended up with a new hundred billion dollar company.
00:34:23 · Eric Vishria
The first question, going back to twenty sixteen, was, "Is AI that big a new?" Workload, because there have been many, many other attempts for specialized chips for other things that really just didn't end up mattering. There's some fine outcomes, but they just didn't really end up mattering, et cetera. Right, exactly. And so it's just like, well, okay, well, you need something that's a really, really big workload. Okay, so that's one. We had a lot of conviction on that. And then two, was the nature of the workload, did it introduce a new constraint or problem in it? What I learned was basically the AI workload But GPUs didn't solve the core-to-core communication, basically the layers of the network problem. You're like, okay, wait a minute, there is a new constraint, which is communications, communication bound problem. Then you're like, okay, is AI a new giant workload that is going to have specialized chips? Everything that I just said was the entire, everything I knew at that time. That's obviously played out, and in each prior generation, we got Intel, we got Nvidia, we got Broadcom, Avago, we got Qualcomm and- Arm in each of these generations, and there will be these giant winner, standalone winners. Obviously, the TBU itself is a winner, Trainium's a winner. We've had Grok and Cerebrus, etch. It'll keep getting fought out, but I think that will end up being big. And actually, I think there's a new sixth one that's coming generation, which is, and I'm really excited, we've made an investment that's unannounced in this, but I think that for the first time. In a long time, there's actually room for a new CPU approach. The thing that's happening right now, and you see this reflected in all the semi stocks and everything else, is the LLMs which are running on accelerators and GPUs generate code. The code runs on CPUs. And right now it's running on classic CPUs we've had around forever, but there's a whole bunch of constraints on CPUs that have existed, and CPUs have dragged all this baggage forward that you might not need to anymore. And so, I'm actually really excited about that possibility. The next category,
00:36:23 · Patrick O'Shaughnessy
[00:36:23] S0: the next category. Does the experience with Cerebras make you wanna do a lot more investing in companies? No, but why not? In twenty nineteen, we're
00:36:33 · Eric Vishria
[00:36:33] S2: sitting in a board meeting and this thing- The thing is, it's melting. It's like, "Fucking melting, okay? " And we've raised five hundred million dollars or something, and it's like, "Wait, what? It's melting or it's burning or something? " And like we're looking at it, and I'm like, "Holy shit! " I realize like five hundred million dollars isn't that much in today's era, but I was just like, "We're gonna lose all this money. This isn't gonna work. " And what that team did Insane. They're built differently, and I have so much respect and thanks to them for what they've done. There are efforts that make you really proud to be a venture capitalist, because you're funding something that makes a difference and matters. I'm an investor because that's a means to work with companies, not because I fundamentally love investing or something like that. I like working with companies, that's my favorite part of it. Working with teams like that and companies like that is so special on these giant ambitious efforts. And I said this well before twenty eighteen or whatever, whether ServiceWorker didn't, I think it was an effort that was worth venture capital. That was the kind of thing you should do. You should try to build something that people have tried for fifty years and haven't been unable to, but now we think we could do, and there's a reason and application for it and everything else. I love that. I do like those kinds of things, joking aside, and we do have a robotics company and then this CPU. Project that we're talking about. I think these kinds of things are actually really fun and interesting and good use of venture capital, but they definitely aren't
00:37:57 · Patrick O'Shaughnessy, Eric Vishria
[00:37:57] S0: easy. The productive naivete that you described there. It's probably a virtue that you didn't know more than you knew, otherwise you wouldn't have done it. This is certainly my experience with, like, in any of these fields, you ask experts, they're gonna tell you don't do it. It sucks, it's too hard, the base rate's too low, young people can't do it, there's forty-seven reasons. All of those things, yeah. Is there anywhere where that is just a bridge too far that could be like bio or something like this where you just aren't willing to invest if you're naive? You know [00:38:24] S2: what's so They're not correct three times out of four, they're correct nineteen times out of twenty, maybe ninety-nine times out of a hundred, like they are correct. The thing that Bruce, one of our founders, always says, "What could go right? We have to ask ourselves, what could go right? And do we see that path? Yeah, young people can't build chips or you shouldn't do a no-semi company or you shouldn't do this or like whatever. All of that stuff is actually totally right, except when it isn't. Apparently there's a saying that someone said to one of my partners, which was, "If it doesn't work, it'll be for all of the reasons that your partner said. If it does work, it will be because those reasons didn't matter. " It's just such a good example of this whole thing, which is, yeah, most of the time when we lay out the reasons that a It's right, but then sometimes they just don't matter.
00:38:24 · Eric Vishria, Patrick O'Shaughnessy
[00:38:24] S2: what's so They're not correct three times out of four, they're correct nineteen times out of twenty, maybe ninety-nine times out of a hundred, like they are correct. The thing that Bruce, one of our founders, always says, "What could go right? We have to ask ourselves, what could go right? And do we see that path? Yeah, young people can't build chips or you shouldn't do a no-semi company or you shouldn't do this or like whatever. All of that stuff is actually totally right, except when it isn't. Apparently there's a saying that someone said to one of my partners, which was, "If it doesn't work, it'll be for all of the reasons that your partner said. If it does work, it will be because those reasons didn't matter. " It's just such a good example of this whole thing, which is, yeah, most of the time when we lay out the reasons that a It's right, but then sometimes they just don't matter. [00:39:18] S0: You and I are both interested in robotics. It's not controversial that if robotics works, it might dwarf what we're currently living through. What do you think has to be true for it to work? Obviously it's exciting. I want a robot in my house folding my laundry. It sounds great. It's kind of one of these classics, it's always ten years away, and it's been that way for a long time. What do you see happening? What has to happen for this to actually be a thing in the near to medium term?