Invest Like the Best with Patrick O'Shaughnessy
01:20:18 · Gavin Uberti
[01:20:18] S1: Yeah, there was a viral tweet by Noah Brown, which said that as these models are having longer and longer time horizons, they can do tasks that take, say, six months, and there's often not enough time to go and evaluate them for such a long period of time because by that point, you'll have a new model, you'll want to go evaluate instead. And with tech like we built our cluster scale memory, you can go ahead and run that six month job much faster. But there's a second piece of this too, talking to him about it, he's now an angel as well, where it's not just the time, it's also the number of people or agents who are working on this. If you're trying to go and evaluate, can a human build a rocket? You will find that the answer is no. No one person can build a rocket, instead you have to go put a team together. And I believe the same thing will be true of agents too. If you want to go ask, can an agent go out and build some crazy futuristic piece of software, you'll probably need a very large team. Maybe that's 10, maybe that's a million. You have to go and have this enormous amount of both cluster scale memory to go ahead and have that very short time per token and a huge amount of flops to be able to go run that whole fleet.
01:21:27 · Rob Wachen
[01:21:27] S3: I'm going be a little futuristic. I firmly believe we are on a global march of inference becoming majority of global GDP. It may take more than ten years, but it's gonna happen. And right now, we measure productivity as a society as GDP per capita. But really, it's going to look much more like agents per megawatt or it may be agents per gigawatt by then. And while we're being futuristic, I think this is the second to last year where a majority of the workforce is going to be human. I think in 2027, you're going to see there's going to be more agents doing knowledge work than humans, And it's going to be extremely interesting to see what happens. You could imagine a world where, for countries, a majority of their energy ends up going into data centers doing inference. And the energy efficiency of those data centers basically governs how many agents and therefore, you know, how big their workforce is. So you're going to see, like as Gavin is saying, one agent or a team of five to 10 agents working on group projects for a couple days. So you can do pretty cool stuff because they're smart, but it's not going to be civilization scale. What happens when you have countries that can have literally a billion concurrent agents, like a billion people in the workforce working twenty four seven concurrently on the same stuff? It's just kind of unfathomable what's going to happen. And it's going to be the biggest proliferation of technology humanity's ever seen.
01:22:45 · Gavin Uberti, Patrick O'Shaughnessy
[01:22:45] S1: I think as well, like, when you have these huge huge amounts of demand, you get this idea of economies of scale again. Yep. Or think about people. I have a brain. I'm not using the whole thing all at the same time. That's only part of us gonna be active and this is the way healthy brains work. MOE models have worked much the same way. On MOE model, only a small fraction of parameters is being used for any given token at any given moment. But if you have a large number of users on a piece of hardware, you can go kind of take that brain, cut it up into many different experts on many different servers, and run a huge amount of volume through it. So you'll have a bunch of different pieces of traffic, you'll have many of them using each part of the brain at any given point in time, and you'll also make the cost per thought, cost per token way, way lower. So I think you're going to end up with these giant scale distributed brains. The form factor of this is a big data center with a bunch of chips, a huge amount of flops, a huge amount of scale up interconnect. [01:23:38] S0: You think we'll see a trillion dollar individual data center? [01:23:41] S1: Absolutely. It is a matter of time. It's like asking, will you see a billion dollar fab or a $10,000,000,000 fab or a $100,000,000,000 fab? It is inevitable that the economies of scale don't stop at, oh, 40,000,000,000 is the magic number for fabs. No, the cost wafer keeps going down as you keep spending more money. And the same thing will be true of plants that go out and make steel or plants that
01:23:41 · Gavin Uberti, Patrick O'Shaughnessy
[01:23:41] S1: Absolutely. It is a matter of time. It's like asking, will you see a billion dollar fab or a $10,000,000,000 fab or a $100,000,000,000 fab? It is inevitable that the economies of scale don't stop at, oh, 40,000,000,000 is the magic number for fabs. No, the cost wafer keeps going down as you keep spending more money. And the same thing will be true of plants that go out and make steel or plants that [01:24:04] S0: go out and make tokens. A very smart alien lands on Earth and wants to know from each of you how you would frame up this opportunity that you guys are tackling. What do you say to them? [01:24:15] S1: Truman is thinking is really valuable, that every company in the world runs on thinking. And we are entering this really unique moment in time where you have machines that can go think almost as good and as soon as good and as soon better than the best humans can. Building these machines is going be a huge opportunity, but more important than that, the way in which you go ahead and run this kind of thinking is going to be very, very different as demand goes higher and higher and higher and higher. There's a unique moment right now to go build a new set of solutions, a new roadmap for how do you run the future quadrillion parameter models for a billion people all at the same time on a gigantic scale up cluster.
01:24:15 · Gavin Uberti, Rob Wachen
[01:24:15] S1: Truman is thinking is really valuable, that every company in the world runs on thinking. And we are entering this really unique moment in time where you have machines that can go think almost as good and as soon as good and as soon better than the best humans can. Building these machines is going be a huge opportunity, but more important than that, the way in which you go ahead and run this kind of thinking is going to be very, very different as demand goes higher and higher and higher and higher. There's a unique moment right now to go build a new set of solutions, a new roadmap for how do you run the future quadrillion parameter models for a billion people all at the same time on a gigantic scale up cluster. [01:24:56] S3: We are in a new era of intelligence where the cost of producing intelligence is dramatically so much cheaper than the value of the intelligence that we are in a many year, probably many decade supply shortage of these tokens. And, basically, any chip or any system that can produce tokens is likely to be extremely valuable, and you should find some part of the supply chain of the token. That can be everything from model training down to what we're doing in silicon and otherwise to spend time on and push the frontier, and that the companies that are the largest are frankly going to be the companies that produce most of the global supply of tokens and own a majority of the supply chain of that token. And importantly, it's people who build systems that as they get more and more chips put together, get cheaper.
01:24:56 · Rob Wachen, Gavin Uberti, Patrick O'Shaughnessy
[01:24:56] S3: We are in a new era of intelligence where the cost of producing intelligence is dramatically so much cheaper than the value of the intelligence that we are in a many year, probably many decade supply shortage of these tokens. And, basically, any chip or any system that can produce tokens is likely to be extremely valuable, and you should find some part of the supply chain of the token. That can be everything from model training down to what we're doing in silicon and otherwise to spend time on and push the frontier, and that the companies that are the largest are frankly going to be the companies that produce most of the global supply of tokens and own a majority of the supply chain of that token. And importantly, it's people who build systems that as they get more and more chips put together, get cheaper. [01:25:46] S1: The way you want this to scale is not that, oh, if I want go serve 10 times more tokens, I buy 10 times more servers. It must be some solution where if I want to go serve 10 times more tokens, then I get some economies of scale benefit with my, say, cluster scale memory tech that allows me to then not charge as much as 10 times more for those fact tokens. [01:26:05] S0: What a ridiculously exciting future that you guys are building to enable. When I did this with Gavin last time, asked him my traditional closing question, so this time I'll ask you. What is the kindest thing that anyone's ever done for you?