Invest Like the Best with Patrick O'Shaughnessy
01:06:49 · Neil Movva
[01:06:49] S2: In a line, would say the labs pay an immense premium to be three to six months ahead of everything else. I think that's probably still worth it. I think it makes perfect sense for an entropic to do what they do. There's a sensitive topic around distillation, which I think is a very core piece of the relationship between closed and open frontier. I'd like to offer an alternative view on that, which is there is the sense that distillation is theft, that you're taking something from the frontier models when you distill on their outputs. And in fact, even if that's not your intent, even if you don't ever try to scrape data from Anthropic, one thing I'll offer is that an increasingly large percentage of the artifacts we put out on the internet are AI generated. You just look at GitHub alone. What percentage of repos created in the last year do we think were created by Cloud Code? Do we consider that to be distillation? Because it's probably all we need. I would not be surprised if you could train a fable class model only on the outputs of code you consider good on GitHub that's open source. And certainly, we take the position that users own the outputs of their interaction with AI and they choose to put that up on GitHub, which a lot of them do, we're gonna have latent distillation for a long time. It seems fundamentally impossible for me. I don't think it's fundamentally possible to prevent the diffusion of information or model capabilities. It will happen, the question is just how fast.
01:08:01 · Patrick O'Shaughnessy, Neil Movva
[01:08:01] S0: And so then the question becomes, do scaling and improvement laws hold forever or for a really long period of time? And if they do, then there's value to being three to months ahead and that will just last as long as it lasts and they can charge a huge premium for those tokens relative to a very cheap open source token. Is that the right way to think about it? [01:08:18] S2: I think it's possible. I don't know that the premium for being three to six months ahead is gonna last that long. I mean, if you look at like enterprise deployments, they don't move at three to six month speed. A lot of enterprises are probably still on like four six, OPUS four six or OPUS four seven. They don't adopt the bleeding edge rapidly. There's a lot of questions that people have around rolling out any change at all. We're just so early in scratching the surface that I don't think there's any way to call a winner in this race. Certainly, I don't even think this is a race that can be decided ever. It's a continual process. Fundamentally, I don't think open source ever goes away. If there's a vacuum because, you know, one leader steps out, a new leader will step in. There's too much incentive and too much tailwinds too. It gets easier every day to train a frontier class model.
01:08:18 · Neil Movva, Patrick O'Shaughnessy
[01:08:18] S2: I think it's possible. I don't know that the premium for being three to six months ahead is gonna last that long. I mean, if you look at like enterprise deployments, they don't move at three to six month speed. A lot of enterprises are probably still on like four six, OPUS four six or OPUS four seven. They don't adopt the bleeding edge rapidly. There's a lot of questions that people have around rolling out any change at all. We're just so early in scratching the surface that I don't think there's any way to call a winner in this race. Certainly, I don't even think this is a race that can be decided ever. It's a continual process. Fundamentally, I don't think open source ever goes away. If there's a vacuum because, you know, one leader steps out, a new leader will step in. There's too much incentive and too much tailwinds too. It gets easier every day to train a frontier class model. [01:09:00] S0: So your hope of what the future looks like is what balance between closed and open? What balance between model companies doing everything because they have the advantage of owning the stack or whatever? Anthropic can do that. It's like the new Google just do that or something. What do you hope the future looks like?
01:09:00 · Patrick O'Shaughnessy, Neil Movva
[01:09:00] S0: So your hope of what the future looks like is what balance between closed and open? What balance between model companies doing everything because they have the advantage of owning the stack or whatever? Anthropic can do that. It's like the new Google just do that or something. What do you hope the future looks like? [01:09:15] S2: I want abundant tokens and diverse harnesses. I want everyone to build their own harness. Every company, every user even. Make the agent your own. We're not that far away from that level of customization and capability. I want people to own their intelligence and I want that intelligence to be customized, probably not through weight fine tuning but probably through more in context learning. That's a more technical detail but the underlying input to this abundance future is about cheap tokens. My job is to make the tokens as cheap as humanly possible. I will achieve that and I will do it through every layer in the stack available to me. I love the supply side levers. I will use every chip, I'll use every source of power and I will use every piece of land in The United States that's suitable for this. In return, people will have the incentive to explore what it's like to have abundant intelligence. We still treat the agent as a person that is expensive to consult and you should ask them when you have a hard question. That's not the way to think about intelligence. It's incredible that the machine can think, and we should try to get that into as many hands as as many people as possible.
01:09:15 · Neil Movva, Patrick O'Shaughnessy
[01:09:15] S2: I want abundant tokens and diverse harnesses. I want everyone to build their own harness. Every company, every user even. Make the agent your own. We're not that far away from that level of customization and capability. I want people to own their intelligence and I want that intelligence to be customized, probably not through weight fine tuning but probably through more in context learning. That's a more technical detail but the underlying input to this abundance future is about cheap tokens. My job is to make the tokens as cheap as humanly possible. I will achieve that and I will do it through every layer in the stack available to me. I love the supply side levers. I will use every chip, I'll use every source of power and I will use every piece of land in The United States that's suitable for this. In return, people will have the incentive to explore what it's like to have abundant intelligence. We still treat the agent as a person that is expensive to consult and you should ask them when you have a hard question. That's not the way to think about intelligence. It's incredible that the machine can think, and we should try to get that into as many hands as as many people as possible. [01:10:13] S0: You sit in such a unique seat and you have such a unique perspective on, like, what you're trying to do to make this future a reality. What What do you think are your most divergent views of the world versus your friends who are really well informed and interested in this stuff? What ideas of yours make your friends look at you like you have three heads? Most of
01:10:13 · Patrick O'Shaughnessy, Neil Movva
[01:10:13] S0: You sit in such a unique seat and you have such a unique perspective on, like, what you're trying to do to make this future a reality. What What do you think are your most divergent views of the world versus your friends who are really well informed and interested in this stuff? What ideas of yours make your friends look at you like you have three heads? Most of [01:10:29] S2: the ideas on chips, I would say. You know, when I talk about building custom chips and they ask me, oh, so what's different? It's about sidestepping the HBM shortage and focusing on more extreme offload to other forms of memory such as flash. I'm quite passionate about that idea. Everyone on my team knows that I keep banging the drum around like, what do we have to change about the model architecture to make offloading KBCh to flash work at a much greater level? That's in the community of like inference people. We have some divergent views on what you can do if you design a system around serving at one to 10 tokens per second, which is our whole north star. More broadly, I think there is this larger sense around how do people consume a trillion tokens per day? That's the world we want to create, the capability for them to do that. [01:11:12] S0: What's a trillion tokens, like, ground us and how much that is? [01:11:14] S2: A trillion tokens. Well, okay. At OpenAI pricing, that's at least $5,000,000 at the very least for 5.5 or 5.6. Yeah. I think the dollars are probably the most Yeah. [01:11:22] S0: It's a good metric. Yeah. [01:11:23] S2: Yeah. It's millions of dollars.