Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: How do you think about this weird world where you mentioned the talent and the leverage, and they're not writing code themselves, and Claude Code's writing its own code? It seems like the last step of that would be you don't even need the talent to tell the thing what to do. It just figures out what to do on its own, and that's the ultimate thing, that it just runs and is only constrained by compute or something. Am I being too crazy about that, or is that future possible, do you think? [00:00:00] S1: The core of our company is still a research lab. Maybe that's not as well understood, maybe it's getting more understood from the outside. We are doing experiments. We are doing things that push the limits of what our models can do, and that research and that engine is upstream of everything else we've talked about. That is enabled by the models today. It's not entirely done by the models. Over time, we think the models will get better and they'll be more helpful in that process. But having the best talent to set the direction and to help effectively guide that process — not just the priorities, but some of the new areas of discovery — it actually makes that research talent even better. I think of it as accentuating and accelerating the talent that we already have. We talk a lot about how talent density beats talent mass, and that's true here. We want the densest collection of AI research talent and inference engineering talent, and that enabled with the best models is a really winning combination.
· Krishna Rao, Patrick O'Shaughnessy
[00:00:00] S1: The core of our company is still a research lab. Maybe that's not as well understood, maybe it's getting more understood from the outside. We are doing experiments. We are doing things that push the limits of what our models can do, and that research and that engine is upstream of everything else we've talked about. That is enabled by the models today. It's not entirely done by the models. Over time, we think the models will get better and they'll be more helpful in that process. But having the best talent to set the direction and to help effectively guide that process — not just the priorities, but some of the new areas of discovery — it actually makes that research talent even better. I think of it as accentuating and accelerating the talent that we already have. We talk a lot about how talent density beats talent mass, and that's true here. We want the densest collection of AI research talent and inference engineering talent, and that enabled with the best models is a really winning combination. [00:00:00] S0: How are scaling laws talked about internally? The consensus has been you've got different components of them. You've got pre-training, post-training, reasoning, and all of these are moving at different paces, and to hit a true wall, they would all need to fall down. That's how the world is now conceptualizing scaling laws. How are they talked about internally? How do you think about them?
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: How are scaling laws talked about internally? The consensus has been you've got different components of them. You've got pre-training, post-training, reasoning, and all of these are moving at different paces, and to hit a true wall, they would all need to fall down. That's how the world is now conceptualizing scaling laws. How are they talked about internally? How do you think about them? [00:00:00] S1: We look at models at various points in their development, so we can see during a pre-training run, how does this model compare to a prior model that we did on these loss curves. That gives us a sense for model capability. You can do the same thing as you think about RL. Probably as important is when customers get their hands on it, what are they seeing? Where are they identifying pain points? Those pain points become training targets for us. We don't train on customer data on the enterprise side. On the prosumer side, it's only if you opt in. But customers tell us things like, "Hey, I wish the model were better at this," or, "I had this particular place where it got stuck, and I could build this other product, but the capability needs to be further than that." What we usually tell them is, "Build your product for that, because we're going to, on the R&D side, improve that over time." There is this connected loop, but internally we're always looking at different models being trained, different snapshots that we have, and comparing them internally and to a lesser extent externally against our own measures, and then ultimately how our customers view them as well.
· Krishna Rao, Patrick O'Shaughnessy
[00:00:00] S1: We look at models at various points in their development, so we can see during a pre-training run, how does this model compare to a prior model that we did on these loss curves. That gives us a sense for model capability. You can do the same thing as you think about RL. Probably as important is when customers get their hands on it, what are they seeing? Where are they identifying pain points? Those pain points become training targets for us. We don't train on customer data on the enterprise side. On the prosumer side, it's only if you opt in. But customers tell us things like, "Hey, I wish the model were better at this," or, "I had this particular place where it got stuck, and I could build this other product, but the capability needs to be further than that." What we usually tell them is, "Build your product for that, because we're going to, on the R&D side, improve that over time." There is this connected loop, but internally we're always looking at different models being trained, different snapshots that we have, and comparing them internally and to a lesser extent externally against our own measures, and then ultimately how our customers view them as well. [00:00:00] S0: And it feels like there's just no slowdown in the scaling laws themselves. Is that a fair characterization?
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: And it feels like there's just no slowdown in the scaling laws themselves. Is that a fair characterization? [00:00:00] S1: For us, that's a fair characterization. Obviously, a bunch of the authors of the scaling laws papers are amongst our founders. Notwithstanding that, we can be a bit of a skeptical bunch. We hold ourselves to a really high standard. It's very scientific method, and people are constantly challenging previously held assumptions. But from what we see, the scaling laws are not slowing down. [00:00:00] S0: So if that's true — you said before it's hard for humans to be exponential in their thinking and not linear. If that continues to be true for however many more turns of the crank here — how do you do that thing of not thinking linear and thinking exponentially, yourself in your job and for the business? The implications are really hard to reason through. Exponential growth rate is one thing, but exponential growth of capability — I don't even know how to get my head around it. How do you get your head around it?
· Krishna Rao, Patrick O'Shaughnessy
[00:00:00] S1: We think about the world as scenarios. It's very hard to have a point estimate in this business. And then having a very low bar for updating your current priors, or your current perspective. It could be the case that something a month ago was true that's just not true today, and that breaks your model, and you have to go back and update it. This old, "Well, we'll forecast once a quarter and we'll revisit this in three months at the next board meeting" — that doesn't work for our business. It's so dynamic that we have to always think about, "Oh, our models couldn't do this before and they could do this now. What does that mean for the TAM?" We've seen this in coding first. Starting with around Sonnet 3.5, 3.6, we started to see this really remarkable jump in capability, which was then followed by adoption and usage and revenue. It was a little hard to predict that, but now we can use coding as an analog for a lot of what's happening elsewhere in the economy and elsewhere in our business. We look at pattern recognition in our own business to try to predict what's going to happen in the future. [00:00:00] S0: Literally 15 minutes before you got here, the news came out about your partnership with xAI and the Tennessee facility. It makes me curious about how you are canvassing the world. That is an opportunity you decided to do. I'm sure there's a universe of things that you've explored. What is the strategy for trying to get more in creative ways? Bring us a little more into that.