Invest Like the Best with Patrick O'Shaughnessy
· Krishna Rao
[00:00:00] S1: We do see progress accelerating. I can't speak for other companies, but for us, the scaling laws are alive and well, and we're seeing that with releases more recently like Mythos. But right now, within the company, 90-plus percent of our code is actually written by Claude Code. A lot of Claude Code's code is written by Claude Code. Why do we allocate compute internally? Why would we forgo revenue for it? It's because the models themselves are helping us build that next generation of models. In addition to this capability leap that you would have from the scaling laws, talent is really important. That talent, with the best models, can really accelerate the development of the capabilities, and we're really seeing that. We don't really think about models as closed or open. We think of them as frontier or not. The ones at the frontier clearly are capturing this economic value, driving meaningful ROI for customers. We are investing behind that thesis, and that means both compute, but it also means talent to use that compute and use our own models to really accelerate the development. That's something we've been doing for a long time. That's not a new phenomenon, but we're seeing the fruits of that in everything we've done. The other piece of it is — it's not just the models; it's the products that get built on top of them. We had 30 different product and feature releases in January. The pace of that has accelerated as well, and that's enabled in part by utilizing the models with the talent that we have, to accelerate ways to access this underlying intelligence. That's our theory of the case on the product side.
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: How do you think about this weird world where you mentioned the talent and the leverage, and they're not writing code themselves, and Claude Code's writing its own code? It seems like the last step of that would be you don't even need the talent to tell the thing what to do. It just figures out what to do on its own, and that's the ultimate thing, that it just runs and is only constrained by compute or something. Am I being too crazy about that, or is that future possible, do you think? [00:00:00] S1: The core of our company is still a research lab. Maybe that's not as well understood, maybe it's getting more understood from the outside. We are doing experiments. We are doing things that push the limits of what our models can do, and that research and that engine is upstream of everything else we've talked about. That is enabled by the models today. It's not entirely done by the models. Over time, we think the models will get better and they'll be more helpful in that process. But having the best talent to set the direction and to help effectively guide that process — not just the priorities, but some of the new areas of discovery — it actually makes that research talent even better. I think of it as accentuating and accelerating the talent that we already have. We talk a lot about how talent density beats talent mass, and that's true here. We want the densest collection of AI research talent and inference engineering talent, and that enabled with the best models is a really winning combination.
· Krishna Rao, Patrick O'Shaughnessy
[00:00:00] S1: The core of our company is still a research lab. Maybe that's not as well understood, maybe it's getting more understood from the outside. We are doing experiments. We are doing things that push the limits of what our models can do, and that research and that engine is upstream of everything else we've talked about. That is enabled by the models today. It's not entirely done by the models. Over time, we think the models will get better and they'll be more helpful in that process. But having the best talent to set the direction and to help effectively guide that process — not just the priorities, but some of the new areas of discovery — it actually makes that research talent even better. I think of it as accentuating and accelerating the talent that we already have. We talk a lot about how talent density beats talent mass, and that's true here. We want the densest collection of AI research talent and inference engineering talent, and that enabled with the best models is a really winning combination. [00:00:00] S0: How are scaling laws talked about internally? The consensus has been you've got different components of them. You've got pre-training, post-training, reasoning, and all of these are moving at different paces, and to hit a true wall, they would all need to fall down. That's how the world is now conceptualizing scaling laws. How are they talked about internally? How do you think about them?
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: How are scaling laws talked about internally? The consensus has been you've got different components of them. You've got pre-training, post-training, reasoning, and all of these are moving at different paces, and to hit a true wall, they would all need to fall down. That's how the world is now conceptualizing scaling laws. How are they talked about internally? How do you think about them? [00:00:00] S1: We look at models at various points in their development, so we can see during a pre-training run, how does this model compare to a prior model that we did on these loss curves. That gives us a sense for model capability. You can do the same thing as you think about RL. Probably as important is when customers get their hands on it, what are they seeing? Where are they identifying pain points? Those pain points become training targets for us. We don't train on customer data on the enterprise side. On the prosumer side, it's only if you opt in. But customers tell us things like, "Hey, I wish the model were better at this," or, "I had this particular place where it got stuck, and I could build this other product, but the capability needs to be further than that." What we usually tell them is, "Build your product for that, because we're going to, on the R&D side, improve that over time." There is this connected loop, but internally we're always looking at different models being trained, different snapshots that we have, and comparing them internally and to a lesser extent externally against our own measures, and then ultimately how our customers view them as well.
· Krishna Rao, Patrick O'Shaughnessy
[00:00:00] S1: We look at models at various points in their development, so we can see during a pre-training run, how does this model compare to a prior model that we did on these loss curves. That gives us a sense for model capability. You can do the same thing as you think about RL. Probably as important is when customers get their hands on it, what are they seeing? Where are they identifying pain points? Those pain points become training targets for us. We don't train on customer data on the enterprise side. On the prosumer side, it's only if you opt in. But customers tell us things like, "Hey, I wish the model were better at this," or, "I had this particular place where it got stuck, and I could build this other product, but the capability needs to be further than that." What we usually tell them is, "Build your product for that, because we're going to, on the R&D side, improve that over time." There is this connected loop, but internally we're always looking at different models being trained, different snapshots that we have, and comparing them internally and to a lesser extent externally against our own measures, and then ultimately how our customers view them as well. [00:00:00] S0: And it feels like there's just no slowdown in the scaling laws themselves. Is that a fair characterization?
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: And it feels like there's just no slowdown in the scaling laws themselves. Is that a fair characterization? [00:00:00] S1: For us, that's a fair characterization. Obviously, a bunch of the authors of the scaling laws papers are amongst our founders. Notwithstanding that, we can be a bit of a skeptical bunch. We hold ourselves to a really high standard. It's very scientific method, and people are constantly challenging previously held assumptions. But from what we see, the scaling laws are not slowing down. [00:00:00] S0: So if that's true — you said before it's hard for humans to be exponential in their thinking and not linear. If that continues to be true for however many more turns of the crank here — how do you do that thing of not thinking linear and thinking exponentially, yourself in your job and for the business? The implications are really hard to reason through. Exponential growth rate is one thing, but exponential growth of capability — I don't even know how to get my head around it. How do you get your head around it?