Invest Like the Best with Patrick O'Shaughnessy
· Dylan Patel
[00:00:00] S1: Anthropic's whole goal in 2025 and even a lot of 2024, they're like, hey, by the end of 2025, we need an L4 software engineer in our model. And they by and large achieved that with 4.6 Opus. What they didn't say is that, and if you look at Mythos, if you compare benchmarks, it's like an L6 engineer. So L4 is like pretty new. L6 is like, quite well experienced. I think Anthropic said that the model internally was available in February. So in two months, they've gone from L4 engineer to L6 engineer. What's next? When you think about the model progress, it's only accelerated. Anthropic's release cadence has compressed. OpenAI's release cadence has compressed. Why? Because generally to make a better model, you need a few things, right? You need amazing compute. Compute is very expensive and it has a time scale that we track and it's like, it's growing, but it's set in stone for the next short term. It's like set in stone what you've already signed and there'll be delays and shifts and somehow you can find a little more, but it's generally pretty set in stone. There's amazing researchers that people are paying tens of millions of dollars for. And then lastly, there's implementation. And implementation historically has been very difficult. If I have an idea, now I have to implement it. Implementing is hard. Now ideas are there. Implementation is very easy. It's expensive, but it's very easy. So how does one decide what ideas to implement? And it turns out if your implementation is just so much easier, now you can just implement more ideas and move on the treadmill faster and faster and faster, whether that is AI model research. And so now your model release cadence is shrunk down to two months from what it was six months before. Or I want to take every power plant in the US and every transmission line and model it and run regressions and see the micro supply and demand. I can also do that. The idea is cheap. Which idea makes sense? Which idea is worth the capital that you have to spend on the tokens? Because the implementation is there. That's the key learning. And if implementation costs continue to tank – which they are. We don't even have those yet. It's only been a handful of hours since Opus 4.7 launched, but my team is pretty excited about it internally. What now comes to the world? It's a complete reordering of how economies work.
· Dylan Patel, Patrick O'Shaughnessy
What used to matter a lot was execution was very, very fucking difficult and ideas were cheap. Now ideas are cheap and plentiful, but execution is very easy. So really only the good ideas are the ones that can justify the spend on super cheap implementation. [00:00:00] S0: So are you actually scared or does it just introduce some uncertainty that's hard to grapple with?
· Dylan Patel
[00:00:00] S1: Uncertainty is there, but I do think that causes some fear in terms of how does society reform itself? How does one exist in a world where actually your ability to implement something is not actually that important? Your ability to choose the correct idea for AI to implement and then your ability to sell that idea or sell what the AI has implemented is what matters. Your ability to garner capital towards that is what matters. And going back to the point of it's very important to have the newest model always, who's going to have access to the newest model? Anthropic's project, I know it's not called Earwig, but I troll Anthropic people by calling it Earwig. Glasswing. Anthropic Earwig – where they only released Mythos to certain companies for cyber, that's just going to be something that continues. Models will have less broad and less broad deployment. I know OpenAI and Anthropic and all these people are like, we want to have great AI for everyone. AI is very fucking expensive. Who's going to pay for the trillion dollars of infrastructure? People who have money and can build useful things with AI. And then you don't want people to distill your model so you don't release them broadly. You release them to fewer and fewer set of customers. Those customers are also now wrestling over the tokens unless Anthropic jacks them. They could double their pricing on Opus and I would continue to pay and most users would continue to pay. I bet that wouldn't solve their humongous capacity problem that they have. So then the question becomes, where does this cycle end where token usage and therefore the benefits of those tokens, the additional value generated on top of those tokens, aggregates among fewer and fewer and fewer companies. I don't have Mythos. You know who has Mythos? Top freaking banks. Now they're only using it for cybersecurity, but at some point I can envision a world where – hey, maybe I, because I have an enterprise Anthropic contract and Anthropic people kind of like me, they're willing to give us slightly earlier access or slightly higher rate limits or something for a model. I hope that's what happens. And then my competitor, whoever that is, doesn't have that and I'm able to fucking crush them. There are people like Ken Griffin of Citadel, who's super well connected and super rich.
· Dylan Patel, Patrick O'Shaughnessy
He goes and signs a deal with OpenAI Anthropic that's like, yeah, I'm going to get access to your models and I'll buy the first 10 billion dollars worth of tokens each year. So whenever you release the model, I'll spend the 10 billion tokens and then everyone else can get the model after that. And it's like, okay, well now what does that do? Now he's going to crush everyone in the markets. That's just an example. It could be any number of things. It could be cyber like Anthropic is worried about. Oh, now I can hack people. It can be information services business like myself where I crush someone else. I think it's such a broad base. We don't know what these models can do. Anthropic doesn't know what these models can do. No one knows what these models can do. It's up to the end-user to figure out where they can leverage the tokens to see what they can build and imagine, which is tremendously productive and uplifting for humanity. But then what happens to the concentration of resources and usage of it? [00:00:00] S0: Presumably right now, robotics or robots consume relatively zero tokens versus everything else. What's your view of that? If that's like a second demand curve that could start to ratchet. There's a new startup every single day within a mile of here trying to build something interesting in robotics.
· Dylan Patel
[00:00:00] S1: There's this concept of software-only singularity, which is that the world has AI singularity, but only in software. And now what about the rest of the world? The vast majority of the world is physical and you can see the world orient around hardware, not software. That's actually why I think software-only singularity is just a blip and not like a, we do get everything else. Because software is super easy, what makes robots really hard? It's programming microcontrollers and actuators and controlling all this stuff is very difficult. And right now, the interesting thing about models, AI models, is they're actually really inefficient in learning. It's just, we're able to give them so much data that they're able to learn and pass us in certain ways. Currently, the robot models – VLAs, Vision Language Action Models, which is very popular right now, is probably not going to be the thing that ultimately scales beyond. They are inefficient in data and we can't scale the data for them fast enough. There is going to be some way to large-scale pre-trained robot models where, just like humans, see all this data throughout their lives. And what's interesting is humans, the reason why we're so good is we're sample efficient. One example, two example, we're good. So applying that to robotics, so once you have this software-only singularity implementation is super cheap, anyone can start to build these models that now robots are actually useful. And so I think in the next six to 18 months, we'll start seeing real breakthroughs in robotics that enable few-shot learning, i.e. there's a pre-trained robot model and now there's a robot that you have hired or bought or whatever. You show it a few examples and it's able to do it. Right now, there's a lot of companies doing robots for advertisement or robots for simple stuff like that, but it'll be like, oh, folding clothes, sure, sure, sure. No, but it's going to get really niche. Robots just for cleaning chalkboards, and it's a rental service. Or it'll be a model package that you download onto your standard robot that then does that. In any ways, there'll be a huge explosion in physical good acceleration and deflationary effects there, but that's ultimately going to keep token demand going crazy. I don't think token demand slows down personally.
· Patrick O'Shaughnessy
[00:00:00] S0: Did you learn anything else about the world based on Mythos' results and how it was built? It's my way of asking if you break down the components of the scaling laws.