Invest Like the Best with Patrick O'Shaughnessy
· Krishna Rao, Patrick O'Shaughnessy
[00:00:00] S1: I think it is this paradigm of how compute is used. Thinking of it as not just something that is a variable cost over some time period, but really this resource that's so fungibly utilized. We run workloads on one day in the morning on a chip for inference, and in the afternoon or evening, we use it for model development. That paradigm does not exist in a company like a software company or a factory. You can't repurpose — if you have a bunch of people doing R&D and that's your R&D expense, they can't go and become COGS, and vice versa, in most traditional companies. Here, you really have that fungibility . That's possible. That's where the return on compute is so important, and I think people are beginning to understand that. But there's still a tendency towards treating it like, "Oh, I have to separate these two costs," when in actuality they're very self-reinforcing, and that flexibility is what helps to drive revenue short term and long term. [00:00:00] S0: If I was to force you out of your role and into an investor seat at a great big investing firm, and then I said, "Your job is to go grill these companies and invest in the best ones." What questions would you be asking of the labs or companies building models to really get at the heart of the points of uncertainty, of skepticism, of things that might not make these the best businesses of all time? Curious from that angle how you would approach it.
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: If I was to force you out of your role and into an investor seat at a great big investing firm, and then I said, "Your job is to go grill these companies and invest in the best ones." What questions would you be asking of the labs or companies building models to really get at the heart of the points of uncertainty, of skepticism, of things that might not make these the best businesses of all time? Curious from that angle how you would approach it. [00:00:00] S1: First, what is the ROI on compute, all up? How are you utilizing it? What return are you seeing today, and how is that coming over time? That's certainly a question I get asked a lot by investors, but I think it's a really important question. These are the massive unprecedented investments that companies like us are making. What's the return that you're generating on that, and when does it come, and what is the shape of it? So I think that's one. A second one is, how do your customers see ROI? I get asked this question a lot as well. Are people just using this for testing? Are they actually deploying this at meaningful scale? For our business, we're seeing that in spades. Our net dollar retention rate is over 500% on an annualized basis. Nine out of the Fortune 10 — these are real customers making significant buying decisions. [00:00:00] S0: These aren't pilots anymore.
· Patrick O'Shaughnessy, Krishna Rao
[00:00:00] S0: These aren't pilots anymore. [00:00:00] S1: Exactly. On the way here, I was in an Uber, and I signed two double-digit million-dollar commits in the car ride, which was 20 minutes. From that perspective, we're seeing it, and we're now being judged by some of the biggest companies in the world, the most sophisticated buyers. And startups — they have choice in the market, and they're choosing us. One question I get a lot — or I would ask from the skeptical investor seat — is, "How are your customers getting return from this?" Maybe a third one is, how do you think about compute in the future, and where does it come from? Some of the places that we buy compute from, they sell the compute to others. They may use the compute internally. What is the balance of that over time? [00:00:00] S0: So your philosophy there is be involved with great players and have flexibility. [00:00:00] S1: That's right. [00:00:00] S0: There's this crazy stat about AI, just the generic concept being less popular than Congress amongst the general populace. It's kind of funny when you first hear it, but when you really think about it, you're like, this is kind of fucked. We need to solve this problem. It doesn't seem like the general world that isn't in technology, doesn't live in the Bay Area or New York — does not yet feel or understand why this is good for them, just as measured by their opinion of it. What do you think we need to do as an industry about that problem?
· Krishna Rao
[00:00:00] S1: If we think about the transformation that's happening — there's been other transformative waves before, all the way back to the Industrial Revolution, the internet, cloud, et cetera. One of the things that's different about AI is it's all happening so quickly. You can have years or decades of progress that are being compressed into months. Going back to humans thinking in terms of exponentials versus linear, that can be jarring. We are very optimistic generally about the potential for this technology. We as an industry can continue to do a better job of articulating it. Dario wrote this essay, "Machines of Loving Grace." It's all about the potential for this technology to transform the way that we live, whether that be in drug development and curing diseases that are more mainstream but also rarer diseases, the ability to accelerate that biological progress. Number two, in healthcare and how healthcare is delivered. To raise our standard of living in the developing world and in places where resources are not as plentiful. How do we actually make sure that the economic gains are accruing to not just a small number of people in the world, but across the world? All of those things are part of the promise and the potential of AI. We could probably do a better job of painting that picture, and we want to show more tangible results for that over time. I think that is coming, and that's one of the things I'm most optimistic about. On the other side, though, we do want to articulate the risks. I don't think we should just tell everyone that everything's going to be great. There are likely to be bumps in the road. People generally gravitate towards more honest and balanced assessments. If I feel like somebody's just telling me all the good news and none of the bad news, then I'm like, "Okay, do I really trust this perspective?" That's where there's a need for balance. Look, these are some of the things that happen when change is compressed over a short amount of time. How do we work across commercial and government to actually come up with some of the solution sets? So I think it's about a clear articulation of the opportunities. It's about thinking about what those solutions may be, and that it's not any one company that can come up with it.
· Krishna Rao, Patrick O'Shaughnessy
We don't have this blueprint that's going to solve everything, but to at least have that dialogue about some of the risks and downsides and what we can do to address it. And then I think it's being transparent about that, about both of those things when we talk about it. I do think that over the long term, the opportunity is going to be significantly higher and greater than some of the risks and the downsides that will happen, but that doesn't mean it's going to be perfectly smooth on the curve. [00:00:00] S0: The release of Mythos was such an interesting moment. It was the first time many people — friends of mine that are careful watchers of this stuff — said something like, "This one kind of makes me scared." So it relates back to the safety question. It's also the first example of you coming out and saying, "We want to make sure this isn't used for bad, and it's maybe the first one that we are worried could be used for bad." I'm curious what that discussion was like internally before the world heard about it — the decision-making process around it. And just using that as an example, talk about the things that do scare you as we continue to advance and the scaling laws continue to hold.
· Krishna Rao
[00:00:00] S1: One of the things about Mythos is that people maybe misconstrue it as just a cyber model. It is an incredibly capable model across many, many different dimensions. What we found was that cyber in particular was a place where it spiked. This was the first model that we decided to release in a different way. The way in which we did that is consistent with our mission, our principles. We wanted to do it in that way. We have this phased approach to it because we think that when a model is this capable — and again, cyber is the thing that people focused on, but there are other things as well — we think it can be used in a positive way to patch code bases. You've seen these examples where we had an open-source code base that a prior model found 22 security vulnerabilities in, and Mythos then found 250. That is kind of scary, but that informed the way in which we released it. We didn't say we're never going to release it. We said, "Let's do it in a phased way. Let's do it to a group that will expand over time, where we can focus on this one cyber capability and how it can actually be used positively in a defensive way, as opposed to in an offensive way." We think that's a template that could be used for the future, but because of this one particular area, we wanted to be cognizant of that in how we released it.