Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: A bear case. [00:00:00] S1: Other than just these scaling laws break. In terms of if we assume scaling laws continue, and we now know they're going to continue for pre-training for at least one more generation ... and we're very early in the two new scaling laws for post-training, mid-training, RLVR, whatever people want to call it, and then test-time computed inference ... we're so early in those, and we're getting so much better at helping the models hold more and more context in their minds as they do this test-time compute. That's really powerful, because everybody's like, "Well, how's the model going to know this?" Well, eventually, if you can hold enough context, you can just hold every Slack message and Outlook message and company manual in a company, in your context, and then you can compute the new task and compare it with your knowledge of the world, what you think, what the model thinks, all this context. It may be that just really, really long context windows are the solution to a lot of the current limitations. That's enabled by all these cool tricks like KV cache offload and stuff. I do think, other than scaling laws slowing down, other than there being low economic returns to ASI, edge AI is to me by far the most plausible and scariest bear case.
· Patrick O'Shaughnessy
[00:00:00] S0: I like to visualize different S-curves. You invested through the iPhone, and I love to see the visual of the iPhone models as it went from this clunky, bricky thing up to what we have now, where each one's a little bit ... obviously we've sort of petered out on its form factor. If you picture something similar for the frontier models themselves, does it feel like it's at a certain part of that natural technology paradigm progression?
· Gavin Baker, Patrick O'Shaughnessy
[00:00:00] S1: If you're paying for Gemini Ultra or SuperGrok and you're getting the good AI, it's hard to see differences. I have to go really deep on something like, "Do you think PCI express or ethernet is a better protocol for scale-up networking, and why? Show me the scientific papers." If you shift between models and you ask a question like that where you know it really deeply, then you see differences. I do play fantasy football, winnings are donated to charity, but it is like these new models are quite a bit better at helping who should I play. They think in much more sophisticated ways. If you're a historically good fantasy football player and you're having a bad season, this is why. This is why, because you're not using it. I think we'll see that in more and more domains, but I do think they're already at a level where, unless you are a true expert or just have an intellect that is beyond mine, it's hard to see the progress. That's why I do think we need to shift from getting more intelligent to more useful, unless more intelligence starts leading to these massive scientific breakthroughs and we're curing cancer in '26 and '27. I don't know that we're going to be curing cancer, but I do think from almost an ROI S-curve, we need to hand off from intelligence to usefulness, and then usefulness will then have to hand off to scientific breakthrough just that creates whole new industries. [00:00:00] S0: What are the building blocks of usefulness, in your mind?
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: What are the building blocks of usefulness, in your mind? [00:00:00] S1: Just being able to do things consistently and reliably. A lot of that is keeping all the context. There's a lot of context if someone wants to plan a trip for me. I've acquired these strange preferences. I follow that guy, Andrew Huberman, so I like to have an east-facing balcony so I can get morning sun. The AI has to remember. Here's how I like to fly, here are my preferences for that. Being on a plane with Starlink is important to me. Okay, here are the resorts I've historically liked. Here are the kinds of areas I've liked. Here are the rooms that I would really like at each. That's a lot of context. To keep all of that and weight those, it's a hard problem. I think context windows are a big part of it. There's this METR task evaluation thing. [00:00:00] S0: How long it can work for.
· Patrick O'Shaughnessy, Gavin Baker
[00:00:00] S0: How long it can work for. [00:00:00] S1: How long it can work for. You could think of that as being related in some way to context, not precisely, but that just task link needs to keep expanding, because booking a restaurant is economically useful but it's not that economically useful. Booking me an entire vacation, and knowing the preferences of my parents, my sister, my niece and my nephew, that's a much harder problem. That's something that a human might spend three or four hours on optimizing that. Then if you can do that, that's amazing. Then again, I just think it has to be good at sales and customer support relatively soon. Then after that, it has to be ... and I think it is already here. I do think we're going to see an acceleration in the awesomeness of various products. Engineers are using AI to make products better and faster. [00:00:00] S0: We both invested in Fortell, the hearing aid company ... [00:00:00] S1: A hundred percent, yeah. [00:00:00] S0: ... which is just absolutely remarkable. It's something I never would've thought of.
· Gavin Baker
[00:00:00] S1: We're going to see, I think, something like that in every vertical, and that's AI being used for the most core function of any company, which is designing the product. Then it will be ... there's already lots of examples of AI being used to help manufacture the product and distribute it more efficiently, whether it's optimizing a supply chain, having a vision system watch a production line. A lot of stuff is happening. The other thing I think is really interesting in this whole ROI part is Fortune 500 companies are always the last to adopt a new technology. They're conservative. They have lots of regulations, lots of lawyers. Startups are always the first. Let's think about the cloud, which was the last truly transformative new technology for enterprises. Being able to have all of your compute in the cloud and use SaaS, so it's always upgraded, it's always great, et cetera, et cetera, you can get it on every device. I think that first AWS re:Invent, I think it was in 2013. By 2014, every startup on Planet Earth ran on the cloud. The idea that you would buy your own server and storage box and router was ridiculous. That probably happened even earlier. That had probably already happened before the first re:Invent. The first big Fortune 500 companies started to standardize on it like maybe five years later. You see that with AI. I'm sure you've seen this in your startups. I think one reason VCs are more broadly bullish on AI than public market investors is VCs see very real productivity gains. There's all these charts that for a given level of revenue, a company today has significantly lower employees than a company of two years ago. The reason is AI is doing a lot of the sales, the support, and helping to make the product. I mean, ICONIQ has some charts, a16z ... by the way, David George is a good friend, great guy ... he has this modelbusters thing. There's very clear data that this is happening, so people who have a lens into the world of ventures see this. I do think it was very important in the third quarter. This is the first quarter where we had Fortune 500 companies outside of the tech industry give specific quantitative examples of AI-driven uplift. C.H. Robinson went up something like 20% on earnings. Should I tell people what C.H. Robinson does?