Invest Like the Best with Patrick O'Shaughnessy
00:08:56 · Neil Movva
[00:08:56] S2: Most questions where you want to have a definitive answer over not 100 sources, not a thousand sources, but 10,000 sources or more. If you want to build an authoritative index of information, like for example, one of our customers parallel web systems seeks to do. They want to build an index over the whole internet. And they want to monitor the Internet in real time for changes. That is the kind of crazy exabyte scale task that you need a very different kind of intelligence or scale of intelligence to achieve. Deep research is a top category for us. And then increasingly, we see cyber security following this direction. If you think about, yes, there's so much code you can generate, but there's exponentially more ways to break that same code than is to generate that code. There are some great customers out there who are working very hard to find agents that can break any piece of software and proactively patch them. When Fable first came out, for example, or Mythos first came out, basically, there was this push in the cybersecurity community to run Fable against every line of code we've ever written and look for bugs in 20 different ways. Meaning, you're looking for both memory errors, you're looking for business logic errors, and looking for like network vulnerabilities. All these things. And these are all actually things that you would write specialized agents for. You wouldn't just have Fable look at the source code once, you'd have it actually set up environments where you can pen test these applications. At some point, people started to make this joke that security has become proof of work. When you want secure software, it's really a question of how many dollars did you spend on Anthropix APIs trying to break into your software. That is the best indication for how secure it is because that's the best tool in the world. And increasingly, we found that open source models, well, the frontier of intelligence here is quite jagged. It's not the case that Fable finds a superset of all bugs in software. You would find some bugs with a very small model that you don't find with the large model. You'd find some bugs with Haiku that you would find with Fable and vice versa. So it encouraged this very diverse approach to sampling and and trying to build cybersecurity agents that break software autonomously such that you can patch them.
00:10:47 · Patrick O'Shaughnessy, Neil Movva
[00:10:47] S0: If you were to get speculative and imaginative about the sorts of things that very cheap, very long running agents can enable, We talked about some very practical examples, deep research, cybersecurity, etcetera. If you get a little bit dreamier about the use cases, new product category that this sort of inference will unlock. I guess the question is just like, so what? If you're maximally successful, dream a little bit about what that might enable. [00:11:13] S2: I think for individual users, what I'm excited about most is this idea of proactive intelligent agents. You can imagine Siri that is running in the background all the time to understand all the emails you received in a day, all the text messages you receive in a day and has a much more encyclopedic view of your life and how to be helpful in that life. Right now, there's still point solutions, you end up doing a lot of prompting. Siri is not very proactive. Something we can fix with abundant inference. If you trust the machine enough that it's reliable and also trustworthy as in private, you might even imagine the machine can understand how you interact with it and proactively surface your next action. Whenever you open your phone, can we build a good model of what you're gonna do next? My estimation is yes, we totally can. [00:11:52] S0: And the key to that is incredibly cheap intelligence.
00:11:52 · Patrick O'Shaughnessy, Neil Movva
[00:11:52] S0: And the key to that is incredibly cheap intelligence. [00:11:54] S2: You have to be willing to spend tokens without any promise of return. That is the unlock. The long lens view to take on this is that we have a form of intelligence that can tackle any verifiable problem. Any verifiable problem means most software. It means a lot of formal math proofs and similar and it could also mean scientific discovery. These are all relatively verifiable problems. And all those things currently have a dollar cost attached to them, essentially, that's a hidden one. It's like how many tokens could you possibly harness to make this work. We have started to bring you within view a dollar cost for these long horizon tasks that is reasonable. It's not millions, it's thousands, and maybe it could be hundreds or even tens of dollars in the near future to have a definitive answer to any scientific question, to any research problem. [00:12:42] S0: So if we dream about that future, we then become limited just by the questions that people can ask basically?
00:12:42 · Patrick O'Shaughnessy, Neil Movva
[00:12:42] S0: So if we dream about that future, we then become limited just by the questions that people can ask basically? [00:12:49] S2: Pretty much. The questions we can ask, the models are on the cusp of basically taking even a high level question and chasing it down. Every possible follow-up, you can have the model essentially take that on its own. And the question is, what is your token budget? And we will solve the token budget problem. What about non verifiable tasks? Those are basically the entire category of human taste into that category. We have not solved human taste yet and I don't know that it fundamentally can be. I'm excited to be surprised here but we are focused on very quantitative problems. We leave the quality of writing, we leave the beauty of art to people.
00:13:22 · Patrick O'Shaughnessy, Neil Movva
[00:13:22] S0: Vanta automates compliance so your team can spend less time on security reviews and more time getting customers. Vanta cuts audit prep by 82% and gives you instant up to date proof of trust without all the manual so you can close deals faster and with less friction. Customers report a 526 return on investment, and more than 16,000 companies use Vanta, including Ramp, Harvey, and Snowflake. Get started with $1,000 off vanta.com/invest. Ridgeline is the first end to end system of record with embedded AI for investment management firms running portfolio accounting, reconciliation, reporting, trading, and compliance on one unified platform. Firms are moving off legacy technology and onto Ridgeline because of how far ahead Ridgeline's AI features are compared to anything else in investment management software. I've been hearing from a lot of investment managers about AI, and they fall roughly into two camps, with some unsure where to even start and others convinced they can build their own order management system over a weekend. The reality is that running an investment firm will always require governance, controls, and a single source of truth for your data, and no amount of AI enthusiasm changes that requirement. Ridgeline is built on exactly that foundation, which is why I believe that the firms that come out ahead in the AI era will be the ones running on Ridgeline's unified platform. If you're serious about your firm's AI strategy, Ridgeline [00:14:45] S2: should
00:14:45 · Neil Movva, Patrick O'Shaughnessy
[00:14:45] S2: should [00:14:45] S0: be part of that conversation, and you can request a demo at ridgeline.ai. Alright. Now let's talk about the very clever stack of solutions that you hope to build ultimately to have this giant token factory, supplier of extremely low cost intelligence. I think you think about this in terms of software, hardware, and power. Talk through what your master plan is to approach this challenge that's so different from what others are thinking about doing. [00:15:09] S2: We always have to start with software. Where is the opportunity on today's data centers to improve efficiency? And the first thing we did was we tried to build the entire LLM software stack around peak GPU efficiency, meaning we're using NVIDIA GPUs. We wanted to squeeze out more tokens from the same chip than anyone else in the world. And that starts with the lowest level of programming, kernels. It's actually my background. I spent my whole professional life working on GPUs and kernels. NVIDIA was my first job while I was in college. I got to see how the tensor cores got to earn their right to be on the chip. [00:15:39] S0: What does that mean? Like, what is a tensor core? [00:15:40] S2: Tensor core is a specialized unit on the GPU that accelerates matrix multiplication. Simple as that. There's been a long history of how we evolve that tensor core over time that we'll get into. [00:15:49] S0: And why is matrix multiplication so important?