Invest Like the Best with Patrick O'Shaughnessy
00:16:59 · Gavin Uberti
[00:16:59] S1: if you look at most other goods, like the iPhone, for example, they've gotten into these economies of scale where as a result, more money does not really buy a better iPhone. That if you're a billionaire or if you're just the average American, you buy the same phone. And tokens aren't like that yet. We're still in the very early days where a general purpose system, relatively small one, is kind of handcrafting these tokens like they made screws back in, like, the Renaissance. And I wanted to live in a world where you have the same economies of scale for token making that you do for making, say, iPhones or cars or anything else. I think that is one of the huge unlocks that allows a huge group of people to go use the best quality models. Economies of scale have all made capitalism very, I don't know, fair. I think that allows you to go ahead and have the same product in many, many different hands, and you're able to go then serve way more users on a single scale up cluster, allows you to get closer to that point for token serving too.
00:17:54 · Rob Wachen
[00:17:54] S3: Yeah. And also just certain products aren't usable if they're slow. So if you want to serve coding models and you want people to actually use them, like, there's a certain number of tokens per second you need to hit. So the question is, while maintaining that per token speed, how many users can I serve at the same time? And you can basically decide, I'm gonna shut off a bunch of the world from using this stuff or everyone's gonna get a worse experience. So fundamentally, you need to find ways to push out the curve, and that's why there's such a pressure for new hardware.
00:18:23 · Patrick O'Shaughnessy
[00:18:23] S0: Vanta automates security and compliance for over 16,000 fast moving companies like Ramp, Cursor, and Harvey, keeping them audit ready around the clock. It's the number one agentic trust platform, and it now helps companies like yours watch for the risks that show up between audits, across your vendors, your AI tools, and your whole environment. Every new tool your team signs up for, every vendor that turns on AI features is an opportunity for something to go wrong, and most security programs weren't built for AI's pace of growth. The Vanta agent works like a twenty four seven GRC engineer in the background, finding issues, drafting fixes for you, and cutting vendor assessment time by up to 50%. Whether you're a fast growing startup or a global enterprise, Vanta helps you earn and prove trust. Invest like the best listeners get a special offer for $1,000 off at vanta.com/invest. Ridgeline is the first end to end system of record with embedded AI for investment management firms, running portfolio accounting, reconciliation, reporting, trading, and compliance on one unified platform. Firms are moving off legacy technology and onto Ridgeline because of how far ahead Ridgeline's AI features are compared to anything else in investment management software. I've been hearing from a lot of investment managers about AI, and they fall roughly into two camps, with some unsure of where to even start and others convinced they can build their own order management system over a weekend. The reality is that running an investment firm will always require governance, controls, and a single source of truth for your data, and no amount of AI enthusiasm changes that requirement. Ridgeline is built on exactly that foundation, which is why I believe that firms that come out ahead in the AI era will be the ones running on Ridgeline's unified platform. If you're serious about your firm's AI strategy, Ridgeline should be part of that conversation. You can request a demo at ridgeline dot a I. I'd like to take some time to step back and hear both of your stories for how you came to this idea and this company, and then kind of walk through what it's been like to build it.
00:20:21 · Patrick O'Shaughnessy
Because I think in so doing, we'll understand the system that you've built for the company itself that will then be able to power subsequent generations of products like this one for this crazy inference feature that we're staring down. Rob, maybe starting with you, just take it however far back you want. But what I'm curious about and your personal story was actually the very first thing I ever heard from either one of you was your personal story many years ago now, which really kind of blew me away. I'm most interested in your motivation ultimately for being here doing this thing.
00:20:53 · Rob Wachen
[00:20:53] S3: It starts back actually in high school for me. I've been very unlucky and lucky at different points in life. This was one of the tougher times. At the end of my sophomore year of high school, I got injured at a martial arts tournament. The next day, I couldn't walk for some reason. I thought there was something wrong with, like, my SI joint or something. I went through physical therapy. I did different types of scans. I couldn't figure it out. And eventually, they found this big bump on my back and an MRI and, you know, told me it was a tumor. Stage four bone cancer was told I had under thirty percent chance of survival. It was like a two year crazy chemotherapy, surgery, learning to walk again experience. And when you go through something like that, it changes the Overton window of human experience and makes you appreciate, like, what actually matters. And you also ask yourself, what are you gonna do if you have the chance to live? If you actually wanna get through something like that, you need to be hoping for something. And I always knew I wanted to do something very impactful if I had the chance to get through it. And it took me a couple years to figure out what that was going to be. And at the same time, as I got to college and met a bunch of other people building cool tech, I got extremely excited by AI models, especially once GPT-three came out. And I was like, wow, this is the first model that can kind of speak English. And these things are gonna get really smart. What happened was when GPT-four came out, there was GPT-four V, which was the first model with image uploading. So I went through my camera roll, and I found a picture of my back with this bump on it before I was diagnosed. And I said, hey. Chat GPT. Pretend you're an expert doctor. A patient comes in, and they says they have this bump on their back. What could it be? And it immediately says, like, this could be a tumor. You should get an MRI immediately. Go to the doctor. And I just kinda sat there still. It was like, that took me six months. And, you know, yesterday, this feature wasn't there. Today, it's here. I go to show my parents, and I got this, like, notification being like, you're all out of image credits today. Like, you need to she didn't get a pro plan. And I was like, holy crap.
00:22:38 · Rob Wachen
Like, this is gonna change everything, and we clearly don't have the infrastructure to serve it. And there's very few things you can work on that can actually bring this technology at scale to the world faster. I mean, there's, like, plenty of people that are super smart working on models, and the fabs seem maybe unreachable to work on, but it seemed like the hardware was all designed before ChatGPT. Every GPU, every TPU, every AI chip that was serving these models were just fundamentally built before this and are retrofit to serve these modern models. There's gonna be an entire new wave of architectures that came out. And what a more exciting thing to work on than bringing this to everybody. Very different angle at the same time. I was running a startup incubator called Prod, which has incubated a bunch of different companies, some of the earliest ones being Cursor and AnySphere, which merged, and MerCore, and Etch went through it, a handful of others. At the time, as these models were getting smarter, it was, yeah, 2022, I was realizing all of these companies are spending all the money they raise on compute. And I had this realization as I was working on some of my own stuff that, like, oh my god, all the products I wanna build are gonna cost tens of millions of dollars a year in inference. Hey. This is not going to be tenable. Like, the cost structure of every software company and the COGS is not gonna be, like, zero anymore for an incremental user. It's gonna be, like, quite high and it's gonna be a function of inference. And then the optics of every business is gonna also be inference as people use more and more coding agents. So fundamentally, it seems like inference is gonna be really important, and it feels like we are on a decade march for inference to become, you know, the biggest market in the world. So when you think about that, ten years from now, there's gonna be these giant projects where everything in that data center fundamentally hasn't been designed today. We should go pick something and work on it. That's kind of how it got started.