Invest Like the Best with Patrick O'Shaughnessy
00:16:34 · Rob Wachen
So when we think about optimizing memory bandwidth for the system, you have to think about how fast these chips can communicate together because if they can only communicate really quickly with themselves and very slowly with other chips, you're not going to actually be able to serve giant models at 10,000, 20,000 tokens per second. So we need multiple orders of magnitude of infrastructure build out throughout the entire stack, the wafer to the watt, transistor to the token, to actually bring this stuff to the world. And I think that
00:16:59 · Gavin Uberti
[00:16:59] S1: if you look at most other goods, like the iPhone, for example, they've gotten into these economies of scale where as a result, more money does not really buy a better iPhone. That if you're a billionaire or if you're just the average American, you buy the same phone. And tokens aren't like that yet. We're still in the very early days where a general purpose system, relatively small one, is kind of handcrafting these tokens like they made screws back in, like, the Renaissance. And I wanted to live in a world where you have the same economies of scale for token making that you do for making, say, iPhones or cars or anything else. I think that is one of the huge unlocks that allows a huge group of people to go use the best quality models. Economies of scale have all made capitalism very, I don't know, fair. I think that allows you to go ahead and have the same product in many, many different hands, and you're able to go then serve way more users on a single scale up cluster, allows you to get closer to that point for token serving too.
00:17:54 · Rob Wachen
[00:17:54] S3: Yeah. And also just certain products aren't usable if they're slow. So if you want to serve coding models and you want people to actually use them, like, there's a certain number of tokens per second you need to hit. So the question is, while maintaining that per token speed, how many users can I serve at the same time? And you can basically decide, I'm gonna shut off a bunch of the world from using this stuff or everyone's gonna get a worse experience. So fundamentally, you need to find ways to push out the curve, and that's why there's such a pressure for new hardware.
00:18:23 · Patrick O'Shaughnessy
[00:18:23] S0: Vanta automates security and compliance for over 16,000 fast moving companies like Ramp, Cursor, and Harvey, keeping them audit ready around the clock. It's the number one agentic trust platform, and it now helps companies like yours watch for the risks that show up between audits, across your vendors, your AI tools, and your whole environment. Every new tool your team signs up for, every vendor that turns on AI features is an opportunity for something to go wrong, and most security programs weren't built for AI's pace of growth. The Vanta agent works like a twenty four seven GRC engineer in the background, finding issues, drafting fixes for you, and cutting vendor assessment time by up to 50%. Whether you're a fast growing startup or a global enterprise, Vanta helps you earn and prove trust. Invest like the best listeners get a special offer for $1,000 off at vanta.com/invest. Ridgeline is the first end to end system of record with embedded AI for investment management firms, running portfolio accounting, reconciliation, reporting, trading, and compliance on one unified platform. Firms are moving off legacy technology and onto Ridgeline because of how far ahead Ridgeline's AI features are compared to anything else in investment management software. I've been hearing from a lot of investment managers about AI, and they fall roughly into two camps, with some unsure of where to even start and others convinced they can build their own order management system over a weekend. The reality is that running an investment firm will always require governance, controls, and a single source of truth for your data, and no amount of AI enthusiasm changes that requirement. Ridgeline is built on exactly that foundation, which is why I believe that firms that come out ahead in the AI era will be the ones running on Ridgeline's unified platform. If you're serious about your firm's AI strategy, Ridgeline should be part of that conversation. You can request a demo at ridgeline dot a I. I'd like to take some time to step back and hear both of your stories for how you came to this idea and this company, and then kind of walk through what it's been like to build it.
00:20:21 · Patrick O'Shaughnessy
Because I think in so doing, we'll understand the system that you've built for the company itself that will then be able to power subsequent generations of products like this one for this crazy inference feature that we're staring down. Rob, maybe starting with you, just take it however far back you want. But what I'm curious about and your personal story was actually the very first thing I ever heard from either one of you was your personal story many years ago now, which really kind of blew me away. I'm most interested in your motivation ultimately for being here doing this thing.
00:20:53 · Rob Wachen
[00:20:53] S3: It starts back actually in high school for me. I've been very unlucky and lucky at different points in life. This was one of the tougher times. At the end of my sophomore year of high school, I got injured at a martial arts tournament. The next day, I couldn't walk for some reason. I thought there was something wrong with, like, my SI joint or something. I went through physical therapy. I did different types of scans. I couldn't figure it out. And eventually, they found this big bump on my back and an MRI and, you know, told me it was a tumor. Stage four bone cancer was told I had under thirty percent chance of survival. It was like a two year crazy chemotherapy, surgery, learning to walk again experience. And when you go through something like that, it changes the Overton window of human experience and makes you appreciate, like, what actually matters. And you also ask yourself, what are you gonna do if you have the chance to live? If you actually wanna get through something like that, you need to be hoping for something. And I always knew I wanted to do something very impactful if I had the chance to get through it. And it took me a couple years to figure out what that was going to be. And at the same time, as I got to college and met a bunch of other people building cool tech, I got extremely excited by AI models, especially once GPT-three came out. And I was like, wow, this is the first model that can kind of speak English. And these things are gonna get really smart. What happened was when GPT-four came out, there was GPT-four V, which was the first model with image uploading. So I went through my camera roll, and I found a picture of my back with this bump on it before I was diagnosed. And I said, hey. Chat GPT. Pretend you're an expert doctor. A patient comes in, and they says they have this bump on their back. What could it be? And it immediately says, like, this could be a tumor. You should get an MRI immediately. Go to the doctor. And I just kinda sat there still. It was like, that took me six months. And, you know, yesterday, this feature wasn't there. Today, it's here. I go to show my parents, and I got this, like, notification being like, you're all out of image credits today. Like, you need to she didn't get a pro plan. And I was like, holy crap.