Invest Like the Best with Patrick O'Shaughnessy
00:54:44 · Gavin Uberti, Rob Wachen, Patrick O'Shaughnessy
[00:54:44] S1: I think this comes down to economies of scale again. A certain part to the stack and there are huge economies of scale and others there aren't. For example, on designing models, huge economies of scale there. For chip fabrication, same story. But for example, if you think about building some small metal part inside of that rack, there's not that same effect. We think the natural boundaries are on the chip side, on the bottom, and the model layer at the top, and we'll fill the whole gap between. [00:55:10] S3: A few weeks ago, there was a guy who was running a next generation AI chip for one of the frontier companies, and he's trying to recruit one of our architects. And, actually kinda did an UNO reverse card and started recruiting the person trying to recruit our guy. Then within a week, we hired him. And I was going on a walk as we were kind of finalizing the offer. I was like, you're leading this super important project. Why are you deciding to join? And his answer is super interesting, which was, it fundamentally is not existential for my company for this product to win. For Google with TPUs, the revenue comes from search [00:55:41] S0: Google won't fail if TPUs fail.
00:55:41 · Patrick O'Shaughnessy, Rob Wachen, Gavin Uberti
[00:55:41] S0: Google won't fail if TPUs fail. [00:55:43] S3: That's right. Meta won't fail if MTIA fails. Microsoft won't fail if Maya fails. And OpenAI won't fail if Jalapeno fails. Ultimately, this is our product. It is completely unsurprising that the best chip in the world is built by a company that only builds that chip. It's NVIDIA. Right? And for us, it is completely existential for us to get as much token capacity online as possible. It recruits a set of talent and recruits a support from suppliers and from customers that view it with the level of intensity that we do. Look at the raw FLOP stats. [00:56:12] S1: You compare any of the chips built by the labs or by the hyperscalers. The FLOP density for say FB8 times FB8 is lower than the Blackwell B300. And that makes sense because they don't have to go take the risk. They just have to go build a similar enough product and not pay the NVIDIA tax.
00:56:30 · Patrick O'Shaughnessy
[00:56:30] S0: Your finance team isn't losing money on big mistakes. It's leaking through a thousand tiny decisions nobody's watching. Ramp puts guardrails on spending before it happens. Real time limits, automatic rules, zero firefighting. Try it at ramp.com/invest. As your business grows, Vanta scales with you, automating compliance and giving you a single source of truth for security and risk. Learn more at vanta.com/invest. Every investment firm is unique and generic AI doesn't understand your process. Rogo does. It's an AI platform built specifically for Wall Street connected to your data, understanding your process, and producing real outputs. Check them out at rogo.ai/invest. The best AI and software companies from OpenAI to cursor to perplexity use WorkOS to become enterprise ready overnight, not in months. Visit workos.com to skip the unglamorous infrastructure work and focus on your product. Ridgeline is redefining asset management technology as a true partner, not just a software vendor. They've helped firms five x in scale, enabling faster growth, smarter operations, and a competitive edge. Visit ridgelineapps.com to see what they can unlock for your firm. As I think about you guys building the solution, the process of doing so is solving a sequence of really hard challenges. What has been the single episode that was the hardest to overcome? We were designing the chip.
00:57:51 · Gavin Uberti, Rob Wachen
[00:57:51] S1: We built this massive, massive FPGA cluster to go out and verify that full chip worked as is. And FPGAs are digital entities. You can go test digital logic, but not analog logic. And it turns out that when the chip came back, we began to go see issues in our retention of adaptation of incorrect results. And we realized, wait a minute, there's a problem with a back pressuring logic across a clock domain crossing is failing, and this is going to cause the ship to produce wrong results and it is very, very hard to solve. We realized there was one and only one way to solve it, as we had to go line up two clock signals on our chip to within fifty picoseconds, that is literally fifty trillionths of a second, and we had to go get these signals aligned to this super small granularity and do it on every chip 2,000,000,000 times a second. [00:58:42] S3: A lot of people said this was impossible. We had people quit.
00:58:42 · Rob Wachen, Gavin Uberti, Patrick O'Shaughnessy
[00:58:42] S3: A lot of people said this was impossible. We had people quit. [00:58:45] S1: Yeah. That people literally were like, this problem is unsolvable, and that's the luck, guys. Well, when you have a problem like that, step one is, okay, let's assume the problem is solvable. How would it be solved? Well, we realized what we have to be able to do is find a way to go ahead and move our clock phase by a picosecond, ten picoseconds. And we had an idea, what if we had these two clocks, they had them just a little bit apart from each other. We figured out that, hey, if you go out and figure out the phase, if we then go out and use the drifting mechanism go ahead and wait for just the right amount of time to get that always lined up, we can do this extremely reliably and then lock the phases exactly where they have to be. We can guarantee this never happens. People were, I think, blown away that this worked, and it worked as well as it actually did, but we made it work. [00:59:35] S0: How long did that take? [00:59:37] S1: It was actually about two weeks. [00:59:39] S3: It was dark two weeks. [00:59:40] S1: It was a very scary two weeks, but it was the kind of thing where when that kind of thing happens, that is the most important time to go ahead and be investing effort, that is the hardest time to go do it when you feel like things are hopeless, but like the sooner you solve that problem, the sooner you can get back to building and scaling up production to mass volumes.
00:59:40 · Gavin Uberti, Rob Wachen
[00:59:40] S1: It was a very scary two weeks, but it was the kind of thing where when that kind of thing happens, that is the most important time to go ahead and be investing effort, that is the hardest time to go do it when you feel like things are hopeless, but like the sooner you solve that problem, the sooner you can get back to building and scaling up production to mass volumes. [00:59:57] S3: I think a lot of our story is like, as Gavin says, assume it is possible. Like, assume it is possible to have a chip with way more flops on it. Assume it is possible to have a system with way lower latency between chips. Assume it is possible to create a shared memory pool that can run at way higher bandwidths, like, how would one do it? A lot of the time when we do experiments, we will do dozens of experiments and all of them will fail. But we only need one to work. There's multiple times I mean, Gavin, I think, was leading the charge and our Chip bring up with, I think, 30 different board experiments, and three of them worked. And all three of them are worth their weight in gold. [01:00:30] S1: This is one of the things that people come to me and say, Gavin, almost none of your experiments works. I only got to get lucky once.