00:19:05 · Aditya Agarwal, Dmitri Dolgov
[00:19:05] S2: You've often said that the hard part of driving isn't just the perception. It's the fact that there are multi agents kind of like each of them kind of playing their own part. You're having to essentially model out not just your own behavior but how the world is going to react to some of your own kind of actions. And then you pair that with kind of the closed loop RL which is what gets you to superhuman safety. But how do you think about the closed loop RL in a world where your bad trajectories lead to very bad outcomes? [00:19:34] S0: First of all, you don't do training in the real world. [00:19:37] S2: You have [00:19:38] S0: to say that. You can't learn the real world is for data collection, for grounding. All of the training and all of the closed loop training especially happens in the simulator. But, of course, you cannot build a realistic high fidelity simulator in a vacuum. It's an accelerator, but it still has to be grounded. And this is where the tight loop of building the driver, the simulator, the evaluation machinery, and kind of letting them evolve together and be grounded and validated in real world data with real deployments is what's required.
00:19:38 · Dmitri Dolgov, Aditya Agarwal
[00:19:38] S0: to say that. You can't learn the real world is for data collection, for grounding. All of the training and all of the closed loop training especially happens in the simulator. But, of course, you cannot build a realistic high fidelity simulator in a vacuum. It's an accelerator, but it still has to be grounded. And this is where the tight loop of building the driver, the simulator, the evaluation machinery, and kind of letting them evolve together and be grounded and validated in real world data with real deployments is what's required. [00:20:17] S2: We've talked publicly about how the fifth generation driver was where you kind of made a big jump, right? Like to kind of like being a I guess you describe it as AI as kind of the more of the backbone. What kind of motivated that change? Like it sounds as though that was in some ways a fairly big architectural kind of like difference as you kind of made that jump. Like why what ended up kind of motivating that?
00:20:40 · Dmitri Dolgov
[00:20:40] S0: Well, it was a natural evolution of the state of the art of the tech. I think around that time, I think it became pretty clear that we wanted to kind of bet on this paradigm of building high capacity outboard models and then distill them into the students, Waymo Driver that runs in the car being the student. You just get better scaling loss that way. And it's not specific to autonomous vehicles. It's just kind of a fairly known property that you start with high capacity models, and then you distill. You get better scaling loss than just from working on the small capacity models directly. And so in order to do that, you want the... Your, you know, your your driver to be an effective student. So that means it needs to have kind of maybe, you know, have a strong ML backbone. Otherwise, you just can't, you know, learn from the teacher so well. Right? And also at that time, you know, we've gone through several big waves of innovation in this project. Right? We've, you know, we've seen ConvNets, Then, you know, we've seen transformers. You know, now there's VLMs, VLAs. Right? So it just seemed like a very natural progression. You mentioned end to end as, you know, one of the kind of design dimensions in this whole space. This was also all of that was coming together.
00:22:00 · Ruchi Sanghvi
[00:22:00] S1: You've been through multiple innovations in the course of this project, but it feels like these innovations were so significant that you would have to rewrite large parts of the system to take into account these innovations? I mean, do you do that? Like, how do you rewrite a system, especially when it already works?
00:22:19 · Dmitri Dolgov
[00:22:19] S0: That's a great question. Look, architecture's come and go. I just mentioned a few really big waves of innovation that we've seen. The foundation is kind of all of the machinery that allows you to grab the fruits of the latest state of the art technology at any moment in time. And that's your evaluation, that's your data, that's infrastructure. If you don't have that, if you don't have those strong fundamentals, then you kind of evolve into a tech stack that kind of works and has certain implicit properties that have organically emerged over time. But it's... It can be very, very difficult, if not impossible, to make big changes to it. Right? And then, you know, there's plenty of companies, I think, in the history of technology development that kind of got stuck in those local minima. So what you want if you have... Well, conversely, if you have those strong fundamentals, that allows you to take really big technical swings. Again, whether it's transformers or convnets or what what what have you. And land kind of big disruptive change in technology without the disruption of momentum of your product and without seeing regressions in the field, which, of course, is so important for us because we're operating in a safety critical environment. So, if anything, I would say that's probably the most difficult thing that we've built at Waymo, and probably the most important muscle that we, as a company, develop over time. It's not, again, the technology of the day. It's this ability to bet on the latest and greatest innovation and then take big disruptive steps in technology development without breaking momentum of product development and deployment.
00:24:14 · Ruchi Sanghvi, Dmitri Dolgov, Aditya Agarwal
[00:24:14] S1: So to repeat, the key would be infrastructure, data, and then evaluation. [00:24:20] S0: That's right. [00:24:20] S2: Given the model that you have trained and then you have distilled to kind of run on these cars, if that was the only IP that Waymo had today, right? Like, Okay, you have these models that have been trained, but you didn't actually have kind of like the historical trove of essentially evals and data and so on, would that be an asset in like, would that be a sufficient asset to start kind of like a new AV company?