Y Combinator Startup Podcast
00:05:26 · Dmitri Dolgov
These systems are not just something that you can build in the lab, you know, get it perfect and then deploy at full scale overnight. So given those two factors, you really need to super clearly and super crisply define the operating conditions and the deployment parameters of your agent, and then build a rigorous framework to guide your deployment so that you can scale in a responsible manner. And this is absolutely critical. This is how you earn trust from your customers, from the communities, from the regulators and yourself. So at Waymo, we see these gaps, of course, in the context of autonomous vehicles, but these gaps will show up in practically any sort of nontrivial physical agent that we will deploy in some shape or form. And driving is just simply the first domain where AI has crossed these four gaps at scale with the public interacting with our product. So let's dive into those lessons that we've learned over the years at Waymo from working on this problem and talk about how we address those gaps. I have seven lessons in this talk. They're all technical. There's a lot more that goes into building a company and building a product, but today I'll just focus on the technical aspects of building AI for the physical world. And each one of those lessons, I think by itself, will not be exactly earth shattering. You know, a lot of it will overlap with likely things you've heard elsewhere, but I hope that the grounding of these lessons in our experience and some of the nuance that I can add about how they showed up in our experience of deploying a physical agent and scaling it safely will be interesting and useful for many of you who are in the space as you build your product, as you build your startup. So let's dive in. The first lesson has to do with this massive, frustrating, sometimes soul crushing difference between a demo and a real product. And a working demo is 1% at best of the work that you have to do. The many of performance, the many nines of reliability that follow, that's where the real work happens. And if you're a founder in the room, chances are you are focused on getting that first prototype, that first demo off the ground. And when you hit that first version of a system that works, that first 90%, when the demo actually works, it feels incredible.
00:08:06 · Dmitri Dolgov
You feel like you solved it, the sky's the limit, you're extrapolating forward, and in our world, we hit that first milestone, that first 90% back around 2010. So when this project started in 2009, before we started building the system, we set a couple of pretty ambitious goals for ourselves. One was to drive a 100 autonomous, a 100,000 miles in autonomous mode. The second goal was to drive 10 routes, each one was a 100 miles long, chosen to cover a variety of conditions across the Bay Area, and we had to do each one from beginning to end without a human intervention. We had at the time a team of about a dozen engineers, and we accomplished both of these goals in about a year and a half. And keep in mind, this was well before any of the AI breakthroughs, before ConvNets, before transformers, before VLMs, before any of the stuff that we talk about today. And yet, you know, we got it done, and kinda by demo standards, autonomous driving was solved in 2010, right? We handled everything. We could drive during the day, during the night, we handled traffic, pedestrians, cyclists, traffic lights, construction zones, on freeways, on surface streets, so we were quote unquote capability complete, and you know, at the time we felt like we're on top of the world, but then we quickly ran, as we started building towards the product, we quickly ran into a brutal reality that there's a massive difference between doing something once or driving 10 routes once and building a scalable service with nobody behind the wheel. It took us about ten more years to begin providing a service, and then five more years to scale to half a million trips per week. So the demo took eighteen months, the product took about fifteen years. But now we're scaling exponentially. To date, we've served well over 20,000,000 fully autonomous trips, and we've driven well over 200,000,000 fully autonomous miles, and we have rider only vehicles operating in 15 cities across The United States, And we're scaling exponentially, it took us fifteen years to get to that first 100,000,000 miles and about seven months to drive the next 100,000,000. It took us about eight years to go from the time when we started our initial writer only operation to the time when we had... When we were serving writers in four cities. Earlier this year, we launched four cities in just one day.
00:10:49 · Dmitri Dolgov
So why does bridging that gap from demo to product take so long? Because there's this harsh engineering reality that you can't really cheat, that reliability and performance lives on this exponential letter of nines. So getting to that first 90% or 99%, that's the easy part. But then every next nine that you wanna add, that takes about 10 times more effort. So you need to know upfront exactly how many nines your product actually needs. So demo might need, you know, one nine, an assist product or, you know, Copilot might need a few, but a fully autonomous AI agent that we're gonna be putting out in the physical world that engages with the public, you know, with kids running around, that needs a whole stack of them. And at scale, the long tail is the problem space, it's your entire problem statement. When you drive millions of miles per week, a rare event that might happen once in a million miles, that just becomes your daily reality. And getting those next nines means doing something different every time. So you don't get to say six nines of performance or reliability by doing the same thing that you did for, you know, to achieve the first two, but longer. You have to do fundamentally different things, it requires a fundamentally different approach. For example, you can take reliability. You can get to the first couple of nines by just doing proper engineering and doing some bug fixes, but to get to the next few, you need to invest in fundamentally different approaches. You need to build fully redundant systems, have tiered fullback architectures and so forth and so on, and the same thing holds for the performance of AI models. So what that actually means is that in this space, it's incredibly easy to get started, but it can be excruciatingly difficult to get to the real product, and that effect is only amplified with every wave of technological breakthroughs, and that naturally leads to hype cycles. So every AI breakthrough from deep learning to cognitive nets to transformers, via lanes, you name it, it makes it that much easier to get started. Your demos, your prototypes, they get a 100 times easier, but the tail, that's where the hard problems are that moves much less. It moves, but the effect is muted, and that's why every hype cycle produces a wave of absolutely spectacular demos and very few real products.
00:13:31 · Dmitri Dolgov
And the recurring mistake of every cycle is spending on the demo what you should be saving for the nines. Now, you know, this being a startup school, the last thing I wanna do is throw too much cold water on the magic and the excitement of those early days. This this time is absolutely magical. It's amazing. Cherish it. Leverage it. But the key is to remain honest about the product that you are building, the number of nines and performance and reliability that that product demands, and not cutting corners to get there. Otherwise, you might be in for a pretty rude awakening later, so count your nines before you count your demo views. And this brings us to the second lesson. Once you know how many nines your product actually needs, it fundamentally dictates the architecture and the core technical approach that you need to pursue. Now every technology has a performance versus effort curve, right? They all tend to start fairly steep and go up, and then they flatten out, And you know, as I just mentioned, every other nine gets an order of magnitude more difficult. So common failure mode is picking the tech that gives you the fastest early ramp, riding that steep curve, feeling like you're winning, projecting that you know, steep slope into the future and feeling like the sky's the limit, and then hitting the plateau and discovering that the technology path that you picked actually flattens out way before the performance that is required by your product. Now you might still choose to be, at least for a while, on that steep curve for a variety of practical reasons. You know, maybe you want to prototype something or demo something or build something in service of learning, but be honest with yourself where you're building for the purpose of a demo, for the purpose of learning, or towards an actual product. So let's take an example from our domain, autonomous vehicle and sensing. There's been a long standing debate about what kind of sensors do you actually need for autonomous driving. Naturally, more sensors means higher performance, but also means higher complexity. So humans of course can drive with just eyes, so there's that proof of existence. Now, and if the goal were to just approximately match human performance or to build an assist product, that's a very reasonable way to go.
00:16:11 · Dmitri Dolgov
However, if you are targeting full autonomy and you're targeting superhuman, strongly superhuman performance, you find that weak sensing just leads to a safety curve that flattens out way too early. So at Waymo, we've taken an approach where we use multiple sensing modalities. We use cameras, LiDARs, and radars, and they all complement each other. Cameras give you high resolution and color, but they're passive and they degrade in darkness and glare. LiDAR gives you a direct measurement of the three d structure of the world around you, and radar is very good at punching through environmental conditions and weather like fog or rain or snow, and it directly can measure velocity using Doppler. LiDAR and radar are active sensors, so that means they see just as well in pitch darkness or, for example, when driving into a blinding sunset. And these different sensing modalities, of course, they're not backups to each other. In our stack, each modality has an encoder and the information from all of those sensors get fused into a single view of the world around us that is much more precise and generally vastly superior to what you get with any one sensor. So let me show you a few examples. Here's the scene where Waymo is driving in a dust storm in Phoenix. So what you see here is what the scene looks like to our fairly advanced high resolution and high dynamic range camera. It's very close to what a can, you know, a human would see in the same conditions, which is not much. And here on the right is what the LIDAR sees for the exact same frame, and you can much more clearly see that there's a pedestrian standing on the side of the road. So if they were to step onto the road, that early detection can make a really big difference in how the situation plays out and the safety of everyone involved. Here's another example. At night, driving along, and there are a couple of pedestrians who are about to jump onto the road over a concrete construction barrier. Again, at the bottom you see the camera, really can't see much, and the lighter view at the top. Again, lighter versus camera. Here's another example. There are a couple of dogs chasing a bull, and a couple of kids chasing the dogs. And big difference, here's what it looks like to the camera.
00:18:48 · Dmitri Dolgov
Here's the LiDAR, and the early detection of the kids is off to the side, and there are no headlights, there are no lamps there, it's complete darkness, so it makes a big difference. Or think about what happens when something physically obstructs the view of your sensors. Yeah, if you don't have redundancy in sensing, you can have, you know, a single leaf land on your sensors and bring your robot to a full stop. Now, so you need redundancy. Redundancy of course does not necessarily mean multiple sensing modalities, but if you need redundancy anyway, you might as well benefit from the complementary physics of the different sensing modalities in the nominal case. So here's a video of one of our cars that picked up a leaf, or actually I think a full branch of a tree that our wipers were unable to shake, and the car detected that and because we have sensing redundancy, it safely was able to get back to the depot for proper cleaning. So specifically, when it comes to hardware, do not anchor to today's components prices. We are on the sixth generation of the Waymo Driver, the Waymo Hardware Suite today, and with every generation, the hardware not only delivered amazing capability, but we're able to drastically simplify and radically reduce the cost of the hardware as well. So betting your company, betting your approach on today's hardware prices is just betting your company on a number that has a fairly short shelf life and is going to expire. So hardware would change. Many components will get commoditized and drop in price, so design for that future and be ready to upgrade. And that brings us to the next lesson, lesson number three, technology moves incredibly fast, especially nowadays. So you need to be ready to ride those tech waves and do that repeatedly. And when you do, as to not only think about the wins in performance and the wins in capability, you have to be very mindful about unification and simplification. Over the years, we've seen a number of major breakthroughs in technology, a lot of them around AI, and with every wave of innovation, we pretty much rebuild the Waymo Driver around that major wave of AI breakthroughs, and we often push the state of the art in those areas forward ourselves. We leveraged ConvNets around 2013 for computer vision and perception.