Y Combinator Startup Podcast
00:21:27 · Dmitri Dolgov
Then when Transformers came about around 2017, we bet big on them both for perception and for the task of behavior prediction and decision making and planning. Turns out the task of driving is not that dissimilar from the task of modeling language because of the social aspects of driving. You're kinda having a conversation with other dynamic actors in the world, but you're doing that in the space, kinda body language of your agent, your car, as opposed to just the language of words. And you operate in sequences and local continuity matters, but so does global context. And today we're leveraging the latest in VLMs and frontier role models. Now using the latest tech for capability and performance wins, I don't wanna say it's easy, but it can be, you know, reasonably straightforward, doing applied research in isolation or starting a tiger team to, you know, prototype some new technology is not the most difficult part. There's many companies, many teams that are excellent in this. The much harder muscle to build is to carry that bleeding edge research into production and deploy it in a safety critical environment without regressions, and do it without breaking stride on the scaling of your product. And adding capability, again, is not the hardest part, but adding capability while at the same time reducing fragmentation and reducing complexity, that is really important. And finally, the hard muscle to build as a company is to be able to do that repeatedly through multiple waves of technical innovation or technical breakthroughs. So on this front, have two bits of advice. The first one, when the technology, a new technology shows up, yeah, it can be very exciting, very tempting to kick off a new effort, a tiger team to pursue it, and that's great, you should absolutely do that. However, when you do, it's very important that you consider what you would do after. Under a success scenario, let's say that effort succeeds, you should be very clear on what the path of that new innovation is for your company, for your entire product, for your entire system. Oftentimes I've seen a failure mode where a project, a very difficult technical project succeeds, and then there's a dead end. So that can be very wasteful, that can be completely deflating.
00:24:04 · Dmitri Dolgov
The second bit of advice that I have here is when pursuing new tech, again, don't just ask what does this new tech give me in terms of capability and performance, also ask has it simplified my stack and has it led to fragmentation or unification? So set your launch bar to demand both breakthrough performance and at the same time, radical simplification and unification. And this exact philosophy and this muscle that we've built at Waymo over the years is what produced our latest core technology, and the heart of it is the Waymo Foundation Model. Now, Waymo Foundation Model is a multimodal world action language model. It's kind of a mouthful, so let me unpack the ingredients. It's a multimodal model because it is able to process these multimodal sensory inputs, cameras, LiDARs, radar. It's a world model because it inherently understands how the world works, the physics, the dynamics, as well as the social and semantic aspect of it. It's an action model because we are not just passively observing how the world evolves, we're an active participant. So the model needs to understand the effects of the actions of our agent on the world and be able to tell the good ones from bad ones. And finally, it's aligned with language, and that allows us to unlock general world knowledge from visual language models, and that's incredibly useful in the long tail of rare semantic situations. So more specifically, this is what the architecture looks like. It's kind of your typical encoder decoder architecture. The encoder part takes in the multimodal sensing and compresses it or encodes it into an efficient representation that retains all of the relevant data, all of the relevant information for the generative part or the decoder. It's an end to end model, which has a couple of nice properties. It allows us to effectively backpropagate the gradient from the task that we actually care about all the way to the early layers of the model, and it allows the encoder to reach, you know, kind of learn the right rich representations for what the generative part needs to solve the task. It uses the system one, system two, think fast, think slow architecture, and it leverages the general world knowledge of VLMs for efficient learning of semantic tasks. So let's dive deeper. First, the think fast path.
00:26:43 · Dmitri Dolgov
That part fuses the raw data from our cameras, our LiDARs, our radars, and that allows for split second safety critical decisions. So So you can think of it as kind of your driving instincts. This is what allows the car to brake instantly if let's say a pedestrian runs into the road or a cyclist that's nearby swerves into your path. This is like the, if you will, the lizard brain of your agent that deals with a lot of geometric tasks and can react in milliseconds. Second is the slow path. That's the part that's responsible for the more complex semantic and scene level understanding type tasks. And these sort of tasks, these things don't typically change in milliseconds, so there you can afford a bit more latency and you can trade that off for higher capability and higher levels of reasoning. So for example, if the Waymo Driver encounters a situation when there's, you know, a vehicle, let's say it's on fire on the side of the road, the fast path might just see it as a generic obstacle and reason that the path ahead of us is clear. And this is where the slow path comes in, and that path can use deep semantic reasoning to understand the semantics of that object, the car being fire, and the broader scene context, and that allows our driver to decide to take a very different action or a different route entirely, even if geometrically the path ahead of us is clear. And finally, there's the generate component. That's the decoder. That's the component that understands and can produce behavior. It understands how other actors behave, and it allows us to make predictions and plan our own driving decisions. And our Waymo Foundation model powers the Waymo Driver that runs on different generations of hardware and runs on different vehicle platforms. You have our fifth generation and sixth generation, the Jaguar I-PACEs, the Ojai and the Hyundai Ioniq, and in the future, will power different products and different commercial applications like trucking and personally owned vehicles. So by leveraging the strategy of focusing on the high capacity foundation upward model, we're able to move a lot of complexity upstream to that large shared foundation, and that allows us to make that specialization layer that's running on the car are pretty lightweight, and that in turn allows us to speed up the development process.
00:29:25 · Dmitri Dolgov
So the most important muscle in this lesson is for your company to not just leverage the tech of the day, but have the ability and build up muscle to repeatedly ride those tech waves and pull in the results of that innovation into production without regression, without breaking stride in deployment and scaling, and without drowning complexity. So let's move to the next lesson. There is a well known lesson in the AI community that general methods that leverage massive compute and massive data will always beat methods that rely on handcrafted engineered human knowledge. That's the so called bitter lesson that Richard Sutton published and formulated in 2019. And we have lived this and we have seen this in every wave of technical breakthroughs. Each time the bitter lesson holds methods that scale best with compute, with data, they always win out. And by the way, this is one of the reasons why we bet on the approach of building the foundation model. There is a well known property that if you bet on high capacity model and you use your data and your compute on that, you just get better scaling laws, and then you distill into smaller, more efficient models that are running on your agent in real time, you just get better scaling laws as opposed to just focusing on the smaller models directly. So one nuance area where this lesson shows up is the user structure in your models. And depending on how you use your structure, you can end up on either side of the bitter lesson. Essentially structure that fights scale will always lose, and structure that channel scale always wins. In particular this comes up around the discussion of end to end models. As I mentioned, an end to end model has some very nice properties. You back propagate gradient from the final tasks all the way through the model, and it allows the API between the encoder and decoder to use rich learn representations. And you know, those are the easiest models to build and train.
00:31:40 · Dmitri Dolgov
You know, you can start, the architectures are known, you can start with doing some imitation learning and kind of a black box end to end model will give you very rapid progress and you'll ride that very, you know, initial steep part of the curve, and for some products that's enough, but if you need to reach superhuman levels of performance in a fully autonomous agent, in a safety critical environment, just doing kind of that basic vanilla end to end is not enough, And this is where structure comes in. And the key question here is, does the structure boost scale or does it fight it? Does it limit and constrain your solution space, or does it help you scale without loss of generality? So let me give you an example, let me illustrate this point with kind of a simple thought exercise and a toy problem. Imagine you're building a robot that will play the game of Go. And it wants, you know, you want it to play the game in the physical world, right? So you have a camera that's observing the board and you have an actuator that will actually move the pieces around. Now one way you can build such a robot is to, you know, have an end to end system that goes directly from pixels to actuation, and maybe you train it by giving it some videos of, you know, how humans play the game. And that could be a very interesting research exercise, however, if your goal was to build the world's best playing Go robot, that's probably not the most efficient way to go. And the reason for that is that there is a very simple intermediate representation that captures completely the state of the game, the state of the task that you're trying to solve, as you know, 19 by 19 board, and that gives you a fully observable and complete state of the world that you care about, at least for the game plan and link part. So leveraging that structure, it doesn't limit your model, it doesn't constrain your solution space, but it gives you a very helpful way to scale. Now that of course was a toy example. Anything that's not trivial that you're trying to deploy in the physical world will not have that property, and the fact that such a simple, clean, engineered representation doesn't exist in the physical world is the whole reason why we need end to end systems and learned representations and learned embeddings, but in the physical world, there does exist structure.
00:34:23 · Dmitri Dolgov
You have laws of physics, you have rules of the road, you have objects that behave in reasonably predictable ways, and you can use that structure in addition to the learned representations to boost your performance, simplify validation, and at the end of the day, just get better scaling laws. And this is the approach that we are pursuing at Waymo, which we call structure augmented end to end. So we go beyond the basic vanilla end to end by augmenting the learned embeddings with materialized structured representations. And that gives us a few very important advantages. So first is validation at inference time. Now because the model isn't just a black box where sensors go in and actuation commands go out, we can create a very powerful correctness and safety validation layer that you can run-in real time when the agent is deployed on our vehicles. And this is really important for any agent that's operating in the physical world. Secondly, we get great wins in efficiency when it comes to large scale training and evaluation of the generative part of the model, the decoder. Now if all you have is a black box end to end system, you are forced to do all of your evaluation and all of your training in the end to end setup all the way from sensors to decisions to actuation. Having that intermediate structured representation allows you to kinda mix and match. You can do some training at larger scale and some evaluation in the space of those compact structured representations, and some in the full space of end to end from sensors to decisions. And finally, we get strong, verifiable feedback signals for both evaluation and for training, training recipes to support things like reinforcement learning. That additional materialized structure just gives you much more powerful tools for evaluation, for metrics, as well as crafting your loss function or, you know, reinforcement learning recipes. So the lesson here is to bet on a system that's maximally learned and minimally constrained, and leverage structure intentionally to boost performance and scaling laws, both in training and in evaluation. Now that raises the question of how do you actually train and evaluate your physical AI agent? And that brings us to the next lesson. To build and safely deploy an agent in the physical world, it is absolutely critical to have a good, large scale, realistic, high fidelity simulator.