Y Combinator Startup Podcast
00:02:12 · Jay
[00:02:12] S2: One of the earliest successful approaches of using AI on robots is the r t two paper, where they use a pretrained language model on web text and images and use that to control robots. And the interesting thing about that is that it's a fine tuned version of a language model, and instead of outputting English, for example, they just output what we call an an effector pose, which is the coordinates that you can then translate into join commands that can control the robots. In some cases, it's... In some sense, it's very similar to what we see now with large language models, just that instead of fine tuning, the models are good enough to just do it out of the box.
00:02:58 · Francois, Jay
[00:02:58] S4: And this is actually quite similar to, like, the mapping that I have in my head is kinda like the COT moment. And so like when we did GSM eight ks and you said, Susie had $8 she spent $5 how much does she have now? It had to output like four hash marks and then the answer and an EOS. There was no ability for it to chain of thought. And so then we allowed it to actually have like, okay, let me go eight minus five is three, blah, blah, And so like, hash, hash, hash, hash, three. And so you allowed it to do this kind of thinking before it actually gave an action or an answer. And similarly now, like VLAs were basically like an RT2 is basically like, it has to output an action. There's no, like I can't allocate more compute for a more complex task. And then now I have this code chain of thought thing that I can do. And I can say, okay, even if you were using the example where you're not actually outputting the code, just giving it to Astra and let it think, think, think, think, think, and then output an action. Similarly now, and then with code, it's a more Kolmogorov complexity or lower Kolmogorov complexity code code length to just use code and really compact code to just, here's the code. Just go. [00:04:07] S2: I totally agree with you on that. I feel like at the end of the day, it's a lot about a bit of less. Right? If you give the agents or you give the AI model more autonomy and you... If you unshackle it a bit more and give it more resources, it can actually do a lot of the things that we fine tune it to do.
00:04:07 · Jay, Vincent
[00:04:07] S2: I totally agree with you on that. I feel like at the end of the day, it's a lot about a bit of less. Right? If you give the agents or you give the AI model more autonomy and you... If you unshackle it a bit more and give it more resources, it can actually do a lot of the things that we fine tune it to do. [00:04:25] S3: I'm just gonna hop in here as well. I think the the really interesting thing about the bitter lesson here is that, like, VLAs by design architecturally, they're built on top of language models as well. Right? They they can potentially reason. They can write code. So perhaps the better lesson here isn't necessarily what architecture you build necessarily, but it's what data is most useful. Right? Like, people have been sort of nagging at this VLA data bottleneck for years now, and we're seeing very, very slow progress. And so the lesson here is maybe we we have a modality of data that we know works very well, and this idea of transferring across different modalities, which Han and I are super excited about. Like, how do you take something that's traditionally out of distribution, like robotics, and make it something that's in distribution? Like, maybe the way of harnessing a bit of that sin is saying, let's pick a data for which we know this modality is pretty blessed. There's a lot of data, these language... What these models understand it well, and then use this as a way of unlocking a lot of other domains as well.
00:04:25 · Vincent, Jay
[00:04:25] S3: I'm just gonna hop in here as well. I think the the really interesting thing about the bitter lesson here is that, like, VLAs by design architecturally, they're built on top of language models as well. Right? They they can potentially reason. They can write code. So perhaps the better lesson here isn't necessarily what architecture you build necessarily, but it's what data is most useful. Right? Like, people have been sort of nagging at this VLA data bottleneck for years now, and we're seeing very, very slow progress. And so the lesson here is maybe we we have a modality of data that we know works very well, and this idea of transferring across different modalities, which Han and I are super excited about. Like, how do you take something that's traditionally out of distribution, like robotics, and make it something that's in distribution? Like, maybe the way of harnessing a bit of that sin is saying, let's pick a data for which we know this modality is pretty blessed. There's a lot of data, these language... What these models understand it well, and then use this as a way of unlocking a lot of other domains as well. [00:05:22] S2: Yeah. And to add to that, the reason why r two... RT two is such a good model is because it is using a language model that taps into the modality of of all the data that a language model is trained on. So all those web images and web tags actually improves the VLA compared to just training a specific robotics foundation model Right. Without any pretraining.
00:05:22 · Jay, S0, Vincent
[00:05:22] S2: Yeah. And to add to that, the reason why r two... RT two is such a good model is because it is using a language model that taps into the modality of of all the data that a language model is trained on. So all those web images and web tags actually improves the VLA compared to just training a specific robotics foundation model Right. Without any pretraining. [00:05:45] S0: So so you're benefiting in the case of r t two, these early VLA approaches from the... From pretraining. I guess, what's the limitation in the, I guess, action taking fine tuning approach that you can now get around if you can directly write code? Like, you're you're referring to, like, a difference in data there. What exactly is that difference in data you're referring to? [00:06:05] S3: There's a few things that models have got much better at since r d two. Like, one, I mean, they're much better tool use, for example. So, you know, one one kind of data is now these models can... Well, I mean, they they write code much better, so now they can write complex policies as code. They're also much better using tools to explore the kinds of environments they're in, like what arms they have access to. And a lot of this comes from in context learning.
00:06:05 · Vincent, Han Ming
[00:06:05] S3: There's a few things that models have got much better at since r d two. Like, one, I mean, they're much better tool use, for example. So, you know, one one kind of data is now these models can... Well, I mean, they they write code much better, so now they can write complex policies as code. They're also much better using tools to explore the kinds of environments they're in, like what arms they have access to. And a lot of this comes from in context learning. [00:06:28] S1: Like, the the key difference between, like, these VLAs and... Or, like, g p d six or LLM isn't necessarily the architecture, but kind of the approach you take towards training. We want to be bitter, less impaled. Right? Like, we want to benefit from all kinds of data. We want to pour in computer computer used data into our robot models. We wanna pour encoding data into our robot models. But when you do that, you just end up getting what's... What we think of as, like, these general purpose LLM agents. And then so to me, it's like, okay. Why not just build a really good foundational LLM and then use that to control robots rather than training, like, a model that's specifically for robotics and that's more dependent on just robot data.