Y Combinator Startup Podcast
00:04:07 · Jay, Vincent
[00:04:07] S2: I totally agree with you on that. I feel like at the end of the day, it's a lot about a bit of less. Right? If you give the agents or you give the AI model more autonomy and you... If you unshackle it a bit more and give it more resources, it can actually do a lot of the things that we fine tune it to do. [00:04:25] S3: I'm just gonna hop in here as well. I think the the really interesting thing about the bitter lesson here is that, like, VLAs by design architecturally, they're built on top of language models as well. Right? They they can potentially reason. They can write code. So perhaps the better lesson here isn't necessarily what architecture you build necessarily, but it's what data is most useful. Right? Like, people have been sort of nagging at this VLA data bottleneck for years now, and we're seeing very, very slow progress. And so the lesson here is maybe we we have a modality of data that we know works very well, and this idea of transferring across different modalities, which Han and I are super excited about. Like, how do you take something that's traditionally out of distribution, like robotics, and make it something that's in distribution? Like, maybe the way of harnessing a bit of that sin is saying, let's pick a data for which we know this modality is pretty blessed. There's a lot of data, these language... What these models understand it well, and then use this as a way of unlocking a lot of other domains as well.
00:04:25 · Vincent, Jay
[00:04:25] S3: I'm just gonna hop in here as well. I think the the really interesting thing about the bitter lesson here is that, like, VLAs by design architecturally, they're built on top of language models as well. Right? They they can potentially reason. They can write code. So perhaps the better lesson here isn't necessarily what architecture you build necessarily, but it's what data is most useful. Right? Like, people have been sort of nagging at this VLA data bottleneck for years now, and we're seeing very, very slow progress. And so the lesson here is maybe we we have a modality of data that we know works very well, and this idea of transferring across different modalities, which Han and I are super excited about. Like, how do you take something that's traditionally out of distribution, like robotics, and make it something that's in distribution? Like, maybe the way of harnessing a bit of that sin is saying, let's pick a data for which we know this modality is pretty blessed. There's a lot of data, these language... What these models understand it well, and then use this as a way of unlocking a lot of other domains as well. [00:05:22] S2: Yeah. And to add to that, the reason why r two... RT two is such a good model is because it is using a language model that taps into the modality of of all the data that a language model is trained on. So all those web images and web tags actually improves the VLA compared to just training a specific robotics foundation model Right. Without any pretraining.
00:05:22 · Jay, S0, Vincent
[00:05:22] S2: Yeah. And to add to that, the reason why r two... RT two is such a good model is because it is using a language model that taps into the modality of of all the data that a language model is trained on. So all those web images and web tags actually improves the VLA compared to just training a specific robotics foundation model Right. Without any pretraining. [00:05:45] S0: So so you're benefiting in the case of r t two, these early VLA approaches from the... From pretraining. I guess, what's the limitation in the, I guess, action taking fine tuning approach that you can now get around if you can directly write code? Like, you're you're referring to, like, a difference in data there. What exactly is that difference in data you're referring to? [00:06:05] S3: There's a few things that models have got much better at since r d two. Like, one, I mean, they're much better tool use, for example. So, you know, one one kind of data is now these models can... Well, I mean, they they write code much better, so now they can write complex policies as code. They're also much better using tools to explore the kinds of environments they're in, like what arms they have access to. And a lot of this comes from in context learning.
00:06:05 · Vincent, Han Ming
[00:06:05] S3: There's a few things that models have got much better at since r d two. Like, one, I mean, they're much better tool use, for example. So, you know, one one kind of data is now these models can... Well, I mean, they they write code much better, so now they can write complex policies as code. They're also much better using tools to explore the kinds of environments they're in, like what arms they have access to. And a lot of this comes from in context learning. [00:06:28] S1: Like, the the key difference between, like, these VLAs and... Or, like, g p d six or LLM isn't necessarily the architecture, but kind of the approach you take towards training. We want to be bitter, less impaled. Right? Like, we want to benefit from all kinds of data. We want to pour in computer computer used data into our robot models. We wanna pour encoding data into our robot models. But when you do that, you just end up getting what's... What we think of as, like, these general purpose LLM agents. And then so to me, it's like, okay. Why not just build a really good foundational LLM and then use that to control robots rather than training, like, a model that's specifically for robotics and that's more dependent on just robot data.
00:06:28 · Han Ming, S0
[00:06:28] S1: Like, the the key difference between, like, these VLAs and... Or, like, g p d six or LLM isn't necessarily the architecture, but kind of the approach you take towards training. We want to be bitter, less impaled. Right? Like, we want to benefit from all kinds of data. We want to pour in computer computer used data into our robot models. We wanna pour encoding data into our robot models. But when you do that, you just end up getting what's... What we think of as, like, these general purpose LLM agents. And then so to me, it's like, okay. Why not just build a really good foundational LLM and then use that to control robots rather than training, like, a model that's specifically for robotics and that's more dependent on just robot data. [00:07:05] S0: On that note, maybe one of the things that I think, if if we could wind the clock a couple years, may have been a good sign that we were trending in a good direction to this is research on code as policies. Right? Do guys wanna talk a little bit about when the research community refers to code as policies, what exactly that means? And I think it... It's worth putting this in the context of when this paper came out, which was coding agents just starting to work. I think this was still in the era in which people were, like, putting comments and auto filling, you know, Python code blocks, which, you know, now we think of as agent history, but this was, like, two years ago. So, yeah, how how does Code as policies work, and how did that inspire some of what you guys are now seeing as possible?
00:07:40 · Francois
[00:07:40] S4: I probably rewind back to Voyager, where like Voyager was like the one of the first... What does a coding agent mean? Coding agent requires good tool use and on the fly tool creation. That's called code. Okay. You have Python built ins. Let's just say I have only Python built ins. Those are tools. And I have sort, and I have like if, and I have for, and I have while, and I have all these things. And I need to use those tools to assemble a new tool called a new Francois. Py. And now that's a new tool, and I get to use that. That was Voyager. And like, yeah, I'm gonna say Voyager was the first one to do this. Voyager was the most popular one to do this for Minecraft. And then they created tools that they could invoke on the fly to help them play the game better, and compressed thinking and experience into a new tool that I can later invoke. And then I think because everyone in 2024 had this insight, was like, okay, if we just pour, you know, Dario and Sam both all realized if I pour all of my heart and soul and resources and compute and intelligence into getting better at coding, we will have liftoff. Then I can automate the ML engineer, then I can have liftoff and I'll have all the things. I'll solve all the things. And so then that's what everyone did. And I don't think that they had the insight. I don't think they were so insightful to know that, oh, if I did this, then we will have LMs that would be good enough to write policies for robotics. And I can displace all of robotics with actual coding agents. And I can displace artists with, like, you know, coding agents that are writing JavaScript to make beautiful, like, art. Like, I'm not sure that they had that insight, be honest.