Y Combinator Startup Podcast
00:14:30 · Vincent
[00:14:30] S3: thing is we've been sort of thinking about this idea of a harness almost as a form of, like, domain specificity. Like, when when you, for example, deploy a robot in a new environment, maybe it's in a wet lab, and it just needs to do a lot of test you're picking up, for example. You can learn a lot of this in context, but then one way of consolidating this context for future agents is sort of packaging each of these skills that you've learned into specific programs, for example. Like, writing out these skills and writing out these memories is a form of consolidating. It... It's it's like a form of distillation from past experience for your future agents to use. So this is quite interesting. The other thing that's quite interesting is it almost seems like there's some relationship to the broader meta learning literature throughout in machine learning history. Like, you have this sort of bigger model that then programs a smaller model to do things. And one of the very interesting things is that, yes, these smaller models are gonna learn in context. They're sort of wrapped inside the harness, and they're doing the task. You can make them maybe smaller. You can make them run faster as long as the bigger model can sort of do this domain specialization of your harness quite well. So so this is something that we're we're pretty interested in right now. And last point as well, I think the the the really interesting threaded question is how far in context learning can get you. Right. And I think it's really anyone's guess as to that. Like, there's papers that show that, you know, these models sort of can approximate gradient descent during in context learning as well. So it's this this line between weight space and symbolic space, I think, is is a little bit blurry.
00:16:00 · S5, S0, Jay, Han Ming
[00:16:00] S5: YC's next batch is now taking applications. Got a startup in you? Apply at ycombinator.com/apply. It's never too early, and filling out the app will level up your idea. Okay. Back to the video. [00:16:14] S0: So what exactly are we watching here? [00:16:16] S2: We are watching Astra controlling the arms to pick a block off the table and put it into the bowl. [00:16:22] S0: So here, Astra is using just these camera inputs, and presumably, it's aware of the robot that it's controlling, and it's gonna write code that controls the robot at its individual joints. [00:16:31] S2: Yes. So it it does look at all the cameras. It has all the camera feeds. Instead of code, it's more like a tool call. So it sends commands to the robot to to control and put the block into the bowl. [00:16:46] S0: So in this situation, we're using Astra directly. But if we were using, say, Waddle's harness, what's the kind of difference between directly using Astra to control this versus doing this through Waddle's API? [00:16:56] S1: Sometimes directly, like, having Astra directly command the post robot should go to is not direct... The optimal tool to use. If it's a repetitive task, you don't want Astra in the loop. Maybe you wanna write code that can run repeatedly very, very fast. Or if it's a task... You've done a similar task before, you should be able to call it a skill that has compiled and use that skill to do the task faster and handle edge cases better. [00:17:17] S0: So here we can see it seems to be approaching in on this guy. Oh, very dramatic.
00:17:17 · S0, Jay, Han Ming
[00:17:17] S0: So here we can see it seems to be approaching in on this guy. Oh, very dramatic. [00:17:22] S2: Yeah. You can look... You can notice that the latency is a bit slow because we are bottlenecked by the latency of Astra. But if you look at the trends, the latency of these models are improving very rapidly. Right. So one thing that we saw is that for Fable class LLMs, their latency is improving by around two x per month, which is very, [00:17:43] S1: very fast. [00:17:44] S0: Very fast. [00:17:44] S2: Yeah. Yeah. If the trends continue, we could get real time control by end of the year. [00:17:50] S0: So now that we saw this task go from end to end, why don't we walk through what that actually entailed? So there's a coding agent in the loop here running this. Yes. What are the steps that the coding agent would have taken? And let's kind of contrast the direct coding agent version versus the Waddle harness in the loop version. [00:18:06] S2: So what just happened was that in a series of turns, Astra received the images from the cameras, and it outputted the end effect of pose for the robot to go to. And it does it over multiple turns to complete the task. This is less of code as policies, but more of tool calls.
00:18:06 · Jay, Vincent
[00:18:06] S2: So what just happened was that in a series of turns, Astra received the images from the cameras, and it outputted the end effect of pose for the robot to go to. And it does it over multiple turns to complete the task. This is less of code as policies, but more of tool calls. [00:18:27] S3: The unscreen part is pretty repetitive, so you could actually automate a lot of that using code. As as well as actually approaching the bottle, picking it up, like, lot of these things are pretty deterministic once you've done a task quite a few times. The interesting part is where you build in the variation into the coder's policy graph. And so we have a few points of variation. This can come from, for example, when you detect an object. Maybe you use a VLM as part of your tool call. Or when something fails, for example, how do you check that it's failed? How do you then potentially do something else depending on the failure? Like, there's kind of more flexible responses. We tend to put a VLM inside a loop to ensure that, know, while while the coding graph is sort of deterministic, there's points of variation that allow it to generalize.
00:19:13 · Francois, Vincent
[00:19:13] S4: The biggest insight I think I've had, a change in worldview on how to perform machine learning and how to get the rest of the distance on AGI, was, like, a lot of conversations I've had with François Chollet. And in 2020, maybe even 2018 when he did On the Measure of Intelligence, he talked a lot about transduction versus just wrong, versus program induction. And transduction just means that I'm learning a function theta that maps from x's to y's. And why is that wrong? It's just slow, and it's information inefficient. And so to go from Xs to Ys, I need a lot of pairs of Xs and Ys. And if I have a very small amount, then you need a lot of inductive bias, and what you need to act is a good generator to map from Xs to Ys. So now my theta takes in three N pairs of Xs and Ys and emits the function that maps from X to Ys. And that's what code is. So that's... Like, if you give me a coding interview, even a whiteboard, you say, okay. Here's a pudding problem. Here's some examples. Okay. Cool. Write the function. And then we are emitting an f that will map from x to y. [00:20:17] S3: And and this is quite interesting. Right? Because I think... I mean, this is a lot of the traditional program synthesis literature. Mhmm. And I feel like a lot of reason why those methods didn't work as well was because that inductive bias, as you said. Mhmm. Right? It's... You're shifting the difficulty of the problem from finding that mapping to finding the right inductive biases to map it to this, like, smaller code to then do the mapping. [00:20:37] S4: Mhmm.
00:20:37 · Francois, Vincent
[00:20:37] S4: Mhmm. [00:20:37] S3: But finding that set of inductive biases is super hard. But maybe that's what Astra is buying us. Like, even in Cox Eye, there's a giant shift in the literature from, you know, specifically neurosymbolic methods to just using Astra to write code to map. And so maybe that's a cool [00:20:51] S4: way to think about this.... Neurosymbolic. That is neurosymbolic. We have neurons, and then they're emitting symbols with