Dream Machines: Physical AI, Robotics, Being Human
00:30:24 · Aravind Kandiah
[00:30:24] S0: so most of the time, that's act... Actually where we start, which is like, can we show you proof that it just works in the real works in the real world? Right? And the the means by which we get there, you know, there's like a whole bunch of things that we can do. Right? So it always starts with, like, bringing in the end user and, like, trying to convince them. And most of the time, they are not even, like, AI engineers. They may be a c suite at, a a large MNC or it may be an end user that is just trying to automate some critical task in a in a factory. Right? So then it's like, okay. I have these 10 tasks that I need to do. Can you show me that these 10 tasks can be done, trained entirely in simulation? So it just becomes a train in sim, show in real life, or if I test in SIM and I test in real, are the performance equivalent? So that's like the the proof is in the pudding part of things. Right? Mhmm. It's like how do we showcase it to customers? It's just by doing it for them. The second part where your question is like, okay. How do you break down this whole symptom real thing? And I think you hit it exactly in the head, which is different domains have different thresholds of symptom real affordances, if you will. Right? And symterial is, like, these four big parts. There's, like, the perception symterial, which like, how real does it look? There's the dynamics, symterial, which is, you know, if I move a robot arm in simulation, is it the same movement in reality? And then there's what I call, like, the object to robot interaction, which is like contact forces. And the last one is, like, all your crazy deformable cloths and all that. Perception, is something that we have solved, like that symptom real gap. So, like, does it look realistic? Right? And you can think, like, you know, you can use world models. You can use diffusion models. You can learn from the actual sensor imagery and transfer that onto your simulation. The second one, which is the robot dynamics, there's a lot of new solvers coming out like Newton and all that that... I wouldn't say they are one to one, but they are good enough approximation that customers have found it a meaningful test and meaningful training data already. Right? So MolmoAct, like this paper that came out as well, showed the real to sim performance deltas are actually pretty good as well.
00:32:41 · Aravind Kandiah, Evan Helda
And the most interesting thing I think I've seen so far is what I call the sim to sim to sim to real, which is you train on one type of physics engine, you show that it generalizes to another physics engine, and then you show that it can jump to another physics engine, and therefore, it improves your generalizability when you actually jump to reality as well. Right? Because reality is just another type of physics engine. Right? So you're training a policy that can generalize across these types of physics engines. [00:33:11] S1: Interesting. [00:33:13] S0: Right. [00:33:13] S1: It's cool. It's... You you put it in different physical realities along the way before it gets to, like, the ultimate one. And let it deal with, the the the variety and the the unknowns, the uncertainties, etcetera. Right?
00:33:13 · Evan Helda, Aravind Kandiah, Charles Wong
[00:33:13] S1: It's cool. It's... You you put it in different physical realities along the way before it gets to, like, the ultimate one. And let it deal with, the the the variety and the the unknowns, the uncertainties, etcetera. Right? [00:33:25] S0: Right. So it's like you... The thing is, what are the things that we can do at the speed of software? Right? You remove as much real world bottlenecks as possible Mhmm. To do things as fast as you can at the speed of software. You just improve iteration time. And all of these are approximations, but if you can get the system to be a very, good approximator of, like, the physics, then you can also approximate, the real world physics and just generalize pretty well as well. Right? So that's kind of, how we've been able to overcome all those gaps. But, you know, coming back to the first part that I mentioned, that is majority. It's just like showing customers it actually works. Mhmm. And that's something that we have we have gotten to be very repeatable where data comes in, we can calibrate the system, and then put it out again. [00:34:13] S2: Yep. Well, I will add as well, like, know, and Aravind is hinting at it a little bit. Like, there's a technology when it comes to Centro. There's also the trust aspect. [00:34:20] S0: Mhmm.
00:34:20 · Charles Wong
[00:34:20] S2: Right? The human part of it. You could show, you know, skeptics, you know, countless benchmarks, metrics, and they still might not feel comfortable deploying their robot into their environment if it's only been trained in sim. Right? They still need a seal of proof of somebody. It's all it's all transitive trust. Right? Mhmm. And one thing that we've been exploring as a company is not just working with developers, but also working with regulators. Right? You can think about heavily regulated spaces like maritime, in in this case, like, you know, warehousing, manufacturing, all that. The regulators, the certification societies, and all that who approve every autonomous system that goes out before, you know, before it's actually deployed, they, you know... And and you could credit them, they've been trying, but it's it's hard for them to to regulate autonomy. Right? There's so much that could go wrong from the perception stack of the controls to everything. Yeah. Right? And that, you know, they're not AI native. Right? They've been... For them, certification has been ship hull integrity, you know, propulsion systems. Mhmm. Right? And now we're asking them to evaluate, you know, perception and autonomy. That's a totally different ask. Right? The only thing we've been exploring is there's a gap here. Can Bifrost fill it? Can we be the platform for developers to battle test their systems and be the same platform that does the precheck so when they go to the evaluation authorities, you know, they already have something ready to go. Right? And there... There's a common language now. Right?
00:35:48 · Evan Helda, Aravind Kandiah
[00:35:48] S1: Got it. Got it. [00:35:49] S0: And just double clicking into Charles' point because I think you brought up a really good point, which is real to him is sure it's a technological problem, but I think more importantly, it's actually a trust building exercise, which is how do you establish a set of experiments that both people can have, like, very high trust in? And, like, how do you incrementally grow that set of experiments? So it's like, okay. You know, we trained in sim. Let's have these 10 scenarios where we demonstrate it. And, like, getting all the stakeholders to actually align for these actual robot deployments in, like, mission critical areas is actually really tricky because there's multiple stakeholders that we're trying to bring to the table. Yeah. [00:36:31] S1: So I can can only imagine that that sales process, especially if that last guy who's responsible for the safety element in the funnel, putting the machine in the field, especially in these defense areas that you guys seem to focus on. [00:36:45] S0: But I think that's the... We, you know, we actually turned it around and it became our unique selling of value proposition. Okay. Right. So it wasn't actually a problem. It was actually the... That was the reason why people are coming to us because it's like, oh, actually, this makes certification much easier. [00:37:02] S1: That's compelling. Yeah. His anxiety was way worse before.
00:37:02 · Evan Helda, Aravind Kandiah, Charles Wong
[00:37:02] S1: That's compelling. Yeah. His anxiety was way worse before. [00:37:06] S0: Way worse before because, like, so many unknowns. Right? But now it's like, okay. Now I know what's tractable. I know how to approach the problem. Yep. And then you're getting all the stakeholders to sit around the same thing, which is like Yeah. [00:37:17] S1: Makes a lot sense. [00:37:17] S0: Okay. These are the set of scenarios. Right? [00:37:19] S1: He he now has a story to tell to his boss. Why does he gonna be safe? [00:37:23] S2: Exactly. [00:37:23] S1: He didn't really have have that justification before. [00:37:26] S0: And in my perspective, I feel like this is one of the biggest problems in physical AI, which is like, you have this system that has a promise of being able to do anything and everything, but no one is defining the bounds of the system. It's like, what can't it do? What can it do? And it says expectation mismanagement. [00:37:48] S2: Yeah. [00:37:48] S0: Yeah. And that's like the the big thing that we always see between, like, a vendor who's, like, building and selling the robotic system and the actual end user. And I said, oh, but you told me it should be able to handle this thing. Why can't it handle this thing? Right? Like, self driving, they got away with this by, you know, having levels. It's like level one, level two, level three, level four. [00:38:08] S2: Yeah. [00:38:09] S0: But we're still... Like, there's still some gray areas and, like, oh, actually, you know, it's Tesla's thing, this or that.