The Robotics Stack
00:05:44 · Oskar, Varun Nair, Leo
[00:05:44] S0: Yeah. Neura Robotics, for example, from Germany, they raised with Tether. Somewhere I read about it that they're kind of planning to put a wallet essentially on each robot, but as you say, you know, it's very far into the future that I see a robot working or going somewhere and paying with their undevised wallet or something like that. [00:06:02] S2: I definitely see like there are potential use cases, and it can happen, it's just that you're in the robotics field, you're like, okay. Can we get, like, more deployments actually, like, out there? And that's a p zero question rather than, like, can we get wallets on the robots deployed? Because that's, a p two question. Yeah. [00:06:18] S1: Yeah. One question I have as a follow-up from your background and your experience is how do you define physical AI and where do you draw the lines between classic robotics, foundation models? Is physical AI the same thing as embodied AI? How do you define the space?
00:06:35 · Varun Nair
[00:06:35] S2: I think they are different. AI so far has been on our screens. It's been in the world of bits. And physical AI is all it's saying is that, hey, can I make intelligent decisions in the real world? I think Waymo, Zoox, Robotaxi are fantastic examples of physical AI, and they're even embodied in a sense, right, they're a core. But I wouldn't call them robotics, even though they're similar. I think it's important to make the distinction that there is robotics, which is a big part of the narrative of physical AI today, but physical AI in general is just any intelligent decisions being made in the real world. We have solved rigid body dynamics through simulations, so that's why things like Waymo's can exist, quadrupeds like Spot can do navigation and locomotion very well because again, bodies have been sort of figured out. But dextrous manipulation is still on the frontier of things because in robotics, that stuff is not fully solved yet. But if you've heard of companies like Spectre, they are not in the robotics field at all. They are an intelligent system for facilities. It plugs into your cameras, your other sensors, everything in like, let's say a factory or a warehouse or a building. It's like Palantir for the building. Like it understands everything. Can say like if there is a robber outside, there's a car that's parked that shouldn't be there, it can like intelligently pick up on those things. Assume a building has just come alive and it is sentient and intelligent because of all the sensor data it has. I think that's physical AI. Like in the Venn diagram, it's clearly physical AI, it's clearly outside of robotics. And that's also a big problem that's getting solved. So largely speaking, physical AI is the entire domain where the world of atoms makes smart decisions, but robotics in specific is the frontier because it's still unsolved for in a lot of ways. And that would be embodied AI, brilliant companies doing some cutting edge work around that. And I would break that down into data, simulation, deployments and foundation models, obviously, that I like to learn from these, broadly speaking. Yeah.
00:08:31 · Leo, Varun Nair
[00:08:31] S1: I see. That also bridges the gap with something interesting you said earlier. You've been into data, you've been into deployment, and lately you've been interested in SIM simulation. Usually, people are into simulation before doing deployment. So how did you, like, get into real world stuff before getting into into SIM? [00:08:50] S2: I mean, you could go either way. The thing is you do deployments or you just, like, evaluate in the real world and you see that policies don't work very well and you're like, hey, there might be a better way to scale this up. There might be better ways to run a thousand different trajectories rather than resetting one manually again and again. And that's how I found my way. It's not like I hadn't dabbled in SIEM before, I have, but it's interesting because you have data on one side, you have deployment on one side and you have SIEM sitting in this middle where learning in SIEM is smart, but there is a real gap that exists between transferring a policy that exists in SIEM versus in actual deployments. But it's the other way too. When something fails in deployments, it needs to be rerouted back to SIEM as well to make the simulation even better. And that's a piece that's missing. So it's not just that SIEM informs deployment, but it's also that deployments will inform how to make better SIEMs. I'm actually putting something out there about this later today or tomorrow. So hopefully it'll give you better context about it as well. Yeah.
00:08:50 · Varun Nair, Leo
[00:08:50] S2: I mean, you could go either way. The thing is you do deployments or you just, like, evaluate in the real world and you see that policies don't work very well and you're like, hey, there might be a better way to scale this up. There might be better ways to run a thousand different trajectories rather than resetting one manually again and again. And that's how I found my way. It's not like I hadn't dabbled in SIEM before, I have, but it's interesting because you have data on one side, you have deployment on one side and you have SIEM sitting in this middle where learning in SIEM is smart, but there is a real gap that exists between transferring a policy that exists in SIEM versus in actual deployments. But it's the other way too. When something fails in deployments, it needs to be rerouted back to SIEM as well to make the simulation even better. And that's a piece that's missing. So it's not just that SIEM informs deployment, but it's also that deployments will inform how to make better SIEMs. I'm actually putting something out there about this later today or tomorrow. So hopefully it'll give you better context about it as well. Yeah. [00:09:48] S1: Yeah. We recently discussed with an engineer saying that simulation was still struggling to accurately represent actuators, for example, since robots are pretty much a bunch of actuators. It tells you a lot about how accurate sim can be. Right?
00:09:48 · Leo, Varun Nair
[00:09:48] S1: Yeah. We recently discussed with an engineer saying that simulation was still struggling to accurately represent actuators, for example, since robots are pretty much a bunch of actuators. It tells you a lot about how accurate sim can be. Right? [00:10:02] S2: Yeah. That's really fair. And the frontier of sim is getting better every other month. Nvidia, they swapped out their PhysX engine for Newton that is supposed to be doing deformables, fiction and contact dynamics better, so it will definitely improve in the next coming times. We have companies like V sim, the creators of Isaac Sim are now building their own simulation in The UK. It's a frontier that's definitely getting worked on. And it might also eat up the data business, to be very honest. If simulations work and are calibrated well enough, think about it this way. If I can model the real world in sim, why can't I just use sim as data? Like, it is totally possible. Like, these are, like, components that interact with each other but can also eat into each other very much so. [00:10:43] S1: Do you include world models such as World Labs Atlas into your simulation category?
00:10:43 · Leo, Varun Nair, Oskar
[00:10:43] S1: Do you include world models such as World Labs Atlas into your simulation category? [00:10:50] S2: That's a good question. And I might not be the best expert to talk about this, though I have spoken to some of them, but it's a spectrum. So you have things like Isaac Sim, MuJoCo on one side, which is like simulating every single object, and then you have world models on the other side. Both have their pros and cons. Have world models can like generate fluid, like fluidity better, so rather deformable objects better because it's pixel generation. So it is actually helpful to have these three d rendering like Atlas exist in world models and that is part of simulation. But when you want to do rigid contact dynamics and physics and understand the physics of it, world models aren't the best in understanding physics because it's, again, it's learned from pixels. It's not learned from action data. It hasn't learned the inherent physics behind it. So yeah, they are two separate things, but a good simulation needs to leverage both of them rather than using just one of them. [00:11:43] S0: I see. Maybe just to understand your role a bit better right now, you said you invested in a friend's company. Are you like creating a company yourself? Are you kind of stealthing around? If you have some practical experiences or some recent learnings that you can share from actual deployments, that will be super interesting as well.