00:08:31 · Steve Xie
[00:08:31] S0: Yeah. Totally. So I I would say, yeah, both of us are building the infrastructure. So I really think that's, you know, simulation is such a big a big problem to solve, but also a big opportunity. And, everyone should be building the simulation infrastructure. And by simulation infrastructure, I mean, you know, the physics solver. So so I mean the SimReady Assets. I also mean the framework, and also the applications to generate successful simulation. Right? And by successful simulation, I I I I don't mean, the typical graphics, pretty much like the gaming simulation rather than the physics. Right? Because, so first, we need to understand about the mission of, simulation for physical AI. The mission is not to, light people to enjoy another game, right, to to, to to enjoy the game in virtual world, but to help accelerate, deploy robotics in the physical world. So physical realism, physical accuracy, physical quality is a key of simulation for physical AI. Right? And that's actually where, you know, we are putting the most of the work. And so I would say that, you know, for Lightwheel, you know, we are focusing on... In order to provide a cycle simulation, which provides the most high quality physics. Right? And in order to provide the... By simulation for physics, then we need a few things there. First, we will need to have a really, really awesome physics server. A really, really awesome physics server. Right? The physics server needs to be real... Like, not only realistic, but also highly efficient to run on GPUs because we need to run them on parallel, right, at a very, very large scale. And, also, the software needs to be connected with a solar ecosystem. Right? It couldn't be a a a... Like like, one independent solver, for example, maybe a rigid body solver, and that's it. No. It it it needs to be connected to every other solver together as a ecosystem. Right? So that's how we collaborate with Newton of NVIDIA. Right? Because Newton is really a great... The next generation GPU accelerated physics solar ecosystem. Right? And by doing that, we not only build SimReady Assets on top of it, we not only calibrate the server, but also we are building the server ourselves on top of a new thing. Right? Together building this open ecosystem. So I think the openness of the ecosystem is really, really important. Right?
00:11:17 · Steve Xie, Jonathan Stephens
So so so back to your point, you know, NVIDIA is building simulation infrastructure as we are too. Right? And all of us need to work together in order to make simulation successful. So then I will say this is the part of the sim... The the fixes over, which is extremely important. The second thing is actually the SimReady Assets. And I think to a lot of people's misconception, especially if the people is coming from gaming industry, coming from self driving industry, they may not put that much attention of SimReady Assets because they can feel... It's just typical technical artist workflow. Right? And I I I I even get people asking me, hey. I can buy this asset, $9.9 on websites. Right? And I was like, no. It's totally different. Right? The $9.9 is just a virtually realistic asset, but what we are looking for is a physically accurate asset. For example, a fridge. Right? You need to have, you know, the magnetic force of this fridge. You need to have all the hinges, the joints. Right, in order to get it, accurate, to connect with the physics solver, in order to generate the most accurate forces. Right? Mhmm. So the SimReady Asset is extremely important because you need to calibrate all those physical parameters, the forces. But, also, you need to, you need to optimize it to make it highly efficient to run on GPU physics hours. So that's the SimReady Asset, which, you know, Lightwheel specializes in. Then there is another thing. [00:12:59] S1: Is that where we started, by the way? Is that where is that where Lightwheel started with the assets?
00:13:04 · Steve Xie
[00:13:04] S0: That that... That's that's, exactly where we started because, you know, we solve a problem that no one else wanted to solve. They they they feel that they are looking for other people to solve, but they are not solving themselves. But the more I look into it at the time, the more I found out it is truly intriguing. It is truly difficult. It is truly difficult. The more I work on it, the the more I look into it, the more I found out it is extremely difficult to build. This high quality SimReady Assets at scale. Because, basically, from my part, I love the problems, which is, I would say, like, we we we... Which is both much needed, but also extremely challenging. I... So so I love to solve those high impact problems. So that's how we jump into why and also how we jump into SimReady Assets. Yeah. But I would say that, you know, that's just just, like, a very superficial level of this physical AI simulation problem. Right? The more we work on SimReady Asset, the more we found out... Actually, there's also another core problem need to be solved in order to build the high quality physical accurate simulation, which is the physical measurement. Right? So why is that? So a lot of people may think, hey. You are building simulation, so everything you are doing is in computer. Right? So I'm I'm just looking at your computer screen so I can find out all of your product, but that's actually not it. We are operating probably one of the world's largest, robotics arm factory. Right? With with hundreds of arms of doing automated data collection, manipulation with all the real objects. For example, the fridges, you know, the microwave, the oven, the dishwasher, the table, you know, all the different type of tools, things like that, in a factory. Right? It's a it's a real factory. Why do we need that? The major is factory, we're calling that. Right? Oh, yeah. We we are are calling it the physical environment factory. We are calling it the physical environment factory. I gotta get that right out. Yeah. Oh, sorry to cut you off. No worries. No worries. So actually, we are showcasing a minimum factory in GTC next week, right, which I'm super excited of. But basically, the, the core concept is the more we work on SimReady Asset, the more we found out, hey. We need to scale the data collection of the real world, especially the real assets.
00:15:51 · Jonathan Stephens, Steve Xie
[00:15:51] S1: Mhmm. [00:15:51] S0: In order for us to build scalable, high quality SimReady assets. So how about we start with this factory? Right? So we build a smaller factory, and then we start rebuilding bigger and bigger factories. Right? So I would say this is extremely important because we we we we... Without all the physical parameters we collect from this factory, there is no way we can scale the productionization of the SimReady Assets. Right? So right now, the SimReady Assets is mainly, I will say, 99% automation work. Right? In that, you know, we have the three d reconstruction. We have the automated physical factory data collection of... Measuring all the core physics of those real objects. Also, we are using the USD, which is the open standards of describing the SimReady Assets introduced by NVIDIA and also a few others.
00:16:47 · Jonathan Stephens
[00:16:47] S1: Mhmm. So so let me just kinda reiterate what you've told me because there's a lot of information you've just gave me, which is really good. So, you know, we're we're we're we're building simulation infrastructure at Lightwheel, and you... We started with you have to have high quality physics solver, right? So that's that's basically an engine that's that does the math to make sure that physics interacting within a simulation environment correctly. So I, we have a great plastic bag with oranges example where it's like that plastic bag to get the physics of that. You can, you can make it look pretty in a game, but it's actually got to work exactly as it would in real life to our normal physics on planet earth, or else when you transfer that simulation to a robot in real life, it will fail. That's the second point. Can't even get the physics right in an engine. This falls apart. But then the next step is we have to take these assets. Have a refrigerator we use as a great example, right? Everyone has one, everyone has opened up a door, it takes a lot of force to open it. But there also takes no force once that seal is broken. So we got to make sure that we are building that into these assets. I'm gonna pause you for a second, ask you a quick question on that. Now we got the physics on that fridge, right? How important is it to look exactly like a like a visual replica? Because I I also wonder if that is not as important because we always create vision models that just know what a refrigerator is for the robot. Is that visual fidelity as important as a physics?
00:18:18 · Steve Xie
[00:18:18] S0: So I would say, visual fidelity is also quite important. The major, yeah. Yeah. Yeah. The major reason there is, you know, we are... So, so, basically, though the VR and VAs, you know, they are getting much more, visually data trained, you know, they still need to, you know, look at exactly, the kind of, fridge, to find on the handle and other, places, in order to manipulate. But but here's the thing. Right? The visual quality is not that difficult to build. Mhmm. Right? Because, you know, there... There's al... Already the gaming industry. There's already the the self driving industry. So it is it is... In my opinion, it is a solved problem. It is a solved problem. The difficulty there is more in, I would say, the optimization of efficiency. Right? So you need to understand about the best trade off between the visual quality versus, you know, the the efficiency of, you know, the the the simulation, the data. Mhmm. So that is the most important thing. Right? If we put, you know, a extremely high quality visual there, for example, a movie level visual, it would look extremely realistic. But the problem there is that it would take too much compute, and it wouldn't be super efficient to run at large parallelizable jobs there. And so from our perspective, you know, we are actually closing a loop in that for every new asset we introduce, we actually close the whole loop of SIM to real, but also real to SIM, meaning that, you know, we would actually leverage, we would actually use the data generated from the simulation to train the algorithm and deploy the algorithm in SIM and also in real of the real robotic arm and manipulate about the SIEM object to test, you know, the efficiency of the data and also the quality of the data. By this, we actually understand, you know, where is the, the best combination that we we we should have good visual, but no need to have the best visual there.