00:24:46 · Jim Fan
So it's a technique that allows us to train a high degree of freedom, robot hands, to do pen spinning at human level, and I want to admit that I am a subpar human. I can never really do pen spinning, and I gave up a long time ago in my childhood. So it's really glad to see my AI avenging my poor skills. Right? How do we get the data to train this? The answer is there is actually no supervised dataset. It's all done by reinforcement learning in massively parallel simulations on the GPU because, luckily, all the physical equations are mostly just matrix modifications. So you can run this on CUDA, simulate reality at 10,000 times faster than real time. And then you can transfer this to the real world. And how do we do that? Through a process called domain randomization, where for the 10,000 different environments, you can vary the gravity, friction, and weights, and then you follow the simulation principle. So suppose an AI model is able to master a million realities with different physical parameters, then it's highly likely that it's gonna zero shot the one million and first reality, which happens to our own physical world. And just to kind of show some quick videos, we can run like a robot parking simulation, and then it transfers to real world. We can control a four finger hand to manipulate a cube, and this one is most interesting. We have a robot dog walking on a yoga ball and trying to stay balanced, and actually, the yoga ball is very hard to simulate. It's bouncy, right? It's soft, but actually, we don't even simulate that. It's just a hard sphere that we do, but we do a lot of domain randomization, and then, you know, we're roboticists, we're weird, so we just transfer and run it in the streets. Yeah. It's an actual robot dog running a yoga ball in the streets. You know, sometimes I feel like I'm the director of a Black Mirror episode. Yeah. It's it's just so weird. Actually, my friend, after seeing this video, he told me that he tried a yoga ball on his own dog, and it couldn't do it. So in a sense, we have achieved super dog performance. And of course, we can scale this to a lot more complex embodiment, like an army of humanoid in sim, and these humanoids are going through ten years worth of intense training in only two hours of simulation time. And then with that much data, we can train a 3,000,000 parameter neural network. No.
00:27:28 · Jim Fan, Molly Welch
That's not billion, only 3,000,000 parameter to control the robot body to stay balanced, walk, run, jump, and dance. So basic... Basically, this small neural net that we train in simulation, it can capture this kind of subconscious motor coordination that we, as humans, do all the time, and it runs at millisecond scale on a GPU onboard. So this is actually shot in the NVIDIA headquarters, the Voyager Building. So if you visit Voyager, you might run into these little guests, and, you know, don't be a stranger. Wave. Say hi to the robots running around. And also don't forget to visit our robot hospital. These little guys work as hard as our researchers, and, you know, no robot harmed during filming. [00:28:19] S0: Too good. Another day in Silicon Valley where you see a robot dog on yoga ball. You see a robot getting CPR at NVIDIA HQ. Yes. We live in an amazing world. [00:28:33] S1: Only happens in Silicon Valley. We are the weirdest people. [00:28:36] S0: Exactly. We're very weird. We're very weird. Taking us in another but related direction, we talked about synthetic data. We talked about Teleop. I wanna hear your view on world models, which is this newer paradigm, pioneered by some of the folks, a few of the folks that we've talked about today and, and may have a role in the data... In helping solve the data problem for robotics. I... I'd love to maybe hear you talk a little bit about world models, where you think world models fit into this overall picture, whether they can help on the data side. I'd love to hear your views there.
00:29:15 · Jim Fan
[00:29:15] S1: Absolutely. So first, let me kind of define world models. I think the term means different things to different researchers. And to us, world model means this kind of pretrained video foundation model that can take as input actions and then predict the future in pixels. That's an operational definition of world model that I I work with on a daily basis. Now why are these models so powerful and and and so useful? I I really like to think of... Just to make analogy, I really like to think of video world models as kind of like Doctor Strange. Right? They have seen almost every possible physical phenomenon from billions of Internet videos, and the model basically learns a superposition of visual realities. It's just like LMs learns a superposition of all kinds of different personalities from all the novels and articles it read on the internet. Right? So the video models is a superposition of visual realities. And when you prompt these models, you basically collapse the infinite possibilities into one chosen future. So in this world, in this way, these video generation models are actually counterfactual simulation engines. Right? Like, they have seen all the physics, and they know how things should act given certain actions. So let's say you drop a bottle of water, you drop a cup of water, it will know that the water will spill. It has this predictive ability. And for robotics, this ability is super important because actually, for humans, when you do certain actions, right, you have this mental model of how the actions... Like, what kind of results would the action yield? And then you plan implicitly to optimize for actions. So we see this world model as kind of an alternative and a very powerful new way to either generate data or even become the backbone for a robot brain. And I would like to actually share screen again. I want to show a very recent work from our lab. So, yeah, let me let me share screen. We actually just showed this video, like, a few minutes ago, and, actually, I I I tricked all of you. There is not a real pixel in this. It's actually generated entirely by a video world model. So we call this idea and and the algorithm GR00T dreams. Right? Like, does humanoid robot dream of pixelated sheep? That's where this this name is inspired by.
00:31:52 · Jim Fan
So how it works is you take a pretrained video generation model, and then you can fine tune it on the robot data that you you you collected. And we found that because of all the pretraining on billions of Internet videos, this world model is actually able to learn the constraints, the physical constraints of the robot, all the mechanical kind of parameters of the robot, and then it's able to simulate how the robot will behave given a small amount of post training data. So here, on left and right, it's actually the exact same initial frame, and the only difference is you give it different props and then to a similar different futures. So on the left, it's pick up the apple and place on the pen, and on the right, pick up the can and place on the pen, and the model knows how to follow these languages, and all these pixels are generated. And also, the other good thing is, because it's an end to end learned model, it doesn't care how complex the scene is, and I think that's its strongest advantage over traditional graphics pipelines, where the more complex the scene is, the more effort artists need to put, and also the more compute you need to do ray tracing and so on. So here is a much more complex scene. The video world model doesn't care. It's gonna simulate just as well. And then we're able to run this on a lot more diverse scenes, and you can see, like, all the reflection, right, all the complex reflection on the silverware is able to do that without implementing a ray tracing algorithm explicitly. And then using this pipeline, we can generate lots and lots of synthetic data that we call neural trajectories. And, you know, we can add this as if they're real robot data and then train on those data together, and we'll be able to perform a lot better on the on the robot task. So I would say that work model is still at at its early stage. We have some initial ideas, early ideas to make it work, but a lot more research needs to be done.
00:34:09 · Molly Welch
[00:34:09] S0: Very cool. Thank you for sharing that. And and, again, to see... Amazing to see the demos. You mentioned robotic brains. I wanna zoom out and talk a little more broadly about robotic foundation models. What are the principles to build robotic foundation models, and where are we in the development of robotic foundation models today?
00:34:29 · Jim Fan
[00:34:29] S1: Absolutely. And I think I'm very much following the vibe that brought me you know, here at this moment in the first place. So I mentioned that I was basically choosing hard problems and then selecting for simple and elegant methods because they scale the best with compute and data. And it's the same principle that I'm applying to robotics, and we can summarize that in four words, data maximalist and model minimalist. I mentioned there are many different ways to collect the data. Right? We can collect through tally operation. We can generate data using simulators like Isaac Lab, and then we can also generate data using world models. So the data pipeline can be very complex, but the model should be as clean as possible, as end to end as possible, because in this way, you can leverage all the different data sources without building separate training recipes and loss functions for the models. So it takes, as input, pixels and then directly output the motor control signals that you saw, like the the high dimensional continuous signals. And earlier this year, we open sourced a model called GR00T n one that operationalizes this this approach. So the GR00T n one model, it's got system one and system two. Right? System two is a VOM that does reasoning. System one is actually a diffusion model that renders the actions at more than a 100 hertz. And despite it's got this kind of modular approach, it's actually an end to end fully trained model through backpropagation. So that's how that's how we approach it, and we're we're building an ecosystem, so we open source everything. We open source the training recipe, the model parameters, and if you search for isogrut online, you can download the model and try