00:20:16 · Siyuan Huang
[00:20:16] S2: So for the data zero, actually, there are several key designs for the models. So first is that the brain and controller, they are co-designed to scale humanoid intelligence. So during years of research in robotics, we have significantly found out that the controller is still actually one of the bottlenecks for the robotic systems, including, I think, including the humanoid robot and also the wheel robot. So it's really... It's really critical to incorporate the design ideas of the controller into the design of the brain. So we call it co-design. We co-design the controller and the brain to scale the human intelligence. We also propose several ways of how to scale such kind of humanoid foundation models. So first is to portray on large-scale diverse full-body human data. As Xiaojian Ma mentioned, we designed our own data collection system. I think I discussed this with Michael previously. We have some shared insight in how to build up the egocentric videos data and also capture the human movement we'll introduce later. And second is the real-to-seem-to-real accelerated humanoid foundation model evaluation and enable humanoid agents. So this is a severe problem for humanoid robots because doing the experiment, repeated experiment or control experiment, ablations for the humanoid robot is really, really hard. I think you need to reset humana robot into a certain accurate space and you need to do the repeated trial for H-robot and for the G1-robot, the machine status is also a problem. In order to have rigorous evaluations, we need better infrastructures to do the experiments. I think that's the key to have a better or more efficient iterations for the foundation models. So we build up the real-to-sim-to-real pipeline to do this kind of repeated experiments and try to loop the development of the foundation model with the post-stream and evaluation I think this is quite mature in previous wheel robots, but still early for the humanoid robot. We'll describe this later. And the third one is the data action control. So we believe that in the future, if we want to really scale up the foundation model and we want to improve the performance towards deployable standards, like 99% of access rate or 100% of access rate, We need RL. We need real-world RL to train a humanoid robot to have a better human learning process.
00:23:14 · Siyuan Huang
So the current, most trackers, they use the trajectory-based tracking or controller. So that is not possible. It's possible for the human interventions while during the deployment or during the rollout of the human robot. So we design a Delta action-based controller so that we can enable the human intervention during every rollout or inference time of the foundation models. I think this is a key design. We'll describe detail. In the following stage. So basically these six points are the designer design principle we have proposed in the downtown zero models. And we can have a brief overview of the human intelligence tasks. And here are some like summaries of the tasks we have seen in the demos like operating our turntable. And so this kind of task is quite challenging because like The, like holding the record and put it into a hole, the very small hole is requires a pretty precise, um, uh, interactions. And also we have this kind of wujiheng. So it's really hard to move the base. And then move the arm, then insert that record into the small hole. And we find out this is one of the most challenging tasks for the zero models. And also making the bat. So this incorporates interactions with the fabrics doing the Moving the pillar and then move the joysticks, interacting with the fabrics and also tackling some randomization or the variations of the forms of the fabric materials. And there are some more tasks like, I can introduce this briefly, like picking up objects from the ground, requiring the robot to have the capability to do the balancing. And also operating the dishwasher. Yeah. Also, as Chris just mentioned, it needs the Delta Zero to brace its left hand against the shelf for this kind of support to have enough force to opening the door. Yeah.
00:25:28 · Michael Cho, Siyuan Huang, Chris Paxton
[00:25:28] S0: I'm curious, like this bracing against the... Sorry. Is that learned from the human data or... Because human probably don't need this, right? Human is strong enough. Our core muscle is strong enough. But probably the G1 needs it. Is this learned from the human data or is it something else? [00:25:42] S2: Yeah, this is learned and emerged from human data. So the humans have various ways of opening the, the dishwashers. Like we have a better core, so may not need the, uh, to do the bracing stuff, but, uh, if you want only using the G1 to do the, like, teleoperations, so we have, we need to have a better way for the robot to, to have a like stable core. So bracing is one of the, one of the way, or have this kind of Compliance to open the door. [00:26:13] S3: So the teleoperators basically are, or not teleoperators, data collectors were aware of the limitations of the G1 and collected that particular motion. Yeah. Yeah. Okay. Makes sense.
00:26:24 · Siyuan Huang
[00:26:24] S2: And then like operating a stab trash can. So this requires a very precise foot movement. So when we talk about human robots, we need to make use of the foot, the feet. To walk through different terrains and also using that to like step on a trash can so we can like, so basically for the scene in the data zero, we did not modify any object from any object or furniture. I know in some demos from certain companies or research demos, we were modifying Maybe the articulations to make it easier for the robot to open, to step on, have a more precise movement. But for this one, for Delta Zero, we did not do any modifications to the articulations or forces, structures of the objects. So it's really exciting to see that a robot can really step on a real trash can and then open it smoothly and throw out the trash into the bin. And also sitting on the sofa. So this requires the whole body coordinations. Like the robot needs to see the positions and the plan from the sofa and then sit on the rest in the hand. Yeah. And these are some summaries of the tasks from the zero. Actually, I think the principle is that we want to have a full coverage of how humans can interact with a certain room. And we want to try to prove that the human robot can really coverage like most of the challenging scenario or Simple, medium and challenging interaction with the room. So we can really trust the robot can do the daily tasks around us. And for the later part, here's some more details about the model design and about how we propose to scaling the human intelligence. And the four sections is about how to co-design the brain. And the controller. So the main belief is that we can have a learned whole body controller, right? Using the reinforcement learning in the simulator and doing the same tutorial. And we can skating up the motions, which is the source of the, the, the, which is the learning source of the controller. We can scan out the motions, we can observe the scaling law of how the controller can behave and can develop across different Amount of hours of motion collections. And using that controller, we can co-design the brain. And we actually use latent word action models. So the latent word action models, it takes the images and also the noise as an input. And then we can have the instructions. We can have the videos as image patches.
00:29:20 · Siyuan Huang
And then we predict the actions. The actions are the motion commands. So it's not just only the motion skeleton, it's the motion command. It indicates how the robot should control its bodies and then try to control the robot. And then the controller converts the motion command into the whole body control signal. And then this can convert the command and the images and instructions into the graphite or into the actions for the robot. And during the inference, we can actually have the human interventions we introduced before we use a data action design. So the humans can use a PICO or a teleoperation system, the motion capture, they can intervene when they think the robot has made some mistakes. So this is critical for the future development and the human in the loop learning. So this is the basic structures of the model. And in the training time, we have designed certain dynamics, like the model in Cosmos, like inverse dynamic models. So we designed this kind of learning dynamics to have a like more like fully, fully exploration of the training data. In this version, we have about, um, you know, 10,000 hours of egocentric and human data to train the Delta-0 Foundation models. And during inference, we can, do, similar to the training we can use the multimodal self-attention to attend the images and instructions, and then we can generate the action output, convert them into the command and control then the GYN robot. So this is the design. And here are more breakdowns for the design principle for the controller. So actually we discussed about control a lot during the last several months. Especially for the development of humanoid models. First from the controller, right? Do this kind of dance, performance, and have amazing behaviors from humanoid foundation models. So we have some Certain principles, design principles. The first is that the control infrastructures should be designed for the brain learning. And then the second is that we can try to make the body movement, body's capabilities available to the policy. So this, this try to make sure that the capability can be connected between the controller and the brainer. And in detail, we have certain requirements for this kind of controller. So the first is the controller can provide an interface for the intent and intervention. So this motivates DeltaBit, DeltaController.
00:32:17 · Siyuan Huang
And second, the controller is a whole body. So we have seen controller that separate the upper body and lower body. That is easier for the VLA models or WAN models. But we really need a whole body controller for what So this is critical for a human robot and critical for the deployment. So this is critical for a human or robot and it's critical for the deployment. So we have seen demos from papers, actually We have a lot of papers as well. But from these papers, we didn't observe a very reliable performance from the controller, which means that the robot can have some jittering performance. They can have turning performance. So this kind of performance, although you can hide this kind of performance during the demo, right? We all know that the robotics demo is tricky, but actually if you want to turn that capability into the deployment, into production, This kind of stability is unacceptable for the humanoid foundation model. So we want the controller to have really reliable and stable performance throughout different tasks and throughout the tasks that requires compliance. This is pretty important for the humanoid foundation model. And the fourth, the first one is execution fidelity and responsiveness. So we, we want the controller have like a better accuracy, especially for the task that requires high, high, high That's just accuracy, like the turning, like the turntable test. And the fifth is the compliance. When the robot is trying really to use more, more forces for opening the door or for some heavy, heavy load. So we need the robot to use extra forces for the interaction. So it requires the robot to have the compliance while maintaining the balance of the whole body coordination. So these five principles are the principles within We think we need to maintain during the scaling process of the controller and it's important not only for the performing certain dance and important for the local manipulations, important for the future deployment of human robots. And here are some examples of Delta action intervention. And it's pretty interesting. We can see that in each task of the deployment, we can roll out the policy, the policy. And then if we found out some problems during the We can stop the actions, stop the policies, and we can do the interventions. And the intervention is basically on the delta position of the anti-factors.