00:58:44 · Siyuan Huang, Xiaojian Ma
[00:58:44] S2: I think it's 90% is done by the agent and 10% by some human neighbors. [00:58:58] S1: So the way we're doing this is actually we're asking the GD6 extra to do this real-to-scene simulator parameter tuning. We actually set a goal for this agent model is that we're actually providing this agent with a trajectory, collecting the real world. The trajectory including both the robot movement and also what the robot actually sees. So basically the camera observation. So what we're asking the agents to do is to try its best to replicate the same motion in the simulator to not just So that kind of metric actually forced the agent to do his best to try to tune the parameters to make sure that the motion can be replicated in a faithful kind of way. But another thing you can see in this is that we are actually tuning the physical parameter of this simulator for this certain motion, for this certain task. So what we're trying right now is that we're trying to expand this approach to other set of tasks, other set of motions well, but what I'm not sure is that maybe when the agent's trying to just replicate this very different kind of motions, it might lead to some contradicting parameters at some point. But we're still working on that.
01:00:16 · Siyuan Huang
[01:00:16] S2: For the last part of the blog is about how to scaling experience. So by manual tasks, we have seen tons of real -world RIL works that are trying to improve the policy successfully from 80% to 99%, 100% for deployment, right? And for humanoid robot, we believe that this kind of system is also pretty cool in the future. So we incorporate this kind of mindset into the design of the controller. It's a Delta Action Controller. And then use this kind of Delta Action Controller, we can do the intervention during the rollout of the policy. And then we can use this kind of human loop data to retrain the policy. And then we can retrain the value model, like the 0.6. And we can learn the values and advantage from the data. And the left side shows a demo of how we can using the value model to estimate the value of the robot test. So we can see the value actually decrease. When the model fails to grasp the cup, and then it increases after it successfully grasps the cup. So I think this is a beginning of how we can really use the Rural Isle to improve the performance of the Himalaya robot. And here is another test. We show the performance of some old generation scenario where the original behavior cloning orientation learning might fail. In this scenario, it's about 20% of performance. Using the human-in-the-loop learning and incorporating this data into the next round of training, we can improve the performance again and again, and then increase the performance into like 65% in this kind of scenario that is easily attached to all the out-of-distribution cases.
01:02:30 · Michael Cho
[01:02:30] S0: I suppose the humans in the loop, does it really need to be in the real world? Because literally the video earlier, you have such good simulation environment, very good parameters, the agents figure out all these parameters. You can also do all this just in sim, right? I mean humans in the loop, like humans doing stuff in sim.
01:02:50 · Siyuan Huang
[01:02:50] S2: Yeah, I think that's, um, if you have like perfect replica of the 3D scene, that's, RL can be done in the simulator. But, uh, actually we, um, a lot of scenarios, we don't have like the full replica of the 3D scenes. I think in that case, if you try to improve the policy into 100% successfully, the real RL is, is critical. I would say it's a must. Because I think we are currently in a stage where we have more and more stable humanoid foundation model system, local manipulation models, but we have not turned that capability into deployment yet. Because we are in lack of this kind of like recipes or designs for turning the 60 or 70% of accuracy into 99%. I think if we have this kind of system, we have these recipes, we can really turn this kind of model capabilities into deployment, into a stable deployment. I think this is also one of the goals we want to achieve at Delta Intelligence. Yeah, I think that's basically the whole picture of the zero block. And in the future, we want to do two major things. At first, of course, is through scanning human intelligence, scanning up the data, scanning up the evaluations, scanning up the human I think this is rarely explored in the humanoid community. And the second one is towards reliable deployment. We have seen sparks in Turning the actions into deployment. In the future, we want to build up better infrastructures, better models, and more stable controller and planner. So we can increase the rate, success rate, increase the efficiency. As I said before, we want the Himalaya robot to have a near Himalaya speed. They are all critical for the reliable deployment. So this kind of deployment system is also one of our vision at Delta. We want to deliver the capability of a human robot into everywhere, into indoor thing, into outdoor thing, into easier and into harder thing.
01:05:07 · Michael Cho, Siyuan Huang
[01:05:07] S0: Yeah. [01:05:08] S2: I think that's the introduction of Delta-0 blog. [01:05:12] S0: Michael, first. Yeah, well, firstly, congrats. I think, you know, it's, yeah, I was kind of mind blown by the very, very long horizon thing that you all showed earlier. Okay. Yeah, I really like the whole co-training between the brain and the whole body controller. I think they kind of need to be tied at the hips. Yeah. Are you guys hiring? Is there something you all want to say for people who are dialing in?
01:05:43 · Siyuan Huang, Michael Cho
[01:05:43] S2: Yeah, of course. We are hiring. We are actually hiring people that have the real convictions for the humanoid robot because I think currently we still have We have the debate about the real robot and human robot. We counter this kind of debate from day to day. But in our mindset that we still run the first principle. We still think that human robot is... Will be the most efficient and most general form for the model, for the research, for the deployment. I think we are holding this belief and I think there will be a turning point after we scaling up this human intelligence into the efficiency and into the capability of the real robot and human robots. We estimate it will be, it will happen in the next year that we'll see. So there is maybe a turning point between the capability and between the deployment potentials of the Shimna robot and the real robot. So we are hiring people. We want to attract talents that has real convictions that a humanoid robot is the form that is the most general and most capable in the future. So if you are interested in this kind of research, this kind of deployment, scaling this kind of humanoid intelligence, please contact us. We have actually some links below of the... So you can also follow us on [01:07:22] S0: X. Any final thoughts if you want to wrap up?