Inquiry about Including Upper Body Joints in Reinforcement Learning for Humanoid Robot

[Ericongic]

Dear Authors,

I have been exploring your humanoid robot project and have some questions about the details shown in your demo. I noticed that in the demonstration video, only the lower body joints of the robot are active, while the upper body joints remain static. I am curious about this design choice and would like to learn more.

1. Does the current design of the project support motion for the upper body joints? If not, what are the reasons behind this decision?

2. Would it be feasible if I added functionality (enabling arm swing movements, etc.) for the upper body joints within your framework and trained them using reinforcement learning? What parts of the code would need modification?

Thank you for your time and assistance. I look forward to your response.

Best Regards,
Eric Wang

[wangyenjen]

Hi @Ericongic,

Our method is capable of supporting joint control in the upper body. The current limitation to the lower body is discussed in our RSS paper.

To enable upper body control, you'll need to modify the joint-related configurations in humanoid/envs/custom/humanoid_config.py and adjust the corresponding number of joints in humanoid/envs/custom/humanoid_env.py. Additionally, ensure that the compute_ref_state index in humanoid/envs/custom/humanoid_env.py is updated accordingly.