Tendon/Muscle Rendering in Brax Visualizer

[ttktjmt]

Hello brax team,

I am exploring the integration of musculoskeletal model simulations using myosuite with brax, and I would like to see tendon and muscle features within the brax simulation system. Although brax now supports tendons and muscles on the backend via MJX compatibility, the visualization (for example, through the brax.io.html.render function) does not display tendons.

Issue

As I investigated the root cause of this problem, it seems that tendons are not currently implemented in the System class in brax/base.py and I cannot refer to them as objects from JavaScript files in brax/visualizer/js/*.js, which presumably contributes to this limitation. There may be other parts of the system that require modification as well.

I found this issue #149 really close to my request, but it is not active, probably because brax supported mjx and tendon was supported in the backend.

Request

Would it be possible to add this feature in development as a new feature for brax? It would significantly improve the visualization aspect of the simulation for researchers and developers working with musculoskeletal models.

If integrating this feature internally is not feasible at the moment, could you please advise on which files beyond brax/base.py and the visualization scripts in brax/visualizer/js/*.js would need modification for comprehensive tendon support?

Planned Implementation

• Visualize tendons as capsules, or with spheres at each node and a cylinder for the path (like mujocoUtils.js#L469-L483 and main.js#L221-L251 in mujoco_wasm)
• Display tendons in red color, with the color intensity/strength changing based on the activation strength of the attached muscle actuator.

Thank you in advance for considering this feature request.

[InckieUltra]

Will mujoco renderer work for you? Here is the example I found:
https://github.com/google-deepmind/mujoco/blob/main/mjx/mujoco/mjx/viewer.py

[ttktjmt]

Hello @InckieUltra, Thank you for the comment.
I should have clarified this up front, but my primary objective is to build an interactive browser-based visualizer using Three.js, similar to what's discussed in issue #107.

I noticed that tendons/muscles are technically supported in the brax System class and accessible via its parent class's (mjx.Model) attributes, so I think I only need to visualize it with the brax html visualizer.

[btaba]

Hi @ttktjmt , we aren't actively developing the brax visualizer anymore. If you'd like to submit a PR, you're more than welcome but we advise to use the MuJoCo renderer when possible.

[btaba]

Hi @ttktjmt , set the issue to "won't fix" but if someone wants to take a look, please feel free