Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Visualize output of forward and select_action #744

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Cadene
Copy link
Collaborator

@Cadene Cadene commented Feb 18, 2025

What this does

  • Add lerobot/scripts/save_inference.py to save outputs of the forward pass or select_action in a directory
  • Add --inference-dir to lerobot/scripts/visualize_dataset_html.py to load these outputs and visualize them

TODO:

  • display loss information
  • save+display predicted actions (action t+1 and chunk)
  • manually adjust shift and scaling of every modality
  • save changes to the visualization to not have to re-do them when changing episodes
  • add a reset changes button
  • fix bug when going down and up episode with arrow keys, which just do +1 on episode index instead of finding the next episode index

How it was tested

Ran commands + manually visualized

How to checkout & try? (for the reviewer)

python lerobot/scripts/save_inference.py \
--dataset.repo_id=pepijn223/mobileso100_drive_forward6 \
--dataset.episodes='[0,1,2,4,10,11,12,20,21,22,30,31,32]' \
--policy.path=pepijn223/mobile_so100_test_100000 \
--output_dir=outputs/save_inference/pepijn223/mobileso100_drive_forward6/pepijn223/mobile_so100_test_100000

python lerobot/scripts/visualize_dataset_html.py \
--repo-id pepijn223/mobileso100_drive_forward6 \
--inference-dir outputs/save_inference/pepijn223/mobileso100_drive_forward6/pepijn223/mobile_so100_test_100000 \
--episodes 0 1 2 4 10 11 12 20 21 22 30 31 32

@Cadene Cadene force-pushed the user/rcadene/2025_02_17_visualize_inference branch from 7b7beef to a66a792 Compare February 18, 2025 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant