Choreographer in discrete action spaces

[swy99]

I am trying to run Choreographer in environments with discrete action spaces.

The implementation was done with the help of the part of DreamerV2 code that are responsible for discrete action space. But my implementation did not work very well, especially lbs shows somewhat collapsing of actions, which means that it selects only one action in pixel based environments.

This link shows a part of the changes I have made besides adding some env wrappers to envs.py: swy99@e27b256

I have mainly tried MiniGrid with different sets of hyperparameters but all my attempts failed.

Did you have any chances to try discrete action space environments? Could you share some working sets of hyperparameters?

[mazpie]

Hi,
I adapted this codebase to several discrete action space environments in the past (Atari, Habitat, Unity environments), and it worked fine, with the same hyperparameters you can find in the current configs files, but to be fair I haven't tested it on MiniGrid yet.

I had a look at the changes you made and they look sensible. Perhaps, there is some issue with the world model's training?
Are you using pixel-based inputs (64x64x3) or are you testing on the low-dimensional Minigrid inputs?

[swy99]

Thanks a lot for taking your time to respond.
I am currently using pixel-based inputs (64x64x3).