Inner otheragent2 resets to initialization

[cooijmanstim]

There is a bug in inner_step_get_grad_otheragent2 where the return value uses trainstate_th2.params and trainstate_val2.params. I think that should be trainstate_th2_.params and trainstate_val2_.params? As it is, it takes params from the trainstates in the global scope which haven't been trained. However, with the way the result of inner_step_get_grad_otheragent2 is used in the caller, I think this bug should have no effect unless args.inner_steps > 2.

POLA/jax_files/POLA_dice_jax.py

Line 541 in 6b07e89

trainstate_th2_, trainstate_th2.params, trainstate_val2_, trainstate_val2.params,

[Silent-Zebra]

Thanks for pointing this out! I think you're right in your comment and the effect. I'll fix this and the other issue you pointed out.

[Silent-Zebra]

Closing as fixes have been implemented and experiments rerun (see updated readme)