Skip to content

issues Search Results · repo:Silent-Zebra/POLA language:Python

Filter by

7 results
 (72 ms)

7 results

inSilent-Zebra/POLA (press backspace or delete to remove)

Silent-Zebra/POLA
Discount Factor Possibly Applied Twice

Thanks to @cool-RR for pointing this out. get_gae_advantages already includes discount factors, then is later multiplied by cum_discount, which is another discount factor. Thus, the discount factor is ...
  • Silent-Zebra
  • Opened 
    on Oct 16, 2024
  • #13

Silent-Zebra/POLA
TypeError: GRUCell.__init__() missing 1 required positional argument: 'features'

$ python ./jax_files/POLA_dice_jax.py --env ipd --n_update 101 --gae_lambda 1.0 --inner_steps 2 --outer_steps 200 --lr_in 0.005 --lr_out 0.003 --lr_v 0.0005 --batch_size 2000 --rollout_len 50 --print_every ...
  • cool-RR
  • 2
  • Opened 
    on Apr 16, 2024
  • #10

Silent-Zebra/POLA
Baseline (Value Function) should account for KL penalty but currently does not

Issue: Because the objective is being modified by the KL penalty, the baseline (value function) should take this into account. The current training scheme has the baseline just learn the values as if there ...
  • Silent-Zebra
  • Opened 
    on Aug 5, 2023
  • #9

Silent-Zebra/POLA
Agent 2 value function not learned?

Here s another bug I found. out_lookahead receives trainstate_val2_copy.params rather than the argument trainstate_val2_copy_params that the containing function is being differentiated with respect to. ...
  • cooijmanstim
  • 3
  • Opened 
    on Mar 7, 2023
  • #8

Silent-Zebra/POLA
Inner agent 1 uses agent 2 as a reference

Here (and line 1097) agent 2 is passed in as a reference for agent 1. IIUC this means that the KL penalty pushes inner agent 1 toward agent 2, and worse, agent 2 s gradient accounts for this. So agent ...
  • cooijmanstim
  • 5
  • Opened 
    on Mar 7, 2023
  • #7

Silent-Zebra/POLA
Inner otheragent2 resets to initialization

There is a bug in inner_step_get_grad_otheragent2 where the return value uses trainstate_th2.params and trainstate_val2.params. I think that should be trainstate_th2_.params and trainstate_val2_.params? ...
  • cooijmanstim
  • 2
  • Opened 
    on Mar 7, 2023
  • #6

Silent-Zebra/POLA
KL div wrong way around?

https://github.com/Silent-Zebra/POLA/blob/6b07e89317b07d91216db9d02c1f915f9313b66a/jax_files/POLA_dice_jax.py#L500 The KL divergence looks like it s the wrong way around. Typically you want the expectation ...
  • cooijmanstim
  • 2
  • Opened 
    on Mar 1, 2023
  • #5
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Press the
/
key to activate the search input again and adjust your query.
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Restrict your search to the title by using the in:title qualifier.
Issue search results · GitHub