Skip to content
This repository has been archived by the owner on Oct 16, 2022. It is now read-only.

3 - Advantage Actor Critic (A2C) [CartPole].ipynb - Returns do not need to be detached #3

Open
nimrare opened this issue Feb 4, 2021 · 0 comments

Comments

@nimrare
Copy link

nimrare commented Feb 4, 2021

Hi Ben

Thanks for the interesting notebooks. Upon studying the "3 - Advantage Actor Critic (A2C) [CartPole].ipynb" notebook, I came to the conclusion that detaching the returns in the update_policy() function is not necessary. The returns are only calculated on the rewards which are environment outputs and therefore not part of the computational graph. So even leaving out the .detach() call should not affect the model. Would you agree?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant