About the average reward

[Tendimension]

I have run 20 million frames(time-steps) in the Breakout environment, but the average reward has not changed. After about 17 million steps, the average reward has changed in Asynchronous Methods for Deep Reinforcement Learning. I do not know where the problem is?

[kkjh0723]

@Tendimension Do you find any reason? I have the same problem. The avg. reward is 2.0 and std. is 0.0 until 20 million frames. Is the reward going up after some period?

[Tendimension]

@kkjh0723 I do not know what the reason is.

[yao62995]

@Tendimension @kkjh0723 I also found this bug. I will check it soon.

[Tendimension]

@yao62995 Thanks a million!

[kkjh0723]

@yao62995 Do you have any updates on this problem?

[andyxzq]

I find the same issue. The average reward is still 0.0 after 1 million steps.