Support computing custom scores and terminating/saving based on them in BaseTrainer

[anyongjin]

This PR introduces a new concept into tianshou training: a best_score. It is computed from the appropriate Stats instance and always added to InfoStats.

Breaking Changes:

• InfoStats has a new non-optional field best_score

Background

Currently, tianshou uses the maximum average return to find the best model. But sometimes it may not meet user needs, for example, the average return only drops by 5%, but the standard deviation drops by 50%. The latter is generally considered to be more stable and better than the former.

[MischaPanch]

Thanks for the PR @anyongjin, it's a good contribution!

Overall, the trainer has to become more flexible, but it would be too much to ask for right now. I think we can merge this after some slight changes and then soon refactor the trainer, taking in consideration the support for custom scoring and custom conditions on terminating the training

[anyongjin]

In essence, Average Reward and Test Score are two different things. The former represents a fixed test result indicator. The latter is a score for the test result. The scoring logic for different tasks and users may be different. For example, some consider the standard deviation and some do not.
Currently, tianshou uses best_reawrd for both average reward and test score. It is difficult for users to implement custom scoring logic. So I suggest that best_reward be used only for average reward, and best_score be added for test score. In this way, best_reward and best_score are two different things. If it is called best_custom_score, people will think that there is a system default score field, so I think it is better not to add 'custom'.

Update:

• Added explanation for InfoStats.best_score.
• Use lambda function when compute_score_fn is None to avoid multiple if-else