Open
Description
- Right now
captureBestHyperparameters
has a side-effect of defining a business logic of how the loss function needs to be adjusted wrt. model size penalty. - For incremental learning, we add another 0.01 times the difference in the old predictions
- We add other model size penalties at multiple other places.
This leads to the situation that we have to follow the execution path to find out how the loss function value is formed. Since it is a critical part of the ML algorithm, I think this business logic should be defined in one place.