Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single model, multiple data #132

Open
fjebaker opened this issue Oct 3, 2024 · 0 comments
Open

Single model, multiple data #132

fjebaker opened this issue Oct 3, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@fjebaker
Copy link
Owner

fjebaker commented Oct 3, 2024

There's often a need to fit a single model to several datasets, where only e.g. the normalisation is changing.

At the moment, the model is evaluated once for each dataset, but there's much room for improvement here: we need only evaluate the model once and can use different normalisations for each one. The caveat is; if the normalisation is being used in a convolution model, the effect is potentially non-linear, but in theory we have all the information to know that at compile time.

For now:

  • A solution would be to introduce a new data wrapper that detects if the domains are all the same / overlapping, and then uses the transformer function to effectively copy the model n times and apply the normalisations.
  • Abuse or implement an alternative AutoCache
  • Just concatenate all the datasets into a new dataset wrapper that implements the dataset API so the model is just evaluated once (but then can't adjust normalisations).
@fjebaker fjebaker added the enhancement New feature or request label Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant