Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata: "Save datastack" should include metadata documents in the archive #1774

Open
davemfish opened this issue Feb 5, 2025 · 1 comment
Milestone

Comments

@davemfish
Copy link
Contributor

Saving a datastack refers to saving the input data for an model. There will not necessarily be metadata documents present, but if there are, we might consider including them in the datastack archive. We want to promote sharing metadata along with datasets.

We might also want to generate a metadata document for the archive itself.

@davemfish davemfish added this to the 3.15.0 milestone Feb 5, 2025
@davemfish
Copy link
Contributor Author

Actually, given that we have the ARGS_SPEC with lots of relevant metadata about model inputs, maybe we should generate metadata documents as part of the Save Datastack operation. That would also make it easier to discover & include in the archive.

It may also be useful to have an option to create metadata for an args dict without compressing & archiving the data.

Any pre-existing metadata should be preserved. geometamaker is designed to do that, but we will need to take care not overwrite descriptions or other human-modified attributes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant