Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic cleaning of .butane/migrations/current #216

Open
jayvdb opened this issue Mar 18, 2024 · 1 comment
Open

Automatic cleaning of .butane/migrations/current #216

jayvdb opened this issue Mar 18, 2024 · 1 comment

Comments

@jayvdb
Copy link
Collaborator

jayvdb commented Mar 18, 2024

This is sort of a continuation of #44 , and a devils advocate for one half of #60 which says

The proc macro runs independently on each #[model], and Cargo does not appear to provide any environment variables with a unique identifier for the build. Thus each .table file is generated independently and there isn't a great way to reap dead ones (that I've identified).

In the last task, one of the outcomes was to create cargo clean which is described as

Clean current migration state. Deletes the current migration working state which is generated on each build. This can be used as a workaround to remove stale tables from the schema, as Butane does not currently auto-detect model removals.
The next build will recreate with only tables for the extant models

This is useful, but to be a solution, needs to be either

  1. very clearly documented as a required step when removing tables or types,
  2. automated, either auto-cleaning, or auto-detecting cleaning is needed.

#44 proposes extra effort is going to be required of developers. If so, it would be easy for developers to create a build.rs and set a unique build identifier variable.

Another approach is to derive a unique-enough id from all files under src/ (and tests/ if CARGO_CFG_TEST is set). Could be file modification timestamps or hashes.

So if we had a build identifier, what can we do? For it to be of any use, it would need to be stored somewhere under .butane/. As it will change frequently, it would need to be a file which isnt checked into VCS.

The "solution" of creating a build.rs to auto-clean .butane/migrations/current caused #56 - Using an IDE while running tests causes test failures. The fix in #215 only works because in that project all of the models are in tests/. It isnt a general solution.

To use this build-id to reap dead tables/types, I guess the either a build.rs or proc-macros would detect that that there is a new build, and delete all the current files - esp types.json which is always json-appended to. Unless done carefully, this is break in the "IDE + running tests" scenario.

What would be interesting is if the proc-macros could write the new files into .butane/builds/<id>, and then some process known to run at the end be injected to replace the current migration dir files if a change was detected.

@jayvdb
Copy link
Collaborator Author

jayvdb commented Apr 6, 2024

Perhaps we can catch the problem at "make-migration" stage.

It is very easy to auto-clean types.json - remove any type which is no longer mentioned in any table.

To get the current list of tables, we may be able to use CrateRoot.impls to get all the implementations of DataObject.

https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/rmeta/struct.CrateRoot.html
https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/rmeta/struct.TraitImpls.html

These files are generated during each build. We would need to find the most recently written one at

$ find target/ -name 'getting_started.rmeta'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant