Skip to content
This repository has been archived by the owner on Oct 23, 2019. It is now read-only.

Detection of implicit file dependencies via pipeline interfaces #122

Open
markfinger opened this issue Jul 1, 2016 · 0 comments
Open

Detection of implicit file dependencies via pipeline interfaces #122

markfinger opened this issue Jul 1, 2016 · 0 comments

Comments

@markfinger
Copy link
Owner

markfinger commented Jul 1, 2016

Caveat: this is quite possibly a terrible idea, but it's amusing me enough to jot it down

Given that compiling a single file, with dependencies, will involve FS calls to:

  • read the text of the file
  • check for the existence of possible files that match its dependency identifiers
  • read package.json files (to identify paths via main or browser fields).

We could use an abstraction over the FS (#88) that detects those calls and annotates the compilation with the dependencies of the outcomes of each FS call (eg: mtime, isFile, etc).

If the system was aware of the implicit file dependencies that are used to produce a compiled asset, we could replace a massive amount of complex cache invalidation logic with simple diffs. Eg: if the compilation of a file relied on fs calls that had x and y outcomes, we can simply check for equality before validating or discarding cached data.

Currently, we cache everything except for path-based dependency identifiers which originate from files within the source root (eg: project files, rather than package files). The reason why we don't cache is down to the implicit ambiguity of ./foo mapping to either ./foo.js or ./foo/index.js. That ambiguity is removed if we can diff against the fs state used for the compilation.

markfinger added a commit that referenced this issue Jul 8, 2016
An fs cache also supports creating contexts, so that file dependencies (isfile, mtime, and content hashes) can be tracked.
markfinger added a commit that referenced this issue Jul 10, 2016
Contexts are now classes, which should help to drop some per-context overhead.
Added validation hooks for context dependencies.
markfinger added a commit that referenced this issue Jul 19, 2016
…g features - re #122 #88

- Provides in-memory caches for FS interactions, which should massively improve performance for dependency resolution.
- Provides dependency tracking, which enables all the files used to compile a unit, and resolve its dependencies, can be used for cache invalidation.
- Provides a facade over the cache so that invalidation of dependencies can be triggered during runtime. By plugging a file watcher into the cache, this feature enables granular cache invalidation of compilation phases.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant