RFC: experiments.lazyCompilation

[JSerFeng]

Motivation

Lazy compilation is a feature that only bundle needed modules. In a large repo, there may be many entries/pages that won't be developed or accessed at all, they don't need to be built. This is also one of reasons why vite so fast, vite only build modules that are imported at runtime.

Implement

Let's see an example, the following module graph

main.js import() foo.js, if foo.js is never used, we don't need to bundle it at all, we can add a proxy module between the importer module and imported module, and make main.js import() the proxy module

At first build, the proxy module do not have any dependency to the foo.js, so all the sub graph starting from foo.js can be skipped.

But we do need to build foo.js if we do accessed it at runtime. We can use http request to notify the server that foo.js is ready to be executed, we need it.

And then server will change the proxy module's content, add a dependency from proxy module to foo.js, and start rebuilding.

After rebuilt, the dependencies become

We can do this by hooking into NormalModuleFactory's factory phase, after factory, a normal module is factorized, we can store the factorize data, and return the proxy module. The proxy module is called LazyCompilationProxyModule, we can control its content by codegen method.

For consistency, we can directly use Webpack's lazyCompilation client code.

Notice that a lazy module proxy's exports is a promise, the promise resolved after the module has been built. The pseudocode is as follows:

let r
module.exports = new Promise(resolve => {r = resolve})

if (module.hot.data) { module.hot.data.resolveSelf = r }

After rebuilt, above code becomes:

let r
module.exports = import('./foo.js')

if (module.hot.data.resolveSelf) { module.hot.data.resolveSelf(module.exports) }

Known issue

The following HMR case will full reload page

import('./foo')
import.meta.hot.accept('./foo')

As we have replaced the original module with lazy module in the normalModuleFactory.module hook, in Rspack, module factorization is parrallel, so it cannot access module_graph at this moment, so we only replace the module with lazy module if its dependency type is ContextElement or dynamic Import, we do not care HMR accept dependency, but in the above example, imported ./foo will become a lazy module while accepted .foo remains normal module, causing this HMR handler is invalid, thus every time hot update populates here will cause page reload

[JSerFeng]

PR: #5915