Add TodoMVC for dart2js & dart2wasm #502
Conversation
✅ Deploy Preview for webkit-speedometer-preview ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
now that we have 3.1 release... let me try to answer some questions :).
Would it be possible to get the build script as part of the PR? This way we can get this re-generated easier in the future. |
|
As for the js-strings builtins, given that safari doesn't ship it yet we could evaluate the polyfill and/or provide a version without the js-strings as comparison. |
Great.
I have now chosen the defaults that flutter uses - namely If there's strong desire, I can also move dart2js to -O2.
I've added a build script and the log from the build.
Thanks for the link. Yes, using a JS proxy is a great idea that allows polyfilling also string constants. PTAL |
|
could you run |
|
How popular is this framework / library? |
|
Hard to say, wasm-gc is still a bit early stages, but that's why this is marked experimental. |
|
@mkustermann could you maybe move your code to the |
This is a Jaspr Dart Web TodoMVC app compiled with dart2js and dart2wasm. We compile with * dart2js in unsound (-O4) mode * dart2wasm in sound (-O2) mode as those are the default settings in e.g. flutter tooling. To remember this difference we specify the optimization mode in the benchmark name. The app can be compiled by installing the Dart SDK and running ``` experimental/dart-jaspr-todomvc % ./build.sh |& tee build.log ... ```
Running I can see that there's a global
The main motivation isn't about the Jaspr framework but about getting a workload to this benchmark suite that runs a WasmGC app that makes heavy use of DOM operations. (**) WasmGC is in many ways a much better compilation target for managed languages than JS is. In the past there was no alternative, languages had to compile to JS, but now we have a better option. Once more and more apps start to target WasmGC instead of JS the question is how performant it is for WasmGC to manipulate DOM or use other browser APIs.
I have now moved it to (Update: Fixed the quoted text where my reply was to) |
Hm... so I've been told that JetStream 3 is going to include a dart test already. Given the popularity of dart as is, we should probably adopt another language's runtime / compilation target instead. |
|
Jetstream does not feature DOM interactions – by definition – there is no other place to put a wasm-gc-DOM interaction workload than Speedometer. And again, this is an experimental workload, we've agreed that there should be limited restrictions on what's added this way, as long as it's fitting the Speedometer sprit (aka heavy on DOM interactions) |
|
Surely there are other WASM GC DOM interaction examples other than dart? If dart is the only example, it seems premature to include such a test in the benchmark. |
|
wasm-gc is not widely adopted, I think we have to take what's offered for now (this is definitely something to revisit if we get more choice in the future). If you can provide a good alternative workload as a PR I'm very happy to accept that. |
In that case, my preference is not to test that in Speedometer v4. While it's okay to include something that's not quite popular in the benchmark, WASM GC DOM interaction seems rather far fetched. I've never seen an example of a popular website that uses WASM, let alone manipulate DOM from WASM. I won't rule out that such a paradigm may become relevant in the future but I don't think it is today. |
|
Please join our regular sync meetings again (happy to reschedule if this works better), so we can discuss this in a more productive format. I think we have a fundamental misunderstanding of what an experimental workload means :). |
|
So on a high level, the way I see it is this (from the perspective of an implementer of a managed language that targets web): Managed languages were for a very long time forced to compile to JS due to lack of alternative. But now there's the option to target WasmGC instead of JS. We have seen very clear evidence that WasmGC is a superior compilation target for managed languages compared to JS. Our observation is that compiling to WasmGC instead of JS leads to:
Those benefits will only become larger over time, as the browser runtimes optimize WasmGC more (improvements in JS are much harder as JS execution has been optimized for decades already and may be close to a ceiling) So IMHO it's inevitable that over time more languages will move towards targeting WasmGC as a backend to capitalize on those benefits. Apps will just run better - a win for all end users. Given this I think it makes sense to have realistic benchmarks of WasmGC in the popular browser benchmark suites - giving browser runtime implementors realistic workload examples that can be used for improving the runtimes. There's two kind of workloads that one may want to benchmark here:
The benchmark that was added to WebKit/JetStream/Dart is of the former category - it's a realistic benchmark derived from flutter web (which is used in production) - the core was extracted to be runnable in commandline JS engines (in general flutter doesn't use DOM much as it draws pixels directly). This PR is adding the other kind of benchmark - one that's heavy on the interaction with browser APIs, specifically the DOM - where we chose the Jaspr Dart framework. It's fair to ask why those should be in Dart. In general I think most high level, class-based, nominal typed languages (C#/Java/Kotlin/Dart/...) will compile to similar WasmGC code. It just happens that we on the Dart team have been experimenting with wasm for quite some time and once WasmGC was being developed we were an early adoptor of that and may be furthest along with our WasmGC backend. But if there's other languages with other frameworks that compile to WasmGC and have realistic workloads for pure WasmGC and WasmGC<->BrowserApi it may be an option to include them as benchmarks. @rniwa are you aware of any?
Actually we also have heard from various teams saying: For DOM based frameworks JS is the only choice and WasmGC wouldn't make sense. But where's the data to back this up? Our initial guess was that the extra overhead of WasmGC calling out to JS to access DOM may be a problem. (Tangential: At some point we had some conversations with folks from Mozilla (@codehag) where this came up tangentially in the context of wasm component model: Would we need a way for WasmGC to talk to DOM directly without going via JS and if so, would that interaction be based on wasm component model or something else, ...) Now Dart happens to be a language where we have a DOM based web framework and optimizing compilers targeting both JS and WasmGC. So we made this TodoMVC app in Jaspr and measured. What's actually interesting and surprising to us, is that the Jaspr TodoMVC compiled with Dart2WasmGC runs faster than the Dart2JS compiled version in Chrome (haven't measured other browsers) - despite the fact that DOM interaction is on the hot path. At some point I built my own Chrome version and profiled it and it seemed that the hot DOM operations are relatively heavyweight, they spend time in C++ blink code, so the extra overhead of going via JS (WasmGC->JS->DOM) seemed to not matter that much compared with the actual C++ DOM code. But it surely adds some overhead. Though that overhead seems to be compensated by faster WasmGC code - so overall for the Jaspr TodoMVC we WasmGC is the better compilation target. (There's of course a chicken-and-egg problem: WasmGC will not get adopted if browsers don't make it fast, and browsers may not want to make it fast if WasmGC isn't adopted. But I think we're already now at a stage where we see the apps compiled to WasmGC to be significantly outperforming JS on some browsers and it's only getting better.) |
|
Maybe also adding @eqrion to hear Mozilla's perspective. |
This is a Jaspr Dart Web TodoMVC app compiled with dart2js and
dart2wasm. We compile with
as those are the default settings in e.g. flutter tooling.
To remember this difference we specify the optimization mode in
the benchmark name.
The app can be compiled by installing the Dart SDK and running