refactor(mc-scripts): manually group some vendor chunks #3615

tylermorrisford · 2024-09-26T19:46:33Z

Summary

This PR revives another PR created by @emmenko to manually group some vendor chunks in the @commercetools-frontend/mc-scripts package. Doing so gives us a few extra files in the bundle but a significantly smaller vendor chunk.

Description

To reduce the main bundle chunk even more, we can instruct the bundlers to group certain vendor packages in specific groups. As a result, the main bundle will have a smaller size. Users (via browsers) will then only need to download the smaller main bundle when it gets updated (caching) as the other vendor chunks didn't change (unless via dependency updates).

I found this article that explains well how and why it's useful like that: https://www.codemzy.com/blog/react-bundle-size-webpack-code-splitting

Using this method to test, I rebuilt the account application before and after making these changes; this resulted in the main app chunk being reduced from ~660KB to ~412KB. Based on this I feel that all monorepo and satellite applications would benefit.

UPDATE
We had to change a bit the packages we wanted to create independent bundles for.
We need to bear in mind that splitting is done at the application level, so what we can split are their direct dependencies so it's not possible to split internals of (for instance) the application-shell package (that's something we can do separately and internally to that package).
We decided to create manual bundles for:

application-shell and application-shell-connectors
moment and moment-timezone
@commercetools-uikit/icons

Building the playground application, here is what biggest files originally looks like (webpack):

and this is how it looks like with the new manual bundles:

Also, this is the comparison when using vite.
BEFORE

AFTER

We're also adding new vite plugins that will help to analyze the built bundles using this env variables:

ANALYZE_BUNDLE
ANALYZE_BUNDLE_TREE

This is how they look like:

changeset-bot · 2024-09-26T19:46:37Z

🦋 Changeset detected

Latest commit: ce14413

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 36 packages

Name	Type
@commercetools-frontend/mc-scripts	Minor
@commercetools-frontend/sentry	Minor
@commercetools-applications/merchant-center-template-starter-typescript	Minor
@commercetools-applications/merchant-center-template-starter	Minor
@commercetools-applications/merchant-center-custom-view-template-starter-typescript	Minor
@commercetools-applications/merchant-center-custom-view-template-starter	Minor
@commercetools-local/playground	Minor
@commercetools-frontend/actions-global	Minor
@commercetools-frontend/application-components	Minor
@commercetools-frontend/application-shell-connectors	Minor
@commercetools-frontend/application-shell	Minor
@commercetools-frontend/i18n	Minor
@commercetools-frontend/l10n	Minor
@commercetools-frontend/permissions	Minor
@commercetools-frontend/react-notifications	Minor
@commercetools-frontend/codemod	Minor
@commercetools-local/visual-testing-app	Minor
@commercetools-website/components-playground	Minor
@commercetools-frontend/cypress	Minor
@commercetools-backend/eslint-config-node	Minor
@commercetools-backend/express	Minor
@commercetools-backend/loggers	Minor
@commercetools-frontend/application-config	Minor
@commercetools-frontend/assets	Minor
@commercetools-frontend/babel-preset-mc-app	Minor
@commercetools-frontend/browser-history	Minor
@commercetools-frontend/constants	Minor
@commercetools-frontend/create-mc-app	Minor
@commercetools-frontend/eslint-config-mc-app	Minor
@commercetools-frontend/jest-preset-mc-app	Minor
@commercetools-frontend/jest-stylelint-runner	Minor
@commercetools-frontend/mc-dev-authentication	Minor
@commercetools-frontend/mc-html-template	Minor
@commercetools-frontend/notifications	Minor
@commercetools-frontend/sdk	Minor
@commercetools-frontend/url-utils	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2024-09-26T19:46:38Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
mc-app-kit-playground	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 10, 2024 4:12pm
merchant-center-application-kit-components-playground	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 10, 2024 4:12pm

tylermorrisford · 2024-09-27T14:58:19Z

packages/mc-scripts/src/config/optimizations.ts

+  react: [
+    'react',
+    'react-dom',
+    'react-router',
+    'react-router-dom',
+    'react-intl',
+    '@reduxjs/toolkit',
+    'redux',
+    'react-redux',
+    'redux-logger',
+    'redux-thunk',
+  ],


@emmenko the only change I made is to lump redux and react together; since redux is fairly small this seems fine to do.

CarlosCortizasCT · 2024-09-30T16:10:29Z

Using this method to test, I rebuilt the account application before and after making these changes; this resulted in the main app chunk being reduced from ~660KB to ~412KB. Based on this I feel that all monorepo and satellite applications would benefit.

@tylermorrisford by default the applications are built using webpack but we also have "experimental" support for Vite.
Could you please also run that comparison by building the application using Vite so we also get those numbers?

ENABLE_EXPERIMENTAL_VITE_BUNDLER=true pnpm build

UPDATE: After some investigation, I think the problem is that we're trying to create bundles for dependencies that are transitive to the application. For instance, @apollo/client is not an application dependency, it is a @commercetools-frontend/application-shell internal dependency.
We're trying to control transitive dependencies form the application bundling which does not seem to be feasible.

We can, for instance, create a bundle for the application-shell packages, but @commercetools-frontend/application-shell is already quite big and I'm not sure we can do any bundling improvement from the application level. I believe we would need to look into that package in isolation and see if we can lazy load more stuff.
Having the bundles bundle visual diagram of that package can help us.

CarlosCortizasCT · 2024-09-30T16:44:53Z

Building an application is failing for both webpack and vite bundlers.

To test it, you can get into the playground directory and run:

// Build with webpack
ENABLE_EXPERIMENTAL_VITE_BUNDLER=false pnpm build

// Build with Vite
ENABLE_EXPERIMENTAL_VITE_BUNDLER=true pnpm build

I think we need to investigate a little further what the updated configuration actually does.

tylermorrisford · 2024-10-02T16:55:31Z

Two items I've discovered in testing the Vite build process by linking the my local version of app-kit with the monorepo and building the account application:

Our Vite config seems to get tripped up on files wit .js extensions, updating those files to .jsx allowed the build to proceed.
There is an export in packages-application/application-account/src/components/organizations/details/custom-extensions/custom-applications/configurations/_shared/form/default-raw-icon.js using !!raw-loader! which causes the build to fail, unless the entire export is added to build.rollupOptions.external.

CarlosCortizasCT · 2024-10-10T08:58:20Z

package.json

@@ -181,7 +181,8 @@
      "@typescript-eslint/parser": "^5.52.0"
    },
    "patchedDependencies": {
-      "[email protected]": "patches/[email protected]"
+      "[email protected]": "patches/[email protected]",
+      "[email protected]": "patches/[email protected]"


vite-bundle-analyzer has a rollup very flexible peer dependency (2.X -> 4.X) but it actually requires 3.X at least because of a typescript type usage.
Since we're using rollup 2.X in the repo (force by rollup), the alternative I took is to change the type usage (very simple change)

CarlosCortizasCT · 2024-10-10T08:59:53Z

packages/mc-scripts/src/commands/build-vite.ts

@@ -77,6 +86,10 @@ async function run() {
      pluginSvgr(),
      pluginDynamicBaseAssetsGlobals(),
      pluginI18nMessageCompilation(),
+      process.env.ANALYZE_BUNDLE === 'true' &&


This plugin create the bundles size visualization:

CarlosCortizasCT · 2024-10-10T09:01:05Z

packages/mc-scripts/src/commands/build-vite.ts

@@ -77,6 +86,10 @@ async function run() {
      pluginSvgr(),
      pluginDynamicBaseAssetsGlobals(),
      pluginI18nMessageCompilation(),
+      process.env.ANALYZE_BUNDLE === 'true' &&
+        analyzer({ defaultSizes: 'parsed', openAnalyzer: true }),
+      process.env.ANALYZE_BUNDLE_TREE === 'true' &&


This plugin creates a bundles dependency visualization:

CarlosCortizasCT · 2024-10-10T09:02:36Z

packages/mc-scripts/src/config/optimizations.ts

+  ],
+};
+
+const removeNonExistantDependencies = (


This is just to make sure the packages we're defining in the manualChunks map actually exists in the application to build.
This is a safety measure so in case one of them are remove (eg: moment) the build does not fail but will only emit a warning about it.

Thanks for the comment. I was fearful we'd remove something else but those we specify at first.

emmenko

Great job, thanks for the contributions! 🙌

emmenko · 2024-10-10T12:36:36Z

packages/mc-scripts/package.json

@@ -108,6 +109,7 @@
    "thread-loader": "3.0.4",
    "url": "^0.11.0",
    "vite": "~4.5.3",
+    "vite-bundle-analyzer": "0.12.1",


I'm not sure if the patch will be used here when someone installs the mc-scripts package.

Fair question. Couldn't find documentation on it. I must say I assume it would be applied.

I don't know either but it won't matter in this case: the change in the source code is about a Typescript type which is not generic in the rollup version we're using but the dependency code expects it to be.
So, the generated code is exactly the same (no typings go there).

emmenko · 2024-10-10T12:39:36Z

packages/mc-scripts/src/config/optimizations.ts

+  manualChunks: Record<string, string[]>,
+  appDependencies: string[]
+) => {
+  return Object.entries(manualChunks).reduce((acc, [chunkName, vendors]) => {


Nit: let's use a more meaningful name

Suggested change

return Object.entries(manualChunks).reduce((acc, [chunkName, vendors]) => {

return Object.entries(manualChunks).reduce((chunkGroups, [chunkName, vendors]) => {

Updated here: f8f062c

emmenko · 2024-10-10T12:40:28Z

packages/mc-scripts/src/config/optimizations.ts

+    const existingVendors = vendors.filter((vendor) =>
+      appDependencies.includes(vendor)
+    );
+    if (existingVendors.length > 0) {


Suggested change

const existingVendors = vendors.filter((vendor) =>

appDependencies.includes(vendor)

);

if (existingVendors.length > 0) {

const hasExistingVendors = vendors.some((vendor) =>

appDependencies.includes(vendor)

);

if (hasExistingVendors) {

Let's picture this scenario:

const manualChunks = { 'chunk-A': ['dep-a', 'dep-b']; }

With the current code, if the app does not have 'dep-b' anymore, the manual chunk would still be created with dep-a. This would not work if we apply your suggestion.

Thanks, nevermind then. I misread the code 🙈

tdeekens

Thanks a lot for the effort on this work!

tdeekens · 2024-10-10T12:50:56Z

.changeset/grumpy-fireants-protect.md

+- [vite-bundle-analyzer](https://github.com/KusStar/vite-bundle-visualizer)
+- [rollup-plugin-visualizer](https://github.com/btd/rollup-plugin-visualizer) (tree visualization)
+
+You can use them by setting this environment variables"


Suggested change

You can use them by setting this environment variables"

You can use them by setting this environment variables:

Updated here: f8f062c

tdeekens · 2024-10-10T12:52:59Z

packages/mc-scripts/package.json

@@ -108,6 +109,7 @@
    "thread-loader": "3.0.4",
    "url": "^0.11.0",
    "vite": "~4.5.3",
+    "vite-bundle-analyzer": "0.12.1",


Fair question. Couldn't find documentation on it. I must say I assume it would be applied.

tdeekens · 2024-10-10T12:54:59Z

packages/mc-scripts/src/config/optimizations.ts

+  ],
+};
+
+const removeNonExistantDependencies = (


Thanks for the comment. I was fearful we'd remove something else but those we specify at first.

refactor(mc-scripts): manually group some vendor chunks

f7ffaf3

tylermorrisford requested review from emmenko, CarlosCortizasCT and a team September 26, 2024 19:46

tylermorrisford self-assigned this Sep 26, 2024

tylermorrisford requested a review from a team as a code owner September 26, 2024 19:46

tylermorrisford added the fe-chapter-rotation Tasks coming from frontend chapter work label Sep 26, 2024

tylermorrisford commented Sep 27, 2024

View reviewed changes

fix(manual-chunks): add vite bundle analyzer

b55c4c6

vercel bot had a problem deploying to Preview – mc-app-kit-playground October 2, 2024 16:46 Failure

vercel bot had a problem deploying to Preview – merchant-center-application-kit-components-playground October 2, 2024 16:46 Failure

fix(manual-chunks): include lockfile

bad24f8

vercel bot had a problem deploying to Preview – merchant-center-application-kit-components-playground October 2, 2024 16:55 Failure

vercel bot had a problem deploying to Preview – mc-app-kit-playground October 2, 2024 16:55 Failure

CarlosCortizasCT assigned CarlosCortizasCT and unassigned tylermorrisford Oct 8, 2024

CarlosCortizasCT marked this pull request as draft October 8, 2024 15:25

CarlosCortizasCT removed request for a team, emmenko and CarlosCortizasCT October 8, 2024 15:25

refactor(mc-scripts): add new vite visualizer plugin

25a30bc

vercel bot deployed to Preview – merchant-center-application-kit-components-playground October 8, 2024 16:35 View deployment

vercel bot deployed to Preview – mc-app-kit-playground October 8, 2024 16:35 View deployment

refactor(mc-scripts): try different manual chunks approach

1c77000

vercel bot deployed to Preview – merchant-center-application-kit-components-playground October 9, 2024 15:39 View deployment

Merge branch 'main' into nm-manually-group-vendor-chunks

ff1069d

CarlosCortizasCT reviewed Oct 10, 2024

View reviewed changes

CarlosCortizasCT added 2 commits October 10, 2024 11:33

chore: add changeset

c793e3b

chore: remove unnecessary changeset

8df0afc

CarlosCortizasCT requested review from emmenko and a team October 10, 2024 09:35

CarlosCortizasCT marked this pull request as ready for review October 10, 2024 09:35

vercel bot deployed to Preview – merchant-center-application-kit-components-playground October 10, 2024 09:37 View deployment

vercel bot deployed to Preview – mc-app-kit-playground October 10, 2024 09:38 View deployment

CarlosCortizasCT approved these changes Oct 10, 2024

View reviewed changes

CarlosCortizasCT requested review from a team October 10, 2024 09:51

emmenko reviewed Oct 10, 2024

View reviewed changes

tdeekens approved these changes Oct 10, 2024

View reviewed changes

refactor(mc-scripts): renaming vars

f8f062c

vercel bot deployed to Preview – merchant-center-application-kit-components-playground October 10, 2024 14:47 View deployment

vercel bot deployed to Preview – mc-app-kit-playground October 10, 2024 14:47 View deployment

emmenko approved these changes Oct 10, 2024

View reviewed changes

CarlosCortizasCT approved these changes Oct 10, 2024

View reviewed changes

CarlosCortizasCT requested a review from a team October 10, 2024 15:00

Merge branch 'main' into nm-manually-group-vendor-chunks

ce14413

vercel bot deployed to Preview – merchant-center-application-kit-components-playground October 10, 2024 16:11 View deployment

vercel bot deployed to Preview – mc-app-kit-playground October 10, 2024 16:12 View deployment

CarlosCortizasCT merged commit f234d49 into main Oct 14, 2024
19 checks passed

CarlosCortizasCT deleted the nm-manually-group-vendor-chunks branch October 14, 2024 14:30

ct-changesets bot mentioned this pull request Oct 14, 2024

Version Packages #3625

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(mc-scripts): manually group some vendor chunks #3615

refactor(mc-scripts): manually group some vendor chunks #3615

tylermorrisford commented Sep 26, 2024 •

edited by CarlosCortizasCT

Loading

changeset-bot bot commented Sep 26, 2024 •

edited

Loading

vercel bot commented Sep 26, 2024 •

edited

Loading

tylermorrisford Sep 27, 2024

CarlosCortizasCT commented Sep 30, 2024 •

edited

Loading

CarlosCortizasCT commented Sep 30, 2024

tylermorrisford commented Oct 2, 2024

CarlosCortizasCT Oct 10, 2024

CarlosCortizasCT Oct 10, 2024 •

edited

Loading

CarlosCortizasCT Oct 10, 2024

CarlosCortizasCT Oct 10, 2024

tdeekens Oct 10, 2024

emmenko left a comment

emmenko Oct 10, 2024

tdeekens Oct 10, 2024

CarlosCortizasCT Oct 10, 2024

emmenko Oct 10, 2024

CarlosCortizasCT Oct 10, 2024

emmenko Oct 10, 2024

CarlosCortizasCT Oct 10, 2024

emmenko Oct 10, 2024

tdeekens left a comment

tdeekens Oct 10, 2024

CarlosCortizasCT Oct 10, 2024

tdeekens Oct 10, 2024

tdeekens Oct 10, 2024

	return Object.entries(manualChunks).reduce((acc, [chunkName, vendors]) => {
	return Object.entries(manualChunks).reduce((chunkGroups, [chunkName, vendors]) => {

	You can use them by setting this environment variables"
	You can use them by setting this environment variables:

refactor(mc-scripts): manually group some vendor chunks #3615

refactor(mc-scripts): manually group some vendor chunks #3615

Conversation

tylermorrisford commented Sep 26, 2024 • edited by CarlosCortizasCT Loading

Summary

Description

changeset-bot bot commented Sep 26, 2024 • edited Loading

🦋 Changeset detected

vercel bot commented Sep 26, 2024 • edited Loading

Choose a reason for hiding this comment

CarlosCortizasCT commented Sep 30, 2024 • edited Loading

CarlosCortizasCT commented Sep 30, 2024

tylermorrisford commented Oct 2, 2024

Choose a reason for hiding this comment

CarlosCortizasCT Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emmenko left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tdeekens left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tylermorrisford commented Sep 26, 2024 •

edited by CarlosCortizasCT

Loading

changeset-bot bot commented Sep 26, 2024 •

edited

Loading

vercel bot commented Sep 26, 2024 •

edited

Loading

CarlosCortizasCT commented Sep 30, 2024 •

edited

Loading

CarlosCortizasCT Oct 10, 2024 •

edited

Loading