Compatibility with broccoli-asset-rev functionality (asset fingerprinting, assetMap.json manifest)

[22a]

In the past it was mentioned that Embroider eliminates the need for broccoli-asset-rev:
ember-cli/broccoli-asset-rev#65 (comment)

Perhaps I’ve lost something along the way with our embroider configuration but it seems like broccoli-asset-rev still fills two roles in our deploy pipeline:

• Static asset revisioning + rewriting (bitmaps, svgs, mp3s, etc.)
• Asset manifest production (dist/assetMap.json)

Static asset revisioning

Is there any way to replicate the behaviour of static assets (eg. in public/) getting checksum filenames?

Asset manifest production (assetMap.json)

The latter is somewhat more trivial and, with a little manual work, can be statically generated from the files on disk post-build but I’m wondering if Embroider considers this a first-class feature.

In our apps we often don’t use the index.html generated by the build process but rather generate the html server-side ourselves, injecting the correct script tags referencing the latest bundle revision. With this setup we rely on the assetMap.json generated by broccoli-asset-rev to allow us to look up the right asset (mapping development build filenames to production asset filenames, eg. appname.js → appname-1A2B3C.js).

To try reach feature parity with this in Embroider land, we configure webpack to limit the initial bundle count to 1 and give it a special name (eg. appname.[chunkhash].js, such that it’s distinguishable from the runtime chunks pulled in by @embroider/router). This works nicely and we can even use a webpack plugin to generate a similar asset manifest but this doesn’t work for the vendor bundles as they’re synthesised outside of webpack.

Is this asset manifest still a useful concept with Embroider and if so could it warrant being emitted as a build artefact? (Even as an opt-in)

[ef4]

The answers to both these questions are similar. In both cases, we don't want to tie embroider core to one particular build & deployment strategy, so features like these don't belong in embroider core. Rather, you can configure them directly on the build tool you're using (webpack being the most mature option at the moment).

For static assets, you can use file-loader. This area will get even easier after switching to webpack 5, which we are nearly ready to switch to as the default. At present, file-loader works best if you co-locate the assets with the components that will consume them (if you try to keep the assets all in /public, for example, the backward-compatibility behaviors for /public gets in the way).

For asset manifests: you could use some webpack plugin that generates a manifest, if you want to. But consider that the index.html we produce can itself work as a manifest, and this has the benefit of being a web standard rather than a quirky custom format. By parsing the HTML you can reliably see exactly which assets of which kinds are needed to boot the app.

but this doesn’t work for the vendor bundles as they’re synthesised outside of webpack.

There are only two vendor scripts that are generated outside webpack (vendor.js and test-support.js). Anything else you're seeing is a real webpack chunk. You may need different configuration settings to control their names though. They are generated by SplitChunksPlugin, which has some defaults for a vendor cache group.

In general when somebody asks how their code can figure out which output file holds which input file, my question is "why are you doing that?" because it usually means your code is taking on too much responsibility, and can instead lean more on the build tools and stop worrying about output paths.

[rwjblue]

For asset manifests: you could use some webpack plugin that generates a manifest, if you want to. But consider that the index.html we produce can itself work as a manifest, and this has the benefit of being a web standard rather than a quirky custom format. By parsing the HTML you can reliably see exactly which assets of which kinds are needed to boot the app.

This would not include any dynamically loaded assets (so it wouldn't be a replacement for what the broccoli-asset-rev manifest). If you really need a concrete manifest, generating one post-build (for example, as part of your deployment pipeline) is likely the way to go.

[achambers]

If you really need a concrete manifest, generating one post-build (for example, as part of your deployment pipeline) is likely the way to go.

https://github.com/ember-cli-deploy/ember-cli-deploy-manifest might be helpful here.

[mydea]

It is still not really clear to me (as somebody who is not really experienced with Webpack, which I guess probably applies to a lot of Ember devs) how I am supposed to migrate my existing, previously auto-fingerprinted code to Embroider/Webpack.

It would be great if there would be a kind of "how to" documentation in how to do that - maybe somebody who knows how these things work can help out there? Some concrete examples:

1. We have a bunch of images, e.g. in a template:

<img src="/assets/img/my-image.png" />

That used to be fingerprinted, but now aren't. So if I update the image without changing the file name, it will not be reflected.

2. We have some static (generated) JSON files we use for translation:

export const jsonMap = {
  de: '/assets/translations/de.json',
  it: '/assets/translations/it.json'
}

// somewhere else...
fetch(jsonMap[locale]);

The JSON files also used to be auto-fingerprinted (when adding .json to the fingerprint extensions), which also doesn't happen anymore either.

I'm happy to update some configuration or do some manual stuff, but I honestly don't really know where to start 😬

[ef4]

We are working on a standardized solution here: emberjs/rfcs#763

[wagenet]

Getting all this figured out is basically our remaining blocker from switching to Embroider.

[kevinkucharczyk]

@ef4 has there been progress on this? We have assets referenced in templates like @mydea's example:

<img src="/assets/icons/my-icon.svg" />

and those aren't being picked up being picked up in an Embroider build. They should get fingerprinted and receive a URL from publicAssetUrl, but they're ignored. Is there some option we can use to include such files?

[amk221]

I'd like to get some more insight into migration paths for existing apps. We too are 99% on Embroider with build times down from 30 seconds to 1 second. But cannot proceed due to the lack of fingerprinting and asset-map generation

[wandroll]

I'd like to get some more insight into migration paths for existing apps. We too are 99% on Embroider with build times down from 30 seconds to 1 second. But cannot proceed due to the lack of fingerprinting and asset-map generation

The most "Embroider" way to get assets fingerprinted is to let the package bundler, webpack, handles it, through a loader.

That means loading

• setting up asset-modulesto load images ( This is where you can tune up the fingerprinting strategy)
• move assets from public folder to app/someAssetFolder as public folder is copied as is for backward compatibility
• make sure that images you use are actually "imported" by a JS file.

I described here , how I did #1333 (comment) but it's descricbed there in the docs #1333 (comment)

[ef4]

As of ember-auto-import 2.7.0, even classic builds can delegate their asset handling to webpack, via the allowAppImports feature. Do that.

[amk221]

@wandroll Thanks, that helps, specificially this bit, which I did not know about:

move assets from public folder to app/someAssetFolder as public folder is copied as is for backward compatibility

After getting the files fingerprinted, it still leaves unanswered questions:

1. dist/assets/fonts/foo-6c0e465fc8a104e16dd2.woff2 is fingerprinted, references to it inside index.html still says <link href="foo.woff2"... (not fingerprinted).
2. No asset map is generated

@ef4 Sorry I am not sure what ember-auto-import's allowAppImports has to do with either of these things?

[ef4]

If you allowAppImports: ["*.png"] you can use webpack to import pngs from your app, which means the instructions shared above about using webpack for assets will work in classic builds (whereas before they only worked with embroider).

[amk221]

@ef4 It took me a whlie to figure out what you meant, but yes, being able to do that would indeed help migrate from classic to emboider.

In case it helps anybody, you can incrementally move your classic Ember config into ember auto-import's webpack config. Then, when ready, you can simply move that config over to the Embroider config.

autoImport: {
  publicAssetURL: `http://cdn.com/`,
  allowAppImports: ['assets/new/**/*.png'],
  // Use ember-auto-import as a stepping stone onto Embroider
  webpack: {
    module: {
      rules: [
        {
          test: /\.png$/i,
          type: 'asset/resource',
          generator: {
            filename: '[path][name]-[hash][ext][query]'
          }
        }
      ]
    }
  },
 fingerprint: {
    enabled: true,
    // Ignore the new, imported assets or you'll end up wth double asset url:
    // as they will be processed by broccoli-asset-rev and webpack
    // https://cdn.com/assets/https://cdn.com/assets/7df9164cae999bf3d289-21664ea8e812ba09e197d8f4c4818118.png
    exclude: ['app/assets/new'],
    extensions: ['png'], 
    prepend: `http://cdn.com/`
  },
},

import Component from '@glimmer/component';
import foo from 'my-app/assets/images/foo.png';

export default class FooComponent extends Component {
  <template>
    <img src={{foo}} />
  </template>
}