Implement app example using webpack #481

rxaviers · 2015-08-14T15:13:16Z

WIP fix for #464

Requires:

ToDo:

About this implementation:

Application doesn't require developer to handle CLDR loading as can be observed at https://github.com/jquery/globalize/blob/fix-464-webpack-example/examples/full-app-npm-webpack/app/index.js. Globalize plugin will handle that.
Webpack relevant configuration is
https://github.com/jquery/globalize/blob/fix-464-webpack-example/examples/app-npm-webpack/webpack-config.js#L38-L44

new globalizePlugin({
    production: options.production, // true: production, false: development
    developmentLocale: "en", // locale to be used for development.
    supportedLocales: [ "en", "es", "zh" ], // locales that should be built support for.
    messages: "messages/[locale].json", // messages (optional)
    output: "globalize-compiled-[locale].[hash].js" // build output.
})

production is a boolean that tells the plugin whether it's on production mode (i.e., to build the precompiled globalize data) or not (i.e., to be in development mode - will use Live AutoReload HRM).

developmentLocale tells the plugin which locale to automatically load CLDR for and have it set as default locale for Globalize (i.e., Globalize.locale(developmentLocale)).

supportedLocales tells the plugin which locales to build/produce compiled-globalize-data for.

output is the name scheme of the built files.

Development mode

Start the Live AutoReload server by running the command below.

npm run start

Then, point your browser to http://localhost:8080/.

Production mode

Build the production bundles by running the command below.

npm run build

The build files are placed in dist. Point your browser at ./dist/index.html to check.

sompylasar · 2015-08-15T08:48:58Z

Using a webpack loader or a function to load the messages (not just from plain JSON).

The real-world use case. I'm migrating a large code base to a set of modern tools, including webpack and, probably, Globalize for i18n, but my messages are in PHP (and that cannot be changed during this migration because they are reused by the backend i18n system). We're currently developing a loader that provides the messages to our frontend i18n system in the required format, and it would fit well if we decide to replace our frontend i18n with Globalize.

Runtime language switching (and asynchronous loading of the required locale which is not loaded during initial page load).

The real-world use case is obvious: language switching without page reload.

rxaviers · 2015-08-17T15:08:15Z

Using a webpack loader or a function to load the messages (not just from plain JSON).

I see. Can you provide examples of the not-just-from-plain-JSON-alternatives please?

Runtime language switching (and asynchronous loading of the required locale which is not loaded during initial page load).

👍

sompylasar · 2015-08-17T15:46:43Z

@rxaviers I'll give you an example but please do not write a specific loader for my specific example. A generic ability to specify a loader (or maybe better a plain function which could delegate to any other code if needed) would be better.

Our i18n system is Yii-based and uses a notion of per-module "category" and "key" (not just "key" which Globalize uses AFAIK), so a transform is required. But the system is highly customized to support multi-level overrides and fallbacks for each language, so this should be somehow handled by our custom loader (we actually delegate to the PHP component which knows how to do overrides and fallbacks, it does this for the server-side).

Here's an example of a single message file (of a single "category" for a "dialogs" module):

<?php

return [

    "_meta" => [
        "selector" => [
            "en",
        ],
    ],

    "dlg.message.title" => "Message",

    "dlg.message.button.close" => "Close",

    "dlg.error.title" => "Error",

    "dlg.info.title" => "Info",

    "dlg.confirm.title" => "Confirmation",

    "dlg.confirm.button.yes" => "Yes",

    "dlg.confirm.button.no" => "No",

    "dlg.prompt.title" => "Input",

    "dlg.prompt.button.yes" => "OK",

    "dlg.prompt.button.no" => "Cancel",

];

It's time for one more unexpressed thought that I've written about before. We need to be able to load different sets of messages for different apps based on a single platform (each app has one or more webpack entry points). Imagine two different applications with a set of common modules, each module depends on its own "category" with translations, each translation may be overridden at the application level. We should be able to build these apps separately and bundle only the translations being required by each app's components.

rxaviers · 2015-08-17T17:48:36Z

Sure, the messages option can be made to either accept the string (as documented above) or a function that passes locale as its first argument. For example:

new globalizePlugin({
    production: options.production, // true: production, false: development
    developmentLocale: "en", // locale to be used for development.
    supportedLocales: [ "en", "es", "zh" ], // locales that should be built support for.
    messages: function( locale ) {
        // Messages in the JSON format for requested locale.
        return getMessagesFor( locale );
    }
    output: "globalize-compiled-[locale].[hash].js" // build output.
});

Will include that...

We should be able to build these apps separately and bundle only the translations being required by each app's components.

The current implementation (rxaviers/globalize-webpack-plugin) is smart to bundle the translations being required only.

sompylasar · 2015-08-17T23:17:55Z

The current implementation (rxaviers/globalize-webpack-plugin) is smart to bundle the translations being required only.

Wow, that's pretty cool if I get you correct! How would you require specific message files from component files then, not having a gigantic "messages" object?

sompylasar · 2015-08-17T23:20:26Z

the messages option can be made to either accept the string (as documented above) or a function that passes locale as its first argument.

Does it by chance support asynchronous loading e.g. via a Promise returned from the "messages" function?
How will that function understand what translation files have been required by the components the entry points of the current compilation depend on?

rxaviers · 2015-08-18T00:23:57Z

Making myself clear, the current implementation requires a gigantic "messages" object to be available for the webpack plugin. But, the plugin is smart to use the messages being required only (and therefore, to generate the final bundle with the translations being required only).

Does it by chance support asynchronous loading e.g. via a Promise returned from the "messages" function?

Async might be tricky in webpack. Most of the compilation plugins (for example, [1], [2]) are sync calls.

I'd like to understand how you handle your custom messages and where async comes in. For example, are you handling files locally or remote? Does the processing require async? We could chat on IRC (http://irc.jquery.org/, #globalize).

How will that function understand what translation files have been required by the components the entry points of the current compilation depend on?

It's possible to pass, as another argument, the filename of the JS file being handled (in webpack terminology, the request variable):

    // For example:
    messages: function( locale, request ) {
        // Messages in the JSON format for requested locale.
        // locale: E.g. "en", "en-GB", etc.
        // request:  E.g., "my-app/index.js", "my-app/components/foo.js", so your function
        //           could look into "my-app/index.messages.json" or
        //           "my-app/components/foo.messages.json" for example
        return getMessagesFor( locale, request );
    }

Did I answer your question?

sompylasar · 2015-08-18T07:32:09Z

Async might be tricky in webpack. Most of the compilation plugins (for example, [1], [2]) are sync calls.

I'd like to understand how you handle your custom messages and where async comes in. For example, are you handling files locally or remote? Does the processing require async? We could chat on IRC (http://irc.jquery.org/, #globalize).

The current implementation executes an external process (a PHP console command which composes the messages using our custom override logic and emits them to stdout) which is an async operation.

I'm sorry for confusing you, we are currently using a plugin, not a loader (we tried loader before but failed). Because of lack of documentation on webpack plugin features we are trying to learn from existing plugins (including yours from now). The current implementation looks hacky and requires us to include the i18n bundle separately, not using webpack autoloading system. The code is not public, but I can show you some generic pieces we came up with.

We use the normal-module-factory to gather the categories from require requests (each module requires a special module path, e.g. i18n/my-category, which can be captured by a RegExp). We have to replace the request with a dummy file.

compiler.plugin('normal-module-factory', function (nmf) {
        langCategories = [];

        nmf.plugin('before-resolve', function (data, beforeResolveCallback) {
// ...
                                langCategories.push(langCategory);
                                data.request = __dirname + '/.i18n-dummy';

Then we use the emit compile step with asynchronous resolution (compilationCallback). We emit a compilation asset with a JS code that uses a globally available object to inject the translations it contains.

compiler.plugin('emit', function (compilation, compilationCallback) {
// ...
            compilation.assets['some-prefix-' + language + '.js'] =  source;
//

It's possible to pass, as another argument, the filename of the JS file being handled (in webpack terminology, the request variable):

Yes, this would be better, I think. This makes the "messages" function return a per-request object instead of the gigantic one, and Globalize then merges all the messages together, right?

rxaviers · 2015-08-18T12:51:42Z

Webpack is very powerful, but it has been really laborious to develop the plugin. I've also been using another plugins as a baseline given the lack of doc, plus I have also been reading the source code and have counted with help from bebraw and sokra.

Thanks for sharing your excerpts. I understood that your require categories return a no-op source. So, I didn't understand how your 'some-prefix-' + language + '.js' assets get loaded by your application? In other words, I didn't understand the link between the code that requires a category and the asset that is generated based on it.

Yes, this would be better, I think. This makes the "messages" function return a per-request object instead of the gigantic one, and Globalize then merges all the messages together, right?

Ok, will include it or suggest something else after thinking more about it.

sompylasar · 2015-08-18T15:51:59Z

I understood that your require categories return a no-op source. So, I didn't understand how your 'some-prefix-' + language + '.js' assets get loaded by your application?

Yes. That's the hacky flavor this solution gets. We access the translations via a singleton component which is populated with the messages. We have to side-load these files by a plain script tag or our own script loader, not by the webpack auto-loader. The messages get injected via a global variable shared between the core JS and the messages JS.

It looks like this (the code is pseudo-code):

// some-prefix-en.js
window.globalI18nManager.addMessages("some-category", {
    "some-key": "Some message"
}, "en");

// bundle.js
window.globalI18nManager.setLanguage("en");
$someElement.text( window.globalI18nManager.formatMessage("some-category", "some-key") );

- Change demo. - Inlcude pt messages.

rxaviers · 2015-08-24T18:23:21Z

examples/app-npm-webpack/index-template.html

+    {%
+        }
+    }
+    %}


It generates stuff like:

<script src="vendor.07c80e6dfcd89796c7d5.js"></script>     <script src="i18n/en.07c80e6dfcd89796c7d5.js"></script> <script src="app.07c80e6dfcd89796c7d5.js"></script>

rxaviers · 2015-08-24T19:13:29Z

@sompylasar your usecase should be tracked by rxaviers/globalize-webpack-plugin#1 and rxaviers/globalize-webpack-plugin#2

Amend README

rxaviers · 2015-08-24T19:24:20Z

Content team, @kswedberg, @arthurvr, if anyone has any spare time, it would be great to count with your continued help reviewing this PR as well. Specially in:

sompylasar · 2015-08-24T19:24:27Z

@rxaviers Thanks!

kswedberg · 2015-08-24T19:28:20Z

thanks, @rxaviers . I should be able to do a quick review this evening.

arthurvr · 2015-08-24T19:30:14Z

examples/app-npm-webpack/package.json

@@ -0,0 +1,18 @@
+{
+  "name": "globalize-full-app-npm-webpack",


Dunno how we generally do that, but if a package is marked private giving a name is kinda moot.

Minor pt-BR messages update

rxaviers · 2015-08-27T19:02:54Z

Thank you all

Closes #488 Ref #481

Ref #481 Closes #490

Includes Simplified Chinese. Ref #481

Ref #481

Ref #481 Closes #493

unindented · 2015-09-01T15:39:17Z

examples/app-npm-webpack/messages/es.json

+		"relative-time-label": "Tiempo Relativo",
+		"message-1": "Un ejemplo de mensaje usando números mixtos \"{number}\", monedas \"{currency}\", fechas \"{date}\", y tiempo relativo \"{relativeTime}\".",
+		"message-2": [
+			"Un ejemplo de mensaje con soporte de pluralizaciónt:",


Small typo, extra t at the end of pluralización.

Thanks, d317164.

juhamust · 2015-09-15T06:03:44Z

I wonder what is the correct place to report the issues with webpack-globalize, so I'll a quick on here.

Feels that globalize compiler is trying to be too smart for its own good (too much optimisation?): Methods like Globalize.numberFormatter are considered undefined time to time. Assigning them to variable like var numberFormatter = Globalize.numberFormatter helps, but only sometimes. I haven't found the pattern, yet.

Also, evaluation may consider variables undefined, while they're not:

// Angular translation filter
angular.module('app')
  .filter('t', function($filter) {
    return function (translationKey) {
      return Globalize.formatMessage(translationKey);
    };
  });

// Compilation error
./node_modules/globalize-webpack-plugin/GlobalizeCompilerHelper.js:72
      throw e;
            ^
ReferenceError: translationKey is not defined
    at eval (eval at extractor (./node_modules/globalize-compiler/lib/extract.js:67:9), <anonymous>:3:36)
    at ./node_modules/globalize-compiler/lib/compile-extracts.js:54:23
    at Array.reduce (native)

If I change the translationKey into valid translation-key string, it works. But obviously always returns the same translation. And all this worked at some point :(

Any wise words to help compiler to make it work like expected?

@unindented

Thanks to @unindented. Amends a6c8a6e Ref #481 Ref #490

rxaviers · 2015-09-15T09:50:48Z

Hi @juhamust,

The problem you're facing is because you're using a dynamic construction in your Globalize options and the compiler is static. It doesn't mean you cannot use dynamic patterns in your code, but you have to change your code like the below:

var availableMessageFormatters = {
  foo: Globalize.messageFormatter(<foo>),
  bar: Globalize.messageFormatter(<bar>)
  ...
};

// Angular translation filter
angular.module('app')
  .filter('t', function($filter) {
    return function (translationKey) {
      return availableMessageFormatters[translationKey];
    };
  });

Note this is actually a good constrain for your code. Because, you could potentially run into a case where you are trying to format a message in production that you have never provided translation for.

Did I answer to your question? Would you change anything in our docs to make this clear?

PS: Anyway, if you need to report errors to webpack-globalize, do so at https://github.com/rxaviers/globalize-webpack-plugin.

juhamust · 2015-09-15T10:39:11Z

@rxaviers thanks a lot for your prompt and informative response. That really took me surprised, so perhaps the documentation really could be improved about the topic.

Duh. I tried building multilingual lookup table, but it still has that dynamic translationKey variable there.

// Build translation message lookup table
var messageLookupTable = {};
var messages = require('json!../common/locales/messages/en.json');

_.each(['fi', 'en', 'es', 'de'], function(languageCode) {
  messageLookupTable[languageCode] = {};
  // Using english translation keys as a reference
  _.each(messages['en'], function(msg, translationKey) {
    messageLookupTable[languageCode][translationKey] = Globalize.messageFormatter(translationKey);
  });
});

So I really need to generate JS versions from all of those JSON message files? That's something I would expect the compiler is doing on my behalf?

Thanks again.

UPDATE: I created a simple from JSON to JS conversion step in building phase and it seems to work as expected.

@AhmedMustafa

Thanks to @AhmedMustafa. Amends a19869f Ref #481

rxaviers · 2015-10-05T21:05:34Z

@juhamust, can you give a little more details on what you are trying to accomplish? Is this part of a frontend or backend application? Where are you going to use messageLookupTable later in your application? I'm asking, because Globalize is pretty flexible and knowing your application helps me to give a more accurate answer. Thx

juhamust · 2015-10-06T07:04:13Z

@rxaviers I responded in separate ticket instead of polluting this PR thread

rxaviers · 2015-10-06T12:28:21Z

👍 I've replied you there.

jquerybot added the CLA: Valid label Aug 14, 2015

rxaviers mentioned this pull request Aug 14, 2015

Full stack example: webpack + Globalize #464

Closed

rxaviers added the Current Sprint label Aug 14, 2015

rxaviers force-pushed the fix-464-webpack-example branch from 7fc75f8 to 0ddb68f Compare August 24, 2015 01:37

Examples: Include app example using webpack

a5cddc2

rxaviers force-pushed the fix-464-webpack-example branch from 0ddb68f to a5cddc2 Compare August 24, 2015 01:38

fixup! Examples: Include app example using webpack

38b5a7d

- Change demo. - Inlcude pt messages.

rxaviers reviewed Aug 24, 2015
View reviewed changes

This was referenced Aug 24, 2015

Bundling with Webpack doesn't work well #441

Closed

Allow a function to be passed in the messages property rxaviers/globalize-webpack-plugin#1

Open

Make function that returns messages async rxaviers/globalize-webpack-plugin#2

Open

fixup! Examples: Include app example using webpack

2573e2b

Amend README

arthurvr reviewed Aug 24, 2015
View reviewed changes

rxaviers and others added 4 commits August 26, 2015 16:49

fixup! Examples: Include app example using webpack

20b49eb

Minor pt-BR messages update

Examples: ar (using Google translate)

cc621b9

Example: add Russian locale to webpack example

e20b6e4

fixup! Example: add Russian locale to webpack example

f2ef378

rxaviers mentioned this pull request Aug 27, 2015

Add Russian locale #493

Closed

rxaviers added 5 commits August 27, 2015 15:38

fixup! Examples: Include app example using webpack

4e3687c

fixup! Examples: ar (using Google translate)

4a07c28

fixup! Examples: zh (using Google Translator)

8be8f18

fixup! Examples: Update es.json

85cd21f

fixup! Examples: Include app example using webpack

e2057e3

jzaefferer added a commit that referenced this pull request Aug 27, 2015

App Example: Include German (de) translation

a9db3ce

Closes #488 Ref #481

rxaviers pushed a commit that referenced this pull request Aug 27, 2015

App Example: Include Spanish (es) translation

a6c8a6e

Ref #481 Closes #490

rxaviers added a commit that referenced this pull request Aug 27, 2015

App Example: Include Chinese (zh) translation (via Google translation)

2bbb1bb

Includes Simplified Chinese. Ref #481

rxaviers added a commit that referenced this pull request Aug 27, 2015

App Example: Include Arabic (ar) translation (via Google translate)

a19869f

Ref #481

rxaviers pushed a commit that referenced this pull request Aug 27, 2015

App Example: Include Russian (ru) translation

4f5248f

Ref #481 Closes #493

unindented reviewed Sep 1, 2015
View reviewed changes

rxaviers added a commit that referenced this pull request Sep 15, 2015

App Example: Fix Spanish (es) translation

d317164

Thanks to @unindented. Amends a6c8a6e Ref #481 Ref #490

rxaviers mentioned this pull request Sep 15, 2015

App example using webpack: Make it more interactive #489

Open

juhamust mentioned this pull request Sep 29, 2015

Generate static files automatically rxaviers/globalize-webpack-plugin#6

Open

rxaviers added a commit that referenced this pull request Oct 1, 2015

App Example: Fix Arabic (ar) translation

ea5273b

Thanks to @AhmedMustafa. Amends a19869f Ref #481

JSFOwner removed the CLA: Valid label Nov 2, 2016

rxaviers removed the Current Sprint label Nov 18, 2016

rxaviers closed this Apr 25, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement app example using webpack #481

Implement app example using webpack #481

rxaviers commented Aug 14, 2015

sompylasar commented Aug 15, 2015

rxaviers commented Aug 17, 2015

sompylasar commented Aug 17, 2015

rxaviers commented Aug 17, 2015

sompylasar commented Aug 17, 2015

sompylasar commented Aug 17, 2015

rxaviers commented Aug 18, 2015

sompylasar commented Aug 18, 2015

rxaviers commented Aug 18, 2015

sompylasar commented Aug 18, 2015

rxaviers Aug 24, 2015

rxaviers commented Aug 24, 2015

rxaviers commented Aug 24, 2015

sompylasar commented Aug 24, 2015

kswedberg commented Aug 24, 2015

arthurvr Aug 24, 2015

rxaviers Aug 24, 2015

rxaviers Aug 24, 2015

rxaviers commented Aug 27, 2015

unindented Sep 1, 2015

rxaviers Sep 15, 2015

juhamust commented Sep 15, 2015

rxaviers commented Sep 15, 2015

juhamust commented Sep 15, 2015

rxaviers commented Oct 5, 2015

juhamust commented Oct 6, 2015

rxaviers commented Oct 6, 2015

Implement app example using webpack #481

Implement app example using webpack #481

Conversation

rxaviers commented Aug 14, 2015

About this implementation:

Development mode

Production mode

sompylasar commented Aug 15, 2015

rxaviers commented Aug 17, 2015

sompylasar commented Aug 17, 2015

rxaviers commented Aug 17, 2015

sompylasar commented Aug 17, 2015

sompylasar commented Aug 17, 2015

rxaviers commented Aug 18, 2015

sompylasar commented Aug 18, 2015

rxaviers commented Aug 18, 2015

sompylasar commented Aug 18, 2015

rxaviers Aug 24, 2015

Choose a reason for hiding this comment

rxaviers commented Aug 24, 2015

rxaviers commented Aug 24, 2015

sompylasar commented Aug 24, 2015

kswedberg commented Aug 24, 2015

arthurvr Aug 24, 2015

Choose a reason for hiding this comment

rxaviers Aug 24, 2015

Choose a reason for hiding this comment

rxaviers Aug 24, 2015

Choose a reason for hiding this comment

rxaviers commented Aug 27, 2015

unindented Sep 1, 2015

Choose a reason for hiding this comment

rxaviers Sep 15, 2015

Choose a reason for hiding this comment

juhamust commented Sep 15, 2015

rxaviers commented Sep 15, 2015

juhamust commented Sep 15, 2015

rxaviers commented Oct 5, 2015

juhamust commented Oct 6, 2015

rxaviers commented Oct 6, 2015