Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENAMETOOLONG when opening tmp-globalize-webpack generated files #42

Open
cprodescu opened this issue Mar 18, 2017 · 6 comments
Open

ENAMETOOLONG when opening tmp-globalize-webpack generated files #42

cprodescu opened this issue Mar 18, 2017 · 6 comments

Comments

@cprodescu
Copy link

We use globalize-webpack-plugin in our project. Everything works great locally, however in CI environment (distributed pants over a mesos fleet), the sandbox root is nested deeply and the file paths generated by globalize-plugin are too long to read (throwing ENAMETOOLONG on fs.openSync).

This is an uncommon issue, affecting very few projects, but I'm wondering what your suggestion would be.

Error: ENAMETOOLONG: name too long, open '$BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/.tmp-globalize-webpack/-var-lib-mesos-slaves-0197ab7c-124d-4305-a1fc-d70e915d0fef-S2720-frameworks-201205082337-0000000003-0000-executors-thermos-devprod-devel-iron_source_centos7-32-63205b9c-0ca3-4edd-af72-cae6ac4f6b1c-runs-12f6437b-7ef2-456a-864f-06f50fc5eb8e-sandbox-build_sandbox-.pants.d-resolve-node-252d64521cf9-project.web.project-web-6d8cfed66902-src-component-nav.js'
                         at Error (native)
                         at Object.fs.openSync (fs.js:640:18)
                         at Object.fs.writeFileSync (fs.js:1333:33)
                         at GlobalizeCompilerHelper.createCompiledDataModule ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/globalize-webpack-plugin/GlobalizeCompilerHelper.js:37:6)
                         at Parser.<anonymous> ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/globalize-webpack-plugin/ProductionModePlugin.js:77:58)
                         at Parser.applyPluginsBailResult ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/tapable/lib/Tapable.js:48:50)
                         at Parser.<anonymous> ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/dependencies/CommonJsRequireDependencyParserPlugin.js:63:22)
                         at Parser.applyPluginsBailResult ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/tapable/lib/Tapable.js:48:50)
                         at Parser.walkCallExpression ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:727:22)
                         at Parser.walkExpression ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:552:40)
                         at Parser.<anonymous> ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:532:12)
                         at Array.forEach (native)
                         at Parser.walkVariableDeclarators ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:517:14)
                         at Parser.walkVariableDeclaration ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:493:8)
                         at Parser.walkStatement ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:378:32)
                         at Parser.<anonymous> ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:371:8)
                         at Array.forEach (native)
                         at Parser.walkStatements ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:370:13)
                         at Parser.parse ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/Parser.js:920:8)
                         at DependenciesBlock.<anonymous> ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/NormalModule.js:104:16)
                         at DependenciesBlock.onModuleBuild ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:310:10)
                         at nextLoader ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:275:25)
                         at $BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:292:15
                         at Object.context.callback ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:148:14)
                         at Object.module.exports ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/babel-loader/index.js:132:8)
                         at WEBPACK_CORE_LOADER_EXECUTION ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:155:71)
                         at runSyncOrAsync ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:155:93)
                         at nextLoader ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:290:3)
                         at $BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:259:5
                         at Storage.finished ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:38:16)
                         at DependenciesBlock.<anonymous> ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack/lib/NormalModule.js:113:20)
                         at DependenciesBlock.onModuleBuild ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:310:10)
                         at nextLoader ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:275:25)
                         at $BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:292:15
                         at Object.context.callback ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:148:14)
                         at Object.module.exports ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/babel-loader/index.js:132:8)
                         at WEBPACK_CORE_LOADER_EXECUTION ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:155:71)
                         at runSyncOrAsync ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:155:93)
                         at nextLoader ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:290:3)
                         at $BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/webpack-core/lib/NormalModuleMixin.js:259:5
                         at Storage.finished ($BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:38:16)
                         at $BUILD_SANDBOX/.pants.d/resolve/node/252d64521cf9/project.web.project-web/6d8cfed66902/node_modules/graceful-fs/graceful-fs.js:78:16
                         at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:445:3)
@rxaviers
Copy link
Owner

Ideally, this plugin shouldn't use the temporary files if we instead knew the answer for this question http://stackoverflow.com/q/31147779/798133

@cprodescu
Copy link
Author

That makes sense.
The closest development I could find on that topic is rmarscher/virtual-module-webpack-plugin. It is built on top of implementation details of enhanced-resolve, so it's fragile.

About the issue here, would you be open to changing the encoding for GlobalizeCompilerHelper . getModuleFilepath (downside of current implementation is that it doubles the length of the path).

@rxaviers
Copy link
Owner

About the issue here, would you be open to changing the encoding for GlobalizeCompilerHelper . getModuleFilepath (downside of current implementation is that it doubles the length of the path).

Sure, what's your proposal? I see in your commit you're using a hash function. Does it handle conflicts?

@cprodescu
Copy link
Author

Good point!
It does not handle conflicts, but it's using the sha-1 algorithm, same as git. The chance of finding a collision is very low (Google has recently published a paper on how they managed to generate a collision after 9 quintillion sha-1 computations). Discussions on sha-1 collisions:
https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html
http://stackoverflow.com/questions/9392365/how-would-git-handle-a-sha-1-collision-on-a-blob

I will add code which throws an error if there is a collision (as unlikely as it is, I agree it shouldn't silently fail). Please let me know if you prefer a different approach altogether.

@rxaviers
Copy link
Owner

Sounds sane, specially if this (i.e., use hash instead of filename) is a feature that could be turned on/off by an option. Would you like to submit a PR?

@necolas do you have any input about it? Thanks

@cprodescu
Copy link
Author

Sure, will look into it!

I think the option is a bit unnecessary for the user, especially since the internal behavior is a workaround for a missing feature in webpack.

However, I don't have a strong opinion, just a few questions:

  • which behavior should be the default?
  • what would be a good name for the option?
  • should it be ON/OFF or allow the user to configure the function?

@cprodescu cprodescu mentioned this issue Mar 26, 2017
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rxaviers @cprodescu