How to embed node.js?

[DiegoBM]

Node.js Version: Latest build
OS: All
Scope: Embed as shared library

Hi all,

Since this issue seems to always end up going unresolved, I'll bump it up to the top again due to the complexity of the topic.
I have been trying to embed node.dll into a simple C++ console project. In my particular case in Windows. All is ok with building the dll both in debug and release versions, but when the time comes to start embedding it, there is absolutely zero documentation in that regard, so I tried to follow Electron's source code which seems to have some extra libraries with initializers where I end up getting lost. In any case I cannot even get to the create environment step since I always have problems either creating the isolated data or creating the platform (V8's version not exposed on the dll), etc, so I must assume I'm doing one or many things wrong.

Could it be possible for someone in the node.js (or electron or node-chakracore) to please give us a brief tutorial/sample-project/something, that helps us all understand how to embed node.js into a C++ project, since it's a far-from-simple job?
Should we understand that the node.js team doesn't really want node.js to be embedded in other projects and hence the secrecy and/or absolute lack of documentation on how to embed it?

Thank you in advance if anyone can help. After a week trying it has started getting quite frustrating. Embedding ChakraCore on the other hand is stupidly simple, but of course you lose the whole environment node.js provides, which is a shame.

Kind regards

[bnoordhuis]

Simple: call node::Start(), done.

Finer-grained: call node::CreateIsolateData() + node::CreateEnvironment() + node::LoadEnvironment() but that makes you responsible for creating the V8 isolate and context, libuv event loop, etc. (which might be good or bad, depending on your needs.)

Some recent changes may have broken the second approach; if so, file an issue or a pull request.

Should we understand that the node.js team doesn't really want node.js to be embedded in other projects and hence the secrecy and/or absolute lack of documentation on how to embed it?

Yes. Yes, that's totally it. facepalm

[DiegoBM]

@bnoordhuis node:Start() (which actually is the only thing that worked) is not an option since it cannot keep a "session" open to stream it commands through time.
The second option (which you called "finer-grained"??) is what I have tried in multiple ways without success.

The sentence you quoted is quite a valid understanding since even the codebase (and some github issue posts) admits not to support the shared library mode (from the configure file):

parser.add_option('--shared', action='store_true', dest='shared', help='compile shared library for embedding node in another project. ' + '(This mode is not officially supported for regular applications)')

In any case you could prove me wrong by pointing us to that vast sea of tutorials, guides and help on how to embed nodejs and save that facepalm for later? That could actually be helpful

[bnoordhuis]

since it cannot keep a "session" open to stream it commands through time

It's not clear to me what this means.

As to the rest of your comment, you are aware it's an open source project? If you think the documentation is lacking, go write it, don't wait for someone else to do it for you.

[DiegoBM]

It means the same thing you can do with the REPL where you can keep typing commands and it will remember all previous declarations until you close your session. But from within my own application. If you use node::Start you either enter the interactive mode where you lose control of your thread, or you send eval commands where you lose the context between subsequent calls.

I'm well aware of the nature of the project. I'm asking if someone with internal knowledge of the project can give the rest of us (who admittedly don't know about its intrinsics) any guidance on the process to follow, that's what usually a helpful community does, people who already know, help people who don't know. There are plenty of open source projects much less well known than node.js out there which are pretty nicely documented and/or where the creators actually help users to use it, which is what I was asking here, if some creator and/or somebody who knows about the embedding process, can help users to use it.

Also, in case you haven't noticed, this entire forum "node/help" is intended to ask for help, which is what I'm doing. If the only help this github offers is "get to the very deeps of the source and solve yourself your problem", then what's the point of it?

[tniessen]

You can find related information at nodejs/roadmap#9 (which is an open(!) issue) and on referenced pages.

We don't currently have instructions on how to embed node into existing applications as far as I know, and I personally never did it. I would suggest to have a look at what node::Start does and to replicate the relevant parts.

What exactly are you trying to achieve by integrating node into your application? What are your requirements?

[DiegoBM]

Thank you for your reply @tniessen. I'm trying to integrate a scripting engine into a 3D Modeler to let users manipulate their models or implement plugins using JavaScript through some sort of internal REPL.
The idea is that the software should expose an api to node and will let the user take it from there from an interactive terminal if they wish to. Also they can install plugin files in JavaScript format that the software will parse on load or on demand. It's pretty similar to what Blender does with Python, in case you are familiar with Blender.

I already checked the link you shared, unfortunately I could't find the help I needed. I also checked the node::Start code but I had problems creating the platform since NodePlatform is not exposed and v8::platform is hidden in node.dll. I also tried to emulate what electron does, but their initialisation doesn't work for me, maybe it's because they are using their own fork of node and I wanted to use the official version.

[bnoordhuis]

NodePlatform is not exposed and v8::platform is hidden in node.dll

You answered your own question, didn't you? This is where you open an issue or a pull request.

[tniessen]

Coincidentally, I am familiar with Blender and also used its scripting capabilities. Integrating node sounds like a reasonable approach, even though you will need to be careful as node is mostly asynchronous and you will probably need a lot of synchronization when communicating with your API (even though libuv makes this a little simpler).

I would suggest to follow @bnoordhuis' suggestion and to open an issue in nodejs/node in order to discuss the necessary code and documentation changes.

[DiegoBM]

Thank you for your answers.

We'll, my assumption was that I must have been doing something wrong and, since I don't know the internals of node at all, that there were very good reasons to keep those hidden from final users and my problem should be somewhere else, that's why I was asking for guidance. I'll try to get experience modifying the node source a bit to not mess it up, and will move the topic to the official repository if I feel confident enough to produce a patch. I'm more of a JS developer in the last few years and my c++ is a bit rusty now, probably not up to the standar of quality of the node.js codebase. In the meantime if anyone can shed some light in the shape of guide or sample project it would be highly appreciated

[MMMMasterM]

Just out of curiosity, can't you take a pointer from your host process's c++ land, hand that address to node::Start as an argument, run a JS script that hands this address off to a c++ child module which in turn uses that address to loop back to the host land?

Host c++ -> node JS -> c++ nodejs addon module -> Host c++

Since it all happens in the same process (and even the same thread? not sure, needs checking), it should be the same address space and same permissions.

It might be a little bit hacky, but it doesn't require any modifications on nodejs and all you need to do to expose your own api of the host program to the JS land is to write a c++ native nodejs addon which exposes those functions and hands them back to the host programm via the single initial object pointer.

[DiegoBM]

I was considering running the full REPL process on a separate thread and send commands back and forth, though I never did it, I reckon I did not consider it being really manageable. In any case thank you for your suggestion :)

[rubys]

See nodejs/node#23265 (comment) for a sketch as to how this could work.

[viferga]

I have solved this issue with a library write on my own. I posted the explanation in another issue. I leave here the link so if any embedder is interested on embedding NodeJS in a program he can do it with the same technique or using the library I have published:

nodejs/node#23265 (comment)

[PopeNobody]

since it cannot keep a "session" open to stream it commands through time

It's not clear to me what this means.

As to the rest of your comment, you are aware it's an open source project? If you think the documentation is lacking, go write it, don't wait for someone else to do it for you.

I suspect that documentation written by somebody who knows the answers will probably be slightly more useful than documentation written by somebody who lacks such knowledge.

[PopeNobody]

dI think a good place to start would be to add a Makefile to test/embedding directory to which what documentation there is refers.

It was easy to compile an object file, you merely need to put -i ${prefix}/include/node into your CFLAGS.

But linking the application, I suspect, would require one or more libraries, and no libraries are installed when you do a make install of the project.

Installing the required libraries in ${prefix}/lib would also be useful.

I would be happy to write such a Makefile, and will make an attempt to figure out the answers so I can do so, but if somebody would tell me what libraries need to be linked in to the application, that would give me a leg up on the project. Also, it would be good to know if there are particular linker flags required.

My project is a very simple one. I am using parse-server in a project, and would like to compile a c++ program, called parse-server, which has the parse-server modules compiled into it.

This would allow me to do things like 'killall parse-server', which is far preferable to 'killall node', which might well have unintended consequences.

[PopeNobody]

Ok, it appears that such a makefile is produced in out/Release.target, and I got it to run.

That's a good thing.