Add gleam support to Mix

[Papipo]

This PR adds support for the gleam language.

• Support for gleam.toml in path deps
• Support gleam in deps loader.
• Support for gleam in deps.compile
• Require specific gleam binary version (hardcoded to 1.9.0 for now, see below)
• Handle absent gleam binary
• Support git deps (support for them in gleam is in main already but not yet released)
• Setup gleam on CI (Otherwise gleam tests will be skipped).

Notes:

• Packages having a gleam binary version requirements are handled automatically when executing gleam compile-package.
• The gleam binary version requirement for mix integration is hardcoded because I could just try to run gleam export package-info but I can't know the reason for a bad exit status (maybe the dep path was wrong and the command was run in a dir without a gleam.toml).
• The required command to export information about gleam packages and their deps is not yet released but has already been merged. Should land on v1.10.0.
• When there are [erlang] options set in the gleam.toml a .app file is generated in the ebin folder of the dependency. Right now this generated file is very basic, having only the vsn, applications (if extra_applications was specified) and mod (if application_start_module was specified). Not sure if there is room for improvement there (like using a .app.src file instead? I don't know the internals of this).

[Papipo]

@josevalim when working on this I was always thinking of erlang target packages, but it would be nice to be able to use javascript ones as well (ie. to use them in Phoenix frontend). Not sure how we can handle that, any hints?

[josevalim]

It should be possible because we use heroicons, a regular JS package, as a dependency on Phoenix applications. So we could probably support a :gleam_targets or similar option that is used by by deps.compile? You will probably need to set app: false true, similar to how heroicons are added.

[lpil]

Hello! You shouldn't need to do anything special target-wise as Gleam performs dead code elimination for code that is for other targets.

@josevalim when working on this I was always thinking of erlang target packages, but it would be nice to be able to use javascript ones as well (ie. to use them in Phoenix frontend). Not sure how we can handle that, any hints?

I would not expect Mix to become a front-end build tool by adding Gleam support. I think the workflow should be the same as it is today etc, with the desired build tool (esbuild, webpack, gleam etc) being run as a Phoenix watcher.

Adding frontend dependencies to the BEAM application dependency tree would either be wasted work compiling and managing them, or the programmer would need to go through every dependency in the lock file and add configuration to their mix.exs for each one they don't want to use on the backend.

We are going to #14262 (comment), so I also need the target config, if any. I think that just exporting everything is the best bet.

There is no such config. A Gleam package doesn't have an explicit target, each function in the public API is available on Erlang, JavaScript, or both, and they get either compiled or eliminated as needed. Whether a package is for Erlang or JavaScript is typically a matter of how the dependant is using them, not a matter of the configuration of the dependency.

There is a target flag in gleam.toml, but that is "if I run code from this page (almost always the tests) what target should be used by default, unless otherwise specified".

[josevalim]

And we should probably skip gleam tests if gleam is not installed, right?

[Papipo]

Done. I guess I need to add gleam to CI (although I need to wait until my gleam PR is merged and released).

[Papipo]

I am kind of copypasting this from the Gleam discord in case someone wants to try this out before merging into main:

What is this?

These changes add first-class support for Gleam in Elixir's Mix, thus allowing gleam path dependencies in mix.exs.
Those dependencies are expected to be 100% valid gleam projects and as such need to have a gleam.toml file.

What's missing?

Probably nothing. Work on the gleam compiler is already merged and work on the Mix side is completed, but there might me rough edges, that's what we need to test.

Instructions

Mix integration will expect you to have gleam >= 1.10.0 installed and that hasn't been released yet. So:

• Install Gleam using the nightly release.
  • Notice that some package managers might allow you to install it this way.
• Grab this elixir branch.
• Edit lib/mix/lib/mix/gleam.ex and replace @required_gleam_version with 1.9.0.
• Run make in the root folder of the Elixir project and add its bin folder to your $PATH

If you were using mix_gleam, undo its setup.

Usage

After running mix deps.get the Mix workflow doesn't change. deps.compile should compile Gleam path deps.
If you use gleam code from your elixir app, it should just work (as long as you have a gleam.toml file - you might want to move gleam code to a sibling folder or into gleam/src) I think that otherwise the mix.exs file will take precedence over the gleam one.

What needs to be tested?

• Anything related to Erlang/OTP shenanigans (.app files, extra_applications and ensuring those start, etc.)
• If you have an existing project using mix_gleam, that's a great opportunity to replace it with this.
• Gleam Git dependencies.
• Just try to break it please.

Thanks

Just report back with your findings and: thanks :person_bowing:

Note: I hope these instructions are enough and correct. Let me know if that's not the case.

[josevalim]

@Papipo thanks for all the work so far!

One question: I thought that gleam shipped as a Hex package, so wouldn't it be possible for the user to depend on the gleam package or similar instead of us depending on a system wide gleam installation?

[Papipo]

@Papipo thanks for all the work so far!

One question: I thought that gleam shipped as a Hex package, so wouldn't it be possible for the user to depend on the gleam package or similar instead of us depending on a system wide gleam installation?

No, gleam is a single binary written in Rust. I don't know if we can provide a hex archive maybe? I don't know how these work.

How is rebar installation handled? I know that mix is able to install it locally, but I am not sure what that means exactly. It's still a binary as well, right?

[josevalim]

For Rebar, we have some binaries which we upload to hex.pm, and we version control them every year or so. Although I don't think this will work for Gleam, because people want to upgrade it more frequently, and Rebar is only a build tool, not really a compiler. So I guess we need to be clear we are using the system one and it is up to them to enforce the team uses the same version across the board (which is how we deal with make and the elixir executable itself).

[Papipo]

For Rebar, we have some binaries which we upload to hex.pm, and we version control them every year or so. Although I don't think this will work for Gleam, because people want to upgrade it more frequently, and Rebar is only a build tool, not really a compiler. So I guess we need to be clear we are using the system one and it is up to them to enforce the team uses the same version across the board (which is how we deal with make and the elixir executable itself).

In fact the binary is also the LS, so you definitely want to have full control of the version you are using, etc.