Init Mobilizon #119132

minijackson · 2021-04-11T12:06:09Z

Motivation for this change

Mobilizon is an online tool developed by Framasoft to help manage your events, your profiles and your groups.

This PR is based on #119049, and is a draft PR until #119049 is merged. I can't really make a PR to nixpkgs that merges into my own branch, so you will find #119049 commits in here too while it is open.

Note that the package and the module has a single maintainer, instead of the whole BEAM team, since I thought this was not a critical part of the BEAM ecosystem in Nixpkgs, and we might not want to have the BEAM team manage every BEAM packages and modules, but this is obviously up for discussion.

CC @NixOS/beam for this exact question, and since it is the first NixOS module integrating an Elixir service.

Things done

pkgs/servers/mobilizon/js.nix

pkgs/servers/mobilizon/default.nix

pkgs/servers/mobilizon/cldrLocales.nix

nixos/modules/services/web-apps/mobilizon.nix

nixos-discourse · 2021-04-11T17:38:24Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/announcing-mixnix-build-elixir-projects-with-nix/2444/20

RaitoBezarius · 2021-04-11T18:22:22Z

I'm okay to step in and become a maintainer if you want, let me know if I can do to help the two PRs to get merged faster.

nixos/modules/services/web-apps/mobilizon.nix

happysalada · 2021-04-12T01:39:03Z

This is great work!
I know see better how you are using the elixir config.
Perhaps in the future, a runtime.exs file can be submitted upstream, so that the only thing needed is environment variable.
I have to say I didn't anticipate this need for elixir configurations, I'm still hoping secretly that packaging other apps won't be that involved.

I see what you mean with the problem around atom and strings. It does seem a little weird. I don't have a solution yet either. I do think what you have is enough to get things started.

happysalada · 2021-04-12T02:30:33Z

I just opened an issue upstream to see if they would be open to adding a runtime.exs
https://framagit.org/framasoft/mobilizon/-/issues/678
(in case you are interested)

minijackson · 2021-04-12T08:42:31Z

@happysalada You reminded me that I wanted to talk about the runtime.exs case with Mobilizon, but then forgot, by bad...

I think Mobilizon's way of handling configuration since 1.1.0 is just straight up better than using runtime.exs, they added a configuration provider that allows you to load configuration from /etc/mobilizon/config.exs (better for NixOS since we don't have to modify a file in the Nix store), or load a configuration from a file specified in the $MOBILIZON_CONFIG_PATH environment variable (even better IMHO).

But about having an upstream configuration which will fetch environment variables for configuration values, I just don't think it's feasible, unfortunately, there simply are to much configuration variables (it would mean a lot of work upstream to add an environment variable for everything, and work downstream to keep up to date). Also this poses some type issues, since the configuration is rich in type, but environment variables are only strings, complex configuration options would need to be parsed, and we kinda go back to the same problem. (For example, I don't see a simple way of using env variables to configure Mobilizon resource providers)

I'm probably biased, but if we're just thinking about the configuration part, apart from having to specify Elixir types, the definition and configuration of Mobilizon's options look like an idiomatic RFC42-style NixOS module.

happysalada · 2021-04-12T08:57:04Z

I see what you mean, Mobilizon looks like a very complex app.
Since you have more experience with it, I'll defer to your opinion on the possibility of having a runtime.exs.

minijackson · 2021-04-12T09:22:05Z

Thanks for stepping in @RaitoBezarius! I don't see you in the maintainer-list.nix file, do you want me to add you?

let me know if I can do to help the two PRs to get merged faster

I think the most important thing right now is reviewing #119049 and see if it looks like something we're fine having as a settings format.

happysalada · 2021-04-12T09:24:57Z

nixos/tests/mobilizon.nix

@@ -0,0 +1,24 @@
+import ./make-test-python.nix ({ lib, ...}:


I'm definitely not saying you should do it here, but maybe add a comment on running the tests from the repo at some point?
I know it would require to repackage the app, so no need to do it in this PR.

Could you expand on what you mean by "from the repo"? Do you mean not using Elixir releases, or not using the NixOS module?

Sorry for being vague here.

When I meant tests from the repo, I was thinking about the test suite, that is run in elixir with mix test. I realize though that I don't think you can do it with a release. You can probably abuse the mixRelease, and override to not output a release but just run mixTest.

Let me know if anything doesn't make sense.

Oh, I see! This seems like something we definitely want to do. I think it would be best to do it directly in the package derivation, or even better, to do it in mixRelease. I don't know yet if Mobilizon wants a particular test environment

The only problem I see with doing it in the mixRelease, is that you need to compile your stuff twice, once for the tests and once for the final release.

happysalada · 2021-04-12T09:25:25Z

nixos/modules/services/web-apps/mobilizon.nix

+                  };
+
+                  http.port = mkOption {
+                    type = elixirTypes.port;


is that correct? Did you add a port type in the elixirTypes?

Oh is that a standard type that has been wrapped in an elixir type. Ok I think I get it.

Yes, this is basically to allow the end user to put Elixir code if they want. That way, they can do:

{ pkgs, ... }: let format = pkgs.formats.elixirConf { }; in { config.services.mobilizon.settings.":mobilizon"."Mobilizon.Web.Endpoint".http.port = format.lib.mkGetEnv { envVariable = "MOBILIZON_LISTEN_PORT"; }; }

happysalada · 2021-04-12T09:30:21Z

@minijackson you still seem to have an ofborg eval failure, https://gist.github.com/GrahamcOfBorg/fc3ab6595c4bd3155a2ecc221198ba39

pkgs/servers/mobilizon/package.json

happysalada · 2021-04-13T02:16:02Z

About the runtime.exs issue,
the maintainers have answered that there is a docker.exs file in config
https://framagit.org/framasoft/mobilizon/-/blob/master/config/docker.exs
that they use as a runtime.exs and in the building of the docker image, they rename that to runtime.exs.
I'm just saying that if you feels it helps you, use it. Otherwise feel free to proceed in the way you think is best, you are the maintainer after all.

happysalada · 2021-04-13T12:34:50Z

nixos/modules/services/web-apps/mobilizon.nix

+            install -m 600 /dev/null "${secretEnvFile}"
+            cat > "${secretEnvFile}" <<EOF
+          # This file was automatically generated by mobilizon-setup-secrets.service
+          export MOBILIZON_AUTH_SECRET='$(${evalElixir} "${genSecret}")'


I was just thinking about this. The problem here is that all the secrets go in the nix store right?
I'm not sure about the file itself, but this makes all the variables world readable.

The elixir commands used to generate the secrets could potentially come from another cli tool. As far as I know there is no strong requirements that the secrets get generated by elixir. Perhaps we should advise the user to generate his secret file himself and just accept it as a configuration input.

The problem is that there is currently no way of specifying secrets except by asking the user to place a file somewhere outside of the nix store. This makes automated deployment more complicated, and I wanted a module where you could just enable the service and specify the configuration required by upstream, and it just works. There is no documentation in the Mobilizon documentation on how to generate the cookie, or how to generate the authentication secret, the instance secret, etc. This would be just extra steps needed to circumvent NixOS' particularities. At the end of the day, I thought the user didn't need to care about that, and that it could be automated.

I think it's a worthwhile compromise, maybe just add a comment to clarify the approach?

nixos/modules/services/web-apps/mobilizon.nix

happysalada · 2021-05-01T02:47:03Z

one question, have you tried running the release with a systemd service?

I'm asking because I'm running into a weird situation where I can start the release with the command line. But for some reason the systemd service keeps on craching with a weird error. It seems to be a problem with the file system being read only, but I can't figure out which path it's trying to write. This is what I have in the logs

May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]: lvl=error msg=GenServer :disksup terminating
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]: ** (ArgumentError) argument error
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]:     :erlang.port_close(#Port<0.36>)
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]:     (os_mon 2.6.1) disksup.erl:169: :disksup.terminate/2
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]:     (stdlib 3.14.2) gen_server.erl:727: :gen_server.try_terminate/3
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]:     (stdlib 3.14.2) gen_server.erl:912: :gen_server.terminate/10
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]:     (stdlib 3.14.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]: Last message: :timeout
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]: State: [data: [{'OS', {:unix, :linux}}, {'Timeout', 1800000}, {'Threshold', 80}, {'DiskData', []}]]
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313461]: [os_mon] cpu supervisor port (cpu_sup): Erlang has closed
May 01 02:41:51 hetzner-AX41-UEFI-ZFS-NVME union[3313283]: lvl=info msg=Application os_mon exited: shutdown

I've verified that RELEASE_TMP is set to /tmp
I've verified that ReadWriteWaths = "/run/postgresql" (checked that this is where the postgres socket is)
I've configured the tz data directory to be the same as StateDirectory.
The RELEASE_COOOKIE is set, and so is the secret key base.
The HTTP_PORT is set as well (to 4080).

If you know a way to inspect the sys calls, or if you have some insights in debugging a similar issue, I'm interested.
I can't find the crash.dump anywhere. If you know where it gets written to, I'm interested.

erictapen · 2023-08-29T14:43:44Z

@happysalada Now that the blocking change to mkYarnPackage is gone, I consider this good to go.

Only question left might be what you think about my change to pkgs/development/beam-modules/build-mix.nix. From how I grew to understand the mix build system, as an application developer you can configure your dependencies in an extremely flexible way by putting build-time configuration into config/. In Mobilizon's case some dependencies need to able to read that config at build time. I made that possible by setting the new appConfig arg to the config/ dir of Mobilizon src for every Mix dependency.

This is of course bad news for Nix users, as every change to the source code is going to invalidate every single Mix dependency. Therefore I made the appConfig option opt-in.

Would you like to have a separate PR for that or would you consider this uncontroversial enough to merge right away? If yes, just give me the sign to squash everything :)

happysalada · 2023-09-04T00:41:57Z

pkgs/development/beam-modules/build-mix.nix

@@ -45,6 +47,14 @@ let
      runHook preConfigure

      ${./mix-configure-hook.sh}
+      ${if isNull appConfigPath then "" else


just a nit, let's use optionalString here if possible.

happysalada · 2023-09-04T00:44:10Z

I just have one nit (the one with optional string).
Could you squash the commits in like 2-3 commits max and rebase on master ?
(ofborg eval is failing and there are rebase conflicts).

regarding the change to build-mix, the honest answer is I have no idea. This looks good on principle so I would say, let's go ahead and merge it, and we will probably understand things better in the future.
I would perhaps add to the comment as to why this was added.
Something like this was added because mobilizon needs to be able to read the config file of some of its dependencies So that we don't loose track of why this was added.

This is added for Mobilizon to be able to let it's dependencies read it's config directory

erictapen · 2023-09-05T16:46:49Z

@happysalada Squashed and pushed :)

happysalada · 2023-09-05T17:59:38Z

ofborg is failing with a weird error https://gist.github.com/GrahamcOfBorg/3f7d64ede074b68e0d68783babe0c3d9
looks like coming from something with yarn2nix.

erictapen · 2023-09-06T12:21:54Z

Let's try reintroducing yarn.nix into the tree, I was already wondering how this would work 🤷‍♀️, but it seemed fine on my system yet.

happysalada · 2023-09-06T12:39:53Z

it doesn't look like this is related, it looks like ofborg is still failing.

erictapen · 2023-09-06T12:52:48Z

Another try: The eval error complains about not being able to read the package.json from source, so I added it in tree (as most of mkYarnPackage calls seem to).

erictapen · 2023-09-06T14:13:55Z

Yay, it worked (with a sha256 for the offlineCache). I'll squash again.

Co-Authored-By: Minijackson <[email protected]> Co-Authored-By: summersamara <[email protected]>

happysalada · 2023-09-07T08:59:58Z

@erictapen thank you for sticking with this for so long!

erictapen · 2023-09-07T10:34:52Z

I can't believe this happened 😁
Thank you and all the other reviewers for showing interest in this! And special thanks to @minijackson, who started this.

minijackson · 2023-09-07T16:13:35Z

@erictapen thank you for actually making it to the end! This wouldn't have been possible without your continuous efforts. Sorry that I haven't been active in this PR lately…

On a side note, can I delete the branch?

happysalada · 2023-09-07T16:16:39Z

Yes as far as im concerned

SuperSandro2000 · 2023-09-12T21:44:45Z

FYI building the docs with mobilizon enabled yields those warnings:

[root@hydra:~]# nix-store --read-log /nix/store/1nnp0p3n7vdxa8hjnqz58nqgj12jlbg0-options.json.drv | cat
warning: option services.mobilizon.enable description uses DocBook
warning: option services.mobilizon.nginx.enable description uses DocBook
warning: option services.mobilizon.settings description uses DocBook
warning: option services.mobilizon.settings.":mobilizon".":instance".email_from description uses DocBook
warning: option services.mobilizon.settings.":mobilizon".":instance".email_reply_to description uses DocBook
warning: option services.mobilizon.settings.":mobilizon".":instance".hostname description uses DocBook
warning: option services.mobilizon.settings.":mobilizon".":instance".name description uses DocBook
warning: option services.mobilizon.settings.":mobilizon"."Mobilizon.Storage.Repo".database description uses DocBook
warning: option services.mobilizon.settings.":mobilizon"."Mobilizon.Storage.Repo".socket_dir description uses DocBook
warning: option services.mobilizon.settings.":mobilizon"."Mobilizon.Storage.Repo".username description uses DocBook
warning: option services.mobilizon.settings.":mobilizon"."Mobilizon.Web.Endpoint".has_reverse_proxy description uses DocBook
warning: option services.mobilizon.settings.":mobilizon"."Mobilizon.Web.Endpoint".http.ip description uses DocBook
warning: option services.mobilizon.settings.":mobilizon"."Mobilizon.Web.Endpoint".http.port description uses DocBook
warning: option services.mobilizon.settings.":mobilizon"."Mobilizon.Web.Endpoint".url.host description uses DocBook
Explanation: The documentation contains descriptions, examples, or defaults written in DocBook. NixOS is in the process of migrating from DocBook to Markdown, and DocBook is deprecated for option documentation. To change your module to use Markdown, apply mdDoc and literalMD and use the *MD variants of option creation functions where they are available. For example:

  example.foo = mkOption {
    description = lib.mdDoc ''your description'';
    defaultText = lib.literalMD ''your description of default'';
  };

  example.enable = mkEnableOption (lib.mdDoc ''your thing'');
  example.package = mkPackageOptionMD pkgs "your-package" {};
  imports = [ (mkAliasOptionModuleMD [ "example" "args" ] [ "example" "settings" ]) ];

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` labels Apr 11, 2021

minijackson mentioned this pull request Apr 11, 2021

Add Elixir configuration settings format #119049

Merged

10 tasks

SuperSandro2000 reviewed Apr 11, 2021

View reviewed changes

happysalada reviewed Apr 12, 2021

View reviewed changes

nixos/modules/services/web-apps/mobilizon.nix Outdated Show resolved Hide resolved

happysalada reviewed Apr 12, 2021

View reviewed changes

nixos/modules/services/web-apps/mobilizon.nix Outdated Show resolved Hide resolved

minijackson requested review from SuperSandro2000 and happysalada April 12, 2021 09:19

happysalada reviewed Apr 12, 2021

View reviewed changes

ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Apr 12, 2021

happysalada reviewed Apr 12, 2021

View reviewed changes

pkgs/servers/mobilizon/package.json Show resolved Hide resolved

happysalada reviewed Apr 13, 2021

View reviewed changes

happysalada reviewed Apr 26, 2021

View reviewed changes

nixos/modules/services/web-apps/mobilizon.nix Show resolved Hide resolved

happysalada reviewed Apr 26, 2021

View reviewed changes

nixos/modules/services/web-apps/mobilizon.nix Show resolved Hide resolved

happysalada reviewed Apr 26, 2021

View reviewed changes

nixos/modules/services/web-apps/mobilizon.nix Show resolved Hide resolved

github-actions bot removed the 6.topic: nodejs label Aug 29, 2023

happysalada reviewed Sep 4, 2023

View reviewed changes

beam/buildMix: add appConfigPath arg

2ce3910

This is added for Mobilizon to be able to let it's dependencies read it's config directory

erictapen force-pushed the init-mobilizon branch from afd2449 to b7d448e Compare September 5, 2023 16:47

github-actions bot added 8.has: documentation This PR adds or changes documentation 8.has: changelog labels Sep 5, 2023

ofborg bot requested a review from erictapen September 6, 2023 13:10

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Sep 6, 2023

erictapen and others added 2 commits September 6, 2023 16:14

mobilizon: init at 3.1.3

8c32950

Co-Authored-By: Minijackson <[email protected]> Co-Authored-By: summersamara <[email protected]>

nixos/mobilizon: add release notes

799e90c

erictapen force-pushed the init-mobilizon branch from c45252e to 799e90c Compare September 6, 2023 14:14

happysalada merged commit fc67d29 into NixOS:master Sep 7, 2023

minijackson deleted the init-mobilizon branch September 8, 2023 16:35

erictapen mentioned this pull request Sep 13, 2023

nixos/mobilizon: convert to mdDoc, remove obsolete override #255004

Merged

12 tasks

Init Mobilizon #119132

Init Mobilizon #119132

Conversation

minijackson commented Apr 11, 2021

Motivation for this change

Things done

nixos-discourse commented Apr 11, 2021

RaitoBezarius commented Apr 11, 2021

happysalada commented Apr 12, 2021

happysalada commented Apr 12, 2021

minijackson commented Apr 12, 2021

happysalada commented Apr 12, 2021

minijackson commented Apr 12, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

happysalada commented Apr 12, 2021

happysalada commented Apr 13, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

happysalada commented May 1, 2021 • edited Loading

erictapen commented Aug 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

happysalada commented Sep 4, 2023 • edited Loading

erictapen commented Sep 5, 2023 • edited Loading

happysalada commented Sep 5, 2023

erictapen commented Sep 6, 2023

happysalada commented Sep 6, 2023

erictapen commented Sep 6, 2023

erictapen commented Sep 6, 2023

happysalada commented Sep 7, 2023

erictapen commented Sep 7, 2023

minijackson commented Sep 7, 2023

happysalada commented Sep 7, 2023

SuperSandro2000 commented Sep 12, 2023

minijackson commented Apr 12, 2021 •

edited

Loading

happysalada commented May 1, 2021 •

edited

Loading

happysalada commented Sep 4, 2023 •

edited

Loading

erictapen commented Sep 5, 2023 •

edited

Loading