Reading the os environment in boot.janet

[jackhill]

I'm working on improving the Janet and jpm-installable module experience on Guix. Currently, we don't have any module packages, but this is something that I'd like change. Guix, like its cousin Nix, installs software differently than traditional operating systems. Essentially, rather than installing all packages into the same central location, each package gets its own directory hierarchy. Multiple views representing different subsets of packages can be created and used for system-wide installed software, software installed for each user, temporary development environments, etc.

For this use case, I don't think setting JANET_PATH would be sufficient since it only contains one directory. I think what I need is a search path that supports a colon separated list of directories where modules can be found. In order to get that, I'm thinking of adding a Guix specific patch to Janet in our package that add such a search path via a GUIX_JANET_PATH environment variable. We've used similar strategies with other software before to good effect.

I'm working on a proof of concept currently, and can fairly easily add entries to module/paths in src/boot/boot.janet. However, I'm stuck on how to read environment variables. In boot.janet, (os/getenv "GUIX_JANET_PATH") returns nil.

I have two questions:

1. How can I read environment variables in boot.janet?
2. What are your thoughts on this general approach?

[jackhill]

Oh, I think I now see part of what I was missing: the boot.janet code runs when building Janet, not when running it later. I guess my question becomes generally: how can I teach Janet about my custom environment variable?

[jackhill]

Ok, I think I've figured out that I should do my custom code in the beginning of cli-main. This is the patch that I'm going to propose we add to our Janet package in Guix:

author: Jack Hill <[email protected]>
diff --git a/src/boot/boot.janet b/src/boot/boot.janet
index feb577ad..fa35cb7d 100644
--- a/src/boot/boot.janet
+++ b/src/boot/boot.janet
@@ -3347,6 +3347,26 @@
   (if-let [jp (getenv-alias "JANET_HEADERPATH")] (setdyn :headerpath jp))
   (if-let [jprofile (getenv-alias "JANET_PROFILE")] (setdyn :profilepath jprofile))
 
+  # Guix customization to make Janet respect GUIX_JANET_PATH
+  # environment variable.  GUIX_JANET_PATH is a native search path
+  # added by Guix to tell Janet where to find modules install in the
+  # same profile via Guix packages.  This is done by augmenting
+  # Janet's module/paths array.
+  (def- guix-janet-path (os/getenv "GUIX_JANET_PATH"))
+
+  (defn- add-guix-path [ext loader]
+    (if guix-janet-path
+      (each elem (reverse (string/split ":" guix-janet-path))
+       (array/insert module/paths 1 [(string elem "/:all:" ext) loader check-is-dep]))))
+
+  # Add the Janet standard extensions and loaders.  List copied from
+  # the calls to module/add-paths in src/boot/boot.janet.
+  (add-guix-path ":native:" :native)
+  (add-guix-path "/init.janet" :source)
+  (add-guix-path ".janet" :source)
+  (add-guix-path ".jimage" :image)
+  # End of Guix customization
+
   # Flag handlers
   (def handlers
     {"h" (fn [&]

Does this seem reasonable? Can you think of a better way for Guix and Janet to play nicely together?

[andrewchambers]

JANET_PATH already supports colon separated lists - not sure where you got the impression it does not.

edit: I seem to be totally wrong and you are right. Looks pretty good to me in that case. Another approach might be to make a symlink farm to create a virtual JANET_PATH.

I think the 'Nix' way would be to have a nix function that takes in a list of janet packages and returns a janet with a wrapper that sets JANET_PATH to these paths.

[jackhill]

Thanks for your reply. Yeah, this isn't the only way to do it, but so far I think it's turning out to be pretty natural. Once I polish it up and submit it to Guix it will get reviewed there as well. One nice thing about using a new variable is that it doesn't get in folks ways if they want to use JANET_PATH for their own purposes as well.

[bakpakin]

The proposed patch looks pretty good to me. However, I think guix packages should go at the end of module/paths and not take precedence over local files. The paths are tried in order, so inserting them at the end would make more sense, or at least at the same position as the :sys: entries

> (each x module/paths (pp x))
(<function is-cached> :preload <function check-is-dep>)
(":cur:/:all:.jimage" :image <function check-relative>)
(":cur:/:all:.janet" :source <function check-relative>)
(":cur:/:all:/init.janet" :source <function check-relative>)
(":cur:/:all::native:" :native <function check-relative>)
# Insert new paths here
(":sys:/:all:.jimage" :image <function check-is-dep>)
(":sys:/:all:.janet" :source <function check-is-dep>)
(":sys:/:all:/init.janet" :source <function check-is-dep>)
(":sys:/:all::native:" :native <function check-is-dep>)
# or here
(".:all:.jimage" :image <function check-project-relative>)
(".:all:.janet" :source <function check-project-relative>)
(".:all:/init.janet" :source <function check-project-relative>)
(".:all::native:" :native <function check-project-relative>)

Should probably come after the preload and current loaders, though.

[jackhill]

Thanks for considering it. I think you're right at the end will be best. I don't have a good reason for putting them at the beginning.

I'll update my patch and then submit it to Guix for review there. I'm pretty excited to see Janet support in Guix improving.

[jackhill]

I adjusted it to add the Guix paths at the end. I also changed it to add all the loaders for each path component next to each other instead of grouping by loader type:

diff --git a/src/boot/boot.janet b/src/boot/boot.janet
index 59e0a016..f9174916 100644
--- a/src/boot/boot.janet
+++ b/src/boot/boot.janet
@@ -3328,6 +3328,30 @@
   (if-let [jp (getenv-alias "JANET_HEADERPATH")] (setdyn :headerpath jp))
   (if-let [jprofile (getenv-alias "JANET_PROFILE")] (setdyn :profilepath jprofile))
 
+  # Guix customization to make Janet respect GUIX_JANET_PATH
+  # environment variable.  GUIX_JANET_PATH is a native search path
+  # added by Guix to tell Janet where to find modules install in the
+  # same profile via Guix packages.  This is done by augmenting
+  # Janet's module/paths array.
+  (def- guix-janet-path (os/getenv "GUIX_JANET_PATH"))
+
+  # The Janet standard extensions and loaders.  List copied from
+  # the calls to module/add-paths in src/boot/boot.janet.
+  (def- default-janet-loaders
+    [[":native:" :native]
+     ["/init.janet" :source]
+     [".janet" :source]
+     [".jimage" :image]])
+
+  (defn- add-guix-path [p]
+    (each [ext loader] default-janet-loaders
+      (array/push module/paths [(string p "/:all:" ext) loader check-is-dep])))
+
+  (if guix-janet-path
+    (each elem (string/split ":" guix-janet-path)
+      (add-guix-path elem)))
+  # End of Guix customization
+
   # Flag handlers
   (def handlers
     {"h" (fn [&]

I still have a little more cleanup to do on the Guix side of things, but I hope to submit it there for review before too long.

[omarbassam88]

I already actually have submitted a patch for Guix and I got it to work in guix shell, guix home and guix system. Could you have a look and let me know if we can improve it using your suggestions. I mean instead of having multiple patches submitted.

I got it to work using the current jpm code with a few substitutions.

https://issues.guix.gnu.org/72925

[amano-kenji]

That's jpm. To make jpm packages for guix, you are going to need janet-build-system.

[omarbassam88]

I was not able to find any information in guix about janet-build-system. Where should I be looking for this information?

[amano-kenji]

I think I am going to create janet-build-system in the future?