on Guix for Fennel

February 9, 2024

For the past two weekends I've been trying Guix out. Which means: I AM NO GUIX EXPERT. Everything stated here about Guix has a non-zero chance of being incorrect.

Here I will gather my takeaways and conclusions about the system, as well as provide a starting point for Lua and Fennel developers that want to use Guix as their package and dependency manager.

NOTICE: This post will only cover .fnl and .lua dependencies.

I will explore adding C dependencies at a further date.

Key takeaways

Concepts

While the Guix documentation is pretty thorough and well written, I've struggled quite a bit with it. I believe the main reason is: You kind of need a big picture of Guix in order to fully understand each part of the system, which are treated with detail in the docs.

I'll provide a brief note on each of the relevant components in order to ease your reading of the documentation.

Packages

Each piece of software available from Guix is a defined package. Defining a package feels similar to instantiating a record or object in any programming language. You fill in their fields. You can see how I defined some packages for a Lua and Fennel libraries in this repo.

Here you get to define how to build the library and where does the source live.

The official Lua build system is supported out of the box with the copy-build-system.

Channels

As you throw your packages in different .scm files and you put them under version control, you get a channel.

Now you can point Guix to the directory under version control, and install your defined packages from your custom channel.

The repo liked above is actually a Guix channel.

Profiles

While channels and packages stand out in the docs, it took me a while to find profiles and figure out them as a key piece for my use case.

Profiles are where you install packages. Make sure to take a look at the section on the cookbook once you have a grasp on the basics. It's a powerful tool.

Working with Guix: Strengths, uncertainties and limitations.

Writing packages

The first thing to do in order to use Guix, is get some packages. If the packages you want to use are not available through the official Guix channels, you get to define your own.

The official Lua build system is supported out of the box with the copy-build-system. The usual workflow for our environment is:

  • Clone the git repo for the dependency you want to add to your project.
  • Copy the relevant files into your project folder.

On a Guix package you get to define the git repo of the sources, which version/tag of the source to checkout and where to copy it.

Strength: Through Guix we get a reproducible way to handle our dependencies.

Here is my definition of the faith Fennel testing library as a Guix package.

(define-module (faith)
  #:use-module (guix build-system copy)
  #:use-module (guix build copy-build-system)
  #:use-module (guix build utils)  
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix packages)
  #:use-module (guix git-download))
(define-public faith
  (let ((faith-git "https://git.sr.ht/~technomancy/faith")
        (version "0.1.2"))
    (package
     (name "faith")
     (version version)
     (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url faith-git)
                    (commit version)))
              (sha256
               (base32
                "1qigr8rjxby9fir2rzh95ndzf293zvmlf8dqchwcmk4zlwpajph7"))))
     (build-system copy-build-system)
     (arguments
      '(#:install-plan
        '(("./faith.fnl" "./faith.fnl"))))
     (synopsis "The Fennel Advanced Interactive Test Helper.")
     (description "The Fennel Advanced Interactive Test Helper.")
     (home-page faith-git)
     (license license:expat))))

One of the notable things to do when defining a Guix package from git is getting the sha256 hash. Guix provides a utility guix hash -rx . that you can run on the cloned and checked out repo of the library. At this point, Guix is just giving us extra steps as compared with the standard workflow. But once you've got it defined, everybody can use your package!

Uncertainties: Decisions to make

The other notable thing to do when defining the Guix package is choosing where to install (I mean, just copy) the library. The above example moves the faith.fnl file from within the package into the PROFILE DIRECTORY. And now is where I start having doubts. Let's explore them together.

I've written a minimal example, available at this repo. Clone it!

  git clone --recurse-submodules https://git.sr.ht/~jiglesias/fennel-guix-example
  cd fennel-guix-example/

There you should have some .fnl files, and the aforementioned Guix channel, provided as a git sub-module.

Now, if you try to run any of the .fnl files, it should not work. You do not have the required dependencies. But we can solve that with Guix.

  guix install lume faith -L./fennel-channel --profile=./deps
  • install will install the following packages
  • -L points to the channel
  • --profile shows where the dependencies will get installed.

Which means the dependencies will be copied into a brand new deps folder on your current directory.

Run the test-loader.fnl script, which uses a bunch of dependencies and:

Behold! Your dependencies got managed!

  fennel test-loader.fnl
    Starting module test with 3 test(s)
    ..F

    FAIL: ./deps/faith.fnl:232: test-fail
      Expected 1, got 2
  
    Testing finished in approximately 0 second(s) with 3 assertion(s)
    2 passed, 1 failed, 0 error(s), 0 skipped
    0.01 second(s) of CPU time used

Now you can do your Fennel interactive development from your favorite text editor.

Trouble in paradise: Guix shell

One of the coolest things about Guix is it's shell with the container option. It offers standalone software environments. Unpolluted by your host system. Pretty much a better Docker container. A playground to share a work environment with other developers.

Try it out:

  guix shell -CP -L./fennel-channel

And try to run the Fennel script. OOOOPS.

  runtime error: module 'deps.faith' not found:
          no field package.preload['deps.faith']
          no file '/usr/local/share/lua/5.3/deps/faith.lua'
          no file '/usr/local/share/lua/5.3/deps/faith/init.lua'
          no file '/usr/local/lib/lua/5.3/deps/faith.lua'
          no file '/usr/local/lib/lua/5.3/deps/faith/init.lua'
          no file './deps/faith.lua'
          no file './deps/faith/init.lua'
          no file '/usr/local/lib/lua/5.3/deps/faith.so'
          no file '/usr/local/lib/lua/5.3/loadall.so'
          no file './deps/faith.so'
          no file '/usr/local/lib/lua/5.3/deps.so'
          no file '/usr/local/lib/lua/5.3/loadall.so'
          no file './deps.so'
          no file './deps/faith.fnl'
          no file './deps/faith/init.fnl'
  stack traceback:
    [C]: in function 'require'
    test-loader.fnl:1: in main chunk
    [C]: in function 'xpcall'
    /home/joaquin/.guix-profile/bin/fennel:6267: in function ?
    [C]: in ?

The deps folder is there. But it's empty. Where are the dependencies? Where you should expect them, where I told you, at the default profile: ~/.guix-profile. You did not specify to use a different profile. Let's go a step further, take a look at the folders where Lua is trying to find our dependency. ls that path. Yup. It does not exist.

Rather than installing the deps at a given profile in my folder ./deps. I could install them at the default one ~/.guix-profile. I feel this is somewhat "non-idiomatic", but I am happy to stand corrected.

The main issue then would be, you would need to set the LUA_PATH environment variable to your profile. BUT, while Guix provides some facilities to set up some env variables, like search paths or files within those search paths. But LUA_PATH can't be a search path. It's not a directory nor a list of files. It's a string starting with the path to search, and ending with "?.lua;".

I have not found a way to convince Guix to do just that… Now, you could just set the LUA_PATH once in the Guix shell. And then things would work. But it's a minor annoyance and definitely a:

Limitation

And it also begs the question to Fennel developers. One of the most valuable assets of Fennel is the interactivity. And I am wondering, how feasible is that once you get into Guix shelling? It would be possible to setup some of our favorite text editors within the shell, but that also sounds overkill.

Post Data

At the point of writing these lines, I have the feeling that all the limitations I see, may be solved with a judicious use of profiles, setting AN environment variable and maybe tweaking a little the Lua package definition. But I am yet to explore that option.

Conclusion

Guix solves the dependency management problem for Fennel and Lua. But it feels like it's an either-or situation. Either you get you libraries conveniently in your path, or you get a containerized development environment.

I'm confident there is a good solution for these use-cases within Guix. It's a pretty large system, with many options I am yet to explore. I believe that the facilities it offers for packaging and distribution will be pretty interesting in the Fennel and Lua space, particularly where C sources are involved.

I hope this post speeds up further research on the topic. It's definitely helped me solidify my understanding of Guix, a tool I will keep tinkering with and I hope you'll do too.