Static Executables with SBCL
Written on 2021-01-05 13:00:00
UPDATE: Make sure to see my follow up post to this).
Common Lisp is an amazing language with many great implementations. The image based development paradigm vastly increases developer productivity and enjoyment. However, there frequently comes a time in a program's life cycle where development pauses and a version must be delivered for use by non-developers. There are many tools available to build an executable in Common Lisp, most of which follow the theme of "construct a Lisp image in memory, then dump it to disk for later reloading". That being said, none of the existing methods fit 100% of my use cases, so this post is dedicated to documenting how I filled the gap by convincing SBCL to generate completely static executables.
There are a variety of reasons to want static executables, but the most common ones I run into personally are:
- I want to archive my executables. I want to have a version of my executables saved that I can dig up at any point in the future, long after I've upgraded my OS (multiple times), and run for benchmarking purposes, to test if old versions exhibited specific behavior, etc. without needing to recompile.
- I want to enable someone to reproduce my results exactly. This is important for reproducibility in academic contexts. Also, some computing contests that conferences organize prefer static executables so they can run tests on their hardware without needing to set up a complicated run time environment.
- I want to make it trivial for someone to install my software. With a static
executable, all anyone running on Linux needs to do is download a single
chmod +xit, and copy it onto their path (preferably after verifying its integrity, but, let's be honest, fewer people do that than should).
There certainly are issues with static executables/linking in general. If you are unaware of what they are, I highly encourage you to read up on the subject before deciding that static executables are the be-all-end-all of application delivery. Static executables are just another tool in a developer's toolbox, to be pulled out only when the time is right.
I'll pause at the moment for a clarification: when I say static executable I
mean a truly static executable. As in I want to be able to run
ldd on it
and have it output
not a dynamic executable and I do not want it to call
any libdl functions (such as
dlsym at runtime). While some
existing methods claim or imply that they make static executables with SBCL
CFFI's static-program-op or
manually linking external libraries into the SBCL runtime while building it),
they by and large mean they statically link foreign code into the runtime, but
the runtime itself is not a static executable.
I have yet to find a publicly documented method of creating a fully static executable with SBCL and it's not too hard to understand why. Creating a static executable requires statically linking in libc and the most common libc implementation for Linux (glibc) does a half-assed job at statically linking itself. While it is possible, many functions will cause your "static" executable to dynamically load pieces of glibc behind your back. Except now you have the requirement that the runtime version must match the compiled version exactly. That defeats the whole point of having a static executable!
For that reason, musl libc is commonly used when creating a truly static executable is important. Unfortunately, musl is not 100% compatible with glibc and for a while SBCL would not work with it. There have been various efforts at patching SBCL to run with musl libc throughout the years, but the assorted (minor!) changes finally got merged upstream in SBCL 2.0.5. This laid the groundwork necessary for truly static executables with SBCL.
Enough with the blabber, show me the code!
I am maintaining
a fork of SBCL that
contains the necessary patches. There is a
static-executable branch which
will always contain the latest version. I plan to rebase this branch on new
SBCL releases or on top of upstream's master branch if it looks like I'm going
to need to do some extra legwork for an upcoming release. There will also be a
series of branches named
static-executable-$VERSION which have my patches
applied on top of the named version, starting with SBCL 2.1.0.
The patch for any SBCL release is also located at
is a detached signature available at
signed with GPG
I would love to get these patches upstreamed, but they didn't get much traction the last time I submitted them to sbcl-devel. Admittedly, they were an early, less elegant version that hadn't seen much use in the real-world. My hope is that other people who desire this capability from SBCL will collaborate to test and refine these patches over time for eventual upstreaming.
Given that most people aren't using musl libc on their development computer,
the quickest, easiest way to get a static executable is to build one with
Docker. After getting the patchset, simply run the following set of commands in
the root of the SBCL repo. This will use
Docker image (another project of mine for a future post) to build a static
executable and then copy it out of the image to your host's file system.
docker build -t sbcl-static-executable -f tools-for-build/Dockerfile.static-executable-example . docker create --name sbcl-static-executable-extractor sbcl-static-executable docker cp sbcl-static-executable-extractor:/tmp/sb-gmp-tester /tmp/sb-gmp-tester docker rm sbcl-static-executable-extractor
You should now be able to examine /tmp/sb-gmp-tester to see that it is a static executable:
$ ldd /tmp/sb-gmp-tester not a dynamic executable
If all goes well, you should also be able to run it, see the sb-gmp contrib tests all pass (fingers crossed), and realize that this worked because libc, the SBCL runtime, and libgmp were all statically linked!
The file README.static-executable (after applying the patchset) has an example of building locally and a set of docker commands that doesn't require tagging images and naming containers.
How does it work??
This approach requires that the target image be be built twice: once to record the necessary foreign symbols, and then again with the newly built static runtime. I can, however, envision ways around this for a sufficiently motivated person.
One way could be to modify the (already in-tree) shrinkwrapping recipe to handle libdl not being available at runtime. I abandoned this approach largely because the shrinkwrapping code is written for x86-64 and does a lot of things with assembly (which I do not know). It is important for me to have static executables on ARM as well. A second way could be to patch out or otherwise improve the check that the runtime version used to build the core matches the runtime version used to run it. I didn't go this approach as it would certainly lead to difficult to debug issues if used incorrectly, plus the Lisp code in the core would need to check the presence/usefulness of libdl functions at runtime.
So, how does this patchset work and why does it require two passes? Apologies to the SBCL devs if I completely butcher the explanation of SBCL internals, but here it goes anyways!
Lisp code routinely calls into C code, whether it is to a runtime provided
function, a libc function, or another library the user has linked and defined
sb-alien package or the portable counterparts in CFFI. In order to
mediate these calls from the Lisp side, SBCL maintains a linkage table. This
table has two components. First is a Lisp-side hash table that maps foreign
names (and an indicator of if it is data or a function) to an integer. The
second is a C-side vector that contains either the address of the symbol (in
the case of data) or the opcodes necessary to call the function (e.g., by
JMPing to its address).
The C-side vector is populated by looking up the symbol's address using
dlsym. This lookup generally happens under two possible scenarios. First,
when the Lisp code defines a foreign symbol it wants to be able to call or
read. Second, every time the runtime starts, it populates the C-side entries
for every symbol contained in the core's hash-table. This second case is how
SBCL handles the dynamic linker changing the address of symbols in between core
This reliance on
dlsym is so baked into SBCL at this point that,
even though the code is nominally conditioned on the internal feature
:os-provides-dlopen, I was unable to build a working SBCL without it (before
these patches, of course).
With these patches, you first build your Lisp image that you want to deliver
like normal. Then, you load the file
into it. Next, you call
sb-dump-linkage-info:dump-to-file to extract the Lisp
side linkage table entries into a separate file (filtered to remove functions
libdl). Once you have this file, you rebuild SBCL, this time with the
intention of creating a static runtime. To do this, you should provide the
- The environment variable
-no-pie -staticin order to build the static runtime.
- Any additional libraries you need should be specified using the environment
- You probably want to set the environment variable
- You need to pass the file containing the linkage table entries to
- Build without the
:os-provides-dladdrfeatures. One way of doing this is to pass
During the build process, the contents of the
file are inserted into the cold SBCL core during second genesis and a C file is
autogenerated containing a single function that populates the C side of the
linkage table using the address of every symbol. This C file is the built into
the runtime and called while the runtime boots, before it starts executing the
core. This means that, if the runtime is a dynamic executable, the system
linker will patch up all the references we need at runtime without SBCL needing
dlsym explicitly. If the runtime is a static executable, then the
symbols are statically linked for us and nothing needs to be done at runtime.
Given how new this approach is, you will certainly run into issues. Many
systems that load foreign code will blindly assume that libraries can be linked
in at runtime and will fail to work (silently or loudly) if that assumption is
not met. Some libraries already have their own homebrew ways of dealing with
this. For instance, if the feature
present, the cl+ssl system will not attempt to load the libraries. To deal with
this issue in a more principled way, I strongly recommend patching systems to
use CFFI's (relatively)
canary argument to
CFFI itself also has some issues with this arrangement because it dives into
sb-alien internals that simply aren't present on
#-os-provides-dlopen. I currently fix this in a kludgy way by commenting out
src/cffi-sbcl.lisp, but if more people
start building static executables, we'll need to come up with a better way of
I would love to get feedback on this approach and any ideas on how to improve it! I strongly believe that better support for building static executables with SBCL should be upstreamed and I doubt I am alone in that belief. Please drop me a line (etimmons on Freenode or daewok on Github/Gitlab) if you have suggestions.
Personally, I have used earlier iterations of these patches to build static executables for some of my grad school work. My next real deployment of these patches will likely be to build CLPM with them and providing static executables starting with v0.4.