libjulia.el

What is this?

NOTE: This project is in-development not ready for production use! See the end of this document for notes on how to contribute.

libjulia.el uses Julia’s C API to embed Julia into Emacs via emacs-ffi.

Why?

Augmenting Julia Development Tools in Emacs

Emacs is a powerful environment for building and using development tools, especially for lispy languages like Julia. This project gives Emacs Lisp direct and efficient access to Juila’s runtime to aid the development of tools like:

• Debuggers
• Structure editors (like paredit, smartparens, and lispy)
• Inline expression evaluation, macro-expansion, and views of lowered forms
• Auto-formatters, linters, refactoring tools

Note that Emacs already has interfaces to Julia via the Julia REPL, a Language Server, and Jupyter kernels. However, none of them provide any of these facilities.

Augmenting Emacs with Julia

Some Emacs extensions could benefit from being written in Elisp + Julia. This project enables this possibility.

Getting Started

Requirements

System

This has only been tested on Ubuntu 16.04.5 with Emacs 27.0.5 and Julia 1.0.2*.

I suspect this works on most modern GNU/Linux systems, and that OSX can be made to work with some tweaks.

Emacs

You’ll need Emacs 25+ built with dynamic module support, as well as the emacs-ffi package installed.

Ensure that ffi-module.so built during the emacs-ffi installation is in your load path. One way to do this is by launching emacs like this:

LD_LIBRARY_PATH=<path-to-emacs-ffi-dir> emacs

See the tests in emacs-ffi to verify your installation.

Compiling Emacs with dynamic module support

TODO: Check if common distros ship Emacs with this option or not.

We need to compile Emacs with module support using the --with-modules configure option:

./configure --with-modules
make

If the build succeeds, your new Emacs executable will be src/emacs. Run make install as root to install it to your system.

NB: You can speed up compilation significantly by using the -j<n> flag to make for a parallel build.

Julia

You’ll need to build Julia 1.0+ to generate libjulia.so.

I’ve been using the following settings in the top-level Make.user file:

prefix=/usr/local
JULIA_THREADS := 0

There is currently an issue initializing Julia via jl_init unless JULIA_THREADS is set to 0.

Development

Re-building the shared library

You should only need to do this if you modify the C sources.

From the top-level directory of this repository, run:

make

In case you need it, make clean wipes away the object files and shared library.

Running the tests

Tests run in a separate Emacs process in “batch” mode (so no new Emacs frame will appear).

From the top-level directory of this repository, run:

make test

Status / Development Notes

Open tasks

• Julia docs mention UInt128 and Int128 types, but there aren’t box/unbox functions for them in julia.h near the others…
• Shell for inspecting Julia’s state
• Async Julia eval
• Determine if we can hold on to several Julia sessions
• Determine if we can clear the state of an existing Julia session

Issues

jl_init, jl_init__threading, and JULIA_THREADS

Currently, the only way I’ve found to successfully call jl_init in the shared library is to compile Julia with JULIA_THREADS set to 0.

Debugging Notes

Compile libjulia with debugging symbols

make debug

Load debug version libjulia

(module-load "/home/dan/treemax/.spacemacs.d/layers/treemax-julia/local/libjulia/libjulia-wrapper.so")
(libjulia--dlopen "/home/dan/julia/usr/lib/libjulia-debug.so")

Run Emacs under gdb

If using LD_LIBRARY_PATH, ensure you update it to point to include the path to libjulia-debug.so

LD_LIBRARY_PATH=... gdb --args ~/emacs-src/src/emacs

Then set a breakpoint somewhere in julia with a gdb command like: b /home/dan/julia/src/module.c:470

Then the run gdb command to launch Emacs.

Load the debug library and do something that calls the code with the breakpoint.

Use gdb to debug.

Upstream Bugs to Report

Embedding Julia docs show the use of jl_get_function, but that doesn’t seem to be exported:

STATIC_INLINE jl_function_t *jl_get_function(jl_module_t *m, const char *name)
{
    return (jl_function_t*)jl_get_global(m, jl_symbol(name));
}

nm /usr/local/lib/libjulia.so |  grep jl_get_function

Resorting to directly calling jl_get_global for now.

Incorrect comment in julia_threads.h regarding JULIA_ENABLE_THREADING.

Actually set to 1 always now in Make.inc

# Enable threading with one thread
JULIA_THREADS := 1

# Threads
ifneq ($(JULIA_THREADS), 0)
JCPPFLAGS += -DJULIA_ENABLE_THREADING -DJULIA_NUM_THREADS=$(JULIA_THREADS)
endif

Which means it’s always on? Not sure how exporting jl_init ever works then:

#ifdef JULIA_ENABLE_THREADING
// this helps turn threading compilation mismatches into linker errors
#define julia_init julia_init__threading
#define jl_init jl_init__threading
#define jl_init_with_image jl_init_with_image__threading
#endif
JL_DLLEXPORT void julia_init(JL_IMAGE_SEARCH rel);
JL_DLLEXPORT void jl_init(void);
JL_DLLEXPORT void jl_init_with_image(const char *julia_bindir,
                                     const char *image_relative_path);

Seems like the name jl_init will always be swapped for julia_init__threading, which has no definition!

Can confirm threading is enabled with:

JL_DLLEXPORT int jl_threading_enabled(void)
{
#ifdef JULIA_ENABLE_THREADING
    return 1;
#else
    return 0;
#endif
}

ccall(:jl_threading_enabled, Cint, ())

Acknowledgments

• emacs-zmq for the first-load automatic compilation code.
• Emacs module documentation