Interfacing LuaTeX with Julia

Posted on December 27, 2017

One cool feature of LuaTeX is ability to interface with external libraries using Lua’s Foreign Function Interface (FFI). An extreme example of this is Luigi Scarso’s LuaTeX lunatic, which provides a two way bridge between Python and LuaTeX. Not being a Python fan, I never looked into the implementation details, in spite of Luigi’s impressive examples.

Recently, Henri Menke posted an interesting example that shows how to use a function from GNU Scientific Library (GSL) to inteface with pgfplots. This got me interested in looking into Lua’s FFI in detail. After a bit of trial and error, I finally figured out how to interface LuaTeX with my current favorite programming language: Julia.

Calling Julia from Lua

The basic idea of calling julia (or any other C library for that matter) from Lua is the following:

local ffi = require("ffi")
local JULIA = ffi.load("julia", true)

ffi.cdef [[
  void jl_init(void);
  typedef struct _jl_value_t jl_value_t;
  jl_value_t *jl_eval_string(const char*);
]]

JULIA.jl_init()

code = [[
  x = [1 2 3]'
  A = [1 0 1; 0 1 1; 1 1 0]

  y = x'*A*x

  print(y[1])
]]

JULIA.jl_eval_string(code)

Save this as, say test-1.lua and run:

luatex test-1.lua

You should see 23 as output.

Phew! That wasn’t difficult. A couple of points to note:

  1. It took me quite some time to figure out that I need ffi.load("julia", true) rather than ffi.load("julia"). Lua’s FFI API documentation says that when the second argument is true, the library symbols are loaded into the global namespace, too.. I don’t completely understand that statement. The only hint that this is needed is a note on Embedding Julia page that says Currently, dynamically linking with the libjulia shared library requires passing the RTLD_GLOBAL option. Since I was not familiar with working with FFI, I could not put two and two together.

  2. I don’t quite understand why one needs to include the function signatures in ffi.cdef. It would be much simpler if I could simply ask ffi to include a C header file. Otherwise, I need to hunt for all the needed function signatures from julia.h file. Some of these functions are defined using #define and I don’t know how to include them in ffi.cdef.

    [Edit: After browsing through some of the old threads in the Lua mailing list, the main concern is that parsing .h files requires the C pre-processor and writing that in pure Lua is a lot of effort. ]

  3. If julia has been compiled with threading support, then jl_init goes under the name of jl_init__threading (note the double underscore). So you have to change jl_init to jl_init__threading in the above example.

Creating a TeX inteface

Now that we know how to call Julia from Lua, the rest is just adding some scafolding to create a TeX inteface. Lua’s FFI interface allows one to call any function defined in a C library and convert return values to Lua objects (strings, numbers, or tables). As a proof of concept, I decided to only pass and receive strings. This is limited (and you need to ensure that the Julia code returns a string), but easier to implement.

To do so, I defined a Lua function julia.eval(str, flag), which takes a string str and evals it using julia. If flag is set to true, then it converts the return value to a lua string and typesets it using ConTeXt.

Note that in the example below, I use ffi.load("/usr/lib/libjulia.so", true) rather than ffi.load("julia", true). For some reason, the latter does not work when compiling a file through context (although, running context test-1.lua on the above example runs fine).

\startluacode
local ffi = require("ffi")
local JULIA = ffi.load("/usr/lib/libjulia.so", true)
local format = string.format

ffi.cdef [[
  void jl_init(void);

  typedef struct _jl_value_t jl_value_t;
  jl_value_t *jl_eval_string(const char*);
  char *jl_string_ptr(jl_value_t *);
]]

JULIA.jl_init()

julia = julia or {}

function julia.eval(str, flag)
  local jval  = JULIA.jl_eval_string(str)
  if flag then
    local lval = ffi.string(JULIA.jl_string_ptr(jval))
    context(lval)
  end
end

\stopluacode

Then, at the TeX end, I define two macros: \julia which simply evaluates its argument using julia and \ctxjulia which evaluates its argument using julia and typesets the result.

\define[1]\ctxjulia{\ctxlua{julia.eval([===[#1]===], true) }}
\define[1]\julia   {\ctxlua{julia.eval([===[#1]===], false)}}

Here is a simple usage example.

\starttext
\julia{a = 10}
\julia{b = 5}
\ctxjulia{string(a*b)}
\stoptext

which, as expected, prints 50. Sure, for something as simple as this, I could have simply used Lua as well. In a future post, I’ll show how to create a nicer user-interface and do show some more interesting examples.

 <Using Font Awesome | Drawing Metapost animations>

This entry was posted in FFI and tagged julia.