Changes

In this article, I explain how to use lua Lua to write macros in [[LuaTeX]]. I give some examples of macros that are complicated in [[PdfTeX]], but can be defined easily using lua Lua in LuaTeX. These examples include macros that do arithmetic on their arguments, use loops, parse their arguments, and manipulate verbatim text.

As its name suggests, [[LuaTeX]] adds luaLua, a programming language, to TeX, the typesetter. This allows us to program TeX in a high-level programming language. For example, consider a TeX macro that divides two numbers. Such a macro is provided by the <tt>fp</tt> package and also by <tt>pgfmath</tt> library of the <tt>TikZ</tt> package. The following comment is from the <tt>fp</tt> package

Since the LuaTeX project started it has been actively supported by ConTeXt. <ref>Not surprising, as two of LuaTeX main developers&mdash;Taco Hoekwater and Hans Hagen&mdash;are also the main ConTeXt developers.</ref> These days, the various <em>How do I write such a macro</em> questions on the ConTeXt mailing list are answered by a solution that uses luaLua. I present a few such examples in this article. I have deliberately avoided examples about [[Fonts in LuaTeX | fonts]] and non-Latin languages. There is already quite a bit of documentation about them. In this article, I highlight how to use LuaTeX to write macros that require some <em>flow control</em>: randomized outputs, loops, and parsing.

= Interaction between TeX and lua Lua =

To a first approximation, the interaction between TeX and lua Lua is straightforward. When TeX (i.e., the LuaTeX engine) starts, it loads the input file in memory and processes it token by token. When TeX encounters <code>\directlua</code>, it stops reading the file in memory, <em>fully expands the argument of <code>\directlua</code></em>, and passes the control to a lua Lua instance. The lua Lua instance, which runs with a few preloaded libraries, processes the expanded arguments of <code>\directlua</code>. This lua Lua instance has a special output stream which can be accessed using <code>tex.print(...)</code>. The function <code>tex.print(...)</code> is just like the lua Lua function <code>print(...)</code> except that <code>tex.print(...)</code> prints to a <em>TeX stream</em> rather than to the standard output. When the lua Lua instance finishes processing its input, it passes the contents of the <em>TeX stream</em> back to TeX.<ref>The output of <code>tex.print(...)</code> is buffered and not passed to TeX until the lua Lua instance has stopped.</ref> TeX then inserts the contents of the <em>TeX stream</em> at the current location of the file that it was reading; expands the contents of the <em>TeX stream</em>; and continues. If TeX encounters another <code>\directlua</code>, the above process is repeated.

On top of these LuaTeX primitives, ConTeXt provides a higher level interface. There are two ways to call lua Lua from ConTeXt. The first is a macro <code>\ctxlua</code> (read as ConTeXt luaLua), which is similar to <code>\directlua</code>. (Aside: It is possible to run the lua Lua instance under different name spaces. <code>\ctxlua</code> is the default name space; other name spaces are explained later.) <code>\ctxlua</code> is good for calling small snippets of luaLua. The argument of <code>\ctxlua</code> is parsed under normal TeX catcodes (category codes), so the end of line character has the same catcode as a space. This can lead to surprises. For example, if you try to use a lua Lua comment, everything after the comment gets ignored.

{-- A lua Lua comment

This can be avoided by using a TeX comment instead of a lua Lua comment. However, working under normal TeX catcodes poses a bigger problem: special TeX characters like &, #, $, {, }, etc., need to be escaped. For example, # has to be escaped with <code>\string</code> to be used in <code>\ctxlua</code>.

As the argument of <code>\ctxlua</code> is fully expanded, escaping characters can sometimes be tricky. To circumvent this problem, ConTeXt defines a environment called <code>\startluacode ... \stopluacode</code>. This sets the catcodes to what one would expect in luaLua. Basically only <code>\</code> has its usual TeX meaning, the catcode of everything else is set to other. So, for all practical purposes, we can forget about catcodes inside <code>\startluacode ... \stopluacode</code>. The above two examples can be written as

-- A lua Lua comment

This environment is meant for moderately sized code snippets. For longer lua Lua code, it is more convenient to write the code in a separate lua Lua file and then load it using luaLua's <code>dofile(...)</code> function.

ConTeXt also provides a lua Lua function to conveniently write to the TeX stream. The function is called <code>context(...)</code> and it is equivalent to <code>tex.print(string.format(...))</code>.

Using the above, it is easy to define TeX macros that pass control to luaLua, do some processing in luaLua, and then pass the result back to TeX. For example, a macro to convert a decimal number to hexadecimal can be written simply, by asking lua Lua to do the conversion.

The percent sign had to be escaped because <code>\ctxlua</code> assumes TeX catcodes. Sometimes, escaping arguments can be difficult; instead, it can be easier to define a lua Lua function inside <code>\startluacode ... \stopluacode</code> and call it using <code>\ctxlua</code>. For example, a macro that takes a comma separated list of strings and prints a random item can be written as

I could have written a wrapper so that the function takes a list of words and chooses a random word among them. For an example of such a conversion, see the <em>sorting a list of tokens</em> page on the [http://luatexLuatex.bluwiki.com/go/Sort_a_token_list LuaTeX wiki]

In the above, I created a name space called <code>userdata</code> and defined the function <code>random</code> in that name space. Using a name space avoids clashes with the lua Lua functions defined in LuaTeX and ConTeXt.

In order to avoid name clashes, ConTeXt also defines independent name spaces of lua Lua instances. They are

Thus, for example, instead of <code>\ctxlua</code> and <code>\startluacode ... \stopluacode</code>, the <code>user</code> instance can be accessed via the macros <code>\usercode</code> and <code>\startusercode ... \stopusercode</code>. In instances other than <code>isolated</code>, all the lua Lua functions defined by ConTeXt (but not the inbuilt lua Lua functions) are stored in a <code>global</code> name space. In the <code>isolated</code> instance, all lua Lua functions defined by ConTeXt are hidden and cannot be accessed. Using these instances, we could write the above <code>\CHOOSERANDOM</code> macro as follows

Since I defined the function <code>random</code> in the <code>user</code> instance of luaLua, I did not bother to use a separate name space for the function. The lua Lua functions <code>os.time</code>, which is defined by a LuaTeX library, and <code>context</code>, which is defined by ConTeXt, needed to be accessed through a <code>global</code> name space. On the other hand, the <code>math.randomseed</code> function, which is part of luaLua, could be accessed as is.

A separate lua Lua instance also makes debugging slightly easier. With <code>\ctxlua</code> the error message starts with

Normally, it is best to define your lua Lua functions in the <code>user</code> name space. If you are writing a module, then define your lua Lua functions in the <code>third</code> instance and in a name space which is the name of your module. In this article, I will simply use the default lua Lua instance, but take care to define all my lua Lua functions in a <code>userdata</code> name space.

Doing simple arithmetic in TeX can be extremely difficult, as illustrated by the division macro in the introduction. With luaLua, simple arithmetic becomes trivial. For example, if you want a macro to find the cosine of an angle (in degrees), you can write

By contrast, in LuaTeX writing loops is easy. Once a lua Lua instance starts, TeX does not see anything until the lua Lua instance exits. So, we can write the loop in luaLua, and simply print the values that we would have typed to the TeX stream. When the control is passed to TeX, TeX sees the input as if we had typed it by hand. Consequently, macro expansion is no longer an issue. For example, we can get the above table by:

The lua Lua functions such as <code>context.bTABLE()</code> and <code>context.bTR()</code> are just abbreviations for running <code>context ("\\bTABLE")</code>, <code>context("\\bTR")</code>, etc. See the [http://www.pragma-ade.com/general/manuals/cld-mkiv.pdf ConTeXt lua Lua document] manual for more details about such functions. The rest of the code is a simple nested for-loop that computes the sum of two dice. We do not need to worry about macro expansion at all!