Changes

In this article, I explain how to use lua to write macros in [[LuaTeX]]. I give some examples The interweaving of ConTeXt and Lua consists of macros two elements: first you tell TeX that are complicated in [[PdfTeX]]you're starting some Lua code; then, but can be defined easily using lua in luaTeX. These examples include macros that do arithmetic on their argumentsonce inside Lua, you need to use loops, and parse their argumentsthe appropriate functions to put things into the TeX stream.

There are two main ways to execute Lua code in a ConTeXt document: The command {{cmd|ctxlua}}, and the environment {{cmd|startluacode}}…{{cmd|stopluacode}}. Both are wrappers around the LuaTeX primitive {{cmd|directlua}}, which you should never need to use. In general, you will define a function inside a {{cmd|startluacode}} block, and then define a TeX command that calls the function using {{cmd|ctxlua</code>, especially because {{cmd|ctxlua}} has a few idiosyncracies. <blockquote>The main thing about Lua code in a TeX document is this: the code is expanded by TeX ''before'' Lua gets to it. '''This means that all the Lua code, even the comments, must be valid TeX!''' A string like {{cmd|undefined}} will cause an immediate failure.</blockquote> = Introduction =Calling a bit of Lua inline: {{cmd|ctxlua}} == The command {{cmd|ctxlua}} is for short inline snippets of Lua, suchas

\def\FP@div#1#$2.#3.#4+ 5 \relax#5.#6.#7neq \relaxctxlua{% % [...] algorithmic idea context(for x>03+5)}$, y>0) % - %determine but is equal to \FP@shift such that % y*10^\FP@shift < 100000000 % <=y*10^ctxlua{context(\FP@shift2+15)}. % - %determine This is \FP@shift' such that % x*10^\FP@shift'< 100000000 % <=x*10^ctxlua{context(string.upper(\FP@shift+1"absolutely")) % - x=x*\FP@shift' % - y=y*\FP@shift % - \FP@shift=\FP@shift-\FP@shift' % - res=0 % - while y>0 %fixed-point representation! % - \FP@times=0 % - while x>y % - \FP@times=\FP@times+1 % - x=x-y % - end % - y=y/10 % - res=10*res+\FP@times/1000000000 % - end % - %shift the result according to \FP@shift} true.

The <tt>pgfmath</tt> library implements {{cmd|ctxlua}} operates under the macro in a similar way, but limits the number of shifts that it doesnormal TeX catcodes (category codes). These macros highlight This means the state of affairs in writing TeX macros. Even simple following two things like multiplying two numbers are hard; you either have to work extremely hard to circumvent for the programming limitations of Lua code inside:* all newlines get treated as spaces* special TeXcharacters like &, #, $, or{, more frequently}, hope that someone else has done the hard work for youetc., need to be escaped.  In luaTeXaddition, such a function can be written using the <warning above still holds. All the Lua code>/</, even the comments, must be valid TeX. Some code> operator (I will explain to illustrate the details later)newline problem:

\def\DIVIDE#1#2ctxlua {-- A Lua comment tex.print("This is not printed")}\directluactxlua {% A Tex comment tex.print(#1/#2"This is printed")}}

ThusThe problem with special TeX characters. (<code>#t</code> is Lua for 'the length of array <code>t</code>.)<texcode>% This doesn't work:%\ctxlua% {local t = {1,2,3,4}% tex.print("length " .. #t)}\ctxlua {local t = {1, with luaTeX ordinary users can write simple macros; and2, perhaps more importantly3, can read and understand macros written by TeX wizards4} tex.print("length " ..\string#t)}</texcode>

= Interaction between TeX = Calling a lua function with {{cmd|cldcontext}} and lua get the return ==

To One can execute a first approximation, the interaction between Lua code from within TeX and lua is straightforward. When TeX (i.e., the luaTeX engine) starts, it loads get back the input file result in memory and processes it token TeX by tokenusing {{cmd|cldcontext}}. When TeX encounters <code>\directlua</code>, it stops reading the file in memoryThus, <em>fully expands the argument of <if {{code>\directlua</code></em>, and passes the control to |myfunction}} is a lua instance. The lua instance, which runs with a few preloaded libraries, processes the expanded arguments function of <code>\directlua</code>. This lua instance has a special output stream which can be accessed using <variable {{code>tex.print|x}} defined in Lua, {{cmd|cldcontext|{myfunction(...5)</code>. The function <code>tex.print(...)</code> is just like }}} returns the lua function <value {{code>print|myfunction(...5)</code> except that <code>tex.print(...)</code> prints to a <em>}} in TeX stream</em> rather than to the standard output. When the lua instance finishes processing its input, it passes the contents of the <em>TeX stream</em> back This is equivalent to TeX.<ref>The output of <code>tex.print{{cmd|ctxlua|{context(myfunction(...5))</code> is buffered and not passed to TeX until the lua instance has stopped.</ref> TeX then inserts the contents of the <em>TeX stream</em> at the current location of the file that it was reading; expands the contents of the <em>TeX stream</em>; and continues. If TeX encounters another <code>\directlua</code>, the above process is repeated}}}.

As an exercise== A larger Lua block: {{cmd|startluacode}}…{{cmd|stopluacode}} == Inside the {{cmd|startluacode}}…{{cmd|stopluacode}} environment, imagine what happens when newlines and special characters behave normally. This solves the following input is processed by luaTeXcatcode problem that {{cmd|ctxlua}} suffers from. <ref>In this exampleApart from these special characters, I used two different kinds of quotations to avoid escaping quotes. Escaping quotes inside <the main warning remains in force: all the Lua code>\directlua</code> is tricky. The above was a contrived example; if you ever need to escape quotes, you can use even the <code>\startluacode ... \stopluacode</code> syntax explained latercomments, must be valid TeX.</ref>

\directlua%startluacode {tex -- The unknown command \undefined will cause this entire block to fail.  -- Print a countdown '10, 8, …, 0!' -- `..` is Lua for string concatenation for i = 10, 2, -2 do context(i ..print", ") end context("Depth 1 0!") -- \\directlua{texpar is equivalent to a blank line in the input -- (Notice the escaped backslash: TeX won't mind the above comment.) context.par()  -- Look! we can use # and $ with impunity! context("Unless we print them, then we must \\#\\$\\& printthe escape characters, too.")\stopluacode</texcode> == Putting Lua code in an external file == You can put your lua code in an external file ('Depth 2'with the <code>.lua</code> extension)}and include it with the <code>require</code> command: <texcode>\startluacode-- include the file my-lua-lib.luarequire("my-lua-lib")\stopluacode</texcode> == Namespaces == It is a good habit to put your custom-defined functions in their own namespace. The traditional namespace for this is <code>userdata</code>:<texcode>\startluacode -- if userdata doesn't exist yet, create it userdata = userdata or {} -- define a shorter synonym u = userdata  -- create my custom function inside the userdata namespace function u.myfunction() -- do stuff end\stopluacode

On top The full list of these luaTeX primitivescanonical namespaces, ConTeXt provides a higher level interfacetaken from [https://distribution. There are two ways to call lua from ConTeXtcontextgarden.net/current/context/latest/tex/context/base/mkxl/luat-ini.lmt luat-ini. The first is a macro lmt]: <code>\ctxlua</codepre> userdata = userdata or { } -- for users (read as ConTeXt luae.g. functions etc), which is similar to thirddata = thirddata or { } -- only for third party modulesmoduledata = moduledata or { } -- only for development teamdocumentdata = documentdata or { } -- for users (e.g. raw data)parametersets = parametersets or { } -- experimental for team<code/pre>\directlua</code>. (Aside: It  If your module, environment, or document is possible going to run the lua instance under different name spacesbe used by other people, you should create your own subnamespaces within these tables.  <code>\ctxlua</codepre> is the default name space; other name spaces are explained latermoduledata['mymodule'] = { }mm = moduledata.mymodulefunction mm.mainfunction() -- do stuffend<code/pre>\ctxlua</code> is good for calling small snippets of lua. The argument of < == Undefined Commands in Lua Comments == Lua code>\ctxlua</code> is parsed under normal invoked inside TeX doesn’t allow TeX catcodes (category codes), so the end of line character has the same catcode as a space. This can lead to surprises. For example, if you try to use a lua comment, everything after the comment gets ignoredundefined commands ''even inside comments''. 

\starttext\startluacode--\undefinedcommandfromme\stopluacodeHello\ctxlua {-- A lua comment\undefinedcommandfromme} tex.print("This is not printed")}\stoptext

This can be avoided by using a TeX comment instead of a lua comment. However, To get the sample above working under normal TeX catcodes poses a bigger problem(as [https: special TeX characters like &, #, $, {, }, etc//www.mail-archive., need to be escapedcom/ntg-context@ntg. For example, # has to be escaped with <code>\string<nl/code> to be used msg103892.html explained by Hans in <code>\ctxlua</code>a NTG-context thread from jan.2023 entitled “Minor bug in Lua or ConTeXt”]), you would need a fallback definition: 

\ctxlua {local t = {1,2,3,4} tex.print("length " .. ifdefined\string#t)}undefinedcommandfromme \else \let\undefinedcommandfromme\relax \fi

As the argument of <code>\ctxlua</code> {{cmd|undefinedcommandfromme}} gets only defined (as {{cmd|relax}}, to do nothing), if and only if it is fully expanded, escaping characters can sometimes be trickyundefined. To circumvent this problem = Calling TeX from Lua = Being a topic on itself, pages are dedicated:* '''[[CLD|ConTeXt defines a environment called <code>\startluacode Lua Documents]]''', or CLD, are way to access TeX from inside Lua scripts.A page give clues about [[CLD_passing_variables|passing variables]] within CLD (2018).. \stopluacode<* [[Lua|Wiki page dedicated to Lua]]** [[Extensions to the Lua I/code>O library]]** [[String manipulation]]** [[Table manipulation]] = Putting stuff in your TeX document from Lua = == Simple printing: context(), tex. This sets the catcodes to what one would expect in luaprint(), and tex. Basically only sprint() ==Use <code>\context(…)</code> has its usual TeX meaning, the catcode of everything else for most things. It is set equivalent to other. So, for all practical purposes, we can forget about catcodes inside <code>\startluacode tex.print(string.. \stopluacodeformat(…))</code>. The above two examples can be written as, so 

-- A lua comment tex.print(name = "This is printed.Jane") local t date = {1,2,3,4}"today" tex.printcontext("length Hello %s, how are you %s?" .. #t, name, date)-- Becomes 'Hello Jane, how are you today?'

This environment is meant for moderately sized More primitively, you have <code>tex.print()</code> and <code snippets>tex. For longer lua sprint()</code>. Either one can take as an argument either a number of strings, it or an array of strings, and will then insert the strings into the TeX stream. The only difference is more convenient to write the that <code>tex.print()</code in > treats each string as a separate lua file and then load it using lua's input line, while <code>dofiletex.sprint(...)</code> functiondoesn't. So the following lines

ConTeXt also provides a lua function to conveniently write to the TeX stream. The function is called <codetexcode>context\ctxlua{tex.print("a", "b")}\ctxlua{tex...print({"a", "b"})}</codetexcode> are both interpreted by TeX as <texcode>ab</texcode> and it is equivalent to  but when we use <code>tex.print(string.format(...))sprint</code>. instead, either of the following

\def\TOHEX#1{\ctxlua{contexttex.sprint("\%Xa",#1"b")}}\TOHEXctxlua{35tex.sprint({"a", "b"})}

The percent sign had to will be escaped because read by TeX as <codetexcode>\ctxluaab</codetexcode> assumes TeX catcodes without any space in between. Sometimes == Context commands == Most commands that you would type with a backslash in plain ConTeXt, escaping arguments you can be difficult; instead, it can be easier to define a lua function inside access from Lua with <code>\startluacode context... \stopluacode</codeem> and call it using command<code/em>\ctxlua</code>. For example, a macro that takes a comma separated list of Unadorned strings and prints a random item can be written end up in TeX as arguments in curly braces; Lua tables end up in TeX as paramater blocks in square brackets. The following two pieces of code are equivalent: 

userdata = userdata or context.chapter({first}, "Some title") math context.randomseedstartcolumns( os{n = 3, rule = "on"}) context("Hello one") context.timecolumn() context("Hello two") function userdata context.randomcolumn(...) context("Hello three") context(arg[math.randomstopcolumns(1, #arg)]) end

<texcode> \CHOOSERANDOMchapter[first]{"Some title} \startcolumns[n=3, rule=on] Hello one", " \column Hello two", " \column Hello three"} \stopcolumns

I could have written For a wrapper so that fuller account of the context.commands, see the function takes a list [http://www.pragma-ade.nl/general/manuals/cld-mkiv.pdf ConTeXt Lua document] manual. It is old, but most of words and chooses a random word among themit still applies. For an example  One final note: arguments can also be specified in the form of such a conversionnested functions. Because LuaTeX evaluates the deepest-nested argument first, see this may cause the <emcode>sorting a list of tokenscontext()</emcode> page calls to be evaluated in the wrong order. For more on this, see the article on [[CLD|ConTeXt Lua documents]], and also, again, the [http://luatexwww.bluwikipragma-ade.comnl/gogeneral/Sort_a_token_list luaTeX wikimanuals/cld-mkiv.pdf CLD manual]. = Passing arguments and buffers: ConTeXt commands that hook into Lua = == Making \command{arg1}{arg2} hook into Lua ==First, define a Lua function:

In the above, I created a name space called <codetexcode>\startluacode -- remember, using the userdata</code> and defined the function <code>random</code> in that name space. Using a name space avoids clashes with the lua functions defined in luaTeX and ConTeXt.namespace prevents conflicts userdata = userdata or {}

In order to avoid name clashes, ConTeXt also defines independent name spaces of lua instances function userdata. They are surroundwithdashes(str) context("--" .. str .. "--") end\stopluacode</texcode>

Then define the TeX command that expands to a {{cmd| |- | '''user''' | a private user instance |- | '''third''' | third party module instance |- | '''module''' | ConTeXt module instance |- | '''isolated''' | an isolated instance |ctxlua}}call:

\startusercodedef\surroundwd#1% math {\ctxlua{userdata.randomseed( global.os.timesurroundwithdashes([==[#1]==]) )}}</texcode> ''NB'': quoting with <code>[==[#1]==]</code> function random([http://www.lua.org/manual/5.2/manual.html#3.1 long strings]) globalworks just like <code>"#1"</code> in most cases, but in addition it is robust against <code>#1</code> containing the quotation mark<code>"</code> which would terminate the Lua string prematurely.Inside {{cmd|protect}}…{{cmd|unprotect}} the macros {{cmd|!!bs}}and {{cmd|!!es} are at your disposition.They are equivalent to <code>[===[</code> and <code>]===]</code> and --being single tokens to TeX -- parsed faster.context(argSee [mathhttp://repo.or.random(1, cz/w/context.git/blob/refs/heads/origin:/tex/context/base/luat-ini.mkiv#arg)l174 <code>luat-ini.mkiv</code>].) end== Making {{cmd|startenv}}…{{cmd|stopenv}} hook into Lua ==The first job is, as ever, to have the Lua function at the ready<texcode>\stopusercodestartluacode userdata = userdata or {}

function userdata.verynarrow(buffer) -- equivalent to \def\CHOOSERANDOM#1%startnarrower[10em] context.startnarrower({\usercode{random"10em"}) context(buffer) context.stopnarrower(#1)}} end\stopluacode

Since I defined the function <code>random</code> in the <code>user</code> instance of lua, I did not bother to use a separate name space for the function. The lua functions <code>os.time</code>, which is defined by a luaTeX library, and <code>context</code>, which is defined by ConTeXt, needed to be accessed through a <code>global</code> name space. On the other handNext, we define the <code>math.randomseed</code> function, which is part start command of lua, could be accessed as is. our custom buffer:

! LuaTeX error &lt;main ctx instance&gt;\def\startverynarrow% {\dostartbuffer [verynarrow] % buffer name [startverynarrow] % command where buffer starts [stopverynarrow]} % command where buffer ends % also:command invoked when buffer stops 

With Lastly, we define the {{cmd|stopverynarrow}} command such that it passes the recently-complated buffer to our <code>\usercodeverynarrow</code> the error message starts withLua function: 

This makes And that's it easier to narrow down the source ! The rest of errorthis article will consist of examples.

Doing simple arithmetic in TeX can be extremely difficult, as illustrated by the division macro in the introduction. With luaLua, simple arithmetic becomes trivial. For example, if you want a macro to find the cosine of an angle (in degrees), you can write

or , if you want less precision (notice the percent sign is escaped):

$\pi = \ctxlua{context("\%letterpercent.6f", math.pi)}$</texcode>Notice that the percent sign is escaped with letterpercent. === mathexpr with LMTX === In LMTX there is a new way to use calculated expressions with mathexpr through ([https://github.com/contextgarden/context-mirror/blob/7fd782dace8f90e7e032ca8f449f8ca4eada450b/doc/context/sources/general/manuals/math/math-fun.tex math-fun]). Some examples are: <texcode>$ \pi = \mathexpr[.40N]{pi} $$ \pi = \mathexpr[.80N]{sqrt(11)} $$ \pi = \decimalexpr[.80N]{sqrt(11)} $$ \pi = \decimalexpr{sqrt(11)} $$ c = \complexexpr{123 + new(456,789)}$

== Loops without worrying about expansion ==

''This example demonstrates using Lua to write a quasi-repetitive piece of ConTeXt code.'' Loops in TeX are tricky , because macro assignments and macro expansion interact in strange ways. For example, suppose we want to typeset a table showing the sum of the roll of two dice and want the output to look like this:

The tedious (but faster!) way to achieve this This is to simply type easy in LuaTeX. Once a Lua instance starts, TeX does not see anything until the whole table by handLua instance exits.  It is however natural to want to So, we can write this table as a the loopin Lua, and compute simply print the valuesthat we would have typed to the TeX stream. When the control is passed to TeX, TeX sees the input as if we had typed it by hand. A first ConTeXt implementation using This is the Lua code for the recursion level might beabove table:

\bTABLE \bTR \bTD $(+)$ \eTD \dorecurse{6} {\bTD \recurselevel \eTD} \eTR \dorecurse{6} {\bTR \bTD \recurselevel \eTD \edef\firstrecurselevel{\recurselevel} \dorecurse{6} {\bTD \the\numexpr\firstrecurselevel+\recurselevel \eTD}% \eTR} setupcolors[state=start]\eTABLE</texcode> HoweversetupTABLE[each][each][width=2em, this does not work as expectedheight=2em, yielding all zeros. A natural table stores the contents of all the cellsalign={middle, before typesetting it. But it does not expand the contents of its cell before storing them. So, at the time the table is actually typeset, TeX has already finished the <code>\dorecurse</code> and <code>\recurselevel</code> is set to 0. The solution is to place <code>\expandafter</code> at the correct location(s) to coax TeX into expanding the <code>\recurselevel</code> macro before the natural table stores the cell contents. The difficult part is figuring out the exact location of <code>\expandafter</code>s. Here is a solution that works: <texcode>\bTABLE \bTR \bTD $(+)$ \eTD \dorecurse{6middle} ] {\expandafter \bTD \recurselevel \eTD} setupTABLE[r][1][background=color,backgroundcolor=gray] \eTR \dorecurse{6} {\bTR \edef\firstrecurselevel{\recurselevel} \expandafter\bTD \recurselevel \eTD \dorecurse{6} {\expandafter\bTD \the\numexpr\firstrecurselevel+\recurselevel \relax \eTD} \eTR} \eTABLE </texcode>setupTABLE[c][1][background=color,backgroundcolor=gray]

OrdinarilyAs an example, getting text into a let's consider typesetting chemical molecules in TeX macro with spaces and newlinesintact requires mucking about with <code>\obeylines</code> and<code>\obeyspaces</code> &mdash; see the [[Inside_ConTeXt#Passing_verbatim_text_as_macro_parameter|Inside Context]] article on this wiki. Luckily for usNormally, however, ConTeXt provides the molecules should be typeset in text mode rather than math mode. If we want :H<codesub>\dostartbuffer[3</codesub>SO<isub>name4</isub><codesup>][+</code><i>bufferstart</isup>,we must type :<code>][<i>bufferstop</i><code>]H{{cmd|low|{3}}}SO{{cmd|lohi|{4}}}{{{cmd|textplus}}}</code> ,but we'd much rather type:{{cmd|molecule|{H_3SO_4^+}}}. commandSo, which creates we need a pair of buffer commands function that write their contents verbatim to can take a named bufferstring like that, parse it, and turn it into the appropriate TeX code. This buffer can then be accessed in LuaTeX via <code>buffersincludes a general parser based on PEG (parsing expression grammar) called [http://www.inf.puc-rio.content(<br/code><i>name<~roberto/i><code>)<lpeg/code>lpeg.html lpeg], and it makes writing little parsers positively joyful. (Once you've got the knack of it, at least. Here is an ) For example that shows how , the buffer definition and the LuaTex call are combinedabove {{cmd|molecule}} macro can be written as follows.

%% Define a buffer \startDedent startluacode -- we will put our molecule function in the userdata namespace.userdata = userdata or { } -- The formatting functions into which the captured-- superscript/subscript blocks will be fedlocal formatters = { } function formatters.low(one) return string. format("\\stopDedent that feeds its contentslow{%% to LuaTex for processing.%% LuaTex finds out by how much the first line is indenteds}", and dedentsone)%% the entire block by that much.end

function formatters.high(one) return string.format("\startluacode\high{%s}", one)end function formatters.lowhigh(one, two) dedent = return string.format("\\lohi{%s}{%s}", one, two)end functionformatters.highlow(nameone, two,three) return string.format("\\lohi{%s}{% Ns}", one,two)end -- These are the characters we may encounter-- The `/` means we want to expand + and - to \textplus c.Bq. This function does \textminus;-- this substition is not handle tabsinstant, but will take place inside the first -- surrounding lpeg.Cs() call.local plus = lpeg.P("+") / "\\textplus "local minus = lpeg.P("-") / "\\textminus "local character = lpeg.R("az", "AZ", "09") -- R is for 'range'local subscript = lpeg.P("_") -- P is simply for 'pattern'local superscript = lpeg.P("^")local leftbrace = lpeg.P("{")local rightbrace = lpeg.P("}") -- a ^ or _ affects either a single character, or a brace-delimited-- block. Whichever it is, call it `content`.local single = character + plus + minuslocal multiple = leftbrace * single^1 * rightbracelocal content = single + multiple

%% read out -- These are our top-level elements: non-special text, of course, and-- blocks of superscript/subscript/both.-- lpeg.Cs(content) does two things:-- (1) not all matches go into the buffer`/ function` construction; only-- *captures* go in. The C in Cs stands for Capture. This way, -- the superscript/subscript mark gets discarded.-- (2) it expands plus/minus before they go into the formatter. The-- s in Cs stands for 'substitute in the replacement values, if any' local code text = bufferssingle^1local low = subscript * lpeg.Cs(content(name) / formatters.low %% First some debugging stuff: dump the input to a text filelocal high = superscript * lpeg.Cs(content) % to check that we got what we wanted / formatters.high logfile local lowhigh = iosubscript * lpeg.openCs("debugcontent) * superscript * lpeg.txt", "w"Cs(content) logfile:write / formatters.lowhighlocal highlow = superscript * lpeg.Cs(codecontent)* logfile:close subscript * lpeg.Cs(content) / formatters.highlow

%% Now for -- Finally, the meat of itroot element: 'moleculepattern'local moleculepattern = lpeg.Cs((lowhigh + highlow + low + high + text)^0) % How many leading spaces in the first line? lead = stringfunction thirddata.matchmolecule(code, '^ +'string) or '' % remove lead from every line -- * `:match` returns the matched string. Our pattern code = -- `moleculepattern` should match the entire input string.gsubAny -- *performed* substitutions are retained. (code, '^' `.Cs()` performs a -- previously defined substitution. lead, '') code = -- * `context()` inserts the resulting stringinto the stream, ready for -- TeX to evaluate.gsub context(code, '\n' .. lead, '\n'moleculepattern:match(string))end

% Tell TeX what strings the dedentBuffer starts and ends with.\def\startDedent molecule#1{\dostartbuffer[dedentBuffer][startDedent][stopDedent]}% Make the ending command (which will also signal the buffer's end)% call the processing LuaTeXctxlua{thirddata.\def\stopDedent {\directlua{dedentmolecule("dedentBuffer#1")}}

\startDedent andra moi ennepe molecule{Hg^+}, \stopDedentmolecule{SO_4^{2-}}

Each <code>\stopDedent</code> immediately uses the <code>dedentBuffer</code>Quite terse and readable by parser standards, isn'st it?contents, and each <code>\startDedent</code> flushes the buffer's contents, sothe construction can safely be used multiple times throughout the document.== Manipulating verbatim text ==

In order Suppose we want to get around write an environment {{cmd|startdedentedtyping}} … {{cmd|stopdedentedtyping}} that removes the weird rules indentation of macro expansion, writing a parser in TeX involves a lot of macro jugglery and catcode trickerythe first line from every line. It is a black artThus, one of the biggest mysteries output of TeX for ordinary users.…

\newbox\chemlowbox startdedentedtyping\def\chemlow #1% include &lt;stdio.h&gt; void main() { print("Hello world \setboxn") ; }\chemlowboxstopdedentedtyping \hbox{{\switchtobodyfont[small]#1}}} </texcode>… should be the same as the output of …

<texcode>\def\chemhighstarttyping#1% include &lt;stdio.h&gt; {\ifvoid\chemlowbox void main() \high{{\switchtobodyfont[small]#1}}% print("Hello world \else n") ; \lohi{\box\chemlowbox} {{\switchtobodyfont[small]#1}}stoptyping</texcode> \fi} … even though the leading whitespace is different.

<texcode>\unexpanded\def\molecule% startluacode {\bgroup -- Initialize a userdata name space to keep our own functions in. \catcode`\_=\active \uccode`\~=`\_ \uppercase{\let~\chemlow}% \catcode`\^=\active \uccode`\~=`\^ \uppercase{\let~\chemhigh}% \dostepwiserecurse {65}{90}{1} {\catcode \recurselevel = \active \uccode`\~=\recurselevel \uppercase{\edef~{\noexpand\finishchem \rawcharacter{\recurselevel}}}}% \catcode`\ -=\active \uccode`\~=`\- That way, we won't interfere with anything ConTeXt keeps in \uppercase{\def~{ --}}% the global name space. \domolecule userdata = userdata or {}%

\def\domolecule# function userdata.dedentedtyping(content) local lines = string.splitlines(content) local indent = string.match(lines[1], '^ +') or '' local pattern = '^' .. indent for i=1{,#1\finishchem\egroup}lines do lines[i] = string.gsub(lines[i],pattern,"")</texcode> end

This monstrosity is a typical TeX parser content = table. Appropriate characters need to be made active; occasionally, <code>\lccode</code> and <code>\uccode</code> need to be set; signaling tricks are needed concat(for instancelines, checking if <code>'\chemlowbox</code> is void); and then magic happens (or so it seems to a flabbergasted usern'). More sophisticated parsers involve creating finite state automata, which look even more monstrous.

With luaTeX, things are different tex. luaTeX includes a general parser based on PEG sprint(parsing expression grammar) called [http://www"\\starttyping\n" .inf.puc-riocontent .br/roberto/lpeg/lpeg.html lpeg]. This makes writing parsers in TeX much more comprehensible. For example, the above <code>"\\stoptyping\molecule</code> macro can be written asn")

-- The typing environment looks for an explicit \type{\stoptyping}. So, -- context.starttyping() context(content) context.stoptyping() -- does not work. But -- context.starttyping() context(content) tex.sprint("\\stoptyping") -- does. end\stopluacode</texcode>\startluacodeuserdata = userdata or {}

local lowercase = lpeg.R("az")local uppercase = lpeg.R("AZ")local backslash = lpeg.P("\\")local csname = backslash * lpeg.P(1) * (1-backslash)^0local plus = lpeg.P("+") / "\\textplus "local minus = lpeg.P("-") / "\\textminus "local digit = lpeg.R("09")local sign = plus + minuslocal cardinal = digit^1local integer = sign^0 * cardinallocal leftbrace = lpeg.P("Here is the code for defining the {")local rightbrace = lpeg.P("{cmd|startdedentedtyping}}")local nobrace = 1 - (leftbrace + rightbrace)local nested = lpeg.P … {{leftbrace * (csname + sign + nobrace + lpeg.V(1))^0 * rightbracecmd|stopdedentedtyping}}local any = lpeg.P(1)pair:

local subscript <texcode>% Create an environment that stores everything % between \startdedentedtyping and \stopdedentedtyping % in a buffer named 'dedentedtyping'.\def\startdedentedtyping = lpeg.P("_"){\dostartbuffer [dedentedtyping]local superscript = lpeg.P("^") [startdedentedtyping]local somescript = subscript + superscript [stopdedentedtyping]}

local content = lpeg% On closing the dedentedtyping environment, call the LuaTeX% function dedentedtyping(), and pass it the contents of % the buffer called 'dedentedtyping'\def\stopdedentedtyping {\ctxlua {userdata.dedentedtyping(buffers.Csgetcontent(csname + nested 'dedentedtyping'))}} + sign + any)</texcode>

local lowhigh = lpeg.Cc("\\lohi{%s}{%s}") * subscript * content * superscript * content / string.formatlocal highlow = lpeg.Cc("\\hilo{%s}{%s}") * superscript * content * subscript * content / string.formatlocal low = lpeg.Cc("\\low{%s}") * subscript * content / string.formatlocal high = lpeg.Cc("\\high{%That's}") * superscript * content / stringall.formatlocal justtext = (1 - somescript)^1local parser = lpegFinally, we will go into a little more detail on how TeX and Lua communicate with each other.Cs((csname + lowhigh + highlow + low + high + sign + any)^0)

userdata.moleculeparser = parser = Other examples ==

function userdata.molecule* [[Calculations_in_Lua|Calculations in Lua]] (strwarning date 2012) return * [[LPeg|Writing a parser:matchwith LPeg]] (strLua Parsing Expression Grammars)end* [[Random|Random numbers]] in ConTeXt and MetaPost\stopluacode* [[SQL|An example with SQL database]]* [[Pascal's Triangle]]

This To a first approximation, the interaction between TeX and Lua is more verbose than straightforward. When TeX (i.e., the LuaTeX engine) starts, it loads the input file in memory and processes it token by token. When TeX solutionencounters {{cmd|directlua}}, it stops reading the file in memory, but is easier <em>fully expands the argument of {{cmd|directlua}}</em>, and passes the control to read and writea Lua instance. With The Lua instance, which runs with a proper parserfew preloaded libraries, I do not have to use tricks to check if either one or both processes the expanded arguments of {{cmd|directlua}}. This Lua instance has a special output stream which can be accessed using <code>tex.print(…)</code>. The function <code>_tex.print(…)</code> and is just like the Lua function <code>^print(…)</code> are presentexcept that <code>tex. More importantly, anyone print(once they know …)</code> prints to a <em>TeX stream</em> rather than to the lpeg syntax) can read standard output. When the parser and easily understand what Lua instance finishes processing its input, it doespasses the contents of the <em>TeX stream</em> back to TeX. This <ref>The output of <code>tex.print(…)</code> is in contrast buffered and not passed to TeX until the implementation based on Lua instance has stopped.</ref> TeX macro jugglery which require you to implement a then inserts the contents of the <em>TeX stream</em> at the current location of the file that it was reading; expands the contents of the <em>TeX stream</em>; and continues. If TeX interpreter in your head to understandencounters another {{cmd|directlua{{cmd|, the above process is repeated.

luaTeX is removing many TeX barriers: using system fonts, reading and writing Unicode files, typesetting non-Latin languages, among others<texcode>\directlua% {tex. However, the biggest feature of luaTeX is the ability to use a high-level programming language to program TeX. This can potentially lower the learning curve for programming TeXprint("Depth 1 \\directlua{tex.print('Depth 2')}")}</texcode>

In For more on this article, I have mentioned only one aspect of programming TeXsee the [http: macros that manipulate their input and output some text to the main TeX stream//wiki.luatex. Many other kinds of manipulations are possible: luaTeX provides access to TeX boxes, token lists, dimensions, glues, catcodes, direction parameters, math parameters, etcorg/index. The details can be found in php/Writing_Lua_in_TeX] article on the [http://wwwwiki.luatex.org/documentationindex.html luaTeX manualphp/Main_Page LuaTeX wiki].