Formatting numbers in Lua

Posted on September 9, 2019

I often use Lua to generate solution for homework assignments. Ideally, I want the solution to look exactly how it would look if it were written by hand. But this can be tricker than it appears at first glance. In this post, I’ll explain the issue and how I solve it.

To illustrate the formatting issue, let me consider an example of writing the solution of how to find roots of a quadratic equation. Let’s start with a simple example. First let’s define

\defineenumeration[question]
\defineenumeration[solution]

Then consider the following example.

\startquestion
  Find the roots of $x^2 + 5x + 6 = 0$.
\stopquestion

\startsolution
  Let's start by computing the determinant.
  \startformula
     Δ = b^2 - 4ac = 1
  \stopformula
  Since $Δ > 0$, the roots are given by
  \startformula
    r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
            = -2 \text{ and } -3.
  \stopformula
\stopsolution

Now suppose I want to generate a homework assignment with four or five such questions. In order to ensure that I don’t make any mistakes, I generate the questions and the answers using Lua. For simplicity, let’s assume that both roots are real. Then, I can use string.formatters to easily generate the assignment and the solution.

\startluacode

local formatters = string.formatters

local question = formatters[ [[
\startquestion
   Find the roots of $%s x^2 + %s x + %s = 0$.
\stopquestion
]] ]

local solution = formatters[ [[
\startsolution
  Let's start by computing the determinant.
  \startformula
     Δ = b^2 - 4ac = %s.
  \stopformula
  Since $Δ > 0$, the roots are given by
  \startformula
    r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
            = %s \text{ and } %s.
  \stopformula
\stopsolution
]] ]

local sqrt = math.sqrt

assignment = assignment or { }

assignment.roots = function(a, b, c)
  context(question(a == 1 and "" or a, b, c))

  D = b^2 - 4*a*c
  r1 = (-b + sqrt(D))/(2*a)
  r2 = (-b - sqrt(D))/(2*a)

  context(solution(D, r1, r2))

end
\stopluacode

Then, in the homework assignment, I can generate the above question and its solution using:

\ctxlua{assignment.roots(1, 5, 6)}

The above generated solution works well when all numbers are integer valued. However, the generated solution is not ideal when some of the calculations result in floats. For example:

\ctxlua{assignment.roots(1, 4, 2)}

will generate:

\startquestion
  Find the roots of $x^2 + 4x + 2 = 0$.
\stopquestion

\startsolution
  Let's start by computing the determinant.
  \startformula
     Δ = b^2 - 4ac = 8.0.
  \stopformula
  Since $Δ > 0$, the roots are given by
  \startformula
    r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
            = -0.5857864376269 \text{ and } -3.4142135623731.
  \stopformula
\stopsolution

Technically, the solution is correct. But one never types a float with a precision of 12 decimal places. Of course, I could change the %s in the template to %.3f as follows:

local solution = formatters[ [[
\startsolution
  Let's start by computing the determinant.
  \startformula
     Δ = b^2 - 4ac = %.3f.
  \stopformula
  Since $Δ > 0$, the roots are given by
  \startformula
    r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
            = %.3f \text{ and } %.3f.
  \stopformula
\stopsolution
]] ]

For the second problem, this will generate:

\startquestion
  Find the roots of $x^2 + 4x + 2 = 0$.
\stopquestion

\startsolution
  Let's start by computing the determinant.
  \startformula
     Δ = b^2 - 4ac = 8.000.
  \stopformula
  Since $Δ > 0$, the roots are given by
  \startformula
    r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
            = -0.586 \text{ and } -3.414.
  \stopformula
\stopsolution

which is partially acceptable (when typesetting the solution by hand, one would uses Δ = 8 rather than Δ = 8.000) but for the first problem, we now get

\startquestion
  Find the roots of $x^2 + 5x + 6 = 0$.
\stopquestion

\startsolution
  Let's start by computing the determinant.
  \startformula
     Δ = b^2 - 4ac = 1.000.
  \stopformula
  Since $Δ > 0$, the roots are given by
  \startformula
    r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
            = -2.000 \text{ and } -3.000.
  \stopformula
\stopsolution

which is not ideal.

So, to typeset such examples, I need to format numbers as follows:

• If the number is an integer, use %d.
• If the number is a float, use %.3f.

However, once you think about it, the above spec is not complete. In addition, what we want is that

• If %.3f gives "0.000", using (the tex equivalent of) %.3e.

Who knew simply formatting numbers could be so complicated! Anyways, here is a simple function that does this formatting:

local mathtype, floor  = math.type, math.floor
local format, strlen, match = string.format, string.len, string.match

formatnumber = function(a)
    if mathtype(a) == "integer" or floor(a) == a then
        return format("%d", a)
    else
        str = format("%s", a)
        fmt = format("%.3f", a)
        exp = format("%.3e", a)
        if strlen(str) < strlen(fmt) then
            return str
        elseif fmt == "0.000" then
             x = match(exp, "(.*)e")
             y = match(exp, "e(.*)")
             return format("%s \\times 10^{%d}", x, y)
        else
            return fmt
        end
    end
end

We first check if the number is an integer (using math.type(a) == "integer") or if the number is a float of the type 8.0 (using math.floor(a) == a) and if so, format the number using %d.

We then next check if casting the number to a string leads to a shorter string than formatting it using %.3f. If so, we format it using %s. This ensures that a number like 8.2 is typeset as 8.2 rather than 8.200.

Finally, we check if formatting the number using %.3f gives "0.000". If so, we typeset the tex equvalent of %.3e.

Phew!

To use this code, I use %s in my templates, and then call the template as

context(solution(formatnumber(D), formatnumber(r1), formatnumber(r2)))

This gives me correct formatting in all the edge cases.

---

 <New Metapost interface | New escape mechanism for t-vim>

This entry was posted in Formatting and tagged luatex, programming.