# Formatting numbers in Lua

Posted on September 9, 2019

I often use Lua to generate solution for homework assignments. Ideally, I want the solution to look exactly how it would look if it were written by hand. But this can be tricker than it appears at first glance. In this post, I’ll explain the issue and how I solve it.

To illustrate the formatting issue, let me consider an example of writing the solution of how to find roots of a quadratic equation. Let’s start with a simple example. First let’s define

```
\defineenumeration[question]
\defineenumeration[solution]
```

Then consider the following example.

```
\startquestion
Find the roots of $x^2 + 5x + 6 = 0$.
\stopquestion
\startsolution
Let's start by computing the determinant.
\startformula
Δ = b^2 - 4ac = 1
\stopformula
Since $Δ > 0$, the roots are given by
\startformula
r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
= -2 \text{ and } -3.
\stopformula
\stopsolution
```

Now suppose I want to generate a homework assignment with four or five such
questions. In order to ensure that I don’t make any mistakes, I generate the
questions and the answers using Lua. For simplicity, let’s assume that both
roots are real. Then, I can use `string.formatters`

to easily generate the
assignment and the solution.

```
\startluacode
local formatters = string.formatters
local question = formatters[ [[
\startquestion
Find the roots of $%s x^2 + %s x + %s = 0$.
\stopquestion
]] ]
local solution = formatters[ [[
\startsolution
Let's start by computing the determinant.
\startformula
Δ = b^2 - 4ac = %s.
\stopformula
Since $Δ > 0$, the roots are given by
\startformula
r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
= %s \text{ and } %s.
\stopformula
\stopsolution
]] ]
local sqrt = math.sqrt
assignment = assignment or { }
assignment.roots = function(a, b, c)
context(question(a == 1 and "" or a, b, c))
D = b^2 - 4*a*c
r1 = (-b + sqrt(D))/(2*a)
r2 = (-b - sqrt(D))/(2*a)
context(solution(D, r1, r2))
end
\stopluacode
```

Then, in the homework assignment, I can generate the above question and its solution using:

```
\ctxlua{assignment.roots(1, 5, 6)}
```

The above generated solution works well when all numbers are integer valued. However, the generated solution is not ideal when some of the calculations result in floats. For example:

```
\ctxlua{assignment.roots(1, 4, 2)}
```

will generate:

```
\startquestion
Find the roots of $x^2 + 4x + 2 = 0$.
\stopquestion
\startsolution
Let's start by computing the determinant.
\startformula
Δ = b^2 - 4ac = 8.0.
\stopformula
Since $Δ > 0$, the roots are given by
\startformula
r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
= -0.5857864376269 \text{ and } -3.4142135623731.
\stopformula
\stopsolution
```

Technically, the solution is correct. But one never types a float with a
precision of 12 decimal places. Of course, I could change the `%s`

in the
template to `%.3f`

as follows:

```
local solution = formatters[ [[
\startsolution
Let's start by computing the determinant.
\startformula
Δ = b^2 - 4ac = %.3f.
\stopformula
Since $Δ > 0$, the roots are given by
\startformula
r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
= %.3f \text{ and } %.3f.
\stopformula
\stopsolution
]] ]
```

For the second problem, this will generate:

```
\startquestion
Find the roots of $x^2 + 4x + 2 = 0$.
\stopquestion
\startsolution
Let's start by computing the determinant.
\startformula
Δ = b^2 - 4ac = 8.000.
\stopformula
Since $Δ > 0$, the roots are given by
\startformula
r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
= -0.586 \text{ and } -3.414.
\stopformula
\stopsolution
```

which is partially acceptable (when typesetting the solution by hand, one
would uses `Δ = 8`

rather than `Δ = 8.000`

) but for the first problem, we now get

```
\startquestion
Find the roots of $x^2 + 5x + 6 = 0$.
\stopquestion
\startsolution
Let's start by computing the determinant.
\startformula
Δ = b^2 - 4ac = 1.000.
\stopformula
Since $Δ > 0$, the roots are given by
\startformula
r_{1,2} = \dfrac{ -b \pm \sqrt{Δ} }{ 2a }
= -2.000 \text{ and } -3.000.
\stopformula
\stopsolution
```

which is not ideal.

So, to typeset such examples, I need to format numbers as follows:

- If the number is an integer, use
`%d`

. - If the number is a float, use
`%.3f`

.

However, once you think about it, the above spec is not complete. In addition, what we want is that

- If
`%.3f`

gives`"0.000"`

, using (the tex equivalent of)`%.3e`

.

Who knew simply formatting numbers could be so complicated! Anyways, here is a simple function that does this formatting:

```
local mathtype, floor = math.type, math.floor
local format, strlen, match = string.format, string.len, string.match
formatnumber = function(a)
if mathtype(a) == "integer" or floor(a) == a then
return format("%d", a)
else
str = format("%s", a)
fmt = format("%.3f", a)
exp = format("%.3e", a)
if strlen(str) < strlen(fmt) then
return str
elseif fmt == "0.000" then
x = match(exp, "(.*)e")
y = match(exp, "e(.*)")
return format("%s \\times 10^{%d}", x, y)
else
return fmt
end
end
end
```

We first check if the number is an integer (using `math.type(a) == "integer"`

)
or if the number is a float of the type `8.0`

(using `math.floor(a) == a`

) and
if so, format the number using `%d`

.

We then next check if casting the number to a string leads to a shorter string than
formatting it using `%.3f`

. If so, we format it using `%s`

. This ensures that
a number like `8.2`

is typeset as `8.2`

rather than `8.200`

.

Finally, we check if formatting the number using `%.3f`

gives `"0.000"`

. If
so, we typeset the tex equvalent of `%.3e`

.

Phew!

To use this code, I use `%s`

in my templates, and then call the template as

```
context(solution(formatnumber(D), formatnumber(r1), formatnumber(r2)))
```

This gives me correct formatting in all the edge cases.

This entry was posted in Formatting and tagged luatex, programming.