Test "perf/let-fold"
Expected: 👍 accept · Size: 316.9 KB · Lines: 6.0 k · lean4export: 3.1.0 · Lean: 4.34.1 · Timeout: 50.0 s · 📄 Declaration · 🔗 Source
The declaration to check is
let d := bits #n 0; (rfl : d + 0 = Nat.add d 0)
where bits #n 0 is a closed Nat whose evaluation makes 2^n recursive
calls, and the rfl is wrapped in id so that its type is compared with
d + 0 = Nat.add d 0 inside the body of the let.
The two sides differ only in how the addition is written. Unfolding
HAdd.hAdd turns the left side into the right, for Θ(1). But both sides
are additions of literals to a closed term, so a checker that folds closed
Nat arithmetic (Nat.add on literals via GMP) will try to evaluate d
first, for Θ(2^n). The official kernel only folds terms without free
variables, and it checks the body of a let with d as a free variable
(bound to its value in the local context), so it never evaluates bits. A
checker that substitutes the value for d before checking the body makes
the comparison closed and pays for the evaluation.
Found by con-leche on openai/math (SquareDifference,
OAI.SquareDifference.source_ordered_contraction), where the bound value is
Fintype.card TupleVertex, whose evaluation enumerates permutations and
takes the checker past 6 GB; the official kernel checks that declaration in
16 ms.
N=16 in the Lean source.
| Checker | Result | ⏱️ | 🧠 | |||
|---|---|---|---|---|---|---|
| sokonanoda | 👍 | 3 ms | (÷17) | 132.1 MB | (×2.0) | |
| mathgraph | 👍 | 4 ms | (÷14) | 87.6 MB | (+33%) | |
| nanoclo | 👍 | 9 ms | (÷6.5) | 65.6 MB | (-1%) | |
| con-ron | 👍 | 1.3 s | (×22) | 351.0 MB | (×5.3) | |
| nanobruijn | 👍 | 1.3 s | (×22) | 287.3 MB | (×4.4) | |
| nanoda | 👍 | 2.1 s | (×36) | 287.8 MB | (×4.4) | |
| lazylean | 👍 | 12 ms | (÷5.0) | 22.9 MB | (÷2.9) | |
| con-leche | 👍 | 3.1 s | (×54) | 346.4 MB | (×5.3) | |
| eink0rn | 👍 | 1.2 s | (×20) | 491.9 MB | (×7.5) | |
| official | 👍 | 58 ms | (0%) | 66.0 MB | (0%) | |
| ind-models | 👍 | 163 ms | (×2.8) | 112.6 MB | (+71%) | |
| lean4lean | 👍 | 67 ms | (+15%) | 99.6 MB | (+51%) | |
| tenet | 👍 | 181 ms | (×3.1) | 53.2 MB | (-19%) | |
| tenet-hs | 👍 | 23 ms | (÷2.5) | 54.7 MB | (-17%) | |
| lean4cobol | 👍 | 317 ms | (×5.5) | 14.9 MB | (÷4.4) | |
| evmlean | ⌛ | 0 ms | 0 B | |||
| mini | 🚫 | 53 ms | 73.6 MB | |||
| overfull | ⌛ | 0 ms | 0 B | |||
| metalean-slow | 🚫 | 72 ms | 75.7 MB | |||
| tenet-jvm | 👍 | 2.2 s | (×38) | 563.8 MB | (×8.5) | |
| nanoclo-fortran | 👍 | 21 ms | (÷2.7) | 6.9 MB | (÷9.6) | |
| kiota | 👍 | 6.3 s | (×108) | 173.4 MB | (×2.6) | |
| metalean-fast | 👍 | 2.4 s | (×42) | 478.5 MB | (×7.3) | |
| canonical-min | ✋ | 411 ms | 1.1 GB | |||
| rpylean | 👍 | 2.2 s | (×37) | 803.4 MB | (×12) | |
| vow-lean-kernel | ⌛ | 0 ms | 0 B | |||
| parse-only | 👍 | 45 ms | (-23%) | 65.9 MB | (0%) |