Lean Kernel Arena / perf/let-fold

Test "perf/let-fold"

Expected: 👍 accept · Size: 316.9 KB · Lines: 6.0 k · lean4export: 3.1.0 · Lean: 4.34.1 · Timeout: 50.0 s · 📄 Declaration · 🔗 Source

The declaration to check is

let d := bits #n 0; (rfl : d + 0 = Nat.add d 0)

where bits #n 0 is a closed Nat whose evaluation makes 2^n recursive calls, and the rfl is wrapped in id so that its type is compared with d + 0 = Nat.add d 0 inside the body of the let.

The two sides differ only in how the addition is written. Unfolding HAdd.hAdd turns the left side into the right, for Θ(1). But both sides are additions of literals to a closed term, so a checker that folds closed Nat arithmetic (Nat.add on literals via GMP) will try to evaluate d first, for Θ(2^n). The official kernel only folds terms without free variables, and it checks the body of a let with d as a free variable (bound to its value in the local context), so it never evaluates bits. A checker that substitutes the value for d before checking the body makes the comparison closed and pays for the evaluation.

Found by con-leche on openai/math (SquareDifference, OAI.SquareDifference.source_ordered_contraction), where the bound value is Fintype.card TupleVertex, whose evaluation enumerates permutations and takes the checker past 6 GB; the official kernel checks that declaration in 16 ms.

N=16 in the Lean source.

Checker Result ⏱️ 🧠
sokonanoda 👍 3 ms (÷17) 132.1 MB (×2.0)
mathgraph 👍 4 ms (÷14) 87.6 MB (+33%)
nanoclo 👍 9 ms (÷6.5) 65.6 MB (-1%)
con-ron 👍 1.3 s (×22) 351.0 MB (×5.3)
nanobruijn 👍 1.3 s (×22) 287.3 MB (×4.4)
nanoda 👍 2.1 s (×36) 287.8 MB (×4.4)
lazylean 👍 12 ms (÷5.0) 22.9 MB (÷2.9)
con-leche 👍 3.1 s (×54) 346.4 MB (×5.3)
eink0rn 👍 1.2 s (×20) 491.9 MB (×7.5)
official 👍 58 ms (0%) 66.0 MB (0%)
ind-models 👍 163 ms (×2.8) 112.6 MB (+71%)
lean4lean 👍 67 ms (+15%) 99.6 MB (+51%)
tenet 👍 181 ms (×3.1) 53.2 MB (-19%)
tenet-hs 👍 23 ms (÷2.5) 54.7 MB (-17%)
lean4cobol 👍 317 ms (×5.5) 14.9 MB (÷4.4)
evmlean ⌛ 0 ms 0 B
mini 🚫 53 ms 73.6 MB
overfull ⌛ 0 ms 0 B
metalean-slow 🚫 72 ms 75.7 MB
tenet-jvm 👍 2.2 s (×38) 563.8 MB (×8.5)
nanoclo-fortran 👍 21 ms (÷2.7) 6.9 MB (÷9.6)
kiota 👍 6.3 s (×108) 173.4 MB (×2.6)
metalean-fast 👍 2.4 s (×42) 478.5 MB (×7.3)
canonical-min ✋ 411 ms 1.1 GB
rpylean 👍 2.2 s (×37) 803.4 MB (×12)
vow-lean-kernel ⌛ 0 ms 0 B
parse-only 👍 45 ms (-23%) 65.9 MB (0%)