GoKawiilDeveloper builds structure-aware fuzzer to hunt bugs in the Gleam compiler
A developer frustrated by inconsistent Erlang and JavaScript outputs from the Gleam compiler first tried using an LLM to generate edge-case test programs, spending about $20 in tokens to surface just one confirmed bug (issue #5613), which was promptly fixed. Dissatisfied with the cost and randomness of that approach, they pivoted to building a structure-aware fuzzer that generates syntactically valid Gleam programs automatically to systematically probe the compiler for hidden defects.
GoKawiil's interpretation of the reporting above, not reported fact.
Compilers that target multiple backends, like Gleam's Erlang and JavaScript outputs, are prone to subtle divergences that manual testing rarely catches, making automated exploration valuable for catching real-world bugs before users do. This experiment also highlights the current limits of LLM-based bug hunting compared to purpose-built fuzzing techniques, informing how open-source language maintainers might invest testing resources going forward.
- LLM-based fuzzing of the Gleam compiler cost about $20 and found only one real bug (issue #5613).
- The developer is now building a structure-aware fuzzer to generate valid Gleam programs automatically for testing.
- The goal is to catch inconsistencies between Gleam's Erlang and JavaScript code generation outputs.
The Fuzzing Book" / "Fuzzing: Brute Force Vulnerability Discovery — If this deep dive into randomly generating Gleam programs to shake out compiler bugs got you curious, this classic book lays out the foundations of fuzzing as a testing discipline. It's a great companion for anyone who wants to build their own generators and harnesses instead of just prompting an LLM and hoping for the best.
See The Fuzzing Book" / "Fuzzing: Brute Force Vulnerability Discovery on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: kurz.net, 2026-08-25
Published there as: “Fuzzing the Gleam Compiler”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.