GoKawiilFuzzing The Gleam Compiler
Can you find bugs in a compiler by generating random programs?
Published on: Tue Aug 25 2026
Introduction
I regularly check on Gleam’s changelog and issue tracker. I am very fond of this project and the people contributing to it. But every time I see an issue that relates to code generation or different outputs between the Erlang and JavaScript, it nags me that there was no way to basically “compute all the Gleam programs”, run them and see if there are any issues.
I imagine it like a chessboard, where you have a quasi-infinite number of possible positions on the board. But we want the chessboard to contain Gleam programs and we want an infinitely big database of those programs to see if they uncover untested edge cases.
My first attempt of doing something adjacent to this was actually prompting an LLM. I instructed it to read through loads of past Gleam issues and find more edge cases by “thinking hard about it”. It came up with all sorts of bit array combinations, nested anonymous functions, nested use patterns. Predictably, this approach did not yield many results.
$20 bucks of tokens later, it found exactly one issue, which was reported and fixed right away: https://github.com/gleam-lang/gleam/issues/5613. One is definitely more than zero. But there are plenty of issues with “LLM fuzzing”: it’s pricey, not deterministic and a bit like pulling the lever on a slot machine.
But there was another idea that I had avoided pursuing, because to be honest it just sounded like a lot of work: structure-aware fuzzing.
Structure-aware Fuzzing
... continue reading