GoKawiilI built buildprof (Github), an open-source tracing tool that shows where the time goes when you compile software on Linux. Here’s a realtime video of it profiling a clean build of ripgrep:
Watch the buildprof demo
Sometimes, builds are slow because there is simply a lot of code to compile. But more often than not, there are fixable problems: poor parallelism, repeated work, dependency downloads or a huge compiler/linker invocation. buildprof makes all of this clearly visible, so you can see what’s worth investigating and optimizing.
You run it by putting buildprof -- in front of any build command you already use:
buildprof -- make -j16 buildprof -- cargo build buildprof -- ninja -C out/target buildprof -- just build buildprof -- ./dev/custom-build-script.sh
buildprof records every process your build command launches, including their subprocesses (and their subprocesses…), and lays them out on one timeline. Time moves from left to right, bar width shows duration, and child processes appear beneath whatever launched them.
I made buildprof because this tweet from Jarred Sumner, chief architect of the Bun JavaScript runtime, was living rent free in my head:
Specifically, the claim that Bun’s new Rust build was >5× faster on Linux than its old Zig build really bothered me. In my experience, Zig projects had usually compiled much faster than Rust projects of similar complexity. That intuition was enough to make me feel there was a mystery to solve.
This was further compounded by another important, yet easily missed, detail in the tweet: the Zig build used Full LTO, while the Rust build used ThinLTO.
Compilers normally optimize separate compilation units largely in isolation. Link-time optimization (LTO) lets them optimize across those boundaries. Full LTO brings those units together into one large optimization job, while ThinLTO preserves more separation so much of the work can run in parallel.
... continue reading