Back in 2017–2020, while I was on the Blaze team at Google, I took on a 20% project that turned into a bit of an obsession: sandboxfs. Born out of my work supporting iOS development, it was my attempt to solve a persistent pain point that frustrated both internal teams and external users alike: Bazel’s poor sandboxing performance on macOS.
sandboxfs was a user-space file system designed to efficiently create virtual file hierarchies backed by real files—a faster alternative to the “symlink forests” that Bazel uses to prepare per-action sandboxes. The idea was simple: if we could lower sandbox creation overhead, we could make Bazel’s sandboxing actually usable on macOS.
Unfortunately, things didn’t play out as I dreamed. Today, sandboxfs is effectively abandoned, and macOS sandboxing performance remains an unsolved problem. In this post, I’ll walk you through why I built sandboxfs, what worked, what didn’t, and why—despite its failure—I still think the core idea holds promise.
Sandboxing 101
To understand how sandboxfs was intended to help with sandboxed build performance, we need to first dive into how Bazel runs build actions. For those unfamiliar with Bazel’s terminology, a build action or action is an individual build step, like a single compiler or linker execution.
To run actions, Bazel uses the strategies abstraction to decouple action tracking in the build graph from how those actions are actually executed. The default strategy for local builds is the sandboxed strategy, which isolates the processes that an action runs from the rest of the system. The goal is to make these processes behave in a deterministic manner.
The sandboxed strategy achieves action isolation via two different mechanisms:
The use of kernel-level sandboxing features to restrict what the action can do (limit network access, limit reads and writes to parts of the file system, etc.). One such mechanism is sandbox-exec on macOS.
The creation of an execution root (or execroot) in which the action runs. The execroot contains the minimum set of files required for the action to run: namely, the toolchain and the action inputs (source files, toolchain dependencies, etc.). One way to do this is via symlink forests.
Symlink forests
... continue reading