Tech News
← Back to articles

The C-Shaped Hole in Package Management

read original related products more articles

System package managers and language package managers are both called package managers. They both resolve dependencies, download code, and install software. But they evolved to solve different problems, and the overlap is where all the friction lives. If you drew a venn diagram, C libraries would sit right in the middle: needed by language packages, provided by system packages, understood by neither in a way the other can use. As Kristoffer Grönlund put it in 2017: “Why are we trying to manage packages from one package manager with a different package manager?”

apt, dnf, pacman, and the rest started as application delivery mechanisms. Users needed to install Firefox or LibreOffice, and decomposing applications into shared libraries emerged as an implementation detail. The dependency graph is there to serve patching and security updates for end users, not developers. A package like python3-requests isn’t there because the Python ecosystem wanted it. It’s there because some GUI application or system tool needed it, and the distro maintainers packaged it.

System package managers generally keep only one version of each package at a time. This massively simplifies dependency resolution, but it means getting a newer or older version is hard for individual users without upgrading the whole system. It’s a stop-the-world model. Hacks like naming packages python3 and python2 exist to work around this when you really need multiple versions, but they’re exceptions.

npm, pip, cargo, and gem went the other direction. They’re dependency assembly tools that help developers build projects. They keep every version around indefinitely, letting projects pin exactly what they need. They’re also cross-platform by design: pip doesn’t know if it’s running on Debian or Fedora or macOS, so it can’t just shell out to the system package manager to install C dependencies even if it wanted to. The fact that you can pip install httpie and get a working command-line tool is a side effect, not the purpose. These tools exist so you can declare requests>=2.28 in a manifest and get a working dependency tree.

The difference shows up in how they handle the same library. A distro maintainer sees a new OpenSSL release and asks: will this break existing applications? Can we backport security fixes without changing behavior? A language package manager asks: does this satisfy version constraints? Can multiple versions coexist? When you need both, the seams show.

Edward Yang wrote about this in 2014: “The different communities don’t talk to each other.” They don’t need to, most of the time. But language packages that wrap C libraries have to deal with C dependencies somehow, and that’s system package manager territory. The language package manager has no vocabulary for it.

The C-shaped hole

C never developed a canonical package registry. It predates the model of “download dependencies from the internet,” and by the time that model became standard, the ecosystem was too fragmented to converge. pkg-config exists as a partial vocabulary for discovering installed libraries, but it’s a query mechanism for what’s already on your system, not a way to declare or fetch dependencies. Conan and vcpkg exist now and are actively maintained, but neither has the cultural ubiquity of crates.io or npm. There’s no default answer to “how do I depend on libcurl” the way there is for “how do I depend on serde.”

System package managers filled this gap by default. If you need libcurl or OpenSSL or zlib, you install them through apt or dnf or brew. This makes your system package manager the de facto C package manager whether it was designed for that or not. And every distro names packages differently: libssl-dev on Debian, openssl-devel on Fedora, openssl on Alpine, openssl@3 on Homebrew. Same library, four names, no mapping between them.

Every language that needs C bindings solves distribution independently. Python has wheels that bundle compiled extensions. Node has node-gyp that compiles against system headers at install time. Rust has build.rs scripts that call pkg-config. Go has cgo with its own linking story. Ruby has native extensions that compile on gem install .

... continue reading