When Go 1.24 was released in early 2025, we were eager to roll it out across our services. The headline feature—the new Swiss Tables map implementation—promised reduced CPU and memory overhead.
Our story begins while the new version was being rolled out internally. Shortly after deploying it to one of our data-processing services, we noticed an unexpected memory usage increase:
We observed the same pattern, a ~20% increase in memory usage, across multiple environments before pausing the rollout. To confirm our suspicions, we conducted a bisect in the staging environment, which pointed directly to the Go 1.24 upgrade as the culprit.
But here's where things got truly puzzling: The increased memory usage wasn't showing up in Go runtime metrics and live heap profiles, which meant that, from the Go runtime's perspective, the service wasn't using more memory. This caught our attention immediately. After all, Go 1.24 was supposed to reduce memory usage thanks to Swiss Tables, not increase it.
In this two-part series, we'll share how we investigated this surprising increase and how the same release ultimately helped us reduce our memory footprint.
In this first post, we'll walk through how we diagnosed a subtle memory allocator regression introduced by a runtime refactor and how we worked with the Go team to identify and confirm the root cause. In Part 2: How Go 1.24's Swiss Tables saved us hundreds of gigabytes, we'll show how Go 1.24's new Swiss Tables implementation dramatically reduced the memory usage of a large in-memory map—yielding a net win across our highest traffic services.
Before diving deeper, we needed to eliminate the most likely suspects. Go 1.24 introduced a couple of major changes that could potentially impact memory usage, so we systematically tested each one:
Swiss Tables : This feature was supposed to reduce memory usage, but we needed to verify it wasn't somehow causing our problem. We created a test build with Swiss Tables disabled by setting the GOEXPERIMENT=noswissmap flag. However, this did not show any improvement in memory usage .
Spin bit mutex: This modified the internal implementation of the runtime mutexes. We also tested reverting the spin bit mutex implementation by creating a build with the GOEXPERIMENT=nospinbitmutex flag. However, we still observed the increased memory usage with this Go experiment flag set.
After ruling out the most likely culprits, we decided to deep dive into Go's runtime metrics to understand what was happening under the hood. These metrics provide valuable insights into Go's runtime, including internal memory management—heap allocations, garbage collector (GC) cycles, and so on. Since Go 1.16, these metrics are exposed by the runtime/metrics package. For more details on Go runtime memory metrics, check out Go memory metrics demystified.
... continue reading