GoKawiilWe've found a new way to speed up parallel Golang continuous integration workflows by taking advantage of the Golang build cache. Replacing GitHub's official actions/setup-go action with a drop-in equivalent cut our test job runtimes by 69%. We're open-sourcing cloudx-io/setup-go (opens in a new tab) so you can do the same.
GitHub's official actions/setup-go step makes parallel jobs interfere with each other's performance, and it continuously loads stale cache values. Backtesting in our monorepo, which has the common situation of a few parallel Golang test jobs (one for lints, one for tests, one for builds), suggests that 86% of the work the default action does is completely unnecessary.
If you manage a moderately complex Go project, you can expect similar performance improvements; see our CI measurement methodology or just try it for yourself.
We Care About Fast CI
We've been shipping a lot of new products and features, and the pace at which we do it is actually increasing over time. This is no accident — we invest heavily in the tools and processes required to make this possible. At the center of every "software factory" are the test suites and Continuous Integration (CI) workflows that ensure code changes won't break in production. If our tests run reliably, and quickly, on every change, we can build at fantastic speed without worrying about breaking things for our customers.
This is important to us, so we measure and invest in the speed of our CI jobs. If you push code to a CloudX repository, our goal is that you get a clear answer as to its acceptability — whether it builds, its tests pass, and it abides by our linter rules — within 90 seconds.
Speed can be achieved in a number of ways, but at the end of the day if you want things to be fast you have to make algorithmic improvements. We're already using Warp Build (opens in a new tab) to run our CI jobs on fast, cost-efficient machines. As our test suite has scaled with our product surface area, we realized that actions/setup-go was not setting us up for success.
How actions/setup-go fails for parallel jobs
GitHub's actions/setup-go (opens in a new tab) is the GitHub-encouraged way to install and run Go in GitHub Actions. It uses actions/cache internally to save and restore the local Go module cache and build cache directories. In principle, that should make downloaded module source code and build/test artifacts from one job run available to all the subsequent job runs in your repo. Here's the default actions/setup-go cache key construction:
setup-go-${os}-${arch}-go-${goVersion}-${hashFiles('**/go.mod')}
... continue reading