GoKawiilIn the previous post, I mentioned that buildcache has some unique properties compared to ccache and sccache. One of them is its Lua plugin system, which lets you write custom wrappers for programs that aren’t compilers in the traditional sense. With Bug 2027655 now merged, we can use this to cache Firefox’s WebIDL binding code generation.
What’s the WebIDL step?
When you build Firefox, one of the earlier steps runs python3 -m mozbuild.action.webidl to generate C++ binding code from hundreds of .webidl files. It produces thousands of output files: headers, cpp files, forward declarations, event implementations, and so on. The step isn’t terribly slow on its own, but it runs on every clobber build, and the output is entirely deterministic given the same inputs. That makes it a perfect candidate for caching.
The problem was that the compiler cache was never passed to this step. Buildcache was only wrapping actual compiler invocations, not the Python codegen.
The change
The fix in Bug 2027655 is small. In dom/bindings/Makefile.in , we now conditionally pass $(CCACHE) as a command wrapper to the py_action call:
WEBIDL_CCACHE = ifdef MOZ_USING_BUILDCACHE WEBIDL_CCACHE = $( CCACHE ) endif webidl.stub : $(codegen_dependencies) $( call py_action,webidl $( relativesrcdir ) , $( srcdir ) ,, $( WEBIDL_CCACHE )) @$( TOUCH ) $@
The py_action macro in config/makefiles/functions.mk is what runs Python build actions. The ability to pass a command wrapper as a fourth argument was also introduced in this bug. When buildcache is configured as the compiler cache, this means the webidl action is invoked as buildcache python3 -m mozbuild.action.webidl ... instead of just python3 -m mozbuild.action.webidl ... . That’s all buildcache needs to intercept it.
Note the ifdef MOZ_USING_BUILDCACHE guard. This is specific to buildcache because ccache and sccache don’t have a mechanism for caching arbitrary commands. Buildcache does, through its Lua wrappers.
The Lua wrapper
... continue reading