At Jane Street we use a pattern/library called “expect tests” that makes test-writing feel like a REPL session, or like exploratory programming in a Jupyter notebook—with feedback cycles so fast and joyful that it feels almost tactile. Having used them for some time now this is the only way I’d ever want to write tests.
Other languages call these “snapshot” tests—see for example Rust’s expect-test, which seems to have been inspired by our library, or Javascript’s Jest. We were first put onto the idea ourselves by Mercurial’s unified testing format, and so-called “cram” tests, for testing shell sessions.
In most testing frameworks I’ve used, even the simplest assertions require a surprising amount of toil. Suppose you’re writing a test for a fibonacci function. You start writing assert fibonacci(15) == ... and already you’re forced to think. What does fibonacci(15) equal? If you already know, terrific—but what are you meant to do if you don’t?
I think you’re supposed to write some nonsense, like assert fibonacci(15) == 8 , then when the test says “WRONG! Expected 8, got 610”, you’re supposed to copy and paste the 610 from your terminal buffer into your editor.
This is insane!
Here’s how you’d do it with an expect test:
printf "%d" ( fibonacci 15 ); [% expect {||}]
The %expect block starts out blank precisely because you don’t know what to expect. You let the computer figure it out for you. In our setup, you don’t just get a build failure telling you that you want 610 instead of a blank string. You get a diff showing you the exact change you’d need to make to your file to make this test pass; and with a keybinding you can “accept” that diff. The Emacs buffer you’re in will literally be overwritten in place with the new contents [1]:
It’s hard to overstate how powerful this workflow is. To “write a test” you just drop an [%expect] block below some code and it will get filled in with whatever that code prints.
Just the other day I was writing a tricky little function that rounds numbers under an unusual set of constraints; it was exactly the kind of thing you’d want to write in a REPL or Jupyter notebook, to iterate quickly against lots of examples. All I had to do was write the following right below my function:
... continue reading