Michael Gibson Senior Engineer, Antithesis Obscure feature + obscure feature + obscure feature = bug Debugging
In the quarter century I’ve been programming professionally, I’ve found three C++ compiler bugs; one each in g++1, clang++, and Microsoft’s MSVC. After finding roughly one problem every ten years, my takeaway is that you have to do something really, really obscure. Or actually, you have to use an obscure feature then pile on another obscure feature and pile on another obscure feature. In fairness, the g++ one was "something weird happened and we googled it and found that someone else had already reported it."
This is the story of something really, really obscure that we do in our C++ SDK, which led to finding the clang++ bug. If you’re not super familiar with C++, you should still be able to follow the main gist and just skim some of the details.
Antithesis provides a series of language-specific SDKs that let you add assertions to your code to specify correctness properties of your software.
Antithesis then runs your software while exercising it with some sort of workload (created using our Test Composer or written by hand). As your software runs, our fault injector disrupts the network, crashes containers, induces random delays in code, and so on. Every time your software hits one of your assertions, Antithesis checks that the assertion holds.
A typical (non-Antithesis) assertion would look like this:
void increment_by_pointer ( int * px ) { assert ( px != nullptr ) ; * px ++; }
Our equivalent of assert here is ALWAYS 2, which includes a message that becomes a test property in our system: void increment_by_pointer ( int * px ) { ALWAYS ( px != nullptr , “Parameter sent to increment_by_pointer should be non - null” ) ; * px ++; } Our SDKs provide many other interesting assertion types too: Actually, our semantically equivalent replacement for assert is ALWAYS_OR_UNREACHABLE , which differs from ALWAYS in how it treats "we never hit this code." Since that distinction isn't important for the rest of this blog post, we'll ignore the additional complication.
ALWAYS_LESS_THAN(x, value) : this is equivalent to ALWAYS(x < val) and analogous to EXPECT_LT in GoogleTest. It tells our system that different values of x are interesting here — we might try maximizing x, for example. The difference is that in ALWAYS(x < val) , the expression x < val is a boolean. But in ALWAYS_LESS_THAN(x, value) , both x and value are integers, so our fuzzer can execute logic on those integers.
REACHABLE() : this asserts that the code is reachable. For example, suppose you’re writing a video game, and there’s a complex series of things you have to do to get to level 3. You could put:
... continue reading