GoKawiilWe represent floating-point numbers using the IEEE standard. For very small numbers, the standard uses special subnormal numbers. Unfortunately, they have a reputation of making operations slow. Thus video game programmers and machine learning specialists sometimes avoid computing with subnormal numbers for performance.
How slow are they? Let me measure. I wrote a small C++ benchmark with a few kernels over arrays of 16384 values (small enough to fit in cache):
multiply each value by 0.75,
add two arrays,
divide each value by 3,
multiply normal values by a tiny constant (2 -1030 ) so that the inputs are normal but the outputs are subnormal,
) so that the inputs are normal but the outputs are subnormal, a dependent chain x *= 0.9999 repeated 16384 times.
For each kernel, I feed either normal values (in [0.5, 1) ), subnormal values, or normal values where one value in a hundred is subnormal. The compiler is allowed to autovectorize the array computations. I use GCC 15 with -O3 -march=native on Linux and Apple clang 17 with the same flags on macOS. I also checked with clang 21 on Linux to make sure.
I ran the benchmark on five processors:
Intel Xeon 6975P-C (Granite Rapids), on an AWS c8i.xlarge instance,
... continue reading