Skip to content
Tech News
← Back to articles

Floats Don't Agree with Themselves

read original get Floating Point Debugging Kit → more articles
Why This Matters

This article highlights the challenges of numerical reproducibility in computational geometry due to the nuances of IEEE 754 floating-point standards and hardware differences. It underscores the importance of exact arithmetic for precise geometric calculations, especially in applications like polygon decomposition, where tiny discrepancies can lead to different outcomes. The development of libraries like exact-poly demonstrates a move towards more reliable, hardware-agnostic geometric computations, benefiting both developers and end-users in critical applications.

Key Takeaways

I was debugging a polygon-overlap test that worked locally and failed on the server. Same code. Same input. Different answer.

The function deciding the answer was small. Three points A, B, C; return the sign of (B - A) cross (C - A) . That sign tells you whether a vertex is convex or reflex, whether a diagonal stays inside the polygon, whether a point is above a line or below. On x86, LLVM had folded the multiply and the subtract into a single fma — one rounding step instead of two. On WASM there is no FMA, so two roundings. A vertex sitting in the epsilon neighborhood of zero fell on opposite sides. From one bit of disagreement, the whole decomposition forked.

This wasn't a bug in my code. It was IEEE 754 working as advertised. The standard pins down the storage format. It does not pin down behavior. Reassociation, FMA contraction, x87 registers at 80 bits, denormal flush flags — four separate ways to get a different answer from the same inputs. Tightening tolerances doesn't help: convex decomposition is discrete. A vertex is reflex or it isn't. An epsilon at the input is a binary difference at the output.

So I wrote exact-poly , a 2D geometry library that uses no floats at all.

A 10-vertex star decomposed into 5 convex parts. Vertex labels ( v0 – v9 ) match input order; part labels ( p0.0 – p5.3 ) are emitted by the cascade. Sum of part areas equals the ring area, exactly.

Try it GitHub — mercaearth/exact-poly

— mercaearth/exact-poly Live demo (7 tabs) — exact-poly.merca.earth

(7 tabs) — exact-poly.merca.earth In production — merca.earth runs exact-poly in WASM client-side and at validation; pick a spot, draw a polygon, watch the same i64 math accept or reject the claim on both sides

IEEE 754 specifies formats and basic operations. The gaps between those operations are where reproducibility leaks out.

Effect Mechanism Where it diverges Intermediate registers x87 FPU keeps values in 80 bits, rounds only on a spill to memory x86 vs ARM, RISC-V, WASM (no extended precision) Fused multiply-add fma(a, b, c) rounds once instead of twice — more accurate, different result LLVM enables on one target, disables on the next Reassociation (a + b) + c rewritten as a + (b + c) changes rounding -ffast-math default, often on without intent Denormals Flush-to-zero avoids the subnormal slowdown Per-process flag, silently flipped

... continue reading