GoKawiilNew tool RealDiff detects hidden runtime behavior changes in pull requests across six languages
RealDiff is an open-source tool that compares two Git revisions by actually running their tests, rather than just diffing source code, to catch behavior changes that ripple beyond the files a pull request touched. It builds a noise baseline from repeated base runs, then flags the first divergent behavior in each call tree, supporting analysis across six programming languages including .NET, Java, and Node.js. A demo shows a one-line change from an unstable to a stable sort algorithm silently altering the output of an unrelated pricing engine elsewhere in the codebase.
GoKawiil's interpretation of the reporting above, not reported fact.
Code review tools today mostly show textual diffs, leaving reviewers blind to downstream effects of seemingly innocuous changes like swapping sort implementations or refactoring shared utilities. By surfacing actual runtime divergences, including in files never edited, RealDiff addresses a real gap where tests pass but silently produce different results, a common source of subtle production bugs. This kind of tooling could become part of CI pipelines to supplement static diff review with behavioral verification.
- RealDiff runs and compares actual test execution between two code versions instead of only diffing source text.
- It establishes a noise baseline using repeated base runs to distinguish real behavior changes from flaky test variance.
- A demo shows how switching List.Sort to OrderBy in a helper function silently changes outputs in an unrelated pricing module.
The Pragmatic Programmer, 20th Anniversary Edition — If RealDiff's example of a "harmless" refactor silently changing pricing output made you wince, this classic is the perfect companion read. It's packed with practical guidance on testing, refactoring safely, and reasoning about how code changes ripple through a system "?" exactly the mindset behind runtime behavior diffing.
See The Pragmatic Programmer, 20th Anniversary Edition on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: github.com, 2026-08-27
Published there as: “Show HN: RealDiff – runtime behavior diffing for pull requests (six languages)”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.