Skip to content
Tech News
← Back to articles

ScopeDB team builds Rust port of Apache DataSketches, reverse-engineers CPC sketch logic

read original more articles
GoKawiil Brief

Developers working on ScopeDB's distinct-counting feature moved from Cloudflare's cardinality-estimator crate to Apache DataSketches, then collaborated with other Rust engineers to create datasketches-rust, starting with an HllSketch implementation. The team is now working through CpcSketch, a more space-efficient algorithm based on Kevin J. Lang's 2017 cardinality estimation research, originally implemented in OCaml and Java/C++ but lacking a Rust version.

Why It Matters

GoKawiil's interpretation of the reporting above, not reported fact.

The effort suggests a gap in Rust's data-sketching ecosystem that practitioners are filling by porting established Java/C++ algorithms rather than building from scratch, which could improve reliability for systems like ScopeDB that need auditable, memory-safe approximate counting. Because CPC sketches rely on intricate bit-matrix logic from older academic code, this kind of reverse-engineering work could also make Lang's algorithm more accessible and maintainable for other Rust-based data systems.

Key Takeaways

Source: tisonkun.io, 2026-11-16

Published there as: “Reverse-Engineering Kevin J. Lang's Lost OCaml Code in Apache DataSketches”

Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.
Keep reading
rust 159 briefs → ocaml 4 briefs →