GoKawiilConviva finds io_uring underperforms mmap in Rust-based DataFusion query engine
Conviva engineers tried replacing mmap with io_uring for reading large Arrow IPC files in their DataFusion-based analytics engine, hoping to fix latency spikes seen under heavy concurrent query loads with mmap. Instead of resolving the problem, the io_uring approach ran slower, and the team's investigation traced the original mmap issue to page-cache thrashing and excessive page faults under high concurrency rather than the memory-mapping mechanism itself.
GoKawiil's interpretation of the reporting above, not reported fact.
The case is a reminder that swapping one I/O primitive for a newer, trendier one (io_uring) isn't automatically a performance win, especially for read-heavy analytical workloads with large sequential files. It highlights how production-scale concurrency can expose subtle OS-level bottlenecks—like shrinking page cache and page fault storms—that don't show up in lighter testing, requiring careful diagnosis rather than assuming a straightforward technology swap will fix things.
- Conviva's DataFusion/Arrow/Rust query engine relies on mmap to read multi-gigabyte Arrow IPC files with zero-copy access.
- Under heavy concurrent production load, mmap caused severe page-cache shrinkage and page faults, spiking p95 latency from 30s to over 150s.
- Switching to io_uring as a fix actually made performance worse, showing the real bottleneck lay elsewhere in page-cache behavior.
Samsung 990 PRO NVMe SSD — The article's whole story hinges on reading multi-gigabyte Arrow IPC files off fast local NVMe, and the 990 PRO is a go-to PCIe 4.0 drive for exactly that kind of throughput-hungry workload. Great for anyone building or benchmarking a query engine at home and wanting local storage that won't be the bottleneck.
See Samsung 990 PRO NVMe SSD on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: conviva.ai — Melissa Pieroni, 2026-09-11
Published there as: “We Replaced MMAP with Io_uring in Our Rust Query Engine. It Got Slower”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.