GoKawiilDataflow Model authors reassess a decade of streaming analytics design
The creators of the original Dataflow Model paper—Tyler Akidau, Rafael Fernández-Moctezuma, Reuven Lax and Daniel Mills—revisited their influential streaming framework eleven years after publication, on the occasion of its VLDB Test of Time award. They conclude that its core ideas, such as prioritizing event time and abandoning the wait for complete data, held up well, but admit that windowing, triggers, and a stream-first worldview were overengineered or misguided compared to database-style approaches like SQL and materialized views.
GoKawiil's interpretation of the reporting above, not reported fact.
This self-critique signals a broader shift in the data engineering field toward simplifying streaming systems by borrowing more directly from database theory rather than building increasingly complex custom mechanisms. It suggests future systems may unify streams and tables more naturally, reducing the operational burden currently placed on developers working with real-time data.
- The original Dataflow Model's emphasis on event time and rejecting data completeness remains valid.
- Triggers and windowing mechanisms are now seen as overengineered solutions to problems users shouldn't have faced.
- Streams and tables should be treated as two views of the same underlying data, echoing database-style incremental view maintenance.
Streaming Systems (O'Reilly) by Tyler Akidau — Written by Tyler Akidau and colleagues behind the Dataflow Model, this O'Reilly book expands the very ideas the paper revisits — event time, windowing, watermarks, triggers, and the streams-and-tables duality. It's the natural next read if the retrospective left you wanting the full argument in depth.
See Streaming Systems (O'Reilly) by Tyler Akidau on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: vldb.org, 2026-09-06
Published there as: “The Dataflow Model Revisited”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.