GoKawiilToday we are releasing the first release candidate for Polars 2.0. The definite 2.0 release will land in the following weeks. We don’t aim to make a big feature release of Polars 2.0. In fact we hope it to be a boring experience for you. The reason we bump this major version is that we can get rid of design decisions made in the past that currently block us and then we want to change defaults to more sensible settings that will benefit a greater audience. The biggest default change will be that all LazyFrame queries now will run on the streaming engine. Casual Polars users can therefore expect huge improvements in memory usage and performance. In aggregate we expect the streaming engine to be easily 5x faster.
To help users transition to 2.0, we have posted a full migration guide. This post will cover a few of the highlights.
Streaming engine as default
This is the biggest impact change of 2.0. Calling collect on a LazyFrame will now default to the streaming engine, leading to massive memory and performance improvements on most queries for users. The reason this required a major version bump is that the streaming engine doesn’t guarantee row-order by default for certain operations ( join , group_by , unpivot , etc.). If you require observable row-order in those operations, you can opt in to that by setting maintain_order=True .
For users who want to keep using the “in-memory” engine as default, they can do so by setting the engine affinity.
lf = pl . LazyFrame ({ "k" : [ 2 , 1 , 0 ], "v" : [ "a" , "b" , "c" ]}) other = pl . LazyFrame ({ "k" : [ 0 , 1 , 2 ], "r" : [ "x" , "y" , "z" ]}) # 2.0: engine="auto" now resolves to the streaming engine. # Row order is no longer guaranteed for joins, group_by, unpivot, ... ( lf . join (other, on = "k" , how = "left" ) . collect () ) # ┌─────┬─────┬─────┐ # │ k ┆ v ┆ r │ <- order may not match `lf`'s original row order # └─────┴─────┴─────┘ # Opt in to observable order for this query: ( lf . join (other, on = "k" , how = "left" , maintain_order = "left" ) . collect () ) # Or keep the old in-memory engine as the default, process-wide: pl . Config . set_engine_affinity ( "in-memory" ) # ...or per query: ( lf . join (other, on = "k" , how = "left" ) . collect (engine = "in-memory" ) )
Stricter Polars
Polars aims to be strict and fail fast. Errors should ideally raise up-front, not 20 minutes into a pipeline. Implicit behavior on data-mismatches should be opt-in, not a default, since those mismatches can hide bugs. This strictness has become even more valuable with the rise of AI-driven development. Agents can validate a query’s structure early by calling collect_schema() , which resolves types and catches schema-level mismatches without materializing any data. This ensures fast feedback for the agents, meaning they can iterate faster. Not all errors can be caught during compilation of the query plan, some depend on data. In these cases Polars defaults to stricter behavior to ensure inconsistencies are caught instead of silently producing different results.
Below are a few examples where Polars has gotten more strict:
is_in lossless type-coercion
... continue reading