Stochastic issue with lazy loaded parquet

[velochy]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

df = pl.DataFrame({ 'g': ['1','1','1','2'], })
df.write_parquet('dummy.parquet')
ldf = pl.scan_parquet('dummy.parquet')
fdf = ldf.filter(True & pl.col('g').is_in(['1']))
print(fdf.collect())

Log output

shape: (0, 2)
┌─────┬─────┐
│ v   ┆ g   │
│ --- ┆ --- │
│ str ┆ str │
╞═════╪═════╡
└─────┴─────┘

Issue description

I ran into this issue in a more complex pipeline, but have managed to condense it down to what seems to be the minimal example.

There are two very odd things about this bug:
a) It behaves stochastically, i.e. gives different results on different runs. On some runs, it works, on others it fails, showing an empty dataframe
b) It seems to require all the parts above, i.e. loading a parquet and having True & in the filter.

In case you are wondering why I'm using True &, I'm programmatically creating a filter and use &= to add sequential conditons, so True is the natural initial starting value.

Expected behavior

It behaves consistently and always returns

shape: (3, 2)
┌─────┬─────┐
│ v   ┆ g   │
│ --- ┆ --- │
│ str ┆ str │
╞═════╪═════╡
│ a   ┆ 1   │
│ a   ┆ 1   │
│ b   ┆ 1   │
└─────┴─────┘

Installed versions

--------Version info---------
Polars:              1.22.0
Index type:          UInt32
Platform:            Linux-6.8.0-52-generic-x86_64-with-glibc2.39
Python:              3.12.2 | packaged by Anaconda, Inc. | (main, Feb 27 2024, 17:35:02) [GCC 11.2.0]
LTS CPU:             False

----Optional dependencies----
Azure CLI            <not installed>
adbc_driver_manager  <not installed>
altair               5.5.0
azure.identity       <not installed>
boto3                <not installed>
cloudpickle          3.1.1
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               2024.12.0
gevent               <not installed>
google.auth          <not installed>
great_tables         <not installed>
matplotlib           3.10.0
numpy                1.26.4
openpyxl             <not installed>
pandas               2.2.3
pyarrow              19.0.0
pydantic             2.10.5
pyiceberg            <not installed>
sqlalchemy           <not installed>
torch                <not installed>
xlsx2csv             <not installed>
xlsxwriter           3.2.2

[cmdlineluser]

I think this is fixed on main #21204

[velochy]

Can confirm. Downloaded and built the current main head and both the toy example as well as the longer pipeline work as intended now.