fix: Fix more global categorical issues

[ritchie46]

@mcrumiller FYI

[ritchie46]

driveby

[mcrumiller]

@ritche the two failing tests in #20514 are fixed with this. Let me do a temp merge with that branch to do a full test.

[github-actions]

The uncompressed binary size was Unknown MB.
The uncompressed binary size after this PR is 1119.4055 MB.

[mcrumiller]

Almost there, test_fast_unique_flag_from_arrow is still failing on global:

import polars as pl
pl.enable_string_cache()
pl.Series(["aaa", "bbb", "ccc"], dtype=pl.Categorical)  # pre-fill cache

# test_fast_unique_flag_from_arrow
df = pl.DataFrame({
    "colB": pl.Series(["1", "2", "3", "4", "5", "5", "5", "5"], dtype=pl.Categorical)
})
filtered = df.to_arrow().filter([True, False, True, True, False, True, True, True])
pl.from_arrow(filtered).select(pl.col("colB").n_unique())
# pyo3_runtime.PanicException: assertion failed: TotalOrd::tot_le(v, self.range.end())

[mcrumiller]

@ritchie46 here is a simpler repro:

import polars as pl
pl.enable_string_cache()
pl.Series(["aaa", "bbb", "ccc"], dtype=pl.Categorical)  # pre-fill cache

s = pl.Series(["1", "2", "3"], dtype=pl.Categorical)
s = s.filter([True, False, True])
s.n_unique()
# pyo3_runtime.PanicException: assertion failed: TotalOrd::tot_le(v, self.range.end())

A few notes:

• pre-filling the cache with only two items doesn't cause the error.
• if s has only two items with a True/False mask it doesn't cause error.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.00%. Comparing base (ca36b66) to head (bd4deaa).
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #20547      +/-   ##
==========================================
- Coverage   79.05%   79.00%   -0.06%     
==========================================
  Files        1564     1564              
  Lines      220627   220618       -9     
  Branches     2502     2502              
==========================================
- Hits       174413   174291     -122     
- Misses      45640    45753     +113     
  Partials      574      574              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ritchie46]

Got it

[mcrumiller]

@ritchie46 all pass now! Nice.

I will note that I don't think we have any tests that cover the Seen::Large case, which requires > 128 categories. Should I try to whip up a test for that, or deal with it later?

[github-actions]

The uncompressed binary size was Unknown MB.
The uncompressed binary size after this PR is 1119.5188 MB.

[ritchie46]

I removed the uses of that kernel, so I think that's the reason it isn't hit.