Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

str.starts_with doesn't filter small prefixes in longer strings correctly #20025

Closed
2 tasks done
harm-matthias-harms opened this issue Nov 27, 2024 · 2 comments
Closed
2 tasks done
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars

Comments

@harm-matthias-harms
Copy link

harm-matthias-harms commented Nov 27, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
pl.DataFrame({"a": ["01000000000000000000", "02000000000000000000"]}).filter(pl.col("a").str.starts_with("01"))
# shape: (0, 1)
# ┌─────┐
# │ a   │
# │ --- │
# │ str │
# ╞═════╡
# └─────

When searching in a shorter string it works:

import polars as pl
pl.DataFrame({"a": ["010000", "0200000"]}).filter(pl.col("a").str.starts_with("01"))
# shape: (1, 1)
# ┌────────┐
# │ a      │
# │ ---    │
# │ str    │
# ╞════════╡
# │ 010000 │
# └────────┘

Log output

No response

Issue description

Searching small prefixes in longer strings doesn't work as expected. Probably this relates to recent changes: #19904

Searching small prefixes in short strings works fine.

Expected behavior

It applies the filter correctly.

Installed versions

--------Version info---------
Polars:              1.15.0
Index type:          UInt32
Platform:            Linux-6.11.8-300.fc41.x86_64-x86_64-with-glibc2.40
Python:              3.12.7 (main, Oct 11 2024, 09:59:21) [GCC 14.2.1 20240912 (Red Hat 14.2.1-3)]
LTS CPU:             False

----Optional dependencies----
adbc_driver_manager  <not installed>
altair               <not installed>
boto3                <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            0.22.0
fastexcel            <not installed>
fsspec               2024.10.0
gevent               <not installed>
google.auth          2.36.0
great_tables         <not installed>
matplotlib           <not installed>
nest_asyncio         1.6.0
numpy                2.1.3
openpyxl             3.1.5
pandas               2.2.3
pyarrow              18.1.0
pydantic             2.10.2
pyiceberg            <not installed>
sqlalchemy           2.0.36
torch                2.5.1+cu124
xlsx2csv             0.8.4
xlsxwriter           <not installed>
@harm-matthias-harms harm-matthias-harms added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Nov 27, 2024
@coastalwhite
Copy link
Collaborator

This is probably already fixed with #20006.

@alexander-beedie
Copy link
Collaborator

alexander-beedie commented Nov 27, 2024

Yup; compiled the latest head to check - it's fixed:

pl.DataFrame({
  "a": [
    "01000000000000000000",
    "02000000000000000000",
]}).filter(
  pl.col("a").str.starts_with("01"),
)
# shape: (1, 1)
# ┌──────────────────────┐
# │ a                    │
# │ ---                  │
# │ str                  │
# ╞══════════════════════╡
# │ 01000000000000000000 │
# └──────────────────────┘

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

3 participants