Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(python): BytecodeParser can now handle mixed/nested and/or control flow #10085

Merged

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented Jul 26, 2023

Ref: #9968.

This was a fun one... enables two additional ops (POP_JUMP_FORWARD_IF_FALSE and POP_JUMP_FORWARD_IF_TRUE if Python >= 3.11, and POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE for earlier versions) in order to support mixed and/or chains (previously only supported all and or all or) with, crucially, nesting resolution being determined from jump offsets.

Example

from polars.utils.udfs import BytecodeParser

fn = lambda x: x > 1 and x != 2 or x % 2 == 0 and x < 3
BytecodeParser( fn,"expr" ).to_expression( col="x" )
# (pl.col("x") > 1) & (pl.col("x") != 2) | ((pl.col("x") % 2) == 0) & (pl.col("x") < 3)

Notice that we maintain the parenthesised inner or block:

fn = lambda x: x > 1 and (x != 2 or x % 2 == 0) and x < 3
BytecodeParser( fn,"expr" ).to_expression( col="x" )
# (pl.col("x") > 1) & ((pl.col("x") != 2) | ((pl.col("x") % 2) == 0)) & (pl.col("x") < 3)

Also:

  • Tidied-up the loose opname-related sets/dicts/etc; all now bundled within a single OpNames class.

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars labels Jul 26, 2023
@alexander-beedie alexander-beedie force-pushed the bytecode-nested-and-or branch from 702ab1e to 482d486 Compare July 26, 2023 07:53
@alexander-beedie alexander-beedie marked this pull request as draft July 26, 2023 07:55
@alexander-beedie
Copy link
Collaborator Author

alexander-beedie commented Jul 26, 2023

Working nicely in 3.11 now; will take care of earlier versions after I come back from watching Oppenheimer :)

@MarcoGorelli MarcoGorelli self-requested a review July 26, 2023 08:14
@ritchie46
Copy link
Member

I just happened to stumble on this SO question. 😄

https://stackoverflow.com/questions/76767485/how-to-achieve-pl-colcode-applylambda-x-x88-if-lenx-2-else-x-with

@alexander-beedie alexander-beedie marked this pull request as ready for review July 26, 2023 15:04
@alexander-beedie alexander-beedie force-pushed the bytecode-nested-and-or branch from 3b17986 to 44f5cd3 Compare July 26, 2023 15:08
py-polars/polars/utils/udfs.py Show resolved Hide resolved
py-polars/polars/utils/udfs.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@MarcoGorelli MarcoGorelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just got a minor comment, but this looks really good!

py-polars/polars/utils/udfs.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants