Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pickling objects by name without __module__ is suboptimal #122459

Closed
serhiy-storchaka opened this issue Jul 30, 2024 · 1 comment
Closed

Pickling objects by name without __module__ is suboptimal #122459

serhiy-storchaka opened this issue Jul 30, 2024 · 1 comment
Labels
3.14 new features, bugs and security fixes performance Performance or resource usage

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Jul 30, 2024

When the object is pickled by name (classes, functions, and other enum-like objects) has no __module__ attribute or it is None, the code searches the object in all imported modules: tries to resolve its name relatively to the module and check that the result is the same object.

Whether the module was get from the __module__ attribute or found in sys.modules, the code checks it again:
Then it checks again: tries to resolve its name relatively to the module and check that the result is the same object.

So in some the work is repeated twice. The following PR makes it only be performed once. It also contains few other minor optimizations and simplifications.

Linked PRs

@serhiy-storchaka
Copy link
Member Author

I did not expect significant effect from this change, but it make pickling a function without __module__ with the Python implementation twice faster:

$ ./python -m timeit -s "import sys; sys.modules['_pickle'] = None; import pickle; global func" -s "def func(): pass" -s "func.__module__ = None" -- "pickle.dumps(func)"
Before: 2000 loops, best of 5: 192 usec per loop
After:  2000 loops, best of 5: 92.3 usec per loop

This affects also pickling Ellipsis and NotImplemented singletons:

$ ./python -m timeit -s "import sys; sys.modules['_pickle'] = None; import pickle" -- "pickle.dumps(...)"
Before: 20000 loops, best of 5: 14.1 usec per loop
After:  20000 loops, best of 5: 10.6 usec per loop

The main purpose is of course cleaning up the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 new features, bugs and security fixes performance Performance or resource usage
Projects
Status: Done
Development

No branches or pull requests

1 participant