Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Keep ob_tid, ob_ref_local, and ob_ref_shared fields valid across allocations in free-threaded build #115491

Closed
Tracked by #108219
colesbury opened this issue Feb 14, 2024 · 1 comment
Closed
Tracked by #108219

Keep ob_tid, ob_ref_local, and ob_ref_shared fields valid across allocations in free-threaded build #115491

colesbury opened this issue Feb 14, 2024 · 1 comment
Assignees
@colesbury
Labels
topic-free-threading type-feature A feature request or enhancement

Comments

@colesbury
Copy link
Contributor

colesbury commented Feb 14, 2024 •
edited by bedevere-app bot
Loading

Feature or enhancement

The free-threaded implementation of dict and list try avoid acquiring locks during read operations. To support this, we need to be able to access the reference count fields of Python objects after they are deallocated (and possibly reallocated). Some of this support is provided by #115103. Additionally, we need to ensure that the debug allocators do not overwrite these fields with "dead" bytes 0xDD, which might make the object look "alive" by having a non-zero reference count.

We still would like to overwrite the rest of the allocation (i.e., from ob_type onwards) to detect use-after-frees in debug builds.

See also: https://peps.python.org/pep-0703/#optimistically-avoiding-locking

Linked PRs
@colesbury colesbury added type-feature A feature request or enhancement topic-free-threading labels Feb 14, 2024
@colesbury colesbury self-assigned this Feb 15, 2024
@colesbury colesbury mentioned this issue Feb 15, 2024
Open
colesbury added a commit to colesbury/cpython that referenced this issue Feb 16, 2024
@colesbury
pythongh-115491: Keep some fields valid across allocations (free-thre…
58a2d1f 
…ading)

This avoids filling the memory occupied by ob_tid, ob_ref_local, and
ob_ref_shared with debug bytes (e.g., 0xDD) in mimalloc in the
free-threaded build.
@bedevere-app bedevere-app bot mentioned this issue Feb 16, 2024
Merged
colesbury added a commit to colesbury/cpython that referenced this issue Feb 16, 2024
@colesbury
pythongh-115491: Keep some fields valid across allocations in obmallo…
20c0c81 
…c (free-threading)
colesbury added a commit that referenced this issue Feb 20, 2024
@colesbury
gh-115491: Keep some fields valid across allocations (free-threading) (…
cc82e33 
…#115573)

This avoids filling the memory occupied by ob_tid, ob_ref_local, and
ob_ref_shared with debug bytes (e.g., 0xDD) in mimalloc in the
free-threaded build.
colesbury added a commit to colesbury/cpython that referenced this issue Feb 20, 2024
@colesbury
pythongh-115491: Keep some fields valid across allocations in obmallo…
85b3647 
…c (free-threading)
@bedevere-app bedevere-app bot mentioned this issue Feb 20, 2024
Merged
DinoV pushed a commit that referenced this issue Feb 21, 2024
@colesbury
gh-115491: Keep some fields valid across allocations in obmalloc (fre…
347acde 
…e-threading) (#115745)
@colesbury
Copy link
Contributor Author

This still needs the mimalloc support in #115103 to fully preserve the fields, but the changes to filling debug bytes is done now.

colesbury added a commit to colesbury/cpython that referenced this issue Feb 29, 2024
@colesbury
pythongh-115491: Fix compiler warning on macOS
36a959e
@bedevere-app bedevere-app bot mentioned this issue Feb 29, 2024
Merged
DinoV pushed a commit that referenced this issue Mar 1, 2024
@colesbury
gh-115491: Fix Clang compiler warning (#116153)
d7ddd90 
gh-115491: Fix compiler warning on macOS
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this issue Mar 4, 2024
@colesbury @woodruffw
pythongh-115491: Keep some fields valid across allocations (free-thre…
be2dcd0 
…ading) (python#115573)

This avoids filling the memory occupied by ob_tid, ob_ref_local, and
ob_ref_shared with debug bytes (e.g., 0xDD) in mimalloc in the
free-threaded build.
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this issue Mar 4, 2024
@colesbury @woodruffw
pythongh-115491: Keep some fields valid across allocations in obmallo…
041aa45 
…c (free-threading) (python#115745)
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this issue Mar 4, 2024
@colesbury @woodruffw
pythongh-115491: Fix Clang compiler warning (python#116153)
625d530 
pythongh-115491: Fix compiler warning on macOS
adorilson pushed a commit to adorilson/cpython that referenced this issue Mar 25, 2024
@colesbury @adorilson
pythongh-115491: Fix Clang compiler warning (python#116153)
18be625 
pythongh-115491: Fix compiler warning on macOS
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@colesbury @diegorusso
pythongh-115491: Keep some fields valid across allocations (free-thre…
de74b97 
…ading) (python#115573)

This avoids filling the memory occupied by ob_tid, ob_ref_local, and
ob_ref_shared with debug bytes (e.g., 0xDD) in mimalloc in the
free-threaded build.
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@colesbury @diegorusso
pythongh-115491: Keep some fields valid across allocations in obmallo…
6e942bd 
…c (free-threading) (python#115745)
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@colesbury @diegorusso
pythongh-115491: Fix Clang compiler warning (python#116153)
b1dd0bb 
pythongh-115491: Fix compiler warning on macOS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@colesbury colesbury

Labels
topic-free-threading type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@colesbury