Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gh-124370: Add "howto" for free-threaded Python #124371
base: main
Are you sure you want to change the base?
gh-124370: Add "howto" for free-threaded Python #124371
Changes from 4 commits
5b4a8aa
0252cf5
df1489e
5658fa8
ae0f637
1554fb9
3b173e0
91ec1ca
6078cfd
e02ed00
526737e
f24bdb7
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"configuration" makes me think of a configuration file I might need to create. Can we say "build of Python"? Or at least "build configuration".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We seem to switch between "free threading" and "free-threaded" kind of randomly. I can't decide if this is OK or if we should choose one and stick with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's fine. We also say both "multithreaded" and "multithreading" depending on the context.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we could add a quick snapshot of the overall plan: if everything works out, eventually free threading will be the only build, etc. Also, maybe a statement about how most programmer won't need to be concerned with this, we're doing a lot to keep everyday Python programs behaving the same, etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would perhaps add a note in the seealso block that refers to the PEP and adds 1 or 2 highlights:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could use more explanation. When might I want to set this environment variable? "an extension module that requires the GIL": is this something I have to figure out, or does Python know this itself somehow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't want to get into the details of the environment variable here. The important point here is that the free-threaded build can run with the GIL enabled -- that's not obvious to most people. The environment variable is mentioned here just as an example of that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it likely that people will write code that examines this variable and chooses between multithreading and multiprocessing? It seems unlikely to me, but maybe library authors will? When should it be used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"How do I distinguish between the free-threaded and GIL-enabled build?" is a fairly common question, although I don't think people make much use of it other than for quick debugging and sanity checks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would break this section into two sections:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like this section is probably among the most important for most users. As I understand it, the general rule is that you write thread-safe code under free-threading the same way that you'd write thread-safe code under previous versions of Python; the rules have not really changed. That could be made more explicit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that's our goal for the implementation, but I'm a bit hesitant to promise too much here, especially because it's not always clear what was thread-safe under the GIL.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've reworked this a bit. Would you please take another look at it? As you wrote, immortalization means reference counts are not modified and objects are not guaranteed not to be deallocated. I wanted to get across that:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's probably worth noting somewhere in here that these objects (and immortalization itself) are implementation details, and very much subject to change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After reading about immortalization, I don't understand the implications for me as a Python programmer. Why do I care that they are now immortal? These all sound like things that would have never been deallocated anyway. Are there unusual circumstances that I should be considering?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hopefully most people won't care, but some programs create these sorts of things in a loop.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you elaborate on how it is not safe? What might happen?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Your program may crash. I'll add it to the guide.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the possible crashing of the interpreter expected to be addressed in 3.14?
As a consequence of iterators not being thread safe, some modules are not safe either (e g.
json
,itertools
). Should this be mentioned somewhere?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is surprising: the lore around removing the GIL has always been that it was easy, the hard part was doing it without affecting single-threaded performance. This seems like a significant hit. I understand it's a work in progress, and performance will be improved, but maybe we could elaborate on that a bit here.