Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-124370: Add "howto" for free-threaded Python #124371

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
119 changes: 119 additions & 0 deletions Doc/howto/free-threading-python.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
.. _freethreading-python-howto:

*********************************
Python Support for Free Threading
colesbury marked this conversation as resolved.
Show resolved Hide resolved
*********************************

Starting with the 3.13 release, CPython has experimental support for running
with the :term:`global interpreter lock` (GIL) disabled in a configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"configuration" makes me think of a configuration file I might need to create. Can we say "build of Python"? Or at least "build configuration".

called :term:`free threading`. This document describes the implications of
free threading for Python code. See :ref:`freethreading-extensions-howto` for
information on how to write C extensions that support the free-threaded build.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We seem to switch between "free threading" and "free-threaded" kind of randomly. I can't decide if this is OK or if we should choose one and stick with it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine. We also say both "multithreaded" and "multithreading" depending on the context.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could add a quick snapshot of the overall plan: if everything works out, eventually free threading will be the only build, etc. Also, maybe a statement about how most programmer won't need to be concerned with this, we're doing a lot to keep everyday Python programs behaving the same, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would perhaps add a note in the seealso block that refers to the PEP and adds 1 or 2 highlights:

  • performance conscious users may wish to try the free threaded version
  • most users can safely continue to use the default CPython version
  • when free threading is no longer experimental, the versions will converge into one version for all.


Installation
============

Starting with Python 3.13.0b2, the offical macOS and Windows installers
colesbury marked this conversation as resolved.
Show resolved Hide resolved
optionally support installing free-threaded Python binaries. The installers
are available at https://www.python.org/downloads/.

.. seealso::

`Installing a Free-Threaded Python
<https://py-free-threading.github.io/installing_cpython/>`_:
A community-maintained installation guide for installing free-threaded
Python.


Identifying Free-Threaded Python
colesbury marked this conversation as resolved.
Show resolved Hide resolved
================================

The free-threaded build of CPython can optionally run with the global
interpreter lock enabled, such as when :envvar:`PYTHON_GIL` is set to ``1``,
or when importing an extension module that requires the GIL.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could use more explanation. When might I want to set this environment variable? "an extension module that requires the GIL": is this something I have to figure out, or does Python know this itself somehow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to get into the details of the environment variable here. The important point here is that the free-threaded build can run with the GIL enabled -- that's not obvious to most people. The environment variable is mentioned here just as an example of that.


The :func:`sys._is_gil_enabled` function will return ``False`` if the global
interpreter lock is currently disabled. This is the recommended mechanism for
decisions like whether to use multithreading or multiprocessing.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it likely that people will write code that examines this variable and chooses between multithreading and multiprocessing? It seems unlikely to me, but maybe library authors will? When should it be used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"How do I distinguish between the free-threaded and GIL-enabled build?" is a fairly common question, although I don't think people make much use of it other than for quick debugging and sanity checks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would break this section into two sections:

  • Identifying free threading builds (with the last paragraph)
  • Reenabling the GIL with a free threading build (first two paragraphs)


The ``sysconfig.get_config_var("Py_GIL_DISABLED")`` configuration variable can
be used to determine whether the build supports free threading. If the variable
is set to ``1``, then the build supports free threading. This is the recommended
mechanism for decisions related to the build configuration.


Thread Safety
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this section is probably among the most important for most users. As I understand it, the general rule is that you write thread-safe code under free-threading the same way that you'd write thread-safe code under previous versions of Python; the rules have not really changed. That could be made more explicit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's our goal for the implementation, but I'm a bit hesitant to promise too much here, especially because it's not always clear what was thread-safe under the GIL.

colesbury marked this conversation as resolved.
Show resolved Hide resolved
=============

The free-threaded build of CPython aims to provide similar thread-safety
behavior at the Python level to the GIL-enabled build. Built-in
colesbury marked this conversation as resolved.
Show resolved Hide resolved
types like :class:`dict`, :class:`list`, and :class:`set` use internal locks
to protect against concurrent modifications in ways that behave similarly to
the GIL. However, Python has not historically guaranteed specific behavior for
concurrent modifications to these built-in types, so this should be treated
as a description of the current implementation, not a guarantee of future
colesbury marked this conversation as resolved.
Show resolved Hide resolved
behavior.

.. note::

It's recommended to use the :class:`threading.Lock` or other synchronization
willingc marked this conversation as resolved.
Show resolved Hide resolved
primitives instead of relying on the internal locks of built-in types, when
possible.



Known Limitations
colesbury marked this conversation as resolved.
Show resolved Hide resolved
=================

This section describes known limitations of the free-threaded CPython build.

Immortalization
---------------

The free-threaded build of the 3.13 release makes some objects :term:`immortal`
in order to avoid reference count contention that would prevent efficient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
in order to avoid reference count contention that would prevent efficient
which have reference counts that are never modified. Since the immortal object is guaranteed not to be
deallocated, efficient multi-threading scaling is possible by avoiding reference count contention between
threads.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reworked this a bit. Would you please take another look at it? As you wrote, immortalization means reference counts are not modified and objects are not guaranteed not to be deallocated. I wanted to get across that:

  • Reference counts not being modified enables efficient multi-threaded scaling.
  • Objects not being deallocated is a potential problem because some applications may leak memory. That's being addressed in 3.14. We don't expect this to be a problem for most applications, but it's something to watch out for if you run your application with refleak checks.

multi-threaded scaling. This means that these objects are never deallocated.
This expected to be addressed in the upcoming 3.14 release with
colesbury marked this conversation as resolved.
Show resolved Hide resolved
`deferred reference counting <https://peps.python.org/pep-0703/#deferred-reference-counting>`_.
ZeroIntensity marked this conversation as resolved.
Show resolved Hide resolved

The objects that are immortalized are:
colesbury marked this conversation as resolved.
Show resolved Hide resolved

* :ref:`function <user-defined-funcs>` objects declared at the module level
* :ref:`method <instance-methods>` descriptors
* :ref:`code <code-objects>` objects
* :term:`module` objects and their dictionaries
* :ref:`classes <classes>` (type objects)
Comment on lines +103 to +107
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably worth noting somewhere in here that these objects (and immortalization itself) are implementation details, and very much subject to change.


The immortalization of these objects happens the first time a thread is started
after the main thread.
ZeroIntensity marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading about immortalization, I don't understand the implications for me as a Python programmer. Why do I care that they are now immortal? These all sound like things that would have never been deallocated anyway. Are there unusual circumstances that I should be considering?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully most people won't care, but some programs create these sorts of things in a loop.


Additionally, numeric and string literals in the code as well as strings
returned by :func:`sys.intern` are also immortalized. This behavior is
expected to remain in the 3.14 free-threaded build.


Frame Objects
colesbury marked this conversation as resolved.
Show resolved Hide resolved
-------------

It is not safe to access :ref:`frame <frame-objects>` objects from other
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on how it is not safe? What might happen?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your program may crash. I'll add it to the guide.

threads. This means that :func:`sys._current_frames` is generally not safe to
ZeroIntensity marked this conversation as resolved.
Show resolved Hide resolved
use in a free-threaded build.

Iterators
---------

Sharing the same iterator object between multiple threads is generally not
safe and threads may see duplicate or missing elements when iterating or crash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the possible crashing of the interpreter expected to be addressed in 3.14?

As a consequence of iterators not being thread safe, some modules are not safe either (e g. json, itertools). Should this be mentioned somewhere?

the interpreter.


Single-Threaded Performance
colesbury marked this conversation as resolved.
Show resolved Hide resolved
---------------------------

The free-threaded build has additional overhead when executing Python code
compared to the default GIL-enabled build. In 3.13, this overhead is about
40% on the `pyperformance <https://pyperformance.readthedocs.io/>`_ suite.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is surprising: the lore around removing the GIL has always been that it was easy, the hard part was doing it without affecting single-threaded performance. This seems like a significant hit. I understand it's a work in progress, and performance will be improved, but maybe we could elaborate on that a bit here.

Programs that spend most of the their time in C extensions or I/O will see
colesbury marked this conversation as resolved.
Show resolved Hide resolved
less of an impact. This overhead is expected to be reduced in the upcoming
3.14 release.
colesbury marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions Doc/howto/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Python Library Reference.
isolating-extensions.rst
timerfd.rst
mro.rst
free-threading-python.rst
free-threading-extensions.rst

General:
Expand All @@ -52,6 +53,7 @@ General:
Advanced development:

* :ref:`curses-howto`
* :ref:`freethreading-python-howto`
* :ref:`freethreading-extensions-howto`
* :ref:`isolating-extensions-howto`
* :ref:`python_2.3_mro`
Expand Down
1 change: 1 addition & 0 deletions Doc/reference/datamodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -846,6 +846,7 @@ this case, the special read-only attribute :attr:`!__self__` is set to the objec
denoted by *alist*. (The attribute has the same semantics as it does with
:attr:`other instance methods <method.__self__>`.)

.. _classes:

Classes
^^^^^^^
Expand Down
Loading