Skip to content

Commit

Permalink
pythonGH-120804: add docs for removal for asyncio child watchers (pyt…
Browse files Browse the repository at this point in the history
…hon#120895)

Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
  • Loading branch information
2 people authored and mrahtz committed Jun 30, 2024
1 parent 4d8d2dc commit ced81fa
Showing 7 changed files with 31 additions and 214 deletions.
5 changes: 0 additions & 5 deletions Doc/library/asyncio-platforms.rst
Original file line number Diff line number Diff line change
@@ -77,11 +77,6 @@ Subprocess Support on Windows
On Windows, the default event loop :class:`ProactorEventLoop` supports
subprocesses, whereas :class:`SelectorEventLoop` does not.

The :meth:`policy.set_child_watcher()
<AbstractEventLoopPolicy.set_child_watcher>` function is also
not supported, as :class:`ProactorEventLoop` has a different mechanism
to watch child processes.


macOS
=====
185 changes: 0 additions & 185 deletions Doc/library/asyncio-policy.rst
Original file line number Diff line number Diff line change
@@ -79,25 +79,6 @@ The abstract event loop policy base class is defined as follows:

This method should never return ``None``.

.. method:: get_child_watcher()

Get a child process watcher object.

Return a watcher object implementing the
:class:`AbstractChildWatcher` interface.

This function is Unix specific.

.. deprecated:: 3.12

.. method:: set_child_watcher(watcher)

Set the current child process watcher to *watcher*.

This function is Unix specific.

.. deprecated:: 3.12


.. _asyncio-policy-builtin:

@@ -139,172 +120,6 @@ asyncio ships with the following built-in policies:
.. availability:: Windows.


.. _asyncio-watchers:

Process Watchers
================

A process watcher allows customization of how an event loop monitors
child processes on Unix. Specifically, the event loop needs to know
when a child process has exited.

In asyncio, child processes are created with
:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec`
functions.

asyncio defines the :class:`AbstractChildWatcher` abstract base class, which child
watchers should implement, and has four different implementations:
:class:`ThreadedChildWatcher` (configured to be used by default),
:class:`MultiLoopChildWatcher`, :class:`SafeChildWatcher`, and
:class:`FastChildWatcher`.

See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>`
section.

The following two functions can be used to customize the child process watcher
implementation used by the asyncio event loop:

.. function:: get_child_watcher()

Return the current child watcher for the current policy.

.. deprecated:: 3.12

.. function:: set_child_watcher(watcher)

Set the current child watcher to *watcher* for the current
policy. *watcher* must implement methods defined in the
:class:`AbstractChildWatcher` base class.

.. deprecated:: 3.12

.. note::
Third-party event loops implementations might not support
custom child watchers. For such event loops, using
:func:`set_child_watcher` might be prohibited or have no effect.

.. class:: AbstractChildWatcher

.. method:: add_child_handler(pid, callback, *args)

Register a new child handler.

Arrange for ``callback(pid, returncode, *args)`` to be called
when a process with PID equal to *pid* terminates. Specifying
another callback for the same process replaces the previous
handler.

The *callback* callable must be thread-safe.

.. method:: remove_child_handler(pid)

Removes the handler for process with PID equal to *pid*.

The function returns ``True`` if the handler was successfully
removed, ``False`` if there was nothing to remove.

.. method:: attach_loop(loop)

Attach the watcher to an event loop.

If the watcher was previously attached to an event loop, then
it is first detached before attaching to the new loop.

Note: loop may be ``None``.

.. method:: is_active()

Return ``True`` if the watcher is ready to use.

Spawning a subprocess with *inactive* current child watcher raises
:exc:`RuntimeError`.

.. versionadded:: 3.8

.. method:: close()

Close the watcher.

This method has to be called to ensure that underlying
resources are cleaned-up.

.. deprecated:: 3.12


.. class:: ThreadedChildWatcher

This implementation starts a new waiting thread for every subprocess spawn.

It works reliably even when the asyncio event loop is run in a non-main OS thread.

There is no noticeable overhead when handling a big number of children (*O*\ (1) each
time a child terminates), but starting a thread per process requires extra memory.

This watcher is used by default.

.. versionadded:: 3.8

.. class:: MultiLoopChildWatcher

This implementation registers a :py:data:`SIGCHLD` signal handler on
instantiation. That can break third-party code that installs a custom handler for
:py:data:`SIGCHLD` signal.

The watcher avoids disrupting other code spawning processes
by polling every process explicitly on a :py:data:`SIGCHLD` signal.

There is no limitation for running subprocesses from different threads once the
watcher is installed.

The solution is safe but it has a significant overhead when
handling a big number of processes (*O*\ (*n*) each time a
:py:data:`SIGCHLD` is received).

.. versionadded:: 3.8

.. deprecated:: 3.12

.. class:: SafeChildWatcher

This implementation uses active event loop from the main thread to handle
:py:data:`SIGCHLD` signal. If the main thread has no running event loop another
thread cannot spawn a subprocess (:exc:`RuntimeError` is raised).

The watcher avoids disrupting other code spawning processes
by polling every process explicitly on a :py:data:`SIGCHLD` signal.

This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O*\ (*n*)
complexity but requires a running event loop in the main thread to work.

.. deprecated:: 3.12

.. class:: FastChildWatcher

This implementation reaps every terminated processes by calling
``os.waitpid(-1)`` directly, possibly breaking other code spawning
processes and waiting for their termination.

There is no noticeable overhead when handling a big number of
children (*O*\ (1) each time a child terminates).

This solution requires a running event loop in the main thread to work, as
:class:`SafeChildWatcher`.

.. deprecated:: 3.12

.. class:: PidfdChildWatcher

This implementation polls process file descriptors (pidfds) to await child
process termination. In some respects, :class:`PidfdChildWatcher` is a
"Goldilocks" child watcher implementation. It doesn't require signals or
threads, doesn't interfere with any processes launched outside the event
loop, and scales linearly with the number of subprocesses launched by the
event loop. The main disadvantage is that pidfds are specific to Linux, and
only work on recent (5.3+) kernels.

.. versionadded:: 3.9


.. _asyncio-custom-policies:

Custom Policies
12 changes: 0 additions & 12 deletions Doc/library/asyncio-subprocess.rst
Original file line number Diff line number Diff line change
@@ -316,18 +316,6 @@ default.
On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default),
:class:`SelectorEventLoop` has no subprocess support.

On UNIX *child watchers* are used for subprocess finish waiting, see
:ref:`asyncio-watchers` for more info.


.. versionchanged:: 3.8

UNIX switched to use :class:`ThreadedChildWatcher` for spawning subprocesses from
different threads without any limitation.

Spawning a subprocess with *inactive* current child watcher raises
:exc:`RuntimeError`.

Note that alternative event loop implementations might have own limitations;
please refer to their documentation.

1 change: 0 additions & 1 deletion Doc/tools/.nitignore
Original file line number Diff line number Diff line change
@@ -17,7 +17,6 @@ Doc/howto/descriptor.rst
Doc/howto/enum.rst
Doc/library/ast.rst
Doc/library/asyncio-extending.rst
Doc/library/asyncio-policy.rst
Doc/library/asyncio-subprocess.rst
Doc/library/collections.rst
Doc/library/dbm.rst
20 changes: 10 additions & 10 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
@@ -652,14 +652,14 @@ asyncio
making some use-cases 2x to 5x faster.
(Contributed by Jacob Bower & Itamar Oren in :gh:`102853`, :gh:`104140`, and :gh:`104138`)

* On Linux, :mod:`asyncio` uses :class:`asyncio.PidfdChildWatcher` by default
* On Linux, :mod:`asyncio` uses :class:`!asyncio.PidfdChildWatcher` by default
if :func:`os.pidfd_open` is available and functional instead of
:class:`asyncio.ThreadedChildWatcher`.
:class:`!asyncio.ThreadedChildWatcher`.
(Contributed by Kumar Aditya in :gh:`98024`.)

* The event loop now uses the best available child watcher for each platform
(:class:`asyncio.PidfdChildWatcher` if supported and
:class:`asyncio.ThreadedChildWatcher` otherwise), so manually
(:class:`!asyncio.PidfdChildWatcher` if supported and
:class:`!asyncio.ThreadedChildWatcher` otherwise), so manually
configuring a child watcher is not recommended.
(Contributed by Kumar Aditya in :gh:`94597`.)

@@ -1162,15 +1162,15 @@ Deprecated

* :mod:`asyncio`:

* The child watcher classes :class:`asyncio.MultiLoopChildWatcher`,
:class:`asyncio.FastChildWatcher`, :class:`asyncio.AbstractChildWatcher`
and :class:`asyncio.SafeChildWatcher` are deprecated and
* The child watcher classes :class:`!asyncio.MultiLoopChildWatcher`,
:class:`!asyncio.FastChildWatcher`, :class:`!asyncio.AbstractChildWatcher`
and :class:`!asyncio.SafeChildWatcher` are deprecated and
will be removed in Python 3.14.
(Contributed by Kumar Aditya in :gh:`94597`.)

* :func:`asyncio.set_child_watcher`, :func:`asyncio.get_child_watcher`,
:meth:`asyncio.AbstractEventLoopPolicy.set_child_watcher` and
:meth:`asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated
* :func:`!asyncio.set_child_watcher`, :func:`!asyncio.get_child_watcher`,
:meth:`!asyncio.AbstractEventLoopPolicy.set_child_watcher` and
:meth:`!asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated
and will be removed in Python 3.14.
(Contributed by Kumar Aditya in :gh:`94597`.)

20 changes: 20 additions & 0 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
@@ -173,6 +173,26 @@ ast

(Contributed by Alex Waygood in :gh:`119562`.)

asyncio
-------

* Remove the following classes and functions. They were all deprecated and
emitted deprecation warnings since Python 3.12:

* :class:`!asyncio.AbstractChildWatcher`
* :class:`!asyncio.SafeChildWatcher`
* :class:`!asyncio.MultiLoopChildWatcher`
* :class:`!asyncio.FastChildWatcher`
* :class:`!asyncio.ThreadedChildWatcher`
* :class:`!asyncio.PidfdChildWatcher`
* :meth:`!asyncio.AbstractEventLoopPolicy.get_child_watcher`
* :meth:`!asyncio.AbstractEventLoopPolicy.set_child_watcher`
* :func:`!asyncio.get_child_watcher`
* :func:`!asyncio.set_child_watcher`

(Contributed by Kumar Aditya in :gh:`120804`.)


collections.abc
---------------

2 changes: 1 addition & 1 deletion Doc/whatsnew/3.9.rst
Original file line number Diff line number Diff line change
@@ -352,7 +352,7 @@ that schedules a shutdown for the default executor that waits on the
:func:`asyncio.run` has been updated to use the new :term:`coroutine`.
(Contributed by Kyle Stanley in :issue:`34037`.)

Added :class:`asyncio.PidfdChildWatcher`, a Linux-specific child watcher
Added :class:`!asyncio.PidfdChildWatcher`, a Linux-specific child watcher
implementation that polls process file descriptors. (:issue:`38692`)

Added a new :term:`coroutine` :func:`asyncio.to_thread`. It is mainly used for

0 comments on commit ced81fa

Please sign in to comment.