async process closing after event loop closed

[kcwu]

BPO	41320
Nosy	@asvetlov, @1st1, @achimnol
Files	cancel_proc.py: code to reproduce this issue

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-07-17.07:38:23.387>
labels = ['type-bug', '3.8', 'expert-asyncio']
title = 'async process closing after event loop closed'
updated_at = <Date 2020-07-19.08:50:01.422>
user = 'https://bugs.python.org/kcwu'

bugs.python.org fields:

activity = <Date 2020-07-19.08:50:01.422>
actor = 'achimnol'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2020-07-17.07:38:23.387>
creator = 'kcwu'
dependencies = []
files = ['49321']
hgrepos = []
issue_num = 41320
keywords = []
message_count = 1.0
messages = ['373799']
nosy_count = 4.0
nosy_names = ['kcwu', 'asvetlov', 'yselivanov', 'achimnol']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41320'
versions = ['Python 3.8']

[kcwu]

(following code is attached as well)
import asyncio
import time

workaround = False

async def slow_proc():
  proc = await asyncio.create_subprocess_exec('sleep', '10', stdout=asyncio.subprocess.PIPE)
  try:
    return await proc.stdout.read()
  except asyncio.CancelledError:
    if workaround:
      proc.terminate()
      time.sleep(0.1)  # hope the machine is not too busy

async def func():
  try:
    return await asyncio.wait_for(slow_proc(), timeout=0.1)
  except asyncio.TimeoutError:
    return 'timeout'

def main():
  while True:
    print('test')
    asyncio.run(func())

main()

---

Run above code, it may work without error message (expected behavior).

However, depends on timing, it may show warning/error messages
case 1.
Loop <_UnixSelectorEventLoop running=False closed=True debug=False> that handles pid 1257652 is closed

case 2.
Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7fcbcfc66160>
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/base_subprocess.py", line 126, in __del__
    self.close()
  File "/usr/lib/python3.8/asyncio/base_subprocess.py", line 104, in close
    proto.pipe.close()
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 536, in close
    self._close(None)
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 560, in _close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
    self._check_closed()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

---

Although running tasks will be cancelled when asyncio.run is finishing, subprocess' exit handler may be invoked after the event loop is closed.

In above code, I provided a workaround. However it just mitigates the problem and not really fix the root cause.

This is related to https://bugs.python.org/issue35539

p.s. My test environment is Debian 5.5.17

[kumaraditya303]

Duplicate of #88050