Subprocess exception on terminate

Classification

Affected Components : subprocess
Operating System : Linux
Python Versions : 2.6.x, 2.7.x, 3.1.x
Reproducible : Yes

Source code

import time
from subprocess import Popen

p = Popen(['/bin/sleep', '1'])

time.sleep(1)

print(repr(p.poll()))

p.terminate()

Steps to Produce/Reproduce

To reproduce the problem copy the source code in a file and execute the script using the following command syntax:

$ python -OOBRtt test.py

Alternatively you can open python in interactive mode:

$ python -OOBRtt <press enter>

Then copy the lines of code into the interpreter.

Description

The code execute a process

p = Popen(['/bin/sleep', '1'])

Waits one second:

p = Popen(['/bin/sleep', '1'])

Checks the status of the process:

print(repr(p.poll()))

Then a command to terminate the process is issued:

p.terminate()

The check of the status of the process would return:

None -> process not completed.
0 -> process completed successfully.
1 -> error.

The test should always return None at the check to indicate that the process is still running, instead in some cases we receive a Traceback indicating an unexpected error.

This happens when the process check returns 0 indicating that the process has completed successfully before we execute the terminate command.

This can be seen in the following examples covering versions of python used in production systems.

TEST WITH PYTHON 2.6

# PYTHON 2.6
# First case: process still running then  killed
python -OOBRttu test.py 
None

# Second case: process exited gracefully
# therefore there is no process to terminate
# and terminate raises an error on exit
python -OOBRttu test.py 
0
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    p.terminate()
  File "/usr/lib/python2.6/subprocess.py", line 1257, in terminate
    self.send_signal(signal.SIGTERM)
  File "/usr/lib/python2.6/subprocess.py", line 1252, in send_signal
    os.kill(self.pid, sig)
OSError: [Errno 3] No such process

TEST WITH PYTHON 2.7

# PYTHON 2.7
# First case: process still running then  killed
python2.7 -OOBRttu test.py 
None

# Second case: process exited gracefully
# therefore there is no process to terminate
# and terminate raises an error on exit
python2.7 -OOBRttu test.py 
0
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    p.terminate()
  File "/usr/lib/python2.7/subprocess.py", line 1532, in terminate
    self.send_signal(signal.SIGTERM)
  File "/usr/lib/python2.7/subprocess.py", line 1527, in send_signal
    os.kill(self.pid, sig)
OSError: [Errno 3] No such process

TEST WITH PYTHON 3.1

PYTHON 3.1
# Only one case: process exited gracefully
# therefore there is no process to terminate
# and terminate raises an error on exit
python3 -OOBRttu test.py 
0
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    p.terminate()
  File "/usr/lib/python3.1/subprocess.py", line 1344, in terminate
    self.send_signal(signal.SIGTERM)
  File "/usr/lib/python3.1/subprocess.py", line 1339, in send_signal
    os.kill(self.pid, sig)
OSError: [Errno 3] No such process

The problem can be seen in around 60% of the times we tested it using python 2.x but was confirmed 100% of the times when python 3.1 was used.

Raising an exception on terminate is a python bug.

The proper behavior would be for the terminate to return a value indicating if the process had already been terminated and not throw an exception in that case.

Workaround

Proper handle of the issue would require implementing a method to detect a possible race conditions between process death and call to terminate() and interception of the exception on terminate.