Skip to content

Commit

Permalink
pythongh-95027: Fix regrtest stdout encoding on Windows (python#98492)
Browse files Browse the repository at this point in the history
On Windows, when the Python test suite is run with the -jN option,
the ANSI code page is now used as the encoding for the stdout
temporary file, rather than using UTF-8 which can lead to decoding
errors.

(cherry picked from commit ec1f6f5)
  • Loading branch information
vstinner committed Sep 2, 2023
1 parent 693a035 commit 73334eb
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 3 deletions.
14 changes: 11 additions & 3 deletions Lib/test/libregrtest/runtest_mp.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
from test.libregrtest.setup import setup_tests
from test.libregrtest.utils import format_duration, print_warning

if sys.platform == 'win32':
import locale


# Display the running tests if nothing happened last N seconds
PROGRESS_UPDATE = 30.0 # seconds
Expand Down Expand Up @@ -267,11 +270,16 @@ def _run_process(self, test_name: str, tmp_dir: str, stdout_fh: TextIO) -> int:
self.current_test_name = None

def _runtest(self, test_name: str) -> MultiprocessResult:
if sys.platform == 'win32':
# gh-95027: When stdout is not a TTY, Python uses the ANSI code
# page for the sys.stdout encoding. If the main process runs in a
# terminal, sys.stdout uses WindowsConsoleIO with UTF-8 encoding.
encoding = locale.getencoding()
else:
encoding = sys.stdout.encoding
# gh-94026: Write stdout+stderr to a tempfile as workaround for
# non-blocking pipes on Emscripten with NodeJS.
with tempfile.TemporaryFile(
'w+', encoding=sys.stdout.encoding
) as stdout_fh:
with tempfile.TemporaryFile('w+', encoding=encoding) as stdout_fh:
# gh-93353: Check for leaked temporary files in the parent process,
# since the deletion of temporary files can happen late during
# Python finalization: too late for libregrtest.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
On Windows, when the Python test suite is run with the ``-jN`` option, the
ANSI code page is now used as the encoding for the stdout temporary file,
rather than using UTF-8 which can lead to decoding errors. Patch by Victor
Stinner.

0 comments on commit 73334eb

Please sign in to comment.