IOError: [Errno 0] Error raised when executing tox with --result-json option on Windows.

[r2dan]

Tox crashes on Windows when invoked with --result-json option. To reproduce the issue tests need to output a lot of text to stdout. Exception is raised only on Windows platform. The attached package reproduces the error on my machine (~30% reproduction rate).
Minimal package to reproduce the issue: tox_issue_result_log.tar.gz
Stack traceback: tox_result.log

The most convenient way to reproduce the error is to run tox in a loop:
for /l %x in (1, 1, 100) do tox --result-json result.json

Please note that pytest is invoked with --verbose flag to increase the amount of text written to stdout.

Platform: Windows 10/Windows 7 (reproduced on 3 machines)

C:\tox_issue_resultlog>pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
apipkg (1.4)
attrs (17.4.0)
backports.shutil-get-terminal-size (1.0.0)
colorama (0.3.9)
decorator (4.0.11)
enum34 (1.1.6)
execnet (1.5.0)
fancycompleter (0.6.3)
funcsigs (1.0.2)
ipython (5.3.0)
ipython-genutils (0.2.0)
ordereddict (1.1)
pathlib2 (2.2.1)
pdbpp (0.8.3)
pickleshare (0.7.4)
pip (9.0.1)
pluggy (0.6.0)
prompt-toolkit (1.0.14)
py (1.5.2)
pycrypto (2.6.1)
Pygments (2.1.3)
pyreadline (2.1)
pytest (3.3.2)
pytest-forked (0.2)
pytest-xdist (1.21.0)
scandir (1.5)
setuptools (38.3.0)
simplegeneric (0.8.1)
six (1.11.0)
tox (2.9.1)
traitlets (4.3.2)
virtualenv (15.1.0)
wcwidth (0.1.7)
wheel (0.29.0)
win-unicode-console (0.5)
wmctrl (0.3)

[asottile]

Related, this function is broken pretty hard in python3:

$ tox --result-json out.json -vvvv -e py35
  removing /home/asottile/workspace/pyupgrade/.tox/log
using tox.ini: /home/asottile/workspace/pyupgrade/tox.ini
using tox-2.9.1 from /home/asottile/opt/venv/lib/python3.6/site-packages/tox/__init__.py
GLOB start: packaging 
GLOB sdist-make: /home/asottile/workspace/pyupgrade/setup.py
  removing /home/asottile/workspace/pyupgrade/.tox/dist
  /home/asottile/workspace/pyupgrade$ /home/asottile/opt/venv/bin/python3.6 /home/asottile/workspace/pyupgrade/setup.py sdist --formats=zip --dist-dir /home/asottile/workspace/pyupgrade/.tox/dist >/home/asottile/workspace/pyupgrade/.tox/log/tox-0.log
GLOB finish: packaging after 0.33 seconds
copying new sdistfile to '/home/asottile/.tox/distshare/pyupgrade-1.2.0.zip'
py35 start: getenv /home/asottile/workspace/pyupgrade/.tox/py35
py35 reusing: /home/asottile/workspace/pyupgrade/.tox/py35
py35 finish: getenv after 0.08 seconds
py35 start: installpkg /home/asottile/workspace/pyupgrade/.tox/dist/pyupgrade-1.2.0.zip
py35 inst-nodeps: /home/asottile/workspace/pyupgrade/.tox/dist/pyupgrade-1.2.0.zip
setting PATH=/home/asottile/workspace/pyupgrade/.tox/py35/bin:/home/asottile/workspace/watch-plz/venv/bin:/home/asottile/.gvm/bin:/home/asottile/bin:/home/asottile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  /home/asottile/workspace/pyupgrade$ /home/asottile/workspace/pyupgrade/.tox/py35/bin/pip install -U --no-deps /home/asottile/workspace/pyupgrade/.tox/dist/pyupgrade-1.2.0.zip >/home/asottile/workspace/pyupgrade/.tox/py35/log/py35-9.log
py35 finish: installpkg after 0.41 seconds
Traceback (most recent call last):
  File "/home/asottile/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/session.py", line 40, in main
    retcode = Session(config).runcommand()
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/session.py", line 392, in runcommand
    return self.subcommand_test()
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/session.py", line 567, in subcommand_test
    self.installpkg(venv, path)
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/session.py", line 507, in installpkg
    venv.installpkg(path, action)
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/venv.py", line 273, in installpkg
    self._install([sdistpath], extraopts=extraopts, action=action)
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/venv.py", line 335, in _install
    action=action)
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/venv.py", line 308, in run_install_command
    redirect=self.session.report.verbosity < 2)
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/venv.py", line 414, in _pcall
    redirect=redirect, ignore_ret=ignore_ret)
  File "/home/asottile/opt/venv/lib/python3.6/site-packages/tox/session.py", line 163, in popen
    sys.stdout.write(data)
  File "/home/asottile/opt/venv/lib/python3.6/codecs.py", line 377, in write
    self.stream.write(data)
TypeError: write() argument must be str, not bytes

[fschulze]

@asottile that's tracked in #426. If you have a way to reproduce that, please report it there.

[fschulze]

@r2dan I wasn't able to reproduce this yet, but I have a suspicion what could cause this.

Could you give some more details?

• From the for loop I assume you use cmd.exe not PowerShell.
• Which Python version do you use?
• Can you run chcp and report which code page is active (for me it's the default 850).

[fschulze]

@r2dan can you try with this patch?

diff --git a/tox/session.py b/tox/session.py
index 35cb4dc..0c8482d 100644
--- a/tox/session.py
+++ b/tox/session.py
@@ -127,7 +127,7 @@ class Action(object):
             fout.write("actionid: %s\nmsg: %s\ncmdargs: %r\n\n" % (self.id, self.msg, args))
             fout.flush()
             self.popen_outpath = outpath = py.path.local(fout.name)
-            fin = outpath.open()
+            fin = outpath.open('rb')
             fin.read()  # read the header, so it won't be written to stdout
             stdout = fout
         elif returnout:
@@ -155,7 +155,6 @@ class Action(object):
                     out = None
                     last_time = time.time()
                     while 1:
-                        fin_pos = fin.tell()
                         # we have to read one byte at a time, otherwise there
                         # might be no output for a long time with slow tests
                         data = fin.read(1)
@@ -173,7 +172,8 @@ class Action(object):
                             break
                         else:
                             time.sleep(0.1)
-                            fin.seek(fin_pos)
+                            # the seek updates internal read buffers
+                            fin.seek(0, 1)
                     fin.close()
                 else:
                     out, err = popen.communicate()

It's against the tox 2.9.1 tag, so it should apply to the release you use.

[r2dan]

@fschulze
I was using cmd.exe, Python 2.7.12.

C:\tox_issue_resultlog>chcp
Active code page: 437

I've tested the patch and it seems to fix the issue. Thanks!

[asottile]

via #734