add test for: Windows output redirection trouble

[tkelman]

Steps to reproduce (this is in Cygwin cross-compile, I'll also check in MSYS2 to see if it's any different there):

$ git clone git://git.kitenet.net/moreutils
$ make clean && make testall1 | moreutils/ts -s

Full output is at https://gist.github.com/tkelman/4d9717ae288c637a6432 - note the jl_uv_writecb() ERROR: invalid argument EINVAL during the later bootstrap stages which seems to be non-fatal, but when running the tests it is fatal. ts is a perl script, but we have similar messages on the Windows buildbots where I think python is driving the process and logging: http://buildbot.e.ip.saba.us:8010/builders/package_win8.1-x64/builds/864/steps/make/logs/stdio

This has been happening for quite a while, I think @staticfloat and I hit this error the last time we tried to enable running the tests on the Windows buildbots so this is probably a blocker to doing that.

versioninfo:

Julia Version 0.4.0-dev+5413
Commit fecadcb* (2015-06-16 17:23 UTC)
Platform Info:
  System: Windows (i686-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz
  WORD_SIZE: 32
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Nehalem)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

---

edit: minimal reproduction:

$ (usr/bin/julia -e "println('a')" && usr/bin/julia -e "println('b')") | cat
a
bERROR: write: invalid argument (EINVAL)
 in uv_write at stream.jl:1024
...

[staticfloat]

Grrr, yes. I remember this. I'm pretty sure it has something to do with the way libuv handles cygwin pipes on windows.

[vtjnash]

is this still broken for you? i currently can't seem to reproduce it on master on my machine.

[tkelman]

Yes, at 298231e this is still happening. Cygwin64 build of win64 Julia (last month was win32 but looked the same), windows 7. I have MARCH = x86-64 in my Make.user but I'd be surprised if that made any difference.

I do recall this being picky about not always happening in an incremental build, are you running the exact command verbatim including the make clean?

[vtjnash]

This seems to be an implementation error with the implementation of UV_HANDLE_EMULATE_IOCP in libuv commit libuv/libuv@54982a2 . In particular, overlapped->hEvent was not initialized prior to ReadFile / WriteFile per https://msdn.microsoft.com/en-us/library/windows/desktop/ms686358(v=vs.85).aspx, so it is still NULL when passed to RegisterWaitForSingleObject, which thus returns ERROR_INVALID_PARAMETER.

This happens here because the NamedPipe handle is getting passed to multiple libuv (Julia) processes and apparently only one process may make an IOCompletionPort for a NamedPipe (not documented, although the restrictions given are pretty troubling -- for example, the handle should not be duplicated, shared, or passed to CreateProcess: https://msdn.microsoft.com/en-us/library/windows/desktop/aa363862(v=vs.85).aspx), forcing libuv to use the fallback emulation code. (partially related, this appear to also indirectly be caused by a resource leak in the Windows kernel where the OS is failing to destroy uv_loop.iocp after program termination, but it is not valid for destruction prior to program termination because we can't close the stdio objects that we've associated with it).

[eschnett]

I've just encountered this error when building the Julia release-0.4 branch with Cygwin. Does this mean that building with Cygwin is currently broken? If there's no immediate prospect of having this corrected, should the build instructions be updated to warn about this?

[vtjnash]

you just can't build with output going to a pipe

[ihnorton]

Additional comments from @vtjnash about this issue:

1. the code to initialize overlapped->hEvent is missing from pipe.c
2. the parallel code in tty.c (from which the pipe.c code was copied) is correct

[tkelman]

^ care to comment or explain there? this needs doing to allow running windows tests on buildbot. does @ihnorton or anyone else know how?

don't just unassign yourself from an important issue you know how to fix without any explanation.

[staticfloat]

@tkelman can you please verify that this PR fixes the problem on windows for you: https://github.com/JuliaLang/libuv/pull/44

In my testing, it seems to have worked, but I'd like to be doubly sure. :)

[tkelman]

It can start running tests, but cmdlineargs is freezing.

[staticfloat]

Are you sure it's actually cmdlineargs and not just the Julia waiting for a worker to join that never does? cmdlineargs is the last test, so if we've got some kind of join() lockup, that might be the problem. -E

…

On Thu, Jun 1, 2017 at 3:09 AM, Tony Kelman ***@***.***> wrote: It can start running tests, but cmdlineargs is freezing. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11727 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAH_aDyuOURVS-6qf_YSj3BNV5re-iVfks5r_o32gaJpZM4FEybF> .

[tkelman]

it hasn't been the last test in a while afaik

[staticfloat]

Really? Looking at the latest linux64 and osx64 testing runs on the buildbot, it's listed as the last.

EDIT: I guess I should specify; it's at least the last to print out. I suppose that's not necessarily the last to be scheduled, but in this case I don't think the distinction matters much.

[tkelman]

run in a single worker, it definitely isn't the last to start

[staticfloat]

Ah, I see, so you're saying that running the tests in a single worker freezes on cmdlineargs?

[tkelman]

It's looking like it's an unrelated problem. make testall1 finishes running enums but never finishes with the next test which is cmdlineargs. Though there was an assertion failure before that for older commits, trying to figure out what fixed that.

[tkelman]

the freeze in the cmdlineargs tests looks like it was caused by 94acb6f

[vtjnash]

upstream's problem now

[tkelman]

it will regress if not tested here. come on, what is so hard about following the standard policy we've always had of adding a test when you fix a bug?