v 0.10.13 do not work

[mond03cn]

After a day running,one process do not work,and console the following:

#0  0x00007f06ed544ccd in write () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00000000006db28a in uv__write (stream=stream@entry=0x268f170) at ../deps/uv/src/unix/stream.c:785
#2  0x00000000006dcbda in uv_write2 (req=0x26a0fa0, stream=0x268f170, bufs=0x7fff0e93d950, bufcnt=<optimized out>, 
    send_handle=<optimized out>, cb=<optimized out>) at ../deps/uv/src/unix/stream.c:1241
#3  0x00000000005af5ec in v8::Handle<v8::Value> node::StreamWrap::WriteStringImpl<(node::encoding)1>(v8::Arguments const&) ()
#4  0x00000000005aede9 in node::StreamWrap::WriteUtf8String(v8::Arguments const&) ()
#5  0x000000000070df11 in v8::internal::Builtin_HandleApiCall(v8::internal::(anonymous namespace)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)1>, v8::internal::Isolate*) ()
#6  0x0000283fe290618e in ?? ()
#7  0x0000283fe29060e1 in ?? ()
#8  0x00007fff0e93dac0 in ?? ()
......
#91 0x0000000800000000 in ?? ()

What is the problem？

[bnoordhuis]

Um, can you be more specific? What does 'once process do not work' mean exactly? You mean it hangs in the write() syscall?

[mond03cn]

Yes, write() isn't return.

[bnoordhuis]

Okay, can you provide me with more details? Do you have a test case that reproduces it? What does uname -a print? What version of glibc are you using? What do you see when you attach strace with strace -p <pid>? Does your application use native add-ons?

[mond03cn]

it need 12 hours to reproduct it...

uname -a
 Linux  3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

ldd node

    linux-vdso.so.1 =>  (0x00007fff333bc000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc73b126000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fc73af1e000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc73ac1a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc73a91e000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc73a708000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc73a4ea000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc73a12b000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fc73b332000)

native addons 
    ws
    toobusy

[trevnorris]

What was the last version of Node where this error did not occur? Has anything else changed since the error started to occur?

[trevnorris]

Also I would like more information about your environment. How many times has the issue reproduced at our around 12 hours running? What type/size of information are your writing?

[mond03cn]

When I use v0.8.0 replace of v.10.13,the problem Problem no longer occurs! Why?

[indutny]

There could be many reasons for it. Can you please post results of strace -p <pid> of your process, when this problem occurs? Otherwise it won't be possible for us to track it down and fix.

[mond03cn]

strace -p show follow

#0  0x00007f06ed544ccd in write () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00000000006db28a in uv__write (stream=stream@entry=0x268f170) at ../deps/uv/src/unix/stream.c:785
#2  0x00000000006dcbda in uv_write2 (req=0x26a0fa0, stream=0x268f170, bufs=0x7fff0e93d950, bufcnt=<optimized out>, 
    send_handle=<optimized out>, cb=<optimized out>) at ../deps/uv/src/unix/stream.c:1241
#3  0x00000000005af5ec in v8::Handle<v8::Value> node::StreamWrap::WriteStringImpl<(node::encoding)1>(v8::Arguments const&) ()
#4  0x00000000005aede9 in node::StreamWrap::WriteUtf8String(v8::Arguments const&) ()
#5  0x000000000070df11 in v8::internal::Builtin_HandleApiCall(v8::internal::(anonymous namespace)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)1>, v8::internal::Isolate*) ()
#6  0x0000283fe290618e in ?? ()
#7  0x0000283fe29060e1 in ?? ()
#8  0x00007fff0e93dac0 in ?? ()
......
#91 0x0000000800000000 in ?? ()

[bnoordhuis]

That's not strace output... the stack trace suggests it's trying to write to a UNIX domain socket. It's not really possible for that to block indefinitely (barring kernel bugs) because node.js puts all file descriptors into non-blocking mode.

In gdb, can you execute the following two commands?

(gdb) frame 1
(gdb) print *stream

Please post the output here.

There is a fd field in the io_watcher sub-struct. What value does it have and what does ls -l /proc/<pid>/fd/<fd> print?

I'm somewhat disinclined to believe that the bug is in node.js. The fact that you're using native add-ons who might be doing $DEITY knows what doesn't help. If you can remove them from the equation, that would be great.

[halfblood369]

I have the same problem.
And the stream show:

(gdb) frame 1
#1  0x000000000099815d in uv__write (stream=0x1e0c830) at ../deps/uv/src/unix/stream.c:785
785 ../deps/uv/src/unix/stream.c: No such file or directory.
    in ../deps/uv/src/unix/stream.c
(gdb) p *stream
$1 = {close_cb = 0, data = 0x1e0c7f0, loop = 0xe4a680, type = UV_NAMED_PIPE, handle_queue = {prev = 0xe461c0,
    next = 0x1e077a0}, flags = 96, next_closing = 0x0, write_queue_size = 873, alloc_cb = 0, read_cb = 0, read2_cb = 0,
  connect_req = 0x0, shutdown_req = 0x0, io_watcher = {cb = 0x998f20 <uv__stream_io>, pending_queue = {prev = 0x1e0c8a8,
      next = 0x1e0c8a8}, watcher_queue = {prev = 0x1e0c8b8, next = 0x1e0c8b8}, pevents = 0, events = 0, fd = 1},
  write_queue = {prev = 0x1e747d8, next = 0x1e747d8}, write_completed_queue = {prev = 0x1e0c8e8, next = 0x1e0c8e8},
  connection_cb = 0, delayed_error = 0, accepted_fd = -1}

and

  $ ls -l /proc/3266/fd/1
  /proc/3266/fd/1 -> pipe:[313010787]

[halfblood369]

@mond03cn Have you resolved the problem?

[cynron]

fd is 1, a.k.a stdout, and it refers to a pipe, so it may be blocking, i guess.

[cynron]

The problem is why main thread hang infinitely and write never return.

[indutny]

Could you please run an strace on your process as I have asked you some time ago? It is quite hard to figure it out, otherwise.

@halfblood369 are you piping data to another process? Seems like the other process isn't reading from the pipe and it becomes full.

[halfblood369]

Yes, the blocked process is another process's child, the code is:

    child = cp.spawn(command, options);
    var prefix = command === Constants.COMMAND.SSH ? '[' + host + '] ' : '';

    child.stderr.on('data', function (chunk) {
      var msg = chunk.toString();
      process.stderr.write(msg);
      cb && cb(msg);
    });

    child.stdout.on('data', function (chunk) {
      var msg = prefix + chunk.toString();
      process.stdout.write(msg);
    });

[indutny]

So, I'd rather look at that process to see why it isn't reading. We are going to make stdio writes non-blocking, but that won't happen in v0.12 . I'll consider this a duplicate of #2598 for now.