-
Notifications
You must be signed in to change notification settings - Fork 632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(node/process): null is not returned when reaching end-of-file in stdin #3113
Conversation
Can you add some custom test case in |
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
Two unit test cases have been added.
However, Github Actions runs the tests without a TTY. So, I have implemented the first unit test case to be ignored on CI, because it cannot run the test as expected. Any solutions to this situation would be appreciated. |
node/process_test.ts
Outdated
async fn() { | ||
const expected = ["65536", "foo", "bar", "null", "end"]; | ||
const scriptPath = "./node/testdata/process_stdin.ts"; | ||
const file = await Deno.readFile(`./node/testdata/process_stdin_dummy.txt`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can replace this with inline Uint8Array like: new TextEncoder().encode("foo\nbar")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test was divided into two tests, one for file and one for pipe. And I used inline Uint8Array in the test for pipe.
https://github.com/denoland/deno_std/pull/3113/files#diff-f27eacddc948ff5c5ce7c851fccf64c00f313b8edc28862b5066c211e2b14d72R402
Deno.test({ | ||
name: "process.stdin readable with piping a file", | ||
async fn() { | ||
const expected = ["65536", "foo", "bar", "null", "end"]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In node.js, the first number becomes 16384
. Is this difference intentional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My confirmation was lacking.
As I mentioned in the issue, the highWaterMark is 65536
with piping a file, But when running the code in child_process with piping stdin, the highWaterMark is a different value.
- tty(tty.ReadStream): 0
- piping files(fs.ReadStream): 65536
- piping stdin(Stream?): 16384
I'll look into it further.
Thank you for pointing this out.
8abb251
to
9ee0509
Compare
9ee0509
to
1c7526f
Compare
e43e22d
to
7f91b62
Compare
The stdin implementation has been changed to be more in line with Node.js. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Nice work! Thanks! @PolarETech
Fixes: #2974
Currently, the
highWaterMark
value ofReadable
forprocess.stdin
is fixed at 0. Therefore, when piping a file,null
is not returned after the file is read to the end, and theend
event is not emitted.This PR will fix this behavior by setting the appropriate highWaterMark value based on the type of stdin.
TODO
Actual
highWaterMark
values in Node.js.- 16384
unresolved (*2)(*1) The accuracy of type identification is insufficient and needs further improvement.
(*2) Non-console Windows applications are assumed (ref), but I couldn't find a way to get this state in Deno. The value in Node.js was checked in an electron app.
Will not be addressed in this PR
tty.ReadStream
Use
Duplex
insteadnet.Socket
withfd
optionUse
Duplex
insteadguessHandleType()
in "node/internal_binding/util.ts"Refs
The highWaterMark value to be set is based on actual Node.js behavior and some source code.
https://github.com/nodejs/node/blob/v18.12.1/lib/internal/bootstrap/switches/is_main_thread.js#L189
https://github.com/nodejs/node/blob/v18.12.1/src/node_util.cc#L257
https://github.com/nodejs/node/blob/v18.12.1/deps/uv/src/unix/tty.c#L333
https://github.com/nodejs/node/blob/v18.12.1/deps/uv/src/win/handle.c#L31
https://github.com/nodejs/node/blob/v18.12.1/lib/tty.js#L60
https://github.com/nodejs/node/blob/v18.12.1/lib/internal/fs/streams.js#L155
https://github.com/nodejs/node/blob/v18.12.1/lib/net.js#L329
https://github.com/nodejs/readable-stream/blob/v4.2.0/lib/internal/streams/state.js#L11-L13
Node test suite
"pseudo-tty/test-readable-tty-keepalive.js" test seems to be related to this change but does not pass the test. The
data
is not received as expected inprocess.stdin.on('readable', ...)
.The test has not been added because it appears to be failing due to factors other than this change, and it may take time to investigate. Perhaps some kind of PTY is needed in addition.
Memo
Some issues with the node compat layer have been found while working on this PR.
stdio
option is ignored inchild_process.spawnSync()
(src)input
option is ignored inchild_process.spawnSync()
(src)