Force line-buffering for stderr #3701

Trott · 2015-11-06T22:23:08Z

On CI, it appears that stderr on SmartOS is not line buffered and that was resulting in some interleaving in test-debug-signal-cluster, causing the test to be flaky. This change forces stderr to be line-buffered, fixing the issue.

Trott · 2015-11-06T22:24:50Z

(Is there a better tag/prefix/whatever for this than src:? node:?)

bnoordhuis · 2015-11-06T22:50:40Z

src/node.cc

@@ -3129,6 +3129,7 @@ void LoadEnvironment(Environment* env) {
  // thrown during process startup.
  try_catch.SetVerbose(true);

+  setvbuf(stderr , NULL , _IOLBF , 1024);


This is not the right place for a call to setvbuf, it's too late after startup. It should probably be the first thing in main() or node::Init() and even that is possibly not soon enough because of printf calls from static constructors.

On a separate testing branch, I moved it to main().

I had it run the test on SmartOS 500 times and there were no failures. https://ci.nodejs.org/job/node-test-commit-smartos/246/

Then I commented out the line.
bf88e85

And ran the test again and got multiple failures.
https://ci.nodejs.org/job/node-test-commit-smartos/247/

I'll move the setvbuf() to main() or anywhere else that seems more appropriate.

Trott · 2015-11-06T23:56:27Z

CI: https://ci.nodejs.org/job/node-test-pull-request/684/

mscdex · 2015-11-07T04:12:11Z

src/node_main.cc

@@ -41,6 +41,7 @@ int wmain(int argc, wchar_t *wargv[]) {
 #else
 // UNIX
 int main(int argc, char *argv[]) {
+  setvbuf(stderr , NULL , _IOLBF , 1024);


I think there shouldn't be a space immediately after each of the first three arguments?

Clearly, I need to up my cut-and-paste game. Extraneous spaces removed. Thanks!

jbergstroem · 2015-11-09T06:24:54Z

@Trott should probably be node: $foo.

SmartOS does not line buffer stderr by default, or at least that is the behavior on the Node project Jenkins server. Force line buffering. This resolves the flakiness observed on SmartOS for test-debug-signal-cluster. Fixes: nodejs#2476 Refs: nodejs#3615

Trott · 2015-11-09T16:47:33Z

@jbergstroem Cool. Changed the commit message, squashed the commits into a single commit and force pushed.

Anyone feel comfortable giving this one a thumbs up (assuming CI comes back green)?

jasnell · 2015-11-09T19:33:10Z

hmm... would this need to be semver-minor or higher?

Trott · 2015-11-09T22:06:02Z

@jasnell I don't think so. I think it only affects messages sent by the Node debugger which bypasses console.error() and uses the raw stderr instead.

Anyway, if it increased your comfort level to have it at semver-minor, I have no objection to that. But if you're asking my opinion, I think it's patch level. I just don't hold that opinion particularly strongly.

bnoordhuis · 2015-11-09T22:10:21Z

LGTM. This is semver-patch IMO, it only affects fprintf statements in src/, not process.stderr.write statements in JS land.

jasnell · 2015-11-09T23:08:01Z

👍 LGTM

Trott · 2015-11-09T23:37:36Z

Kewl. I'm waiting on the return of the CI server to do one last rebased-against-master CI run before landing...

jbergstroem · 2015-11-10T02:23:04Z

LGTM

jasnell · 2015-11-10T16:02:18Z

New CI Run: https://ci.nodejs.org/job/node-test-pull-request/689/

Trott · 2015-11-10T23:51:29Z

OK to land despite OSX + Arm buildbot problems on CI?

jbergstroem · 2015-11-11T01:26:39Z

@Trott if you've tested locally on a mac I'd say go for it. I'm +1 to patchlevel.

SmartOS does not line buffer stderr by default, or at least that is the behavior on the Node project Jenkins server. Force line buffering. This resolves the flakiness observed on SmartOS for test-debug-signal-cluster. PR-URL: nodejs#3701 Fixes: nodejs#2476 Refs: nodejs#3615 Reviewed-By: Johan Bergström <bugs@bergstroem.nu> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Trott · 2015-11-11T01:58:50Z

Landed in 0966ab9.

SmartOS does not line buffer stderr by default, or at least that is the behavior on the Node project Jenkins server. Force line buffering. This resolves the flakiness observed on SmartOS for test-debug-signal-cluster. PR-URL: #3701 Fixes: #2476 Refs: #3615 Reviewed-By: Johan Bergström <bugs@bergstroem.nu> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

MylesBorins · 2015-12-01T01:08:11Z

should this land in LTS?

/cc @jasnell @bnoordhuis @Trott

rvagg · 2015-12-01T02:12:48Z

hey, this is a good one to discuss at an LTS WG meeting—the change itself is a good one but who knows what weird use is out there that may be depending on existing behaviour, perhaps it's not worth the trouble to backport?

MylesBorins · 2015-12-15T00:21:48Z

looks like we missed talking about this at todays meeting. @jasnell should we add a label for LTS discussions?

jasnell · 2015-12-15T02:23:24Z

Yeah, if you'd like go ahead and create an lts-agenda label for this repo.
On Dec 14, 2015 4:22 PM, "Myles Borins" notifications@github.com wrote:

looks like we missed talking about this at todays meeting. @jasnell
https://github.com/jasnell should we add a label for LTS discussions?

—
Reply to this email directly or view it on GitHub
#3701 (comment).

jasnell · 2015-12-17T19:38:41Z

This is flagged as lts-watch but needs to be discussed before landing...

MylesBorins · 2016-02-17T18:51:59Z

@nodejs/lts can you please weigh in on this one?

rvagg · 2016-02-18T02:13:26Z

From these two comments

@jasnell I don't think so. I think it only affects messages sent by the Node debugger which bypasses console.error() and uses the raw stderr instead.

LGTM. This is semver-patch IMO, it only affects fprintf statements in src/, not process.stderr.write statements in JS land.

If you grep fprintf src/* the impact is pretty low. So +1 from me for LTS.

jasnell · 2016-02-19T18:15:10Z

LGTM to v4.4

SmartOS does not line buffer stderr by default, or at least that is the behavior on the Node project Jenkins server. Force line buffering. This resolves the flakiness observed on SmartOS for test-debug-signal-cluster. PR-URL: #3701 Fixes: #2476 Refs: #3615 Reviewed-By: Johan Bergström <bugs@bergstroem.nu> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Trott added c++ Issues and PRs that require attention from people who are familiar with C++. test Issues and PRs related to the tests. labels Nov 6, 2015

bnoordhuis reviewed Nov 6, 2015
View reviewed changes

Trott mentioned this pull request Nov 6, 2015

debugger,test: fix flaky debug logging on SmartOS #3615

Closed

mscdex reviewed Nov 7, 2015
View reviewed changes

mscdex removed the test Issues and PRs related to the tests. label Nov 7, 2015

Trott force-pushed the buffer-stderr branch from 55b0377 to c9d47f2 Compare November 9, 2015 16:46

Trott closed this Nov 11, 2015

Fishrock123 mentioned this pull request Nov 11, 2015

Propose v5.1.0 #3736

Merged

evanlucas mentioned this pull request Nov 16, 2015

ensure preload modules are always preloaded #2253

Closed

MylesBorins added the lts-watch-v4.x label Dec 1, 2015

MylesBorins added the lts-agenda label Dec 15, 2015

rvagg mentioned this pull request Dec 17, 2015

Release 4.2.4 Planning #4321

Closed

MylesBorins mentioned this pull request Jan 11, 2016

LTS WG Meeting 2015-01-11 nodejs/Release#67

Closed

MylesBorins added land-on-v4.x and removed lts-agenda labels Feb 22, 2016

Trott mentioned this pull request Feb 23, 2016

test-debug-signal-cluster.js fails sometimes (investigate flaky test) #3796

Closed

Trott deleted the buffer-stderr branch January 13, 2022 22:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Force line-buffering for stderr #3701

Force line-buffering for stderr #3701

Trott commented Nov 6, 2015

Trott commented Nov 6, 2015

bnoordhuis Nov 6, 2015

Trott Nov 6, 2015

Trott commented Nov 6, 2015

mscdex Nov 7, 2015

Trott Nov 7, 2015

jbergstroem commented Nov 9, 2015

Trott commented Nov 9, 2015

jasnell commented Nov 9, 2015

Trott commented Nov 9, 2015

bnoordhuis commented Nov 9, 2015

jasnell commented Nov 9, 2015

Trott commented Nov 9, 2015

jbergstroem commented Nov 10, 2015

jasnell commented Nov 10, 2015

Trott commented Nov 10, 2015

jbergstroem commented Nov 11, 2015

Trott commented Nov 11, 2015

MylesBorins commented Dec 1, 2015

rvagg commented Dec 1, 2015

MylesBorins commented Dec 15, 2015

jasnell commented Dec 15, 2015

jasnell commented Dec 17, 2015

MylesBorins commented Feb 17, 2016

rvagg commented Feb 18, 2016

jasnell commented Feb 19, 2016

Force line-buffering for stderr #3701

Force line-buffering for stderr #3701

Conversation

Trott commented Nov 6, 2015

Trott commented Nov 6, 2015

bnoordhuis Nov 6, 2015

Choose a reason for hiding this comment

Trott Nov 6, 2015

Choose a reason for hiding this comment

Trott commented Nov 6, 2015

mscdex Nov 7, 2015

Choose a reason for hiding this comment

Trott Nov 7, 2015

Choose a reason for hiding this comment

jbergstroem commented Nov 9, 2015

Trott commented Nov 9, 2015

jasnell commented Nov 9, 2015

Trott commented Nov 9, 2015

bnoordhuis commented Nov 9, 2015

jasnell commented Nov 9, 2015

Trott commented Nov 9, 2015

jbergstroem commented Nov 10, 2015

jasnell commented Nov 10, 2015

Trott commented Nov 10, 2015

jbergstroem commented Nov 11, 2015

Trott commented Nov 11, 2015

MylesBorins commented Dec 1, 2015

rvagg commented Dec 1, 2015

MylesBorins commented Dec 15, 2015

jasnell commented Dec 15, 2015

jasnell commented Dec 17, 2015

MylesBorins commented Feb 17, 2016

rvagg commented Feb 18, 2016

jasnell commented Feb 19, 2016