ERR_HTTP2_INVALID_STREAM

[StephenLynx]

Node 10.13
CentOS 7
HTTP2

When trying to use writeHead on the compatibility layer, I get a ERR_HTTP2_INVALID_STREAM sometimes that crashes the worker process.

Error [ERR_HTTP2_INVALID_STREAM]: The stream has been destroyed
at Http2ServerResponse.writeHead (internal/http2/compat.js:569:13)

The header is in {key:[value,value],anotherkey:[value]} format.

[Trott]

@nodejs/http2 (Should this be transferred to the help repo?)

[StephenLynx]

Trott, keep in mind that this isn't the first time someone has issues with this.
#22135
Now, it should have been fixed on 10.12, so it was either a regression or I am doing something very sketchy with the request object that is causing it to happen.

Edit: the issue happened on ubuntu 16, actually. I am trying to reproduce on CentOS 7 now.
Edit2: it just happened again on CentOS 7 too.

[StephenLynx]

I would also like to point out that I am using the cluster module to spawn one worker process for each CPU core.

[StephenLynx]

And after installing 10.12 and 10.13 again I can't seem to reproduce it anymore.
Edit: nevermind, just happened again.

[antsmartian]

@StephenLynx Could you please share a reproducible code?

[StephenLynx]

I can not, aside from my software gitgud.io/LynxChan/LynxChan

[Trott]

@nodejs/http2 Is it likely that the source of the problem can be determined with the information given?

[apapirovski]

If writeHead is crashing the process then in all likelihood your stream is getting closed in advance of you calling the function. Given that you're using compatibility mode, it's very much possible that some other code is causing an issue that leads to the stream prematurely closing. Note that this was also the case with the issue you linked above.

I'm going to close this out as there's no easily usable reproduction and there's not much likelihood this is a Node core issue.

[StephenLynx]

How could one close it? I can look around my code, I use the request and response exclusively as I have used them for http1 for years.

[StephenLynx]

So while I am not discarding the possibility to be with my code, I am not so sure it isn't with node. I remember finding some similar intermittent crash around version 5 that I also couldn't consistently reproduce but ended up being a DoS on node itself.

[mcollina]

I think this is a bug in our compat API.

As you can see in

node/lib/_http_outgoing.js

Lines 240 to 247 in 9920dbc

	OutgoingMessage.prototype._writeRaw = _writeRaw;
	function _writeRaw(data, encoding, callback) {
	const conn = this.connection;
	if (conn && conn.destroyed) {
	// The socket was destroyed. If we're still trying to write to it,
	// then we haven't gotten the 'close' event yet.
	return false;
	}

, http1 writeHead does not throw if the stream is destroyed, it just returns false.

I think this is a proper bug.

[mcollina]

@StephenLynx can you please upload a script to reproduce the crash?

[StephenLynx]

I tried, but it wouldn't crash with a simple wait to hold concurrent requests and another script hammering it.
The crashes seem to happen a bit at random with my software running. It works, then suddenly crash. It also seemed to be related with time. It would hardly crash right away, but once a worker crashed, it seems all the others became much more prone to crashing too.

[mcollina]

Can you please include a full stacktrace?

[StephenLynx]

I'll try, but I remember it being most of the time here https://gitgud.io/LynxChan/LynxChan/blob/master/src/be/form/captcha.js#L11

[StephenLynx]

GOT IT
@mcollina
https://pastebin.com/vCbVS1LT
It spent 30 minutes working fine, then crashed when outputting from gridfs, then crashed again 14 seconds later when outputting some error 500 page.

[StephenLynx]

Also, notice how the error it threw originated from trying to output a response on the cache handler. But that one is inside a try catch, so it didn't crash there.
I'd also like to point that I had about 3 tabs open, no sure if that influenced anything.

[StephenLynx]

So, the issue here is that is expected for the browser to sometimes interrupt the connection before the response and is up to what handles the connection to deal with it?

[mcollina]

The issue is just a mismatch between writeHead in http1 and http2. See #24723

[StephenLynx]

Btw, did you check if the write and end also don't crash?

[mcollina]

@StephenLynx please check the test in my PR.

[StephenLynx]

Ah yeah, you do a write head, write and end on the test.
The name of the pr made me think if you weren't looking just at writeHead.

[Trott]

Fixed in 32fed93