Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

https: evict cached sessions on error #4982

Closed

Conversation

indutny
Copy link
Member

@indutny indutny commented Jan 30, 2016

Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of #3692, until
OpenSSL fix will be merged/released.

See: #3692

Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of nodejs#3692, until
OpenSSL fix will be merged/released.

See: nodejs#3692
@indutny
Copy link
Member Author

indutny commented Jan 30, 2016

@indutny
Copy link
Member Author

indutny commented Jan 30, 2016

cc @nodejs/crypto @nodejs/collaborators

@@ -84,6 +84,13 @@ function createConnection(port, host, options) {

self._cacheSession(options._agentKey, socket.getSession());
});

// Evict session on error
socket.once('close', function(err) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use an arrow function here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, fixed.

@evanlucas
Copy link
Contributor

LGTM, but you probably want to get someone else from crypto to sign off

port: common.PORT,
rejectUnauthorized: false
}, function(res) {
res.resume();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding more strict asset check?

--- a/test/parallel/test-https-agent-session-eviction.js
+++ b/test/parallel/test-https-agent-session-eviction.js
@@ -77,7 +77,12 @@ function third(server) {
     rejectUnauthorized: false
   }, function(res) {
     res.resume();
+    assert(!req.socket.isSessionReused());
     server.close();
   });
+  req.on('error', function(err) {
+    // never called
+    assert(false);
+  });
   req.end();
 }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, agree!

@shigeki
Copy link
Contributor

shigeki commented Jan 31, 2016

Just commented minor nits. LGTM.

@targos targos added the https Issues or PRs related to the https subsystem. label Jan 31, 2016
@indutny
Copy link
Member Author

indutny commented Feb 2, 2016

Landed in 165b33f, thank you!

@indutny indutny closed this Feb 2, 2016
@indutny indutny deleted the fix/https-session-eviction-on-error branch February 2, 2016 02:53
indutny added a commit that referenced this pull request Feb 2, 2016
Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of #3692, until
OpenSSL fix will be merged/released.

See: #3692
PR-URL: #4982
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
@tristanls
Copy link
Contributor

\o/

rvagg pushed a commit that referenced this pull request Feb 8, 2016
Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of #3692, until
OpenSSL fix will be merged/released.

See: #3692
PR-URL: #4982
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
MylesBorins pushed a commit that referenced this pull request Feb 18, 2016
Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of #3692, until
OpenSSL fix will be merged/released.

See: #3692
PR-URL: #4982
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
MylesBorins pushed a commit that referenced this pull request Feb 18, 2016
Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of #3692, until
OpenSSL fix will be merged/released.

See: #3692
PR-URL: #4982
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
@MylesBorins MylesBorins mentioned this pull request Feb 18, 2016
MylesBorins pushed a commit that referenced this pull request Feb 18, 2016
Notable changes:

This update to the LTS line includes a number of semver minor changes
that have been staged for a number of months. This includes:

  * deps: backport 9da3ab6 from V8 upstream (Ali Ijaz Sheikh)
    - #3609
  * http: handle errors on idle sockets (José F. Romaniello)
    - #4482
  * src: add BE support to StringBytes::Encode() (Bryon Leung)
    - #3410
  * tls: add `options` argument to createSecurePair (Коренберг Марк)
    - #2441

There are also quite a large number of semver patch changes
including over 20 doc fixes and almost 50 test fixes.

Notable semver patch changes include:

  * deps: upgrade to npm 2.14.18 (Kat Marchán)
    - #5245
  * https: evict cached sessions on error (Fedor Indutny)
    - #4982
  * process: support symbol events (cjihrig)
    - #4798
  * querystring: improve parse() performance (Brian White)
    - #4675

PR-URL: #5301
MylesBorins pushed a commit that referenced this pull request Feb 23, 2016
In December we announced that we would be doing a minor release in
order to get a number of voted on SEMVER-MINOR changes into LTS.
Our ability to release this was delayed due to the unforeseen security
release v4.3. We are quickly bumping to v4.4 in order to bring you the
features that we had committed to releasing.

Notable changes:

The SEMVER-MINOR changes include:
  * **deps**:
    - An update to v8 that introduces a new flag
      --perf_basic_prof_only_functions (Ali Ijaz Sheikh)
      #3609
  * **http**:
    - A new feature in http(s) agent that catches errors on
      *keep alived* connections (José F. Romaniello)
      #4482
  * **src**:
    - Better support for Big-Endian systems (Bryon Leung)
      #3410
  * **tls**:
    - A new feature that allows you to pass common SSL options to
      `tls.createSecurePair` (Коренберг Марк)
      #2441

Notable semver patch changes include:

  * **npm**
    - upgrade to npm 2.14.19 (Kat Marchán)
      #5335
  * **https**:
    - A potential fix for #3692)
      HTTP/HTTPS client requests throwing EPROTO (Fedor Indutny)
      #4982
  * **process**:
    - Add support for symbols in event emitters. Symbols didn't exist
      when it was written ¯\_(ツ)_/¯ (cjihrig)
      #4798
  * **querystring**:
    - querystring.parse() is now 13-22% faster! (Brian White)
      #4675

PR-URL: #5301
MylesBorins pushed a commit that referenced this pull request Mar 1, 2016
In December we announced that we would be doing a minor release in order to
get a number of voted on SEMVER-MINOR changes into LTS. Our ability to release this
was delayed due to the unforeseen security release v4.3. We are quickly bumping to
v4.4 in order to bring you the features that we had committed to releasing.

This release also includes security updates to openssl. More information can be found [on nodejs.org](https://nodejs.org/en/blog/vulnerability/openssl-march-2016/)

This release also includes over 70 fixes to our docs and over 50 fixes to tests.

The SEMVER-MINOR changes include:
  * deps:
    - An update to v8 that introduces a new flag --perf_basic_prof_only_functions (Ali Ijaz Sheikh) #3609
  * http:
    - A new feature in http(s) agent that catches errors on *keep alived* connections (José F. Romaniello) #4482
  * src:
    - Better support for Big-Endian systems (Bryon Leung) #3410
  * tls:
    - A new feature that allows you to pass common SSL options to `tls.createSecurePair` (Коренберг Марк) #2441
  * tools
    - a new flag `--prof-process` which will execute the tick processor on the provided isolate files (Matt Loring) #4021

Notable semver patch changes include:

  * buld:
    - Support python path that includes spaces. This should be of particular interest to our Windows users who may have python living in `c:/Program Files` (Felix Becker) #4841
  * https:
    - A potential fix for #3692 HTTP/HTTPS client requests throwing EPROTO (Fedor Indutny) #4982
  * installer:
    - More readable profiling information from isolate tick logs (Matt Loring) #3032
  * *npm:
    - upgrade to npm 2.14.20 (Kat Marchán) #5510
  * *openssl:
    - upgrade openssl to 1.0.2g (Ben Noordhuis) #5507
  * process:
    - Add support for symbols in event emitters. Symbols didn't exist when it was written ¯\_(ツ)_/¯ (cjihrig) #4798
  * querystring:
    - querystring.parse() is now 13-22% faster! (Brian White) #4675
  * streams:
    - performance improvements for moving small buffers that shows a 5% throughput gain. IoT projects have been seen to be as much as 10% faster with this change! (Matteo Collina) #4354
  * tools:
    - eslint has been updated to version 2.1.0 (Rich Trott) #5214

PR-URL: #5301
MylesBorins pushed a commit to MylesBorins/node that referenced this pull request Mar 1, 2016
In December we announced that we would be doing a minor release in order to
get a number of voted on SEMVER-MINOR changes into LTS. Our ability to release this
was delayed due to the unforeseen security release v4.3. We are quickly bumping to
v4.4 in order to bring you the features that we had committed to releasing.

This release also includes security updates to openssl. More information can be found [on nodejs.org](https://nodejs.org/en/blog/vulnerability/openssl-march-2016/)

This release also includes over 70 fixes to our docs and over 50 fixes to tests.

The SEMVER-MINOR changes include:
  * deps:
    - An update to v8 that introduces a new flag --perf_basic_prof_only_functions (Ali Ijaz Sheikh) nodejs#3609
  * http:
    - A new feature in http(s) agent that catches errors on *keep alived* connections (José F. Romaniello) nodejs#4482
  * src:
    - Better support for Big-Endian systems (Bryon Leung) nodejs#3410
  * tls:
    - A new feature that allows you to pass common SSL options to `tls.createSecurePair` (Коренберг Марк) nodejs#2441
  * tools
    - a new flag `--prof-process` which will execute the tick processor on the provided isolate files (Matt Loring) nodejs#4021

Notable semver patch changes include:

  * buld:
    - Support python path that includes spaces. This should be of particular interest to our Windows users who may have python living in `c:/Program Files` (Felix Becker) nodejs#4841
  * https:
    - A potential fix for nodejs#3692 HTTP/HTTPS client requests throwing EPROTO (Fedor Indutny) nodejs#4982
  * installer:
    - More readable profiling information from isolate tick logs (Matt Loring) nodejs#3032
  * *npm:
    - upgrade to npm 2.14.20 (Kat Marchán) nodejs#5510
  * *openssl:
    - upgrade openssl to 1.0.2g (Ben Noordhuis) nodejs#5507
  * process:
    - Add support for symbols in event emitters. Symbols didn't exist when it was written ¯\_(ツ)_/¯ (cjihrig) nodejs#4798
  * querystring:
    - querystring.parse() is now 13-22% faster! (Brian White) nodejs#4675
  * streams:
    - performance improvements for moving small buffers that shows a 5% throughput gain. IoT projects have been seen to be as much as 10% faster with this change! (Matteo Collina) nodejs#4354
  * tools:
    - eslint has been updated to version 2.1.0 (Rich Trott) nodejs#5214

PR-URL: nodejs#5301
MylesBorins pushed a commit that referenced this pull request Mar 2, 2016
Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of #3692, until
OpenSSL fix will be merged/released.

See: #3692
PR-URL: #4982
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
MylesBorins pushed a commit that referenced this pull request Mar 2, 2016
In December we announced that we would be doing a minor release in order to
get a number of voted on SEMVER-MINOR changes into LTS. Our ability to release this
was delayed due to the unforeseen security release v4.3. We are quickly bumping to
v4.4 in order to bring you the features that we had committed to releasing.

This release also includes over 70 fixes to our docs and over 50 fixes to tests.

The SEMVER-MINOR changes include:
  * deps:
    - An update to v8 that introduces a new flag --perf_basic_prof_only_functions (Ali Ijaz Sheikh) #3609
  * http:
    - A new feature in http(s) agent that catches errors on *keep alived* connections (José F. Romaniello) #4482
  * src:
    - Better support for Big-Endian systems (Bryon Leung) #3410
  * tls:
    - A new feature that allows you to pass common SSL options to `tls.createSecurePair` (Коренберг Марк) #2441
  * tools
    - a new flag `--prof-process` which will execute the tick processor on the provided isolate files (Matt Loring) #4021

Notable semver patch changes include:

  * buld:
    - Support python path that includes spaces. This should be of particular interest to our Windows users who may have python living in `c:/Program Files` (Felix Becker) #4841
  * https:
    - A potential fix for #3692 HTTP/HTTPS client requests throwing EPROTO (Fedor Indutny) #4982
  * installer:
    - More readable profiling information from isolate tick logs (Matt Loring) #3032
  * *npm:
    - upgrade to npm 2.14.20 (Kat Marchán) #5510
  * process:
    - Add support for symbols in event emitters. Symbols didn't exist when it was written ¯\_(ツ)_/¯ (cjihrig) #4798
  * querystring:
    - querystring.parse() is now 13-22% faster! (Brian White) #4675
  * streams:
    - performance improvements for moving small buffers that shows a 5% throughput gain. IoT projects have been seen to be as much as 10% faster with this change! (Matteo Collina) #4354
  * tools:
    - eslint has been updated to version 2.1.0 (Rich Trott) #5214

PR-URL: #5301
MylesBorins pushed a commit that referenced this pull request Mar 3, 2016
In December we announced that we would be doing a minor release in order to
get a number of voted on SEMVER-MINOR changes into LTS. Our ability to release this
was delayed due to the unforeseen security release v4.3. We are quickly bumping to
v4.4 in order to bring you the features that we had committed to releasing.

This release also includes over 70 fixes to our docs and over 50 fixes to tests.

The SEMVER-MINOR changes include:
  * deps:
    - An update to v8 that introduces a new flag --perf_basic_prof_only_functions (Ali Ijaz Sheikh) #3609
  * http:
    - A new feature in http(s) agent that catches errors on *keep alived* connections (José F. Romaniello) #4482
  * src:
    - Better support for Big-Endian systems (Bryon Leung) #3410
  * tls:
    - A new feature that allows you to pass common SSL options to `tls.createSecurePair` (Коренберг Марк) #2441
  * tools
    - a new flag `--prof-process` which will execute the tick processor on the provided isolate files (Matt Loring) #4021

Notable semver patch changes include:

  * buld:
    - Support python path that includes spaces. This should be of particular interest to our Windows users who may have python living in `c:/Program Files` (Felix Becker) #4841
  * https:
    - A potential fix for #3692 HTTP/HTTPS client requests throwing EPROTO (Fedor Indutny) #4982
  * installer:
    - More readable profiling information from isolate tick logs (Matt Loring) #3032
  * *npm:
    - upgrade to npm 2.14.20 (Kat Marchán) #5510
  * process:
    - Add support for symbols in event emitters. Symbols didn't exist when it was written ¯\_(ツ)_/¯ (cjihrig) #4798
  * querystring:
    - querystring.parse() is now 13-22% faster! (Brian White) #4675
  * streams:
    - performance improvements for moving small buffers that shows a 5% throughput gain. IoT projects have been seen to be as much as 10% faster with this change! (Matteo Collina) #4354
  * tools:
    - eslint has been updated to version 2.1.0 (Rich Trott) #5214

PR-URL: #5301
MylesBorins pushed a commit that referenced this pull request Mar 8, 2016
In December we announced that we would be doing a minor release in order to
get a number of voted on SEMVER-MINOR changes into LTS. Our ability to release this
was delayed due to the unforeseen security release v4.3. We are quickly bumping to
v4.4 in order to bring you the features that we had committed to releasing.

This release also includes over 70 fixes to our docs and over 50 fixes to tests.

The SEMVER-MINOR changes include:
  * deps:
    - An update to v8 that introduces a new flag --perf_basic_prof_only_functions (Ali Ijaz Sheikh) #3609
  * http:
    - A new feature in http(s) agent that catches errors on *keep alived* connections (José F. Romaniello) #4482
  * src:
    - Better support for Big-Endian systems (Bryon Leung) #3410
  * tls:
    - A new feature that allows you to pass common SSL options to `tls.createSecurePair` (Коренберг Марк) #2441
  * tools
    - a new flag `--prof-process` which will execute the tick processor on the provided isolate files (Matt Loring) #4021

Notable semver patch changes include:

  * buld:
    - Support python path that includes spaces. This should be of particular interest to our Windows users who may have python living in `c:/Program Files` (Felix Becker) #4841
  * https:
    - A potential fix for #3692 HTTP/HTTPS client requests throwing EPROTO (Fedor Indutny) #4982
  * installer:
    - More readable profiling information from isolate tick logs (Matt Loring) #3032
  * *npm:
    - upgrade to npm 2.14.20 (Kat Marchán) #5510
  * process:
    - Add support for symbols in event emitters. Symbols didn't exist when it was written ¯\_(ツ)_/¯ (cjihrig) #4798
  * querystring:
    - querystring.parse() is now 13-22% faster! (Brian White) #4675
  * streams:
    - performance improvements for moving small buffers that shows a 5% throughput gain. IoT projects have been seen to be as much as 10% faster with this change! (Matteo Collina) #4354
  * tools:
    - eslint has been updated to version 2.1.0 (Rich Trott) #5214

PR-URL: #5301
MylesBorins pushed a commit that referenced this pull request Mar 9, 2016
In December we announced that we would be doing a minor release in order to
get a number of voted on SEMVER-MINOR changes into LTS. Our ability to release this
was delayed due to the unforeseen security release v4.3. We are quickly bumping to
v4.4 in order to bring you the features that we had committed to releasing.

This release also includes over 70 fixes to our docs and over 50 fixes to tests.

The SEMVER-MINOR changes include:
  * deps:
    - An update to v8 that introduces a new flag --perf_basic_prof_only_functions (Ali Ijaz Sheikh) #3609
  * http:
    - A new feature in http(s) agent that catches errors on *keep alived* connections (José F. Romaniello) #4482
  * src:
    - Better support for Big-Endian systems (Bryon Leung) #3410
  * tls:
    - A new feature that allows you to pass common SSL options to `tls.createSecurePair` (Коренберг Марк) #2441
  * tools
    - a new flag `--prof-process` which will execute the tick processor on the provided isolate files (Matt Loring) #4021

Notable semver patch changes include:

  * buld:
    - Support python path that includes spaces. This should be of particular interest to our Windows users who may have python living in `c:/Program Files` (Felix Becker) #4841
  * https:
    - A potential fix for #3692 HTTP/HTTPS client requests throwing EPROTO (Fedor Indutny) #4982
  * installer:
    - More readable profiling information from isolate tick logs (Matt Loring) #3032
  * *npm:
    - upgrade to npm 2.14.20 (Kat Marchán) #5510
  * process:
    - Add support for symbols in event emitters. Symbols didn't exist when it was written ¯\_(ツ)_/¯ (cjihrig) #4798
  * querystring:
    - querystring.parse() is now 13-22% faster! (Brian White) #4675
  * streams:
    - performance improvements for moving small buffers that shows a 5% throughput gain. IoT projects have been seen to be as much as 10% faster with this change! (Matteo Collina) #4354
  * tools:
    - eslint has been updated to version 2.1.0 (Rich Trott) #5214

PR-URL: #5301
@mikemorris
Copy link

Moving the discussion from #4482 (comment) over to here, as I think this is the relevant pull request that introduced the problematic code.

[Wed, 09 Mar 2016 18:50:55 GMT] [fatal] [default] TypeError: this._evictSession is not a function
    at TLSSocket.<anonymous> (https.js:83:12)
    at TLSSocket.g (events.js:260:16)
    at emitOne (events.js:82:20)
    at TLSSocket.emit (events.js:169:7)
    at TCP._onclose (net.js:475:12)

We're seeing this error relatively frequently on active servers running Node.js v4.4.0. Previously, we were seeing unhandled errors on idle sockets, but that was addressed in #4482. I'm curious if the volume of idle sockets could have been exacerbated prior to the fix in 2e1ae3e too.

I'm guessing could be as simple as an incorrectly bound this context. I initially thought it could be the fault of the arrow function in the socket.once call, but I think I disproved that theory. I haven't been able to distill a simple test case to reproduce this yet, but it looks like it originates from the attempted destruction of an idle socket initially created with a KeepAlive session.

Wondering if #2534 could be at all relevant here?

@indutny
Copy link
Member Author

indutny commented Mar 23, 2016

Indeed it is most likely some incorrectly propagated this. I suspect that you are using some module that does call agent.createConnection without the context, because core itself sets up the context properly.

@mikemorris
Copy link

Thanks @indutny - it looks like it was node-modules/agentkeepalive#34. Do you think #2534 would eliminate the need for the agentkeepalive dependency if/when it lands?

@indutny
Copy link
Member Author

indutny commented Mar 24, 2016

@mikemorris I don't think that #2534 has anything to do with the http agent. What is the problem that agentkeepalive solves?

@mikemorris
Copy link

@indutny I'm not too familiar with the details here, but my understanding is that this might be a solved problem in Node.js v4+ that we just haven't tested yet. agentkeepalive was being used to keep sockets available for reuse instead of immediately closing them if there was not already another request pending.

node 0.10.x does need it, and until a certain point, node 0.12 needed it as well
in a nutshell, the way the agent worked in 0.10 was that whenever the agent would make a request, it would only ever reuse a socket if, in the same tick, there were other requests in the request queue. agentkeepalive changed this by allowing sockets to be reused up until the specified timeout.
so this greatly reduced the number of sockets that are ever being opened and closed, and increased the number of sockets that'd get reused

scovetta pushed a commit to scovetta/node that referenced this pull request Apr 2, 2016
Instead of using the same session over and over, evict it when the
socket emits error. This could be used as a mitigation of nodejs#3692, until
OpenSSL fix will be merged/released.

See: nodejs#3692
PR-URL: nodejs#4982
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
https Issues or PRs related to the https subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants