Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native addon seems to use Node.js included openssl version instead of my own, but only on Node.js 10 #1559

Closed
JCMais opened this issue Oct 4, 2018 · 13 comments

Comments

@JCMais
Copy link

JCMais commented Oct 4, 2018

I have a Node.js bindings for libcurl, and currently I'm trying to fix an issue that I cannot find a solution, even tho I'm building OpenSSL directly, as see here and here, the version being used at runtime is the one bundled with Node.js. This seems to occur only on Windows.

The version set as dependency is 1.0.2o:
https://github.com/JCMais/curl-for-windows/tree/445e537cb2f5656e3a3ede09e9e4782a6b62c299/openssl

However, by building the addon, and running the following

const Curl = require("../lib/Curl");
console.error(Curl.getVersion());

I can see that the one being used is the same used in Node.js 10:

libcurl/7.57.0-DEV OpenSSL/1.1.0h zlib/1.2.11 libssh2/1.7.0_DEV

node -p process.versions

{ http_parser: '2.8.0',
  node: '10.8.0',
  v8: '6.7.288.49-node.19',
  uv: '1.22.0',
  zlib: '1.2.11',
  ares: '1.14.0',
  modules: '64',
  nghttp2: '1.32.0',
  napi: '3',
  openssl: '1.1.0h',
  icu: '62.1',
  unicode: '11.0',
  cldr: '33.1',
  tz: '2018e' }

This started happening only with Node.js 10, I've rechecked if this is really the case by going back to Node.js 8 and recompiling the addon there.

The js file prints:

libcurl/7.57.0-DEV OpenSSL/1.0.2o zlib/1.2.11 libssh2/1.7.0_DEV

node -p process.versions

{ http_parser: '2.8.0',
  node: '8.12.0',
  v8: '6.2.414.66',
  uv: '1.19.2',
  zlib: '1.2.11',
  ares: '1.10.1-DEV',
  modules: '57',
  nghttp2: '1.32.0',
  napi: '3',
  openssl: '1.0.2p',
  icu: '60.1',
  unicode: '10.0',
  cldr: '32.0',
  tz: '2017c' }

This is currently causing some segfaults, since libssh2 1.7, which is being used to build the addon, is not compatible with OpenSSL v1.1.0h, the one shipped with Node.js 10.

I know that if upgrade libssh2 to 1.8 the problem would be resolved, but I still think that there is something wrong there, since it does not feel right to use the OpenSSL from Node.js.


Also I think this section of the wiki is outdated: https://github.com/nodejs/node-gyp/wiki/Linking-to-OpenSSL#windows

Looks like the symbols are exported by Node now, even on Windows. See nodejs/node@b4d4fd9

  • Platform: Windows 10 Pro
  • Compiler:

msbuild 15.1.1012.6693
cl 19.10.25019 x64
Visual Studio 2017

I have tested on linux, and this problem does not happen. I used the following Dockerfile for that:

FROM node:10-stretch

RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl libcurl4-openssl-dev libssl-dev && rm -rf /var/lib/apt/lists/*

COPY package.json yarn.lock ./
RUN yarn install --ignore-scripts

COPY . ./
RUN yarn pregyp rebuild

This is probably something wrong I'm doing, but I would love some help, or guidance on what I should do to debug/fix this.

@JCMais
Copy link
Author

JCMais commented Oct 4, 2018

After more investigation, this is probably the same problem mentioned here: nodegit/nodegit#1490, to be more exact, on this comment: nodegit/nodegit#1490 (comment)

Other issues that seems to be related to this problem I'm having:
nodejs/node#7608
nodejs/help#254

@refack
Copy link
Contributor

refack commented Oct 6, 2018

but I still think that there is something wrong there, since it does feel right to use the OpenSSL from Node.js.

I'm assuming you mean that is _doesn't" feel right, yes?
You would like to link your addons to your provided openSSL and not to the one embedded in node?

@JCMais
Copy link
Author

JCMais commented Oct 6, 2018

@refack yep! Just fixed it, thanks.

You would like to link your addons to your provided openSSL and not to the one embedded in node?

Exactly


I just used dumpbin and I can see that Node.js really exports many OpenSSL symbols, here is an excerpt:

DUMPBIN /EXPORTS node.lib

output Microsoft (R) COFF/PE Dumper Version 14.10.25019.0 Copyright (C) Microsoft Corporation. All rights reserved.
          Dump of file node.lib

          File Type: LIBRARY

              Exports

                ordinal    name

                            // ... many symbols ...
                            SSL_CIPHER_get_cipher_nid
                            SSL_CIPHER_get_digest_nid
                            SSL_CIPHER_get_id
                            SSL_CIPHER_get_kx_nid
                            SSL_CIPHER_get_name
                            SSL_CIPHER_get_version
                            SSL_CIPHER_is_aead
                            SSL_COMP_add_compression_method
                            SSL_COMP_get0_name
                            SSL_COMP_get_compression_methods
                            SSL_COMP_get_id
                            SSL_COMP_get_name
                            SSL_COMP_set0_compression_methods
                            SSL_CONF_CTX_clear_flags
                            SSL_CONF_CTX_finish
                            SSL_CONF_CTX_free
                            SSL_CONF_CTX_new
                            SSL_CONF_CTX_set1_prefix
                            SSL_CONF_CTX_set_flags
                            SSL_CONF_CTX_set_ssl
                            SSL_CONF_CTX_set_ssl_ctx
                            SSL_CONF_cmd
                            SSL_CONF_cmd_argv
                            SSL_CONF_cmd_value_type
                            SSL_CTX_add_client_CA
                            SSL_CTX_add_client_custom_ext
                            SSL_CTX_add_server_custom_ext
                            SSL_CTX_add_session
                            SSL_CTX_callback_ctrl
                            SSL_CTX_check_private_key
                            SSL_CTX_clear_options
                            SSL_CTX_config
                            SSL_CTX_ctrl
                            SSL_CTX_dane_clear_flags
                            SSL_CTX_dane_enable
                            SSL_CTX_dane_mtype_set
                            SSL_CTX_dane_set_flags
                            SSL_CTX_flush_sessions
                            SSL_CTX_free
                            SSL_CTX_get0_certificate
                            SSL_CTX_get0_param
                            SSL_CTX_get0_privatekey
                            SSL_CTX_get0_security_ex_data
                            SSL_CTX_get_cert_store
                            SSL_CTX_get_ciphers
                            SSL_CTX_get_client_CA_list
                            SSL_CTX_get_client_cert_cb
                            SSL_CTX_get_default_passwd_cb
                            SSL_CTX_get_default_passwd_cb_userdata
                            SSL_CTX_get_ex_data
                            SSL_CTX_get_info_callback
                            SSL_CTX_get_options
                            SSL_CTX_get_quiet_shutdown
                            SSL_CTX_get_security_callback
                            SSL_CTX_get_security_level
                            SSL_CTX_get_ssl_method
                            SSL_CTX_get_timeout
                            SSL_CTX_get_verify_callback
                            SSL_CTX_get_verify_depth
                            SSL_CTX_get_verify_mode
                            SSL_CTX_has_client_custom_ext
                            SSL_CTX_load_verify_locations
                            SSL_CTX_new
                            SSL_CTX_remove_session
                            SSL_CTX_sess_get_get_cb
                            SSL_CTX_sess_get_new_cb
                            SSL_CTX_sess_get_remove_cb
                            SSL_CTX_sess_set_get_cb
                            SSL_CTX_sess_set_new_cb
                            SSL_CTX_sess_set_remove_cb
                            SSL_CTX_sessions
                            SSL_CTX_set0_security_ex_data
                            SSL_CTX_set1_param
                            SSL_CTX_set_alpn_protos
                            SSL_CTX_set_alpn_select_cb
                            SSL_CTX_set_cert_cb
                            SSL_CTX_set_cert_store
                            SSL_CTX_set_cert_verify_callback
                            SSL_CTX_set_cipher_list
                            SSL_CTX_set_client_CA_list
                            SSL_CTX_set_client_cert_cb
                            SSL_CTX_set_client_cert_engine
                            SSL_CTX_set_cookie_generate_cb
                            SSL_CTX_set_cookie_verify_cb
                            SSL_CTX_set_default_passwd_cb
                            SSL_CTX_set_default_passwd_cb_userdata
                            SSL_CTX_set_default_read_buffer_len
                            SSL_CTX_set_default_verify_dir
                            SSL_CTX_set_default_verify_file
                            SSL_CTX_set_default_verify_paths
                            SSL_CTX_set_ex_data
                            SSL_CTX_set_generate_session_id
                            SSL_CTX_set_info_callback
                            SSL_CTX_set_msg_callback
                            SSL_CTX_set_not_resumable_session_callback
                            SSL_CTX_set_options
                            SSL_CTX_set_psk_client_callback
                            SSL_CTX_set_psk_server_callback
                            SSL_CTX_set_purpose
                            SSL_CTX_set_quiet_shutdown
                            SSL_CTX_set_security_callback
                            SSL_CTX_set_security_level
                            SSL_CTX_set_session_id_context
                            SSL_CTX_set_ssl_version
                            SSL_CTX_set_timeout
                            SSL_CTX_set_tmp_dh_callback
                            SSL_CTX_set_trust
                            SSL_CTX_set_verify
                            SSL_CTX_set_verify_depth
                            SSL_CTX_up_ref
                            SSL_CTX_use_PrivateKey
                            SSL_CTX_use_PrivateKey_ASN1
                            SSL_CTX_use_PrivateKey_file
                            SSL_CTX_use_RSAPrivateKey
                            SSL_CTX_use_RSAPrivateKey_ASN1
                            SSL_CTX_use_RSAPrivateKey_file
                            SSL_CTX_use_certificate
                            SSL_CTX_use_certificate_ASN1
                            SSL_CTX_use_certificate_chain_file

                            // ... many other symbols ...

@refack
Copy link
Contributor

refack commented Oct 6, 2018

After using dumpbin I can see that Node.js really exports many OpenSSL symbols, here is the full output:

Yes, node embbeds and exports the full openSSL library. From a design perspective to me it actually makes sense to reuse it. Otherwise you get two different copies of the library loaded to your process...
But I agree that it should be your choice. I'll research this a little bit and try to give you an answer.

@JCMais
Copy link
Author

JCMais commented Oct 6, 2018

Thank you for taking some time to look into this @refack, if I can help in any way let me know.

@Not4me
Copy link

Not4me commented Oct 16, 2018

@JCMais Problem solved? I need some help too....
I build addon that use custom functions from custom OpenSSL. It works fine on windows 10 with node 8.12.0 or 10.9.0. But i have problems with ubuntu 17.10

I compiled library on Windows and Linux. They works. After i compiled node addon with node-gyp.

{
  "targets": [
    {
      "target_name": "cryptowrap",
      "sources": [ "libcryptowrap.cc" ],
      "cflags!": [ "-fno-exceptions" ],
      "cflags_cc!": [ "-fno-exceptions" ],
      "conditions": [
        ['OS=="win"', {
          'include_dirs': [
            "<!(node -e \"require('nan')\")",
            '<(module_root_dir)/deps/include'
          ],
          'msvs_settings': {
            'VCCLCompilerTool': {
              'AdditionalOptions': [ '/EHsc' ]
            }
          },
          'conditions': [
            ['target_arch=="ia32"', {
              'libraries': [
                '<(module_root_dir)/deps/win/x86/libcryptowrap.lib',
                '<(module_root_dir)/deps/win/x86/libcrypto-1_1.lib'
              ],
              'copies': [{
                'destination': '<(PRODUCT_DIR)',
                'files': [
                  '<(module_root_dir)/deps/win/x86/libcryptowrap.dll',
                  '<(module_root_dir)/deps/win/x86/libcrypto-1_1.dll',
                ]
              }]
            }],
            ['target_arch=="x64"', {
              'msvs_configuration_platform': 'x64',
              'msbuild_toolset': 'v141_xp',
              'libraries': [
                '<(module_root_dir)/deps/win/x64/libcryptowrap.lib',
                '<(module_root_dir)/deps/win/x64/libcrypto-1_1.lib'
              ],
              'copies': [{
                'destination': '<(PRODUCT_DIR)',
                'files': [
                  '<(module_root_dir)/deps/win/x64/libcryptowrap.dll',
                  '<(module_root_dir)/deps/win/x64/libcrypto-1_1.dll',
                ]
              }]
            }]
          ]
        }],
        ['OS=="linux"', {
          'include_dirs': [
            "src",
            "<!(node -e \"require('nan')\")",
            '<(module_root_dir)/deps/include'
          ],
          'ldflags': [
            '-Wl,-R<(module_root_dir)/deps/linux',
            '-L<(module_root_dir)/deps/linux'
          ],
          'conditions': [
            ['target_arch=="ia32"', {
              'libraries': [ '-lcryptowrap' ]
            }],
            ['target_arch=="x64"', {
              'libraries': [ '-lcryptowrap' ]
            }]
          ]
        }]
      ]
    }
  ]
}

After i call my function in node via addon from libcryptowrap.dll and custom libcrypto.dll in Windows. And it's all fine. On Linux Node call openssl functions from her dependencies, not from custom libcrypto.so. What should I change on Linux ? I want to call a function that uses functions from my library and custom libcrypto.so

@JCMais
Copy link
Author

JCMais commented Oct 17, 2018

@Not4me I still have the issue, I have not found a workaround yet.

@Not4me
Copy link

Not4me commented Oct 17, 2018

@JCMais Why all work on Windows 10 with my files, but work in linux. How i can recompile node without shared openssl?

@JCMais
Copy link
Author

JCMais commented Nov 13, 2018

@Not4me from what I remember it's not possible to do that easily, but I might be wrong here.

@refack sorry to bother, but have you had any luck with this? I kind of gave up trying already.

@Not4me
Copy link

Not4me commented Nov 23, 2018

I created custom openssl with some my functions. Recompile nodejs with custom openssl. Now it works fine for me

@hsgreen
Copy link

hsgreen commented Dec 11, 2018

I'm having the same kind of problem.
I have a shared library "A" that links against the system installed OpenSSL library that I link to in my custom node addon "B". My shared library "A" is used by other applications and libraries on my system.

I have tried compiling the same system level OpenSSL version as a static library, then linking it directly into my shared library "A". It appears to work when I use "A" in any non-NodeJS applications.

However, when I use library "A" in my node addon with node versions 10.x+, I get segfaults because the build seems to prefer the OpenSSL symbols in the node binary over my shared library "A", even within the shared library "A".

I really think NodeJS should NOT export OpenSSL symbols:

  • It requires all native addons to be built against the binary compatible version as the NodeJS - every NodeJS version update that changes openSSL ABI means you have to install a new OpenSSL and rebuild all your dependent libraries to this new OpenSSL version.
  • The current system does not give you a clean way to use other libraries in native addons that use other OpenSSL versions. For example, I should not have to go rebuild the system provided libcurl to match the OpenSSL version NodeJS happens to be using.

I have my original issue here

@JCMais
Copy link
Author

JCMais commented Jan 17, 2019

So I've some updates. I upgraded libssh2 to the latest version, to check if it was going to work like I thought, and tried to rebuild the addon, however I got another error about undefined symbols.

Turns out that Node.js does not export the whole OpenSSL library, but just a subset of it, I've opened the following issue on Node.js repo asking how to specify new symbols to be exported: nodejs/node#23293, and made a PR exporting the new ones here: nodejs/node#23344

But even with that, older versions would still not have those symbols, so my idea was to basically build OpenSSL anyway, using a similar OpenSSL version (1.1.0) and the compiler probably would grab the symbols, but I still could not build my own OpenSSL.

The error this time was related to more symbols not being defined, however the cause was different now. The OpenSSL version I was trying to compile were including files that didn't have the symbols it expected them to have, and the motive for that was because node-gyp adds the directory <(node_root_dir)/include/node as include_dir (seen here https://github.com/nodejs/node-gyp/blob/53b607426/addon.gypi#L20), and inside that dir there is an openssl directory. So any include on the C++ header files, like #include <openssl/buffer.h>, would search there, instead of the dirs of my own OpenSSL version. The solution was to just rename the openssl folder while the addon was being built, and rename it back afterwards.

The TLDR here is that everything seems very hacky and I really hope a better way to use a different version of OpenSSL is found / shown.

@bnoordhuis
Copy link
Member

I'm closing this because ultimately it's not a node-gyp issue. It's been discussed at length at nodejs/node and nodejs/help, see the linked issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants