Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support-Request: node:12.18.1-alpine does not compile node-re2 #72

Closed
Uzlopak opened this issue Jun 19, 2020 · 8 comments
Closed

Support-Request: node:12.18.1-alpine does not compile node-re2 #72

Uzlopak opened this issue Jun 19, 2020 · 8 comments
Assignees
@uhop
Labels
confirmed A confirmed bug. enhancement

Comments

@Uzlopak
Copy link

Uzlopak commented Jun 19, 2020

My dockerfile is

FROM node:12.18.1-alpine as re2-builder

WORKDIR /opt

RUN apk add python make g++ \
	&& npm install re2@1.15.0

When building I get this:

Writing to build/Release/re2.node ...

> re2@1.15.0 verify-build /opt/node_modules/re2
> node scripts/verify-build.js

internal/modules/cjs/loader.js:1188
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /opt/node_modules/re2/build/Release/re2.node)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1188:18)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:879:14)
    at Module.require (internal/modules/cjs/loader.js:1026:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/opt/node_modules/re2/re2.js:3:13)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:879:14)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! re2@1.15.0 verify-build: `node scripts/verify-build.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the re2@1.15.0 verify-build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-06-19T21_07_53_335Z-debug.log
Building locally ...

> re2@1.15.0 rebuild /opt/node_modules/re2
> node-gyp rebuild
...

Does somebody has a working docker file?
@uhop uhop self-assigned this Jun 20, 2020
@uhop uhop added the bug A reported bug. label Jun 20, 2020
@uhop
Copy link
Owner

uhop commented Jun 20, 2020

Could you share your Dockerfile so I can repro the problem? All I need is the OS + Node part.

@Uzlopak
Copy link
Author

Uzlopak commented Jun 20, 2020

Thank you for your fast feedback.

The error occurs in the few lines I provided in the above Dockerfile. Don't get me wrong... I get a binary but I don't know if RE2 is potentially buggy when used, as my c++ skills are limited. Also I have to potentially explain my colleagues if this is a problem or not.

In later build steps I copy the node_modules/re2 folder into the production image.

@uhop
Copy link
Owner

uhop commented Jun 20, 2020

The error in the original post indicates that a downloaded image does not work. Next, it proceeds to build from sources. That's the expected behavior. I would look into why the downloaded image didn't work, but it shouldn't matter.

Regarding copying the folder into the production image — that is unsupported. re2 is a C++ project based on NAN and built by node-gyp. It is compiled to a platform-specific image. It means that the result depends on the details of OS, installed system libraries, a CPU architecture, processor supported features, Node version, and so on. That's why there is a fail-safe mechanism, which verifies the binary and rebuilds it if there is any problem with it.

For example, I used to build my project (not re2-related) on one type of AWS EC2 VM and deploy on bigger AWS EC2 VM only to get random crashes related to "unsupported instructions". So generally I would advise against it.

@uhop
Copy link
Owner

uhop commented Jun 20, 2020

I googled for the error and it turned out to be a common problem for Alpine:

It looks like there is a missing link in /lib and Alpine doesn't come with the glibc loader by default requiring installing libc6-compat explicitly. I don't know if both are required as in the last link, or only one of them is sufficient (first two links). Please try and let me know so I can update Wiki with this information.

Thank you in advance!

@Uzlopak
Copy link
Author

Uzlopak commented Jun 22, 2020

I already had tried those potential solutions, but none of them worked. Maybe it is no issue, as I get a binary file but just get that error.

FROM node:12.18.1-alpine as re2-builder

WORKDIR /opt

RUN apk add python make g++ libc6-compat \
	&& npm install re2@1.15.0

Building docker image...
Sending build context to Docker daemon  669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
 ---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
 ---> Using cache
 ---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ libc6-compat    && npm install re2@1.15.0
 ---> Running in d82870d4a550
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/22) Installing binutils (2.33.1-r0)
(2/22) Installing gmp (6.1.2-r1)
(3/22) Installing isl (0.18-r0)
(4/22) Installing libgomp (9.2.0-r4)
(5/22) Installing libatomic (9.2.0-r4)
(6/22) Installing mpfr4 (4.0.2-r1)
(7/22) Installing mpc1 (1.1.0-r1)
(8/22) Installing gcc (9.2.0-r4)
(9/22) Installing musl-dev (1.1.24-r2)
(10/22) Installing libc-dev (0.7.2-r0)
(11/22) Installing g++ (9.2.0-r4)
(12/22) Installing libc6-compat (1.1.24-r2)
(13/22) Installing make (4.2.1-r2)
(14/22) Installing libbz2 (1.0.8-r1)
(15/22) Installing expat (2.2.9-r1)
(16/22) Installing libffi (3.2.1-r6)
(17/22) Installing gdbm (1.13-r1)
(18/22) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(19/22) Installing ncurses-libs (6.1_p20200118-r4)
(20/22) Installing readline (8.0.1-r0)
(21/22) Installing sqlite-libs (3.30.1-r2)
(22/22) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 38 packages
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

> re2@1.15.0 install /opt/node_modules/re2
> node scripts/install-from-cache.js --artifact build/Release/re2.node

Trying https://github.com/uhop/node-re2/releases/download/1.15.0/linux-x64-72.br ...
Writing to build/Release/re2.node ...

> re2@1.15.0 verify-build /opt/node_modules/re2
> node scripts/verify-build.js

internal/modules/cjs/loader.js:1188
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /opt/node_modules/re2/build/Release/re2.node)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1188:18)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:879:14)
    at Module.require (internal/modules/cjs/loader.js:1026:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/opt/node_modules/re2/re2.js:3:13)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:879:14)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! re2@1.15.0 verify-build: `node scripts/verify-build.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the re2@1.15.0 verify-build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-06-22T05_51_14_903Z-debug.log
Building locally ...

> re2@1.15.0 rebuild /opt/node_modules/re2
> node-gyp rebuild

make: Entering directory '/opt/node_modules/re2/build'
  CXX(target) Release/obj.target/re2/lib/addon.o
  CXX(target) Release/obj.target/re2/lib/new.o
  CXX(target) Release/obj.target/re2/lib/exec.o
  CXX(target) Release/obj.target/re2/lib/test.o
  CXX(target) Release/obj.target/re2/lib/match.o
  CXX(target) Release/obj.target/re2/lib/replace.o
  CXX(target) Release/obj.target/re2/lib/search.o
  CXX(target) Release/obj.target/re2/lib/split.o
  CXX(target) Release/obj.target/re2/lib/to_string.o
  CXX(target) Release/obj.target/re2/lib/accessors.o
  CXX(target) Release/obj.target/re2/lib/util.o
  CXX(target) Release/obj.target/re2/vendor/re2/bitstate.o
  CXX(target) Release/obj.target/re2/vendor/re2/compile.o
  CXX(target) Release/obj.target/re2/vendor/re2/dfa.o
  CXX(target) Release/obj.target/re2/vendor/re2/filtered_re2.o
  CXX(target) Release/obj.target/re2/vendor/re2/mimics_pcre.o
  CXX(target) Release/obj.target/re2/vendor/re2/nfa.o
  CXX(target) Release/obj.target/re2/vendor/re2/onepass.o
  CXX(target) Release/obj.target/re2/vendor/re2/parse.o
  CXX(target) Release/obj.target/re2/vendor/re2/perl_groups.o
  CXX(target) Release/obj.target/re2/vendor/re2/prefilter.o
  CXX(target) Release/obj.target/re2/vendor/re2/prefilter_tree.o
  CXX(target) Release/obj.target/re2/vendor/re2/prog.o
  CXX(target) Release/obj.target/re2/vendor/re2/re2.o
  CXX(target) Release/obj.target/re2/vendor/re2/regexp.o
  CXX(target) Release/obj.target/re2/vendor/re2/set.o
  CXX(target) Release/obj.target/re2/vendor/re2/simplify.o
  CXX(target) Release/obj.target/re2/vendor/re2/stringpiece.o
  CXX(target) Release/obj.target/re2/vendor/re2/tostring.o
  CXX(target) Release/obj.target/re2/vendor/re2/unicode_casefold.o
  CXX(target) Release/obj.target/re2/vendor/re2/unicode_groups.o
  CXX(target) Release/obj.target/re2/vendor/util/pcre.o
  CXX(target) Release/obj.target/re2/vendor/util/rune.o
  CXX(target) Release/obj.target/re2/vendor/util/strutil.o
  SOLINK_MODULE(target) Release/obj.target/re2.node
  COPY Release/re2.node
make: Leaving directory '/opt/node_modules/re2/build'
npm WARN saveError ENOENT: no such file or directory, open '/opt/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/opt/package.json'
npm WARN opt No description
npm WARN opt No repository field.
npm WARN opt No README data
npm WARN opt No license field.

+ re2@1.15.0
added 101 packages from 74 contributors and audited 101 packages in 83.967s

1 package is looking for funding
  run `npm fund` for details

found 0 vulnerabilities

Removing intermediate container d82870d4a550

FROM node:12.18.1-alpine as re2-builder

WORKDIR /opt

RUN apk add python make g++ libc6-compat \
	&& ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2 \
	&& npm install re2@1.15.0

Building docker image...
Sending build context to Docker daemon  669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
 ---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
 ---> Using cache
 ---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ libc6-compat     && ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2  && npm install re2@1.15.0
 ---> Running in 1ebed36ae9ab
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/22) Installing binutils (2.33.1-r0)
(2/22) Installing gmp (6.1.2-r1)
(3/22) Installing isl (0.18-r0)
(4/22) Installing libgomp (9.2.0-r4)
(5/22) Installing libatomic (9.2.0-r4)
(6/22) Installing mpfr4 (4.0.2-r1)
(7/22) Installing mpc1 (1.1.0-r1)
(8/22) Installing gcc (9.2.0-r4)
(9/22) Installing musl-dev (1.1.24-r2)
(10/22) Installing libc-dev (0.7.2-r0)
(11/22) Installing g++ (9.2.0-r4)
(12/22) Installing libc6-compat (1.1.24-r2)
(13/22) Installing make (4.2.1-r2)
(14/22) Installing libbz2 (1.0.8-r1)
(15/22) Installing expat (2.2.9-r1)
(16/22) Installing libffi (3.2.1-r6)
(17/22) Installing gdbm (1.13-r1)
(18/22) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(19/22) Installing ncurses-libs (6.1_p20200118-r4)
(20/22) Installing readline (8.0.1-r0)
(21/22) Installing sqlite-libs (3.30.1-r2)
(22/22) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 38 packages
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

> re2@1.15.0 install /opt/node_modules/re2
> node scripts/install-from-cache.js --artifact build/Release/re2.node

Trying https://github.com/uhop/node-re2/releases/download/1.15.0/linux-x64-72.br ...
Writing to build/Release/re2.node ...

> re2@1.15.0 verify-build /opt/node_modules/re2
> node scripts/verify-build.js

terminate called after throwing an instance of 'std::system_error'
  what():  No error information
Done.
npm WARN saveError ENOENT: no such file or directory, open '/opt/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/opt/package.json'
npm WARN opt No description
npm WARN opt No repository field.
npm WARN opt No README data
npm WARN opt No license field.

+ re2@1.15.0
added 101 packages from 74 contributors and audited 101 packages in 8.323s

1 package is looking for funding
  run `npm fund` for details

found 0 vulnerabilities


FROM node:12.18.1-alpine as re2-builder

WORKDIR /opt

RUN apk add python make g++ gcompat \
	&& ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2 \
	&& npm install re2@1.15.0


Building docker image...
Sending build context to Docker daemon  669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
 ---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
 ---> Using cache
 ---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ gcompat     && ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2       && npm install re2@1.15.0
 ---> Running in ca6d2a41f183
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/23) Installing binutils (2.33.1-r0)
(2/23) Installing gmp (6.1.2-r1)
(3/23) Installing isl (0.18-r0)
(4/23) Installing libgomp (9.2.0-r4)
(5/23) Installing libatomic (9.2.0-r4)
(6/23) Installing mpfr4 (4.0.2-r1)
(7/23) Installing mpc1 (1.1.0-r1)
(8/23) Installing gcc (9.2.0-r4)
(9/23) Installing musl-dev (1.1.24-r2)
(10/23) Installing libc-dev (0.7.2-r0)
(11/23) Installing g++ (9.2.0-r4)
(12/23) Installing libucontext (0.1.3-r1)
(13/23) Installing gcompat (0.4.0-r0)
(14/23) Installing make (4.2.1-r2)
(15/23) Installing libbz2 (1.0.8-r1)
(16/23) Installing expat (2.2.9-r1)
(17/23) Installing libffi (3.2.1-r6)
(18/23) Installing gdbm (1.13-r1)
(19/23) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(20/23) Installing ncurses-libs (6.1_p20200118-r4)
(21/23) Installing readline (8.0.1-r0)
(22/23) Installing sqlite-libs (3.30.1-r2)
(23/23) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 39 packages
ln: /lib/ld-linux-x86-64.so.2: File exists
The command '/bin/sh -c apk add python make g++ gcompat     && ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2       && npm install re2@1.15.0' returned a non-zero code: 1

FROM node:12.18.1-alpine as re2-builder

WORKDIR /opt

RUN apk add python make g++ gcompat \
&& npm install re2@1.15.0

Building docker image...
Sending build context to Docker daemon  669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
 ---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
 ---> Using cache
 ---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ gcompat         && npm install re2@1.15.0
 ---> Running in 94d1f3eee966
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/23) Installing binutils (2.33.1-r0)
(2/23) Installing gmp (6.1.2-r1)
(3/23) Installing isl (0.18-r0)
(4/23) Installing libgomp (9.2.0-r4)
(5/23) Installing libatomic (9.2.0-r4)
(6/23) Installing mpfr4 (4.0.2-r1)
(7/23) Installing mpc1 (1.1.0-r1)
(8/23) Installing gcc (9.2.0-r4)
(9/23) Installing musl-dev (1.1.24-r2)
(10/23) Installing libc-dev (0.7.2-r0)
(11/23) Installing g++ (9.2.0-r4)
(12/23) Installing libucontext (0.1.3-r1)
(13/23) Installing gcompat (0.4.0-r0)
(14/23) Installing make (4.2.1-r2)
(15/23) Installing libbz2 (1.0.8-r1)
(16/23) Installing expat (2.2.9-r1)
(17/23) Installing libffi (3.2.1-r6)
(18/23) Installing gdbm (1.13-r1)
(19/23) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(20/23) Installing ncurses-libs (6.1_p20200118-r4)
(21/23) Installing readline (8.0.1-r0)
(22/23) Installing sqlite-libs (3.30.1-r2)
(23/23) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 39 packages
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

> re2@1.15.0 install /opt/node_modules/re2
> node scripts/install-from-cache.js --artifact build/Release/re2.node

Trying https://github.com/uhop/node-re2/releases/download/1.15.0/linux-x64-72.br ...
Writing to build/Release/re2.node ...

> re2@1.15.0 verify-build /opt/node_modules/re2
> node scripts/verify-build.js

terminate called after throwing an instance of 'std::system_error'
  what():  No error information
Done.
npm WARN saveError ENOENT: no such file or directory, open '/opt/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/opt/package.json'
npm WARN opt No description
npm WARN opt No repository field.
npm WARN opt No README data
npm WARN opt No license field.

+ re2@1.15.0
added 101 packages from 74 contributors and audited 101 packages in 8.691s

1 package is looking for funding
  run `npm fund` for details

@uhop
Copy link
Owner

uhop commented Jun 22, 2020

Thank you for samples. While I assume it works for you, I will look at what is going on Alpine. In the past we had problems with versions of glibc. I suspect that we have something like that again.

I can suppress errors so users are none the wiser, yet the output can provide a valuable information in cases like this one.

@uhop uhop added confirmed A confirmed bug. enhancement and removed bug A reported bug. labels Jul 5, 2020
@uhop
Copy link
Owner

uhop commented Jul 6, 2020

That was very useful. The problem is that Alpine uses musl rather than glibc. There is a compat library, but it doesn't emulate glibc at the level we need. In the upcoming version I am planning to produce musl versions to save the compilation time.

Meanwhile, in https://github.com/uhop/node-re2/tree/1.15.1 I started to suppress "scary" errors and made the verification more robust.

@uhop uhop closed this as completed Jul 6, 2020
@uhop
Copy link
Owner

uhop commented Jul 8, 2020

I added precompiled versions for musl-based distros (like Alpine) in 1.15.2. Could you check if it works for you?

BTW, when I was debugging it I've noticed some random segmentation faults when running in the cloud. According to Google it is some kind of a weird stack-related Node/Alpine bug, which was fixed in Node 13.x. Previous versions work fine on Alpine 3.9, but not on newer versions.

For example:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@uhop uhop

Labels
confirmed A confirmed bug. enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@uhop @Uzlopak