zeromq 5 runs on bitbucket pipeline, but 6 doesn't

[mehiatt]

We have a simple pipeline that installs zeromq with

• npm i

If we use 5.3.1 version it works fine. If we install the latest:

npm ERR! code 1
npm ERR! path /opt/atlassian/pipelines/agent/build/server/node_modules/zeromq
npm ERR! command failed
npm ERR! command sh -c -- (shx test -f ./script/build.js || run-s build.js) && cross-env npm_config_build_from_source=true node-gyp-build
npm ERR! make: Entering directory '/opt/atlassian/pipelines/agent/build/server/node_modules/zeromq/build'
npm ERR! TOUCH 7722eaf02fcc073df28ffd1fa831e71544edfd42.intermediate
npm ERR! ACTION binding_gyp_libzmq_target_build_libzmq 7722eaf02fcc073df28ffd1fa831e71544edfd42.intermediate
npm ERR! Downloading libzmq source from https://github.com/zeromq/libzmq/archive/20de92ac0a2b2b9a1869782a429df68f93c3625e.tar.gz
npm ERR! Building libzmq Release
npm ERR! cmake -S "libzmq-20de92ac0a2b2b9a1869782a429df68f93c3625e" -B ./build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX="/opt/atlassian/pipelines/agent/build/server/node_modules/zeromq/build/libzmq" -DCMAKE_INSTALL_LIBDIR=lib -DBUILD_STATIC=ON -DBUILD_TESTS=OFF -DBUILD_SHARED=OFF -DWITH_DOCS=OFF -DWITH_LIBSODIUM=ON -DWITH_LIBSODIUM_STATIC=ON
npm ERR! cmake --build ./build --config Release --target install
npm ERR! TOUCH Release/obj.target/libzmq.stamp
npm ERR! CXX(target) Release/obj.target/zeromq/src/context.o
npm ERR! rm 7722eaf02fcc073df28ffd1fa831e71544edfd42.intermediate
npm ERR! make: Leaving directory '/opt/atlassian/pipelines/agent/build/server/node_modules/zeromq/build'
npm ERR! /lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.33' not found (required by /opt/atlassian/pipelines/agent/build/server/node_modules/zeromq/prebuilds/linux-x64/node.napi.glibc.node)
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.1.0
npm ERR! gyp info using node@16.19.1 | linux | x64
npm ERR! gyp info find Python using Python version 3.7.3 found at "/usr/bin/python3"
npm ERR! gyp http GET https://nodejs.org/download/release/v16.19.1/node-v16.19.1-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.19.1/node-v16.19.1-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v16.19.1/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.19.1/SHASUMS256.txt
npm ERR! gyp info spawn /usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/opt/atlassian/pipelines/agent/build/server/node_modules/zeromq/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/root/.cache/node-gyp/16.19.1/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/16.19.1',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/16.19.1/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/opt/atlassian/pipelines/agent/build/server/node_modules/zeromq',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! /bin/sh: 1: cmake: not found
npm ERR! /bin/sh: 1: cmake: not found
npm ERR! In file included from ../src/context.h:4,
npm ERR! from ../src/context.cc:2:
npm ERR! ../src/prefix.h:8:10: fatal error: zmq.h: No such file or directory
npm ERR! #include <zmq.h>
npm ERR! ^~~~~~~
npm ERR! compilation terminated.
npm ERR! make: *** [[zeromq.target.mk:132](http://zeromq.target.mk:132/): Release/obj.target/zeromq/src/context.o] Error 1
npm ERR! gyp ERR! build error
npm ERR! gyp ERR! stack Error: 'make' failed with exit code: 2
npm ERR! gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:201:23)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)
npm ERR! gyp ERR! System Linux 5.15.0-1028-aws
npm ERR! gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /opt/atlassian/pipelines/agent/build/server/node_modules/zeromq
npm ERR! gyp ERR! node -v v16.19.1
npm ERR! gyp ERR! node-gyp -v v9.1.0
npm ERR! gyp ERR! not ok

I am on the latest node and npm version. I don't run into this locally either so it makes testing hard.

[mehiatt]

I found with bitbucket, despite their base image, I needed to install cmake. There is also not much information on this elsewhere online.

I guess previous versions had binary included that bitbucket could use, but 6 doesn't?

adding

• apt-get update && apt-get install cmake -y
  before the install resolved this issue.

[Ashilex]

I have same problem installing zeromq on mac.
I try npm install zeromq@6.0.0-beta.16 --build-from-source
but get kinda same errors as @mehiatt does.

`npm ERR! code 1
npm ERR! path /Users/ghiffo/armadio-del-codice/proveJS/casciaro-mammino/middleware-menager-0mq/node_modules/zeromq
npm ERR! command failed
npm ERR! command sh -c -- (shx test -f ./script/build.js || run-s build.js) && cross-env npm_config_build_from_source=true node-gyp-build
npm ERR! dlopen(/Users/ghiffo/armadio-del-codice/proveJS/casciaro-mammino/middleware-menager-0mq/node_modules/zeromq/prebuilds/darwin-arm64/node.napi.glibc.node, 0x0001): Library not loaded: /Users/runner/arm-target/opt/libsodium/lib/libsodium.23.dylib
npm ERR! Referenced from: <908F1187-CACA-38F3-B226-4E16BB991AD8> /Users/ghiffo/armadio-del-codice/proveJS/casciaro-mammino/middleware-menager-0mq/node_modules/zeromq/prebuilds/darwin-arm64/node.napi.glibc.node

npm ERR! Reason: tried: '/Users/runner/arm-target/opt/libsodium/lib/libsodium.23.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/arm-target/opt/libsodium/lib/libsodium.23.dylib' (no such file), '/Users/runner/arm-target/opt/libsodium/lib/libsodium.23.dylib' (no such file), '/usr/local/lib/libsodium.23.dylib' (no such file), '/usr/lib/libsodium.23.dylib' (no such file, not in dyld cache)
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.1.0
npm ERR! gyp info using node@18.13.0 | darwin | arm64
npm ERR! gyp info find Python using Python version 3.9.4 found at "/Library/Frameworks/Python.framework/Versions/3.9/bin/python3"
npm ERR! gyp info spawn /Library/Frameworks/Python.framework/Versions/3.9/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/Users/ghiffo/.nvm/versions/node/v18.13.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/ghiffo/armadio-del-codice/proveJS/casciaro-mammino/middleware-menager-0mq/node_modules/zeromq/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/ghiffo/.nvm/versions/node/v18.13.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/ghiffo/Library/Caches/node-gyp/18.13.0/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/Users/ghiffo/Library/Caches/node-gyp/18.13.0',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/Users/ghiffo/.nvm/versions/node/v18.13.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/Users/ghiffo/Library/Caches/node-gyp/18.13.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/Users/ghiffo/armadio-del-codice/proveJS/casciaro-mammino/middleware-menager-0mq/node_modules/zeromq',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! /bin/sh: pkg-config: command not found
npm ERR! gyp: Call to 'pkg-config libsodium --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: gyp failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.onCpExit (/Users/ghiffo/.nvm/versions/node/v18.13.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:284:16)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Darwin 22.3.0
npm ERR! gyp ERR! command "/Users/ghiffo/.nvm/versions/node/v18.13.0/bin/node" "/Users/ghiffo/.nvm/versions/node/v18.13.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /Users/ghiffo/armadio-del-codice/proveJS/casciaro-mammino/middleware-menager-0mq/node_modules/zeromq
npm ERR! gyp ERR! node -v v18.13.0
npm ERR! gyp ERR! node-gyp -v v9.1.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/ghiffo/.npm/_logs/2023-02-28T23_22_39_163Z-debug-0.log`

[mehiatt]

@Ashilex
pkg-config: command not found
is the thing you are missing. For me it was cmake. I had this locally on my windows machine, but in bitbucket I needed to add

• sudo apt-get install cmake
  And then it worked. The documentation for this wasn't great - and it's not entirely on zmq but the errors here could be better. It would be nice if ZMQ checked for needed packages/libs/binaries before attempting to build. If it's not on the system throw package missing in a more readable error.

I would look into install pkg-config on your mac and see if that helps.

For ZMQ: check if packages needed to build binaries are present and throw more readable errors if they are not.

[Ashilex]

@mehiatt Will give it a look tomorrow. thank you for now!

[Ashilex]

Thanks again to @mehiatt !

Installing zeromq was solved on M1 mac installing pkg-config and cmake.

After npm install zeromq@6.0.0-beta.16 --build-from-source
this error was prompt.

but get kinda same errors as @mehiatt does.

Do:
brew install pkg-config

and later, after verifying that pkg-config was not enough alone:

brew install cmake.

try to npm install with same command as before and it worked fine.

Don't know if this is just a workaround or if it should work differently when prebuilts will be available for m1 as well.