Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Yarn doesn't correctly handle optional dependencies #1075

Closed
lgeiger opened this issue Jan 3, 2017 · 6 comments · May be fixed by qcif/data-curator#563
Closed

Yarn doesn't correctly handle optional dependencies #1075

lgeiger opened this issue Jan 3, 2017 · 6 comments · May be fixed by qcif/data-curator#563
Labels
feature

Comments

@lgeiger
Copy link
Contributor

lgeiger commented Jan 3, 2017

  • Version: 10.16
  • Target: all

First of all thanks for this great tool! 🥇

I experience one problem only present when using electron-builder together with yarn:

electron-builder will fail if a optional dependency errors during install. npm handles this correctly and doesn't throw a error.
yarn itself also handles optional dependencies correctly if electron-builder's postinstall/rebuild script isn't used.

Here is a concrete example:
We are using electron-builder inside nteract but one of our dependencies requires canvas as a optional dependency (which we don't need inside Electron). Installing canvas will likely fail on some systems and will make our whole build fail if we use yarn for installing.
@develar
Copy link
Member

develar commented Jan 3, 2017

Please set env DEBUG=electron-builder and attach log of the terminal output.

@lgeiger
Copy link
Contributor Author

lgeiger commented Jan 3, 2017 •
edited
Loading

Thanks for the quick response!

Here is the log of a fresh install:

yarn install v0.18.1
info No lockfile found.
[1/4] 🔍  Resolving packages...
warning plotly.js > mapbox-gl > feature-filter@2.2.0: This package has been merged into the 'mapbox-gl-style-spec' package
warning plotly.js > mapbox-gl > mapbox-gl-function@1.3.0: This package has been merged with the 'mapbox-gl-style-spec' package
warning plotly.js > mapbox-gl > vector-tile@1.3.0: This module is now under the @mapbox namespace: install @mapbox/vector-tile instead
warning plotly.js > mapbox-gl > vt-pbf > vector-tile@1.3.0: This module is now under the @mapbox namespace: install @mapbox/vector-tile instead
warning electron-builder > progress-stream > through2 > xtend > object-keys@0.4.0:
warning react-json-tree > react-base16-styling > color-space > husl@5.0.3: Project renamed to HSLuv
warning plotly.js > mapbox-gl > geojson-rewind > geojson-area@0.1.0: This module is now under the @mapbox namespace: install @mapbox/geojson-area instead
[2/4] 🚚  Fetching packages...
warning 7zip-bin-win@2.0.2: The platform "darwin" is incompatible with this module.
info "7zip-bin-win@2.0.2" is an optional dependency and failed compatibility check. Excluding it from installation.
warning 7zip-bin-linux@1.0.3: The platform "darwin" is incompatible with this module.
info "7zip-bin-linux@1.0.3" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
[1/5] ⠠ zeromq
[-/5] ⠠ waiting...
[3/5] ⠠ canvas
[4/5] ⠠ fsevents
error Error running install script for optional dependency: "/Users/lukasgeiger/code/nteract/node_modules/canvas: Command failed.\nExit code: 1\nCommand: sh\nArguments: -c node-gyp rebuild\nDirectory: /Users/lukasgeiger/code/nteract/node_modules/canvas\nOutput:\ngyp info it worked if it ends with ok\ngyp info using node-gyp@3.4.0\ngyp info using node@7.3.0 | darwin | x64\ngyp info spawn /usr/local/bin/python2\ngyp info spawn args [ '/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp/gyp/gyp_main.py',\ngyp info spawn args   'binding.gyp',\ngyp info spawn args   '-f',\ngyp info spawn args   'make',\ngyp info spawn args   '-I',\ngyp info spawn args   '/Users/lukasgeiger/code/nteract/node_modules/canvas/build/config.gypi',\ngyp info spawn args   '-I',\ngyp info spawn args   '/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp/addon.gypi',\ngyp info spawn args   '-I',\ngyp info spawn args   '/Users/lukasgeiger/.node-gyp/7.3.0/include/node/common.gypi',\ngyp info spawn args   '-Dlibrary=shared_library',\ngyp info spawn args   '-Dvisibility=default',\ngyp info spawn args   '-Dnode_root_dir=/Users/lukasgeiger/.node-gyp/7.3.0',\ngyp info spawn args   '-Dnode_gyp_dir=/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp',\ngyp info spawn args   '-Dnode_lib_file=node.lib',\ngyp info spawn args   '-Dmodule_root_dir=/Users/lukasgeiger/code/nteract/node_modules/canvas',\ngyp info spawn args   '--depth=.',\ngyp info spawn args   '--no-parallel',\ngyp info spawn args   '--generator-output',\ngyp info spawn args   'build',\ngyp info spawn args   '-Goutput_dir=.' ]\nPackage cairo was not found in the pkg-config search path.\nPerhaps you should add the directory containing `cairo.pc'\nto the PKG_CONFIG_PATH environment variable\nNo package 'cairo' found\ngyp: Call to './util/has_lib.sh freetype' returned exit status 0 while in binding.gyp. while trying to load binding.gyp\ngyp ERR! configure error \ngyp ERR! stack Error: `gyp` failed with exit code: 1\ngyp ERR! stack     at ChildProcess.onCpExit (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp/lib/configure.js:305:16)\ngyp ERR! stack     at emitTwo (events.js:106:13)\ngyp ERR! stack     at ChildProcess.emit (events.js:191:7)\ngyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)\ngyp ERR! System Darwin 16.3.0\ngyp ERR! command \"/usr/local/Cellar/node/7.3.0/bin/node\" \"/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/
success Saved lockfile.
$ install-app-deps
Rebuilding native production dependencies for darwin:x64
  electron-builder Spawning /usr/local/Cellar/node/7.3.0/bin/node /usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/bin/yarn.js run install -- +0ms
yarn run v0.18.1
$ node-gyp rebuild
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: /usr/local/Cellar/node/7.3.0/bin/node exited with code 1
Error output:
gyp info it worked if it ends with ok
gyp info using node-gyp@3.4.0
gyp info using node@7.3.0 | darwin | x64
gyp info spawn /usr/local/bin/python2
gyp info spawn args [ '/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/lukasgeiger/code/nteract/node_modules/canvas/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/lukasgeiger/.electron-gyp/.node-gyp/iojs-1.4.13/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/lukasgeiger/.electron-gyp/.node-gyp/iojs-1.4.13',
gyp info spawn args   '-Dnode_gyp_dir=/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=iojs.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/lukasgeiger/code/nteract/node_modules/canvas',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
Package cairo was not found in the pkg-config search path.
Perhaps you should add the directory containing `cairo.pc'
to the PKG_CONFIG_PATH environment variable
No package 'cairo' found
gyp: Call to './util/has_lib.sh freetype' returned exit status 0 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp/lib/configure.js:305:16)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR! System Darwin 16.3.0
gyp ERR! command "/usr/local/Cellar/node/7.3.0/bin/node" "/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/lukasgeiger/code/nteract/node_modules/canvas
gyp ERR! node -v v7.3.0
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok
error Command failed with exit code 1.

    at ChildProcess.childProcess.once.code (/Users/lukasgeiger/code/nteract/node_modules/electron-builder/src/util/util.ts:136:14)
    at ChildProcess.g (events.js:292:16)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:885:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
From previous event:
    at spawn (/Users/lukasgeiger/code/nteract/node_modules/electron-builder/src/util/util.ts:99:3)
    at /Users/lukasgeiger/code/nteract/node_modules/electron-builder/src/yarn.ts:140:50
    at runCallback (timers.js:651:20)
    at tryOnImmediate (timers.js:624:5)
    at processImmediate [as _immediateCallback] (timers.js:596:5)
From previous event:
    at /Users/lukasgeiger/code/nteract/node_modules/electron-builder/src/yarn.ts:140:27
From previous event:
    at rebuild (/Users/lukasgeiger/code/nteract/node_modules/electron-builder/out/yarn.js:81:22)
    at /Users/lukasgeiger/code/nteract/node_modules/electron-builder/src/yarn.ts:15:11
    at Generator.next (<anonymous>)
    at runCallback (timers.js:651:20)
    at tryOnImmediate (timers.js:624:5)
    at processImmediate [as _immediateCallback] (timers.js:596:5)
From previous event:
    at installOrRebuild (/Users/lukasgeiger/code/nteract/node_modules/electron-builder/out/yarn.js:33:21)
    at /Users/lukasgeiger/code/nteract/node_modules/electron-builder/src/cli/install-app-deps.ts:34:2
    at Generator.next (<anonymous>)
From previous event:
    at main (/Users/lukasgeiger/code/nteract/node_modules/electron-builder/out/cli/install-app-deps.js:36:21)
    at Object.<anonymous> (/Users/lukasgeiger/code/nteract/node_modules/electron-builder/out/cli/install-app-deps.js:76:1)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.runMain (module.js:605:10)
    at run (bootstrap_node.js:420:7)
    at startup (bootstrap_node.js:139:9)
    at bootstrap_node.js:535:3
error Command failed with exit code 255.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

If I manually remove every mention canvas(the failing package) in yarn.lock the install works fine. But manually editing yarn.lock isn't recommended at all.

@lgeiger
Copy link
Contributor Author

lgeiger commented Jan 3, 2017

It looks like we're running into yarnpkg/yarn#2274.
Since yarn doesn't cleanup correctly if a module fails to install, electron-builder tries to install it again and will fail.

@develar
Copy link
Member

develar commented Jan 4, 2017 •
edited
Loading

It is not yarn issue, but electron-builder issue — because yarn doesn't have rebuild command, we invoke yarn for each native package to rebuild it. And, we require that ALL invocations must be successful. But for optional dep we should just warn about it and ignore.

Not moved to backlog — will be fixed soon.

@develar develar added the feature label Jan 4, 2017
@lgeiger
Copy link
Contributor Author

lgeiger commented Jan 4, 2017

I already made a PR to yarn which does a proper cleanup of failed to install optional modules wich will fix this issue.

👍 For fixing it in electron-builder as well.

@develar develar closed this as completed in f67b7d2 Jan 9, 2017
@develar
Copy link
Member

develar commented Jan 9, 2017

First of all thanks for this great tool!

Thank you for warm words :) Fix landed in the 11.2.1.

@dependabot-preview dependabot-preview bot mentioned this issue Mar 14, 2018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Bump electron-builder from 19.55.2 to 20.5.1 qcif/data-curator
2 participants
@develar @lgeiger