Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(*): bump of sharp for Node v12 compatibility #13646

Merged
merged 18 commits into from
May 3, 2019
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/gatsby-plugin-manifest/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
},
"dependencies": {
"@babel/runtime": "^7.0.0",
"sharp": "^0.21.3"
"sharp": "^0.22.1"
},
"devDependencies": {
"@babel/cli": "^7.0.0",
Expand Down
2 changes: 1 addition & 1 deletion packages/gatsby-plugin-manifest/src/gatsby-node.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import fs from "fs"
import path from "path"
import sharp from "sharp"
import sharp from "./safe-sharp"
pieh marked this conversation as resolved.
Show resolved Hide resolved
import createContentDigest from "gatsby/dist/utils/create-content-digest"
import { defaultIcons, doesIconExist, addDigestToPath } from "./common"

Expand Down
137 changes: 137 additions & 0 deletions packages/gatsby-plugin-manifest/src/safe-sharp.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
// this is very hacky (don't hate me), but at least it's not in [JSFuck](http://www.jsfuck.com/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😆

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I will remove that ;)


const childProcess = require(`child_process`)
const semver = require(`semver`)

const originalConsoleError = console.error
const restoreConsoleError = () => {
console.error = originalConsoleError
}

const handleMessage = msg => {
if (msg.includes(`Incompatible library version: sharp.node requires`)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a better cross platform heuristic here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned in the comment in PR - this issue seems to be OSX specific, I tested my reproduction on Ubuntu and Windows and it didn't happen there, so while not ideal check - should be good enough

restoreConsoleError()

let msg = [
`It looks like there are multiple versions of "sharp" module installed.`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`It looks like there are multiple versions of "sharp" module installed.`,
`It looks like there are multiple versions of the "sharp" module installed.`,

`Please update packages that depend on "sharp".`,
eclectic-coding marked this conversation as resolved.
Show resolved Hide resolved
``,
]

try {
let tmpMsg = []
// npm list seems to work in yarn installed projects as well
const { dependencies } = JSON.parse(
childProcess.execSync(`npm list sharp --json`, {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this factor in a lock file? Would this be inconsistent in case there are both lock files?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, it will make use of package-lock.json if it's available and potentially show not accurate results. If lock file doesn't exist, it will traverse node_modules. This makes it very hard to support, but I think maybe opting out of it if both yarn and npm lock files exist seems reasonable?

encoding: `utf-8`,
})
)

const findSharpVersion = dependency => {
if (dependency.dependencies.sharp) {
return dependency.dependencies.sharp.version
}

for (let depName of Object.keys(dependency.dependencies)) {
const v = findSharpVersion(dependency.dependencies[depName])
if (v) {
return v
}
}

return null
}

const { latestVersion, topLevelPackages } = Object.keys(
dependencies
).reduce(
(acc, depName) => {
const sharpVersion = findSharpVersion(dependencies[depName])
if (sharpVersion) {
acc.topLevelPackages[depName] = sharpVersion

if (
!acc.latestVersion ||
semver.gt(sharpVersion, acc.latestVersion)
) {
acc.latestVersion = sharpVersion
}
}

return acc
},
{
latestVersion: undefined,
topLevelPackages: {},
}
)

let packagesToUpdate = []
// list top level dependencies
tmpMsg = tmpMsg.concat([
`List of installed packages that depend on sharp:`,
...Object.keys(topLevelPackages).map(depName => {
const sharpVersion = topLevelPackages[depName]
if (sharpVersion !== latestVersion) {
packagesToUpdate.push(depName)
}
return ` - ${depName}${
sharpVersion
? ` (${sharpVersion})${
sharpVersion !== latestVersion ? ` - needs update` : ``
}`
: ``
}`
}),
])

if (packagesToUpdate.length > 0) {
tmpMsg = tmpMsg.concat([
``,
`If you are using npm, run:`,
``,
`npm install ${packagesToUpdate.join(` `)}`,
``,
`If you are using yarn, run:`,
``,
`yarn add ${packagesToUpdate.join(` `)}`,
])
}

msg = msg.concat(tmpMsg)
} catch {
msg = msg.concat([
`To get a list of installed packages that depend on "sharp" try running:`,
` - npm list sharp (if you use npm)`,
` - yarn why sharp (if you use yarn)`,
` and update packages that depend on version older than latest listed in output of above command.`,
])
}

msg = msg.concat([
``,
`If an older version of "sharp" still persists and this error is displayed after updating your packages, open an issue in the package's repository and request them to update the "sharp" dependency.`,
])

console.error(msg.join(`\n`))
}
}

let sharp
try {
// sharp@0.22.1 uses console.error and then process.exit and doesn't throw
pieh marked this conversation as resolved.
Show resolved Hide resolved
// so to capture error and provide meaningful troubleshooting guide
// we intercept console.error calls and add special handling.
console.error = (msg, ...args) => {
originalConsoleError(msg, ...args)
handleMessage(msg)
}
sharp = require(`sharp`)
} catch (e) {
handleMessage(e.toString())
throw e
} finally {
restoreConsoleError()
}

module.exports = sharp
31 changes: 31 additions & 0 deletions packages/gatsby-plugin-sharp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,37 @@ limited to the latitude/longitude information of where the picture was taken
can either leave `stripMetadata` to its default of `true`, or manually
pre-process your images with a tool such as [ExifTool][17].

## Troubleshooting

### Incompatible library version: sharp.node requires version X or later, but Z provides version Y

This means that there are multiple incompatible versions of the `sharp` package installed in `node_modules`. The complete error typically looks like this:

```
Something went wrong installing the "sharp" module

dlopen(/Users/misiek/dev/gatsby-starter-blog/node_modules/sharp/build/Release/sharp.node, 1): Library not loaded: @rpath/libglib-2.0.dylib
Referenced from: /Users/misiek/dev/gatsby-starter-blog/node_modules/sharp/build/Release/sharp.node
Reason: Incompatible library version: sharp.node requires version 6001.0.0 or later, but libglib-2.0.dylib provides version 5801.0.0
```

To fix this, you'll need to update all Gatsby plugins in the current project that depend on the `sharp` package. Here's a list of official plugins that you might need to update in case your projects uses them:

- `gatsby-plugin-sharp`
- `gatsby-plugin-manifest`
- `gatsby-remark-images-contentful`
- `gatsby-source-contentful`
- `gatsby-transformer-sharp`
- `gatsby-transformer-sqip`

To update these packages, run:

```sh
npm install gatsby-plugin-sharp gatsby-plugin-manifest gatsby-remark-images-contentful gatsby-source-contentful gatsby-transformer-sharp gatsby-transformer-sqip
```

If updating these doesn't fix the issue, your project probably uses other plugins from the community that depend on a different version of `sharp`. Try running `npm list sharp` or `yarn why sharp` to see all packages in the current project that use `sharp` and try updating them as well.

[1]: https://alistapart.com/article/finessing-fecolormatrix
[2]: http://blog.72lions.com/blog/2015/7/7/duotone-in-js
[3]: https://ines.io/blog/dynamic-duotone-svg-jade
Expand Down
2 changes: 1 addition & 1 deletion packages/gatsby-plugin-sharp/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"potrace": "^2.1.1",
"probe-image-size": "^4.0.0",
"progress": "^1.1.8",
"sharp": "^0.21.3",
"sharp": "^0.22.1",
"svgo": "^1.2.0"
},
"devDependencies": {
Expand Down
2 changes: 1 addition & 1 deletion packages/gatsby-remark-images-contentful/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"cheerio": "^1.0.0-rc.2",
"is-relative-url": "^2.0.0",
"lodash": "^4.17.10",
"sharp": "^0.21.3",
"sharp": "^0.22.1",
"unist-util-select": "^1.5.0"
},
"devDependencies": {
Expand Down
2 changes: 1 addition & 1 deletion packages/gatsby-transformer-sharp/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"fs-extra": "^7.0.0",
"potrace": "^2.1.1",
"probe-image-size": "^4.0.0",
"sharp": "^0.21.3"
"sharp": "^0.22.1"
},
"devDependencies": {
"@babel/cli": "^7.0.0",
Expand Down
62 changes: 27 additions & 35 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -5017,13 +5017,6 @@ binary-extensions@^1.0.0:
resolved "https://registry.yarnpkg.com/binary-extensions/-/binary-extensions-1.11.0.tgz#46aa1751fb6a2f93ee5e689bb1087d4b14c6c205"
integrity sha1-RqoXUftqL5PuXmibsQh9SxTGwgU=

bindings@^1.3.1:
version "1.4.0"
resolved "https://registry.yarnpkg.com/bindings/-/bindings-1.4.0.tgz#909efa49f2ebe07ecd3cb136778f665052040127"
integrity sha512-7znEVX22Djn+nYjxCWKDne0RRloa9XfYa84yk3s+HkE3LpDYZmhArYr9O9huBoHY3/oXispx5LorIX7Sl2CgSQ==
dependencies:
file-uri-to-path "1.0.0"

bl@^1.0.0:
version "1.2.2"
resolved "https://registry.yarnpkg.com/bl/-/bl-1.2.2.tgz#a160911717103c07410cef63ef51b397c025af9c"
Expand Down Expand Up @@ -6246,10 +6239,10 @@ color@^3.0.0:
color-convert "^1.9.1"
color-string "^1.5.2"

color@^3.1.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/color/-/color-3.1.0.tgz#d8e9fb096732875774c84bf922815df0308d0ffc"
integrity sha512-CwyopLkuRYO5ei2EpzpIh6LqJMt6Mt+jZhO5VI5f/wJLZriXQE32/SSqzmrh+QB+AZT81Cj8yv+7zwToW8ahZg==
color@^3.1.1:
version "3.1.1"
resolved "https://registry.yarnpkg.com/color/-/color-3.1.1.tgz#7abf5c0d38e89378284e873c207ae2172dcc8a61"
integrity sha512-PvUltIXRjehRKPSy89VnDWFKY58xyhTLyxIg21vwQBI6qLwZNPmC8k3C1uytIgFKEpOIzN4y32iPm8231zFHIg==
dependencies:
color-convert "^1.9.1"
color-string "^1.5.2"
Expand Down Expand Up @@ -9358,11 +9351,6 @@ file-type@^8.1.0:
resolved "https://registry.yarnpkg.com/file-type/-/file-type-8.1.0.tgz#244f3b7ef641bbe0cca196c7276e4b332399f68c"
integrity sha512-qyQ0pzAy78gVoJsmYeNgl8uH8yKhr1lVhW7JbzJmnlRi0I4R2eEDEJZVKG8agpDnLpacwNbDhLNG/LMdxHD2YQ==

file-uri-to-path@1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz#553a7b8446ff6f684359c445f1e37a05dacc33dd"
integrity sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==

filename-regex@^2.0.0:
version "2.0.1"
resolved "https://registry.yarnpkg.com/filename-regex/-/filename-regex-2.0.1.tgz#c1c4b9bee3e09725ddb106b75c1e301fe2f18b26"
Expand Down Expand Up @@ -14897,11 +14885,16 @@ name-all-modules-plugin@^1.0.1:
resolved "https://registry.yarnpkg.com/name-all-modules-plugin/-/name-all-modules-plugin-1.0.1.tgz#0abfb6ad835718b9fb4def0674e06657a954375c"
integrity sha1-Cr+2rYNXGLn7Te8GdOBmV6lUN1w=

nan@^2.10.0, nan@^2.12.1:
nan@^2.10.0:
version "2.12.1"
resolved "https://registry.yarnpkg.com/nan/-/nan-2.12.1.tgz#7b1aa193e9aa86057e3c7bbd0ac448e770925552"
integrity sha512-JY7V6lRkStKcKTvHO5NVSQRv+RV+FIL5pvDoLiAtSL9pKlC5x9PKQcZDsq7m4FO4d57mkhC6Z+QhAh3Jdk5JFw==

nan@^2.13.2:
version "2.13.2"
resolved "https://registry.yarnpkg.com/nan/-/nan-2.13.2.tgz#f51dc7ae66ba7d5d55e1e6d4d8092e802c9aefe7"
integrity sha512-TghvYc72wlMGMVMluVo9WRJc0mB8KxxF/gZ4YYFy7V2ZQX9l7rgbPg7vjS9mt6U5HXODVFVI2bOduCzwOMv/lw==

nan@^2.9.2:
version "2.11.0"
resolved "https://registry.yarnpkg.com/nan/-/nan-2.11.0.tgz#574e360e4d954ab16966ec102c0c049fd961a099"
Expand Down Expand Up @@ -14992,10 +14985,10 @@ no-case@^2.2.0, no-case@^2.3.2:
dependencies:
lower-case "^1.1.1"

node-abi@^2.2.0:
version "2.4.3"
resolved "https://registry.yarnpkg.com/node-abi/-/node-abi-2.4.3.tgz#43666b7b17e57863e572409edbb82115ac7af28b"
integrity sha512-b656V5C0628gOOA2kwcpNA/bxdlqYF9FvxJ+qqVX0ctdXNVZpS8J6xEUYir3WAKc7U0BH/NRlSpNbGsy+azjeg==
node-abi@^2.7.0:
version "2.8.0"
resolved "https://registry.yarnpkg.com/node-abi/-/node-abi-2.8.0.tgz#bd2e88dbe6a6871e6dd08553e0605779325737ec"
integrity sha512-1/aa2clS0pue0HjckL62CsbhWWU35HARvBDXcJtYKbYR7LnIutmpxmXbuDMV9kEviD2lP/wACOgWmmwljghHyQ==
dependencies:
semver "^5.4.1"

Expand Down Expand Up @@ -16895,18 +16888,18 @@ potrace@^2.1.1:
dependencies:
jimp "^0.2.24"

prebuild-install@^5.2.2:
version "5.2.2"
resolved "https://registry.yarnpkg.com/prebuild-install/-/prebuild-install-5.2.2.tgz#237888f21bfda441d0ee5f5612484390bccd4046"
integrity sha512-4e8VJnP3zJdZv/uP0eNWmr2r9urp4NECw7Mt1OSAi3rcLrbBRxGiAkfUFtre2MhQ5wfREAjRV+K1gubvs/GPsA==
prebuild-install@^5.3.0:
version "5.3.0"
resolved "https://registry.yarnpkg.com/prebuild-install/-/prebuild-install-5.3.0.tgz#58b4d8344e03590990931ee088dd5401b03004c8"
integrity sha512-aaLVANlj4HgZweKttFNUVNRxDukytuIuxeK2boIMHjagNJCiVKWFsKF4tCE3ql3GbrD2tExPQ7/pwtEJcHNZeg==
dependencies:
detect-libc "^1.0.3"
expand-template "^2.0.3"
github-from-package "0.0.0"
minimist "^1.2.0"
mkdirp "^0.5.1"
napi-build-utils "^1.0.1"
node-abi "^2.2.0"
node-abi "^2.7.0"
noop-logger "^0.1.1"
npmlog "^4.0.1"
os-homedir "^1.0.1"
Expand Down Expand Up @@ -19013,19 +19006,18 @@ shallowequal@^1.0.2:
resolved "https://registry.yarnpkg.com/shallowequal/-/shallowequal-1.1.0.tgz#188d521de95b9087404fd4dcb68b13df0ae4e7f8"
integrity sha512-y0m1JoUZSlPAjXVtPPW70aZWfIL/dSP7AFkRnniLCrK/8MDKog3TySTBmckD+RObVxH0v4Tox67+F14PdED2oQ==

sharp@^0.21.3:
version "0.21.3"
resolved "https://registry.yarnpkg.com/sharp/-/sharp-0.21.3.tgz#381937de66c123687f2ac7186f85921c6bb19cdd"
integrity sha512-5qZk8r+YgfyztLEKkNez20Wynq/Uh1oNyP5T/3gTYwt2lBYGs9iDs5m0yVsZEPm8eVBbAJhS08J1wp/g+Ai1Qw==
sharp@^0.22.1:
version "0.22.1"
resolved "https://registry.yarnpkg.com/sharp/-/sharp-0.22.1.tgz#a67c0e75567f03dd5a7861b901fec04072c5b0f4"
integrity sha512-lXzSk/FL5b/MpWrT1pQZneKe25stVjEbl6uhhJcTULm7PhmJgKKRbTDM/vtjyUuC/RLqL2PRyC4rpKwbv3soEw==
dependencies:
bindings "^1.3.1"
color "^3.1.0"
color "^3.1.1"
detect-libc "^1.0.3"
fs-copy-file-sync "^1.1.1"
nan "^2.12.1"
nan "^2.13.2"
npmlog "^4.1.2"
prebuild-install "^5.2.2"
semver "^5.6.0"
prebuild-install "^5.3.0"
semver "^6.0.0"
simple-get "^3.0.3"
tar "^4.4.8"
tunnel-agent "^0.6.0"
Expand Down